Splunk Search

How to get extracted fields count per index?

harshal_chakran
Builder

Hi,
Trying to get the count of extracted fields per index. I am using the following search for this:

index=*|fieldsummary|stats count
This gives me the entire list of all fields in all index.

Also "stats count by index" doesnt work as fieldsummary doesnt have index value. How can we get the field count per index.

Labels (1)
0 Karma

BDein
Explorer

How about this one:

index=summary sourcetype=stash source IN (Summary_Error_*) 
| fields + * 
| stats mode(*) AS #* by search_name 
| transpose header_field=search_name column_name=fieldName 
| eval type=if(match(fieldName,"#(_raw|date_.*|eventtype|tag.*|index|sourcetype|host|info_.*|punct|time*.?pos|search_name|search|search_now|splunk_server.*|linecount)"),"internal","custom") 
| stats count(*) AS * BY type 
| transpose column_name=search_name header_field=type 
| rename "row 1" AS fieldCount 
| search search_name!=fieldName

It was used for summary index by might as well be used other ways.

 

Tags (3)
0 Karma

BDein
Explorer

Can also be accomplished over time by:

index=summary sourcetype=stash source IN (Summary_Error_*) 
| fields + * 
| bin _time span=1d 
| eval search_name=search_name+"#"+_time
| stats mode(*) AS * by  search_name
| transpose 1000 header_field=search_name column_name=fieldName 
| eval type=if(match(fieldName,"(_raw|_time|date_.*|eventtype|tag.*|index|sourcetype|host|info_.*|punct|time*.?pos|search_name|search|search_now|splunk_server.*|linecount)"),"internal","custom")
| stats count(*) AS * BY type
| transpose 1000  column_name=search_name header_field=type
| rename "row 1" AS fieldCount
| search search_name!=fieldName
| rex field=search_name "(?<search_name>[^#]+)#(?<_time>.*)"
| stats max(*) as * BY _time search_name
0 Karma

somesoni2
SplunkTrust
SplunkTrust

Give this a try

index=* | chart limit=0 count(*) as * by index | untable index field value | stats count as fieldcount by index
0 Karma

renems
Communicator

Hi, you just like to know how many times a field has a value? (for a certain index)
I like simple, so how about this?

index= sourcetype= field_name=* | stats count(field_name)

By adding the wildcard for your field, you only search on events that have a value for your field.
If the load on this index is too heavy, or you like to do this regularly, you could also consider writing the results to a summary index. ( | collect index= sourcetype=)

0 Karma

jplumsdaine22
Influencer

This one has me stumped in terms of how to achieve this with a search. Bear in mind though even if this is possible that the count of extracted fields will vary depending on the app context, the user that you run the search as, and could change constantly depending on the time ranges that you are using.

Is there any chance you could share your use case? There may be another way to achieve what ever it is you are trying to do!

If you're not fussed to do it in a single search then you could run the following search:

 index=* |   stats values(*) as * by index

and then process the results elsewhere (eg excel, shell script, etc)

0 Karma

harshal_chakran
Builder

I am planning to get the extract fields count per index for past 7 days duration and then compare it with the fields count for today.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...