Splunk Search

list all fields within a sourcetype

a212830
Champion

Hi,

Is there a way to display all fields being used by a sourcetype, without the values?

Runals
Motivator

If you have 6x you could run the following search. Adjust the initial search to limit scope and then adjust the dedup piece to attempt to capture enough of your sourcetype to get the majority of the fields. There is a balance to be struck /shrug

<search to limit scope> | dedup 20 sourcetype punct | table * | fields - _raw date_* index linecount punct eventtype time*pos splunk_server  | foreach * [eval <<FIELD>> = '<<FIELD>>'."##".sourcetype."##||" ] | stats max(*) as * |  transpose | rename "row 1" as sourcetype column as field | rex field=sourcetype "##(?<sourcetype>[^#]+)" | eval fieldsort = lower(field) | dedup sourcetype field | sort sourcetype fieldsort | table sourcetype field

_d_
Splunk Employee
Splunk Employee

It depends on the version of Splunk that you're running. If you're on 5.0 or above, you can use the new fieldsummary command.

Ex. index=my_index sourcetype=my_sourcetype | fieldsummary

fieldsummary does not work on metadata so it needs to scan all events for all possible fields. This means that it can be pretty expensive if not scoped properly in time.
Additional info here: http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Fieldsummary

MonkeyK
Builder

I like it! if we are only interested in fields likely on every record of a sourcetype, we could speed it up by only taking the first record found:

index=my_index sourcetype=my_sourcetype | head 1| fieldsummary

furthermore, if we are only interested in a summary the field values (say to search on where certain named fields appear), we can aggregate those

index=pan_logs sourcetype=pan:traffic 
| head 1
| fieldsummary 
| table field
| mvcombine delim=", " field 
| nomv field

jaxjohnny2000
Builder

This works well. You can remove some extraneous information if you want just the fields, count and value

index=my_index sourcetype=my_sourcetype | fieldsummary | fields - max mean min stdev numeric_count is_exact

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

No. The fields that exist are determined dynamically depending on the data, and so can not be determined without looking at sufficient data from that sourcetype. There is no way without looking at every piece of data to know with 100% certainty that you have seen all possible fields.

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to May Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars this month. This ...

They're back! Join the SplunkTrust and MVP at .conf24

With our highly anticipated annual conference, .conf, comes the fez-wearers you can trust! The SplunkTrust, as ...

Enterprise Security Content Update (ESCU) | New Releases

Last month, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...