Hi,
I am uploading logs in JSON format into Splunk.
I want to enable automatic field extraction.
Is there any setting for this, or does Splunk always enable automatic field extraction by default?
Thanks,
Deepak
Hi deepak02!
Splunk has both indexed extractions and searchtime extractions for json.
They are found in props.conf.
http://docs.splunk.com/Documentation/Splunk/6.5.2/Admin/Propsconf
INDEXED_EXTRACTIONS = < CSV|W3C|TSV|PSV|JSON >
* Tells Splunk the type of file and the extraction and/or parsing method
Splunk should use on the file.
CSV - Comma separated value format
TSV - Tab-separated value format
PSV - pipe "|" separated value format
W3C - W3C Extended Extended Log File Format
JSON - JavaScript Object Notation format
* These settings default the values of the remaining settings to the
appropriate values for these known formats.
* Defaults to unset.
*If you are using a forwarder, be sure to put the props.conf on the forwarder! not just the indexer!
Also as an FYI, Splunk has a searchtime extractions available:
KV_MODE = [none|auto|auto_escaped|multi|json|xml]
* Used for search-time field extractions only.
* Specifies the field/value extraction mode for the data.
* Set KV_MODE to one of the following:
* none: if you want no field/value extraction to take place.
* auto: extracts field/value pairs separated by equal signs.
* auto_escaped: extracts fields/value pairs separated by equal signs and
honors \" and \\ as escaped sequences within quoted
values, e.g field="value with \"nested\" quotes"
* multi: invokes the multikv search command to expand a tabular event into
multiple events.
* xml : automatically extracts fields from XML data.
* json: automatically extracts fields from JSON data.
* Setting to 'none' can ensure that one or more user-created regexes are not
overridden by automatic field/value extraction for a particular host,
source, or source type, and also increases search performance.
* Defaults to auto.
* The 'xml' and 'json' modes will not extract any fields when used on data
that isn't of the correct format (JSON or XML).
OR
AUTO_KV_JSON = [true|false]
* Used for search-time field extractions only.
* Specifies whether to try json extraction automatically.
* Defaults to true.
What ever way you decide, I encourage you to try a sample of your json using the Add Data wizard, to ensure you are getting the extractions you expect.
http://docs.splunk.com/Documentation/Splunk/6.5.2/Data/Setsourcetype