Splunk Search

Search-Time Field Extraction

A4orce84
New Member

Hello Everyone,

I had a quick question about Field Extraction and replication (copying) the specific field extraction setup to a different Splunk Environment.

Scenario:
We currently have a sandbox with a Splunk environment and I was testing getting field extraction to work correctly with a specific set of logs. I copied the logs to the sandbox, and was able to successfully get the field extraction for this specific log working correctly.

I tried to copy the .conf files (specifically the props.conf and the inputs.conf) which detailed this specific Field Extraction to our real PRODUCTION Splunk environment, to see if I could get the new field extractions to display.

HOWEVER, on search-time the field extractions refuse to come up in the search-head and I am unsure why.

If anyone could offer any assistance, it would be greatly appreciated!

Thanks,

--Asif Ahmad

Tags (3)
0 Karma

A4orce84
New Member

friendly bump! 😃

Thanks everyone for your time and help!

0 Karma

A4orce84
New Member

Hey Kristian,

Sorry for the delayed response. Yes, the files are exactly the same, it is basically a log file which contains some apache log file information.

The source/sourcetype stanza is also the same file that I copied between both the sandbox and production environment. We are currently using heavy forwarders and the props.conf does in fact exist on the forwarder as well.

If you could let me know which files you are needing, I can provide them to you. Thank you in advance to all your time and help, it is greatly appreciated!

--Asif

0 Karma

kristian_kolb
Ultra Champion

see update above /k

0 Karma

kristian_kolb
Ultra Champion

If the log files are identical, and the regexes in the conf files are corretly copied, you may wish to verify that the sandbox and production environments are similar.

Is the source/sourcetype the same? (that is what you use in the props.conf stanza)
Are you using heavy forwarders in the production environment? In that case the props.conf settings should be on the forwarder instead of the indexer, since that is where the parsing phase takes place.

Please tell us more about your environments and perhaps post the extractions/conf-files you are using.


UPDATE:

Hm, I might have been a bit unclear in my previous answer. And indeed, the props.conf file is used in several different stages in the data processing pipeline. In a Heavy Forwarder -> Indexer -> Search Head setup, all settings regarding line breaking, timestamps should be on the Forwarder side, since Heavy Forwarders take care of the parsing stage. All things regarding field extraction should be on the Search Head, since field extraction is normally only performed during searches.

If you want to keep it simple, you could have the same complete props.conf file on all three, though some of the settings would be ignored at each stage.

For more information see;

http://docs.splunk.com/Documentation/Splunk/latest/Deploy/Datapipeline
http://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings

Hope this helps,

Kristian

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...