Deployment Architecture

Splunk Search Head and Fields

sumnerm
Path Finder

I have a configuration on a splunk indexer including search time fields extractions (using a DELIMS/FIELDS config in transforms.conf via sourcetype stanza in props.conf). The props.conf/transforms.conf file are in /etc/system/local

This works well locally on the indexer. However, I've now added a search head, and the fields are not available to pick in the search head, and searches which refer to the fields yield no results.

Copying the props.conf/transforms.conf files to the /etc/system/local on the search head doesn't appear to change the behaviour.

Where should I define search-time field extractions on a search head where the data is being indexed on a peer? Do I need to define with an app?

Do I need to remove the local configuration from the indexer, and if so will direct searches against that indexer still work?

Thanks

Martin


Update from comment:

The search head has been restarted. I've simplified down the props.conf and transforms.conf file so it covers just the relevant stanzas:

props.conf

[Test-PageData] 
TIME_PREFIX = ^ 
TIME_FORMAT = %m/%d/%y %H:%M:%S.%3N 
MAX_TIMESTAMP_LOOKAHEAD = 30 
SHOULD_LINEMERGE = false 
REPORT-csv = Test-PageData-FieldSplit 

transforms.conf

[Test-PageData-FieldSplit] 
DELIMS = "," FIELDS = "Timestamp","SessionID","PageURL"....

Using these search-time field extractions works fine on the search peer locally, just not from the search head.

Tags (1)
1 Solution

sumnerm
Path Finder

I have this working now.

I changed the props.conf file to be based on [host::...] rather than sourcetype and it worked straight away.

This may be related to the numeration of the sourcetypes. The sourcetype is being listed in Splunk as Test-PageData-[nnn] with [nnn] being some numeric reference. So it looks like this gets picked up with the local search but not via the search-peer.

Looking at other answers the sourcetype issue is potentially related to the fact that the files being monitored are .csv files. The sourcetype is manually configured in the monitor stanza.

Some related questions:

http://answers.splunk.com/questions/490/why-do-variations-in-sourcetype-appear http://answers.splunk.com/questions/723/how-to-override-splunk-renaming-sourcetypes-xxx-1-if-field-n...

Thanks for those who have offered help, I'm happy with the resolution now.

View solution in original post

0 Karma

sumnerm
Path Finder

It is in there. Thanks for the help, but I think I have the answer now (see below)

0 Karma

sumnerm
Path Finder

I have this working now.

I changed the props.conf file to be based on [host::...] rather than sourcetype and it worked straight away.

This may be related to the numeration of the sourcetypes. The sourcetype is being listed in Splunk as Test-PageData-[nnn] with [nnn] being some numeric reference. So it looks like this gets picked up with the local search but not via the search-peer.

Looking at other answers the sourcetype issue is potentially related to the fact that the files being monitored are .csv files. The sourcetype is manually configured in the monitor stanza.

Some related questions:

http://answers.splunk.com/questions/490/why-do-variations-in-sourcetype-appear http://answers.splunk.com/questions/723/how-to-override-splunk-renaming-sourcetypes-xxx-1-if-field-n...

Thanks for those who have offered help, I'm happy with the resolution now.

0 Karma

hulahoop
Splunk Employee
Splunk Employee

Is 'Test-PageData' in the output when you run the show command on the CLI ("splunk show config props")?

0 Karma

Simeon
Splunk Employee
Splunk Employee

Fields can be extracted at search or indexing time. For indexed fields, the extraction configuration needs to reside on the indexer. For search time extraction, the configuration should reside on the search head. If you have only copied the files without reloading the config, you will likely not see the field extractions. There are other possibilities, but without seeing the exact extractions (config files) it will be hard to debug. You can reload the extractions on the fly with the following command:

| extract reload=true

http://www.splunk.com/base/Documentation/latest/SearchReference/Extract

In lieu of this, you could also restart the search head to reload the configuration.

0 Karma

sumnerm
Path Finder

The search head has been restarted. I've simplified down the props.conf and transforms.conf file so it covers just the relevant stanzas:

props.conf

[Test-PageData]
TIME_PREFIX = ^
TIME_FORMAT = %m/%d/%y %H:%M:%S.%3N
MAX_TIMESTAMP_LOOKAHEAD = 30
SHOULD_LINEMERGE = false
REPORT-csv = Test-PageData-FieldSplit

transforms.conf

[Test-PageData-FieldSplit]
DELIMS = ","
FIELDS = "Timestamp","SessionID","PageURL"....

Using these search-time field extractions works fine on the search peer locally, just not from the search head.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...