Splunk Search

can i use index time and search time field extraction for a particular source type?

tpsplunk
Communicator

For a particular sourcetype I need to have two fields extracted at index time and also 10+ fields extracted at search time. what is the syntax to do this? should I have multiple sourcetype stanzas in props.conf for the same sourcetype or can i combine index and search time extraction into the same stanza?

1 Solution

jbsplunk
Splunk Employee
Splunk Employee

You can have different extractions in the same stanza, that isn't going to be a problem for you. Here is an example of something you might do

[sourcetype]

EXTRACT-searchtime = (?msi)search\s+time:\s+(?<searchtime>[^\r\n]+)[\r\n]
TRANSFORMS-indextime = indextimeextraction

View solution in original post

jbsplunk
Splunk Employee
Splunk Employee

You can have different extractions in the same stanza, that isn't going to be a problem for you. Here is an example of something you might do

[sourcetype]

EXTRACT-searchtime = (?msi)search\s+time:\s+(?<searchtime>[^\r\n]+)[\r\n]
TRANSFORMS-indextime = indextimeextraction

tpsplunk
Communicator

note that if you have a distributed environment you will end up with the index time props and transforms.conf on your indexers and the search time props and transforms.conf + fields.conf on your searchhead(s)

0 Karma

jbsplunk
Splunk Employee
Splunk Employee

That is correct.

0 Karma

tpsplunk
Communicator

oh and i should say i'd like to keep the delimiter based search time extraction because its very simple for me to maintain (i.e. i don't have to do anything when devs add new logging fields as long as they follow the delimiter format)

0 Karma

tpsplunk
Communicator

i don't want to get into the "should i be using index time extraction" discussion. let's just assume that i need to and focus on how/if i can use delimiter based search time field extraction and index time field extraction where the index time field extracted field will also be picked up by the delimiter based search time extraction. from the docs it looks like i need to set a fields.conf stanza for that field with INDEXED=FALSE, but that seems counter-intuitive (http://www.splunk.com/base/Documentation/4.2.2/Data/Configureindex-timefieldextraction ).

0 Karma

jbsplunk
Splunk Employee
Splunk Employee

Then, I guess my question becomes why is search time field extraction us a delimiter not sufficient to meet your requirements, and how is the index time extraction going to meet that requirement?

0 Karma

tpsplunk
Communicator

i don't really need to do both- its just that the delimiter based search time extraction is also going to pick up the field that i'm adding to the index time extraction.

0 Karma

jbsplunk
Splunk Employee
Splunk Employee

I am not sure why you'd need to do both a search and index time field extraction at the same time, but this could definitely cause some wierdness. Most of the time search time field extraction is the way to go. I'd say a good 80% of the time, index time field extraction isn't the right solution. It can be quite expensive, and usually isn't worth the cost.

0 Karma

tpsplunk
Communicator

ok excellent, that makes sense. currently i'm using a delimited based search time extraction. this will probably cause an overlap where the field i want to change to index time extraction will also be search time extracted. will that cause any weirdness?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...