can i use index time and search time field extraction for a particular source type?

[tpsplunk]

For a particular sourcetype I need to have two fields extracted at index time and also 10+ fields extracted at search time. what is the syntax to do this? should I have multiple sourcetype stanzas in props.conf for the same sourcetype or can i combine index and search time extraction into the same stanza?

[jbsplunk · Accepted solution]

You can have different extractions in the same stanza, that isn't going to be a problem for you. Here is an example of something you might do

[sourcetype]

EXTRACT-searchtime = (?msi)search\s+time:\s+(?<searchtime>[^\r\n]+)[\r\n]
TRANSFORMS-indextime = indextimeextraction

[tpsplunk]

note that if you have a distributed environment you will end up with the index time props and transforms.conf on your indexers and the search time props and transforms.conf + fields.conf on your searchhead(s)

[jbsplunk]

That is correct.

[tpsplunk]

oh and i should say i'd like to keep the delimiter based search time extraction because its very simple for me to maintain (i.e. i don't have to do anything when devs add new logging fields as long as they follow the delimiter format)

[tpsplunk]

i don't want to get into the "should i be using index time extraction" discussion. let's just assume that i need to and focus on how/if i can use delimiter based search time field extraction and index time field extraction where the index time field extracted field will also be picked up by the delimiter based search time extraction. from the docs it looks like i need to set a fields.conf stanza for that field with INDEXED=FALSE, but that seems counter-intuitive (http://www.splunk.com/base/Documentation/4.2.2/Data/Configureindex-timefieldextraction ).

[jbsplunk]

Then, I guess my question becomes why is search time field extraction us a delimiter not sufficient to meet your requirements, and how is the index time extraction going to meet that requirement?

[tpsplunk]

i don't really need to do both- its just that the delimiter based search time extraction is also going to pick up the field that i'm adding to the index time extraction.

[jbsplunk]

I am not sure why you'd need to do both a search and index time field extraction at the same time, but this could definitely cause some wierdness. Most of the time search time field extraction is the way to go. I'd say a good 80% of the time, index time field extraction isn't the right solution. It can be quite expensive, and usually isn't worth the cost.

[tpsplunk]

ok excellent, that makes sense. currently i'm using a delimited based search time extraction. this will probably cause an overlap where the field i want to change to index time extraction will also be search time extracted. will that cause any weirdness?