Splunk Search

using source for a field extractions

kkalmbach
Path Finder

I seem to be having some problems with extracting fields from the "source"

In by props.conf, I have:

[my_source]
SHOULD_LINEMERGE = false
LINE_BREAKER = (---------*)
REPORT-get_service = get_service

in transforms.conf, I have:

[get_service]
SOURCE_KEY=source
REGEX = ([^/]*)_sndmsg.out
FORMAT = service::"$1"

This only seems to be half working. In the search, if I do

index=myindex sourcetype=my_source service=abc

nothing comes back.

If I split it out into a separate search:

index=myindex sourcetype=my_source | search service=abc

then everything works fine.

Also if I prepend a wildcard, everything works fine, so this also works..

index=myindex sourcetype=my_source service=*abc

If I do an

eval l=len(service)

it always come back with what I expect. Everything looks fine when I print out the service field, I just can't put it in the inital search.

Any ideas would be greatly appreciated.

Thanks

-Kevin

Tags (1)
0 Karma
1 Solution

gkanapathy
Splunk Employee
Splunk Employee

That's because field searches don't look properly at fields that aren't extracted from the raw event text (vs the source field). You can deal with this by setting

[service]
INDEXED_VALUE=false

in fields.conf. But this is terrible and will make your searches run terribly terribly slow. It will work, but the price is way too high.

You can also modifying search term from

service="abc" 

to

(service="abc" source="*abc*")

This is inconvenient. But you could instead define a macro:

[service(1)]
args = sv
definition = ((service="$sv$" source="*$sv$*"))

and invoke it in your search string with:

index=myindex myterm1 `service(abc)`

instead.

This is something that should have a prettier solution in Splunk, but that's what we can do today. It would not be bad if you filed an Enhancement Request with Splunk Support to help get this in a future release.

View solution in original post

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

That's because field searches don't look properly at fields that aren't extracted from the raw event text (vs the source field). You can deal with this by setting

[service]
INDEXED_VALUE=false

in fields.conf. But this is terrible and will make your searches run terribly terribly slow. It will work, but the price is way too high.

You can also modifying search term from

service="abc" 

to

(service="abc" source="*abc*")

This is inconvenient. But you could instead define a macro:

[service(1)]
args = sv
definition = ((service="$sv$" source="*$sv$*"))

and invoke it in your search string with:

index=myindex myterm1 `service(abc)`

instead.

This is something that should have a prettier solution in Splunk, but that's what we can do today. It would not be bad if you filed an Enhancement Request with Splunk Support to help get this in a future release.

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

There is a big difference performance-wise, yes. Doing a second search tells Splunk to actually bring back all the data (in the initial search), then filter it. Specifying it all in the initial search makes use of the full-text index, which means retrieving only the data you specify in the first place. Note that the cost of the initial query is what typically dominates search time. If there a a lot of stuff in the index that does not match the "service", then there will be a significant difference.

0 Karma

kkalmbach
Path Finder

I think that makes sense. Thanks for you answer.
I will probably go the macro route.
Also, is there a big difference (performance wise)
in doing index="..." | search service="abc"
compared to putting it all in the same inital search?
Thanks again,

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...