Splunk Search

Is there any way to *selectively* avoid automatic field extraction?

sideview
SplunkTrust
SplunkTrust

I have multiline events where there's a fair bit of auto-kv extraction that is good, but then there's a lot of noise as well.

I can create regexes to match the really noisy bits and this works well. I nearly get perfect coverage on the high-value fields that I actually need.

The problem is that even when I have a regex matching, sometimes the same field appears in a foo=bar pair further down into the event, and the autoKV match is clobbering my more explicit regex match. Can someone point me in the right direction? (Obviously the answer is to make the logging less deranged, but it's not an option atm unfortunateley)

-------------------------------------
Fields: Field=GoodValue;foo=bar;jackiechan=theman
AnotherGoodField = AnotherGoodValue
User = bob
.....
Field : BadNoisyValueThatClobbersMyGoodValue
-------------------------------------

One idea is - can I tell the autokv stuff not to pay attention to colons? All the colon stuff is hideously noisy in this sourcetype.

Tags (1)

Ron_Naken
Splunk Employee
Splunk Employee

You could disable KV discovery for a particular source, host, or sourcetype in props.conf. Maybe this would help:

PROPS.CONF:

[mysourcetype]
KV_MODE = none 

gkanapathy
Splunk Employee
Splunk Employee

The colon matching isn't handled by the KV_MODE switch, but by a different search-time extract.

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

It does the colon matching in WinEventLog:: (and maybe WMI::) sourcetypes.

sideview
SplunkTrust
SplunkTrust

The trouble is that there's a huge number of fields for which I need the normal equals sign autokv extraction to work. I tried specifying a manual regex for equals but there's a bunch of subtlety that autokv just does really well when you look at it under a microscope and I couldnt get the manual regex to the desired standard.

0 Karma

sideview
SplunkTrust
SplunkTrust

From what I can tell it's definitely matching colons all over the place.

0 Karma

Ron_Naken
Splunk Employee
Splunk Employee

I don't believe it looks for colons, by default.

0 Karma
Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...