Splunk Search

Is there any way to *selectively* avoid automatic field extraction?

sideview
SplunkTrust
SplunkTrust

I have multiline events where there's a fair bit of auto-kv extraction that is good, but then there's a lot of noise as well.

I can create regexes to match the really noisy bits and this works well. I nearly get perfect coverage on the high-value fields that I actually need.

The problem is that even when I have a regex matching, sometimes the same field appears in a foo=bar pair further down into the event, and the autoKV match is clobbering my more explicit regex match. Can someone point me in the right direction? (Obviously the answer is to make the logging less deranged, but it's not an option atm unfortunateley)

-------------------------------------
Fields: Field=GoodValue;foo=bar;jackiechan=theman
AnotherGoodField = AnotherGoodValue
User = bob
.....
Field : BadNoisyValueThatClobbersMyGoodValue
-------------------------------------

One idea is - can I tell the autokv stuff not to pay attention to colons? All the colon stuff is hideously noisy in this sourcetype.

Tags (1)

Ron_Naken
Splunk Employee
Splunk Employee

You could disable KV discovery for a particular source, host, or sourcetype in props.conf. Maybe this would help:

PROPS.CONF:

[mysourcetype]
KV_MODE = none 

gkanapathy
Splunk Employee
Splunk Employee

The colon matching isn't handled by the KV_MODE switch, but by a different search-time extract.

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

It does the colon matching in WinEventLog:: (and maybe WMI::) sourcetypes.

sideview
SplunkTrust
SplunkTrust

The trouble is that there's a huge number of fields for which I need the normal equals sign autokv extraction to work. I tried specifying a manual regex for equals but there's a bunch of subtlety that autokv just does really well when you look at it under a microscope and I couldnt get the manual regex to the desired standard.

0 Karma

sideview
SplunkTrust
SplunkTrust

From what I can tell it's definitely matching colons all over the place.

0 Karma

Ron_Naken
Splunk Employee
Splunk Employee

I don't believe it looks for colons, by default.

0 Karma
Get Updates on the Splunk Community!

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...

Updated Team Landing Page in Splunk Observability

We’re making some changes to the team landing page in Splunk Observability, based on your feedback. The ...