All Apps and Add-ons

Fields Extraction Not Working

fabiocaldas
Contributor

I'm colleting WinEventLog:Application from windows machine using UniveralForwarder, the collected data is been sent to a HeavyForwarder cluster where the data is analised and if reach my criteria the data is changed using some SEDCMD and transformation, after this the data is sent to a Splunk Indexer.

Doing this, when data arrive to Splunk Indexer the automatic extraction field doesn't work. You can check that my events has well formatted messsages, like:

18/10/2013 01:25:28.7974852 PM AppName="vtex.requestcapture.windowsservice" AppVersion="7.0.1.0" Env="local" AZ="us-east-1e" WorkflowType="events processing" WorkflowInstance="lojaadcos" Level="Warn" LoggerVersion="7.0.0.0" OperationId="1789f3b5-d82a-4aa0-b6de-60ac693f5fb2" ExceptionType="applicationexception" VtexLog

18/10/2013 01:32:07.1257497 PM AppName="vtex.requestcapture.windowsservice" AppVersion="7.0.1.0" Env="local" AZ="us-east-1b" PageLoadTime="30986" Count="5" Browser="Chrome" TimeType="NetworkingTime" Account="EletronicaSantana" VtexMetric

As you can see, my messages only has a Timestamp and a sequence of Key/Values. If I do a simples search using extract auto=t like index=main | extract auto=t all fields are extracted correctly, but honestly I need it done to all data.

Now let me show my configs, to see if someone can help me to have this automatic extraction working again.

HeavyForwarder - props.conf

[WinEventLog:Application]

SEDCMD-remove_eol = s/[\n\r]/ /g

SEDCMD-remove_non_logger_event = s/(.)(VtexLog|VtexMetric|VtexMachineMetric)(.)/\1\2\3\2/g

SEDCMD-remove_primary = s/(.*)(\d\d\/\d\d\/\d\d\d\d\s\d\d:\d\d:\d\d.\d\d\d\d\d\d\d)(.+?$)/\2\3/1

SEDCMD-remove_exception_stacktrace = s/(.+)(ExceptionStack=)(.*)/\1/g

REPORT-field_extraction = logger_field_extraction

KV_MODE = none

TRANSFORMS-set = setNull,setLog,setMetric,setMachineMetric,setLogSource,setMetricSource,setMachineMetricSource,setRequestCaptureIndex

HeavyForwarder - transform.conf

[logger_field_extraction]

DELIMS = " "

[setNull]

REGEX = .

DEST_KEY = queue

FORMAT = nullQueue

[setLog]

REGEX = VtexLog

DEST_KEY = queue

FORMAT = indexQueue

[setMetric]

REGEX = VtexMetric

DEST_KEY = queue

FORMAT = indexQueue

[setMachineMetric]

REGEX = VtexMachineMetric

DEST_KEY = queue

FORMAT = indexQueue

[setLogSource]

REGEX = VtexLog

FORMAT = sourcetype::VtexLog

DEST_KEY = MetaData:Sourcetype

[setMetricSource]

REGEX = VtexMetric

FORMAT = sourcetype::VtexMetric

DEST_KEY = MetaData:Sourcetype

[setMachineMetricSource]

REGEX = VtexMachineMetric

FORMAT = sourcetype::VtexMachineMetric

DEST_KEY = MetaData:Sourcetype

[setRequestCaptureIndex]

REGEX = (AppName="vtex.requestcapture.windowsservice"|AppName="vtex.requestcapture.processwebsite")

FORMAT = rc

DEST_KEY = _MetaData:Index

As you guys can see on my HF if data matches what I'm looking for I change the source type and finally and chosse index.

Reading yestarday about how to use DELIMS to extract fields I set the DELIMS on HeavyForwarder, but I also have this on my Indexer:

Indexer - transforms.conf

[logger_field_extraction]

DELIMS = " "

Indexer - props.conf

[WinEventLog:Application]

REPORT-field_extraction = logger_field_extraction

KV_MODE = none

[source::WinEventLog:Application]

REPORT-field_extraction = logger_field_extraction

KV_MODE = none

[VtexLog]

REPORT-field_extraction = logger_field_extraction

KV_MODE = none

[VtexMetric]

REPORT-field_extraction = logger_field_extraction

KV_MODE = none

[VtexMachineMetric]

REPORT-field_extraction = logger_field_extraction

KV_MODE = none

As you guys can see I tried to apply this Delim anyway but it isn't working.

Again, I need to have all my Key/Values avaliable on my Indexer, and I'm losing this because I use a HF.

Any ideias?

0 Karma
1 Solution

fabiocaldas
Contributor

First I would like to say thanks to alacercogitatus and emiller42, because of your anwsers I knew that I was going on right direction.

So, the reason why my props and transforms conf files weren't working on my indexer/search server was the missing of metadata file exporting props to be used by system.

As soon I added:

[]

export = system

to default.meta.conf file to my app and redeployed it, the extraction started to work.

Thanks God now it's 100% now ...

View solution in original post

fabiocaldas
Contributor

First I would like to say thanks to alacercogitatus and emiller42, because of your anwsers I knew that I was going on right direction.

So, the reason why my props and transforms conf files weren't working on my indexer/search server was the missing of metadata file exporting props to be used by system.

As soon I added:

[]

export = system

to default.meta.conf file to my app and redeployed it, the extraction started to work.

Thanks God now it's 100% now ...

emiller42
Motivator

With the way your events are formatted, Splunk should have no issue automatically extracting the fields with zero configuration. I think the primary problem with your setup is the

KV_MODE = none

setting. This tells Splunk not to do any automatic key-value pair extraction, which is the opposite of what you want.

Get rid of that setting in all of your props.conf stanzas, and get rid of the [logger_field_extraction] stanza in transforms. I would expect your fields to work as expected after that.

If you insist on keeping KV_MODE = none, then change your transforms stanza as follows:

[logger_field_extraction]
DELIMS = " ", "="

Keep in mind that may get inconsistent where your values can contain spaces. Auto-parsing handles this when the values are quoted, as yours are.

0 Karma

fabiocaldas
Contributor

Hi emiller42, I tried first without KV_MODE and DELIM, since it didn't worked I tried with DELIM and them with KV_MODE. I will be re-doing my tests withou KV_MODE and with DELIM as you told. I will be back telling the results.

0 Karma

alacercogitatus
SplunkTrust
SplunkTrust

DELIMS won't help you in this case. Make sure your searching with Verbose mode, and not fast or smart. If you have key="value", splunk will pick it up. Additionally, DELIMS is search-time so it needs to be on a search head.

http://docs.splunk.com/Documentation/Splunk/6.0/Search/Changethesearchmode

0 Karma

fabiocaldas
Contributor

alacercogitatus, I had tried it before, and now I tried again but it doesn't work too.

0 Karma

alacercogitatus
SplunkTrust
SplunkTrust

DELIMS is best used with csv, but can be used with other values. If you want to use DELIMS set it equal to " ", "=" and see if that helps.

0 Karma

fabiocaldas
Contributor

Since I'm using my Indexer server as a search server, why DELIMS aren't working?

0 Karma

fabiocaldas
Contributor

alacercogitatus, even using verbose mode the fields aren't been extracted.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...