Help parsing incoming data

[secphilomath1]

We have data coming in that we need to alert on, however because of the formatting of the data, this is very hard to do.   The data is coming in as key value pairs but the values are not encapsulated in quotes and is being truncated.  For example

_Raw - filepath=c:\program files\abc123\

What we end up getting is

Parsed - filepath=c:\program

Everything after the space is ignored.

If I wanted to find all occurrences where the path was c:\program files\abc123, I can't.

We are sending the data via syslog to the splunk servers

Thanks in advance!

[richgalloway]

Please share the props and transforms for that sourcetype as well as a couple of sanitized sample events. 

[secphilomath1]

@richgalloway 

The data is coming from a FIM product called Tripwire.  Here is the raw data;

Dec 22 02:30:34 10.62.32.10 1 2023-12-22T10:30:34.771Z servernameTW_ES - - - CEF:0|Tripwire|Enterprise|5.5|6|Audit Event|1|UserName=NT AUTHORITY\NETWORK SERVICE UserNameLabel=User Name ElementName=null ElementNameLabel=Element Name VersionTimeStamp=null VersionTimeStampLabel=Version Timestamp Message='C:\Windows\System32\Tasks\Microsoft\Windows\SoftwareProtectionPlatform\SvcRestartTask' accessed by 'NT AUTHORITY\NETWORK SERVICE'. Type 'Set Security'. Application: 'C:\Windows\System32\svchost.exe' Details: DACL Category=Audit Event CategoryLabel=Category rt=12/22/23 2:25 AM Level=Information LevelLabel=level dhost=trip.cs.ad.domain.com

I don't have any props or transforms yet because I am not sure where to start with this.

Thanks

[richgalloway]

Everything ingested by Splunk should have props.conf settings.  Start with the "Great 8": LINE_BREAKER, SHOULD_LINEMERGE, TIME_PREFIX, TIME_FORMAT, MAX_TIMESTAMP_LOOKAHEAD, TRUNCATE, EVENT_BREAKER_ENABLE, and EVENT_BREAKER.

Field extraction from events like this are tricky because the field delimiter is also an allowed character within a field.  It means using lookahead to determine if the current character is part of a field name or field value.  As it turns out, Splunk is not great with lookahead.  Try these settings to see if they work for you.

Props.conf:

[mysourcetype]
SHOULD_LINEMERGE=true
LINE_BREAKER=([\r\n]+)
NO_BINARY_CHECK=true
TIME_PREFIX=\s\d\s
TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%3N%Z
TRANSFORMS-extract = tripwire_fields
TRUNCATE = 10000
EVENT_BREAKER_ENABLE = true
EVENT_BREAKER = ([\r\n]+)

Transforms.conf:

[tripwire_fields]
REGEX = (\w+)=(.*?)(?=\s\w+=)
FORMAT = $1::$2

[secphilomath1]

Thank you, I think this solved it!

[richgalloway]

Please note: I added a missing "2" at the end of the transforms.conf code.

If your problem is resolved, then please click the "Accept as Solution" button to help future readers.

[gcusello]

Hi @secphilomath1 ,

what technology are you using for these data?

if they are standard, you can use the related add-on that gives you al the parsing rules.

If it's custom, you have t omanually parse it.

Ciao.

Giuseppe