Getting Data In

log file not parsing properly - multiple lines per event

dglass0215
Path Finder

Hello,

I have a file monitor for a log file where I am getting indexed data with multiple lines. Example of one event:

2019-12-30 09:16:41:908: Requestor: IMM_Mobile, IsLocal: False
2019-12-30 09:16:41:908: 637132942019089151: Scanned CID: BARCODE:

Now i notice that it is the same time but they should still be separate events. i have read where someone suggested SHOULD_LINEMERGE = false, however if I am reading the documentation correctly, the SHOULD_LINEMERGE = true and BREAK_ONLY_BEFORE_DATE = true being the defaults should be processing the above as two separate events. What am I misunderstanding?

I am hesitant to configure SHOULD_LINEMERGE = false because I think it may be needed for other events that span multiple lines.

only other thing I can think of is possibly my props/transforms might be screwing with the data in some other way. Below are what I think are the relevant portions of my props/transforms:

Props:
[mySourceType]
TRANSFORMS-set= setnull,setparsing
TRANSFORMS-sourcetype= setNewSourceType

Transforms:
[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = \b(?:offline|online|\d{4}-d{2}-d{2}\s+\d{2}:\d{2}:\d{2}:\d{3}:\s+\d{18}:\s)\b
DESK_KEY = queue
FORMAT = indexQueue

[setNewSourceType]
REGEX = \b(\d{4}-d{2}-d{2}\s+\d{2}:\d{2}:\d{2}:\d{3}:\s+\d{18}:\s)
FORMAT = sourcetype::NewSourceType
DEST_KEY = MetaData:Sourcetype

Thanks for any assistance!
David

0 Karma

mydog8it
Builder

Take a look at this data onboarding cheat sheet published by Aplura, it is a great guide to establish your data onboarding practices:
https://www.aplura.com/assets/pdf/onboarding_cheatsheet.pdf

0 Karma

dglass0215
Path Finder

While this might be slightly useful it does not answer my question. Do you know why the two lines above are one event? And do you know how I can fix it? Thanks!

0 Karma
Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...