Dashboards & Visualizations

multiline log: break on return char, not timestamp

alexmartinez
New Member

Sorry new to Splunk...I've a single logfile with entries that look like this:

"15/11/2017 20:20:59","0","1803.xml","Copied to Amazon S3",5,"O"
"15/11/2017 20:21:00","0","1260.xml","Copied to Amazon S3",5,"O"
"15/11/2017 20:21:00","0","2415.xml","Copied to Amazon S3",5,"O"
"15/11/2017 20:21:01","0","134.xml","Copied to Amazon S3",5,"O"
"15/11/2017 20:21:01","0","808.xml","Copied to Amazon S3",5,"O"
"15/11/2017 20:21:02","0","261.xml","Copied to Amazon S3",5,"O"
"15/11/2017 20:21:02","0","646.xml","Copied to Amazon S3",5,"O"
"15/11/2017 20:21:03","0","1157.xml","Copied to Amazon S3",5,"O"

Splunk is breaking this into events by timestamp (field 1) but because the above entries have repeating timestamps I only get the first event for each date.

How can I insure that EACH line gets its own event?

Tags (1)
0 Karma

niketn
Legend

@alexmartinez, what is your current props.conf file settings for this sourcetype?

If you want to break events on every line you should turn off line merge setting:

SHOULD_LINEMERGE=false

Refer to Splunk Documentation: https://docs.splunk.com/Documentation/Splunk/latest/Data/Configureeventlinebreaking

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

alexmartinez
New Member

Thanks @niketn that worked when i configured a new sourcetype with that attribute and also removed the BREAK_ONLY_BEFORE attribute . Then I pointed the datasource to this new sourcetype. Its all working now as expected.

0 Karma

niketn
Legend

@alexmartinez, I have converted to answer, please accept.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

niketn
Legend

@alexmartinez, please accept the answer if your issue is resolved.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

alexmartinez
New Member

Thanks niketnilay

I had to add a new sourcetype with
1. SHOULD_LINEMERGE = false
2 removed BREAK_ONLY_BEFORE attribute

...then pointed our data to this new sourcetype. Worked a treat.

Alex

0 Karma

alexmartinez
New Member

In my edit Source TYpe/advanced settings on the console for Splunk Enterprrse:

BREAK_ONLY_BEFORE ([\n]+)
FIELD_NAMES logtimestamp,is_control_message,filename_message,status,file_status,error_type
INDEXED_EXTRACTIONS csv
NO_BINARY_CHECK true
SHOULD_LINEMERGE true
TIMESTAMP_FIELDS logtimestamp
TIME_FORMAT %d/%m/%Y %H:%M:%S
category Structured
disabled false
pulldown_type true

I tried:
SHOULD_LINEMERGE = false as an admin user in the console but it reverts back to true! I also tried removing the attribute BREAK_ONLY_BEFORE but it won't let me. Can't I edit Advanced settings?

0 Karma
Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...