Hi all,
I have configured the line breaking parameter as (SHOULD_LINEMERGE = true) to read a log file that contains the below data for each update. It will allow to have all data in the same event to run my searches:
S Doing: print 1597931/1
S --> Print Job @>SPOREQ:1597931@
S print job @>SPOREQ:1597931@
S replace user SAPSYS by NFEX
S print job @>SPOREQ:1597931@
S --------> db_rtab Error 128, table TSPEVJOB, action rspogdio_insert (expected)
S SpTSP01Select() - 1597931 is OTF/SMART/RDI/ADSx job, do not read TSP02L!
S Sß: (2017120211393600) sending job @>SPOREQ:1597931@DEV:JCD1@<'
S 1 processed
And I would like to filter the input data to index only the below lines of the log file disregarding the others lines:
S Doing: print 1597931/1
S replace user SAPSYS by NFEX
S Sß: (2017120211393600) sending job @>SPOREQ:1597931@DEV:JCD1@<'
S 1 processed
I was thinking to configure my REGEX just using the phases "Doing: print" , "replace user", "sending job", however it will not work for the line merge configuration as dont have the end of the line. So my question is: how I should configure the regex in transform file to allow just index 3 lines using SHOULD_LINEMERGE ?
I am using this below configuration on the props and transform files and it is not working:
[sourcetype]
SHOULD_LINEMERGE = true
BREAK_ONLY_BEFORE=processed
TRANSFORMS-set= setIndexnull,setIndexparsing
[setIndexnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue
[setIndexparsing]
REGEX = print
DEST_KEY = queue
FORMAT = indexQueue
Thanks and regards,
Danillo Pavan
I still didn´t find any solution for this case. Still need to trash certain lines and then create multiline events out of what is left...
Any proposal??
@danillopavan this seems similar to other question your have posted: https://answers.splunk.com/answers/597389/filtering-data-using-should-linemerge.html
I would request you to consolidate required details against single question and keep only one of them open.
Any answer?
Write your props and transforms.conf like the below and try this.
props.conf
[sourcetype]
TRANSFORMS-set = setnull, print1,print2,print3
Transforms.conf
[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue
[print1]
REGEX = S\sDoing:
DEST_KEY = queue
FORMAT = indexQueue
[print2]
REGEX = S\sreplace
DEST_KEY = queue
FORMAT = indexQueue
[print3]
REGEX = S \sSß:
DEST_KEY = queue
FORMAT = indexQueue
Hello Sandy,
Now no lines are being indexed. The regex commands that you suggested seems to be wrong. Anyway, the instructions that you suggested, in my opinion, would not work. I need to conciliate MERGED LINES with FILTERING.
I am using this below configuration on the transform files and it is not working:
[setIndexnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue
[setIndexparsing]
REGEX = print
DEST_KEY = queue
FORMAT = indexQueue