In the log file I have below mentioned a line :
EVENT_SESH;0;04/01/2018 06:30:23:5000;1;;1;0;;;END OF IMPORT PROCESS FOR THE MASTER STORE - PENDING_TXT(0).
I want to ignore all the lines after above mentioned line before indexing.
Hi saibal6,
let me understand:
is it correct?
In this case, you could use in props.conf
SEDCMD-delete = s/(?ms)EVENT_SESH.*//g
Or a pre-parsing script to run before indexing
If instead you want to discard only the event with EVENT_SESH, you can follow http://docs.splunk.com/Documentation/Splunk/7.0.2/Forwarding/Routeandfilterdatad
Edit props.conf and add the following:
[your_sourcetype]
TRANSFORMS-set= setparsing,setnull
Edit transforms.conf and add the following:
[setnull]
REGEX = EVENT_SESH
DEST_KEY = queue
FORMAT = nullQueue
[setparsing]
REGEX = .
DEST_KEY = queue
FORMAT = indexQueue
Bye.
Giuseppe
Hi saibal6,
let me understand:
is it correct?
In this case, you could use in props.conf
SEDCMD-delete = s/(?ms)EVENT_SESH.*//g
Or a pre-parsing script to run before indexing
If instead you want to discard only the event with EVENT_SESH, you can follow http://docs.splunk.com/Documentation/Splunk/7.0.2/Forwarding/Routeandfilterdatad
Edit props.conf and add the following:
[your_sourcetype]
TRANSFORMS-set= setparsing,setnull
Edit transforms.conf and add the following:
[setnull]
REGEX = EVENT_SESH
DEST_KEY = queue
FORMAT = nullQueue
[setparsing]
REGEX = .
DEST_KEY = queue
FORMAT = indexQueue
Bye.
Giuseppe
Thanks Giuseppe for the answer.
In our case your first suggestion is useful.
we added the line in " SEDCMD-delete = s/(?ms)EVENT_SESH.*//g" in props.conf but it didn't work. It's picking up all the lines from my Log file.
Could you please suggest the next step that we have to do.
one additional information:
let me understand: do you have one line for each event or one event in the file?
do you want to maintain "EVENT_SESH" event and discard the others?
this regex discards all the events where there's "EVENT_SESH", it doesn't take more events, so if you want to discard all the events from the one with "EVENT_SESH" to the end of file, you have to find a different solution: for this reason I suggested also the second solution.
If instead you want to maintain "EVENT_SESH" event and discard the others, you have to try with a different regex, to test because I don't know if the event format is always the same:
SEDCMD-delete = s/(?ms)EVENT_SESH.*/EVENT_SESH;(.*;){8}.*\./g
Bye.
Giuseppe