Splunk Search

REGEX to extract null/empty field as it has values

larmesto
Path Finder

Hello folks,

I was wondering if you could help me with an issue regarding to the field extraction technique.
I have this multiline log (below). My problem is that when I trying to extract the empty fields at the same time as the valued ones.

This is my LOG:

14/03/16 06:02:03 VSED CRPDRETR RETRANS B F XCOMJOB OK+A P $IJDPRT 02014
14/03/16 06:02:03 VSED CRPDRETR RETRANS B F XCOMJOB OK+A P ISTMMCHS 02014
14/03/16 06:02:03 VSED CRPDRETR RETRANS B F XCOMJOB WRITE ALL OK+A D OPE294 VCIP3.SORTWK1.F4 02014
14/03/16 06:02:03 VSED CRPDRETR RETRANS B F XCOMJOB READ OK+A ? PRD2.XCOM310.C230FI02.CONFI 02014
14/03/16 06:02:03 VSED CRPDRETR RETRANS B F XCOMJOB READ OK+A ? PRD2.XCOM310.C230FI02.CONFI 02014
14/03/16 06:02:03 VSED CRPDRETR RETRANS B F XCOMJOB READ OK+A ? PRD2.XCOM310.DESTMODE.CONFI 02014
14/03/16 06:02:03 VSED CRPDRETR RETRANS B F XCOMJOB READ OK+A ? PRD2.XCOM310.DESTMODE.CONFI 02014
14/03/16 06:02:03 VSED CRPDRETR RETRANS B F XCOMJOB OK+A P IJDANCHX 02014
14/03/16 06:02:03 VSED CRPDRETR RETRANS B F $JOBACCT OK+A P IESWAIT 02014
14/03/16 06:04:03 VSED CRPDRETR RETRANS B F $JOBACCT OK+A TRM 02014
14/03/16 08:17:04 VSED CRPDRETR DUMMY B F $JOBACCT READ OK+A q SUBMIT.VCIP3.RETRANS.CRPDRE 43327
14/03/16 08:17:04 VSED CRPDRETR RETRANS B F $JOBACCT OK+A T VMRDR00C 43327 VMRDR00C
14/03/16 08:17:04 VSED CRPDRETR RETRANS B F $JOBACCT READ OK+A e VMRDR00C 43327 VMRDR00C
14/03/16 08:17:04 VSED CRPDRETR RETRANS B F $JOBACCT OK+A VFX 43327 VMRDR00C
14/03/16 08:17:04 VSED CRPDRETR RETRANS B F $JOBACCT OK+A , VSED 43327
14/03/16 08:17:04 VSED CRPDRETR RETRANS B F $JOBACCT OK+A U CPU.VSED 43327
14/03/16 08:17:04 VSED CRPDRETR RETRANS B F $JOBACCT OK+A ? F3 43327
14/03/16 08:17:04 VSED CRPDRETR RETRANS B F $JOBACCT OK+A U VSEPART. 43327
14/03/16 08:17:04 VSED CRPDRETR RETRANS B F $JOBACCT READ OK+A e VMRDR00C 43327
14/03/16 08:17:04 VSED CRPDRETR RETRANS B F $JOBACCT OK+A INI NAME=TRANSM/USO COMUN 43327
14/03/16 08:17:04 VSED CRPDRETR RETRANS B F $JOBACCT READ OK+A ? PRD2.CONFIG.SETXCOM.PROC 43327

What I'm trying to do is to extract, for example, in the first row Column1=WRITE and Column2=ALL and so on.... With empty values or not.

Here's my current REGEX:
(\d+\/\d+\/\d+\s\d+:\d+:\d+)\s+([^\s]+)\s+([^\s]+)\s+([^\s]+)\s+([^\s]+\s+[^\s]+)\s+([^\s]+)\s+([^\s]+)

As you can see in the image attached when is use the final \s+ the matches for 'spaces' breaks my entire formula. (See image)
I'm using regex101 site to test this.

Any help is welcome!
Best regards,

Tags (1)
0 Karma

somesoni2
Revered Legend

Give this a try

(\d+\/\d+\/\d+\s\d+:\d+:\d+)\s+([^\s]+)\s+([^\s]+)\s+([^\s]+)\s+([^\s]+\s+[^\s]+)\s+([^\s]+)\s+(.*)\s*OK\+A\s+([^\s]+)

Updated
Try this

(\d+\/\d+\/\d+\s\d+:\d+:\d+)\s+([^\s]+)\s+([^\s]+)\s+([^\s]+)\s+([^\s]+\s+[^\s]+)\s+([^\s]+)\s+(.*)\s*(OK\S*)\s+(\S+)\s+(.+\s+)+(\S+)$
0 Karma

larmesto
Path Finder

Hi somesoni2,
Thanks for your reply. I've give that regex a try but It only works for some lines.
alt text

Regards,

0 Karma

alemarzu
Motivator

Somesoni's regex should work, the problem was your data sample, all the records ended with "OK+A" but it looks like not after all.

So, you have to change/remove \+A\s+after the OK from the regular expresion.

0 Karma

acharlieh
Influencer

In your screenshot, that file looks to be tab delimited? would matching based on individual tab characters instead of any number of whitespace characters help?

0 Karma

larmesto
Path Finder

Thanks acharlieh
I've tryed this, but is not tab delimited. I believe the only option is to parse with static positions...

0 Karma

alemarzu
Motivator

Are those fixed length fields ?

Take a look at this,

alt text

Edit: regex added

main search | rex field=_raw "\d{2}\/\d{2}\/\d{2}\s\d{2}:\d{2}:\d{2}\s(?<A_TEMP>.*?)\sOK" | rex field=A_TEMP "(?<a_group1>.{4})\s(?<a_group2>.{8})\s(?<a_group3>.{7})\s+(?<a_group4>.{3})\s+(?<a_group5>.*?)\s+(?<a_group6>.*)" | table _time a_group1 a_group2 a_group3 a_group4 a_group5 a_group6
0 Karma

larmesto
Path Finder

Thanks for your reply. The problem is that I need to use only plain regex for extraction because I'm using the props.conf file to achive this, before any use of search & reporting app.

regards,

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to May Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars this month. This ...

They're back! Join the SplunkTrust and MVP at .conf24

With our highly anticipated annual conference, .conf, comes the fez-wearers you can trust! The SplunkTrust, as ...

Enterprise Security Content Update (ESCU) | New Releases

Last month, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...