How to pick overall message

[vikas_gopal]

Hi Experts,
I have syslog file and I want to generate a table from this log file .This file contains log like

2014-12-26 16:22:42 Local7.Debug    172.28.70.132   CEF:0|Microsoft|Microsoft Windows||Microsoft-Windows-Security-
Auditing:4634|An account was logged off.|Low| eventId=1 externalId=4634 msg=Network: A user or computer logged on to this computer from the network.

I want to pick complete "msg" from this log but in the table when I pick msg it shows only one word "Network". I want to show complete msg field like "Network: A user or computer logged on to this computer from the network."
Please help on this .

[MuS · Accepted solution]

Hi vikas_gopal,

how about this:

your base search here | rex "\smsg\=(?<myMsg>.+?)\." | table myMsg

assumption there is no new line, line break after msg=

hope this helps ...

cheers, MuS

[vikas_gopal]

This is what I have done , but with this message is blank.
sourcetype="CEF" | table cat,msg | rex field=msg (?i)msg=(?P.*)
even I tried
sourcetype="CEF" | table cat,msg | rex field=msg "\smsg=(?.+?)."

[MuS]

try this :

sourcetype="CEF" | rex "\smsg\=(?<msg>.+?)\." | table cat, msg

[vikas_gopal]

yup it works , the only problem is it is picking everything after msg field . Can we do something like it picks line between "msg" and "categorySignificance" fields . I observed that message ends just before "categorySignificance" field.
Here is the overall log

2014-12-26 16:22:42    Local7.Debug    172.28.70.132    CEF:0|Microsoft|Microsoft Windows||Microsoft-Windows-Security-Auditing:4634|An account was logged off.|Low| eventId=1 externalId=4634 msg=Network: A user or computer logged on to this computer from the network. categorySignificance=/Informational categoryBehavior=/Access/Stop categoryDeviceGroup=/Operating System catdt=Operating System categoryOutcome=/Success categoryObject=/Host/Operating System art=1419639752421 cat=Security deviceSeverity=Audit_success rt=1419639736000 dhost=WIN-789Z3VMFC02 dst=172.28.70.132 destinationZoneURI=/All Zones/ArcSight System/Private Address Space Zones/RFC1918: 172.16.0.0-172.31.255.255 dntdom=WIN-789Z3VMFC02 duser=Administrator duid=0x1d11f9 cs2=Logon/Logoff:Logoff cn1=3 c6a4=fe80:0:0:0:8045:2eed:5e73:a468 cs1Label=Accesses cs2Label=EventlogCategory cs4Label=Reason or Error Code cs5Label=Authentication Package Name cn1Label=LogonType cn2Label=CrashOnAuditFail cn3Label=Count c6a4Label=Agent IPv6 Address ahost=WIN-789Z3VMFC02 agt=1

[MuS]

Try this regex here, see above ↑

[gabetheISguy]

Splunk auto extracts fields when fields=value criteria is met. Since the event has "msg=Network: " it auto extracts the msg field as Network

To overcome this I would use a regular expression with named capturing groups to extract specific fields as per the requirement

For e.g: to Capture the message using a named capturing group i would use this. Having said that using .* is not always the best method as it captures everything else after the msg actually ends. The field named as Message will capture the messages. LMK if this works

(?i)msg=(?P<Message>.*)

[vikas_gopal]

this is my first time that I am using REX . This is what I have done , but with this message is blank.
sourcetype="CEF" | table cat,msg | rex field=msg (?i)msg=(?P.*)

[MuS]

this is because your field msg only contains Network use the rex like this:

sourcetype="CEF" | rex  "(?i)msg=(?P<msg>.*)" | table cat,msg

[vikas_gopal]

yup it works , the only problem is it is picking everything after msg field . Can we do something like it picks line between "msg" and "categorySignificance" fields . I observed that message ends just before "categorySignificance" field.
Here is the overall log

2014-12-26 16:22:42    Local7.Debug    172.28.70.132    CEF:0|Microsoft|Microsoft Windows||Microsoft-Windows-Security-Auditing:4634|An account was logged off.|Low| eventId=1 externalId=4634 msg=Network: A user or computer logged on to this computer from the network. categorySignificance=/Informational categoryBehavior=/Access/Stop categoryDeviceGroup=/Operating System catdt=Operating System categoryOutcome=/Success categoryObject=/Host/Operating System art=1419639752421 cat=Security deviceSeverity=Audit_success rt=1419639736000 dhost=WIN-789Z3VMFC02 dst=172.28.70.132 destinationZoneURI=/All Zones/ArcSight System/Private Address Space Zones/RFC1918: 172.16.0.0-172.31.255.255 dntdom=WIN-789Z3VMFC02 duser=Administrator duid=0x1d11f9 cs2=Logon/Logoff:Logoff cn1=3 c6a4=fe80:0:0:0:8045:2eed:5e73:a468 cs1Label=Accesses cs2Label=EventlogCategory cs4Label=Reason or Error Code cs5Label=Authentication Package Name cn1Label=LogonType cn2Label=CrashOnAuditFail cn3Label=Count c6a4Label=Agent IPv6 Address ahost=WIN-789Z3VMFC02 agt=1

[MuS]

try my below provided regex, this will stop at the .

[vikas_gopal]

I tried above command only it is still picking everything after "msg" ,like till the last word of the overall log