Getting Data In

Line Break multiple access logs

dperry
Communicator

I need to line break, starting at the IP and end with the time. ex:

74.100.11.60 xx.x.xxx.xxx:59726 - Unauthenticated [15/Jul/2014:17:53:26 -0700] "GET /wps/wcm/connect/4ebe8f0047818b77a890a9332342f25b/ew+-+pub+home+-+family+refer+-+225x130.jpg?MOD=AJPERES&CACHEID=4ebe8f0047818b77a890a9332342f25b HTTP/1.1" 304 - TS:0 WAS:backend_server:10029 TIME:3738
- 127.0.0.1:37296 - - [15/Jul/2014:17:53:26 -0700] "GET / HTTP/1.1" 200 3216 TS:0 WAS:- TIME:286
- 127.0.0.1:47220 - - [15/Jul/2014:17:53:26 -0700] "GET / HTTP/1.1" 200 3216 TS:0 WAS:- TIME:314
46.4.94.230 xx.x.xxx.xxx:38896 - Unauthenticated [15/Jul/2014:17:53:26 -0700] "POST /wps/portal/PublicSearch HTTP/1.0" 200 148284 TS:0 WAS:backend_server:10053 TIME:230882
107.185.76.225 10.4.102.144:59724 - 1205026 [15/Jul/2014:17:53:26 -0700] "GET /SchoolsFirst_Theme_Main/themes/html/SchoolsFirst_Theme_Main/shelfInit.html HTTP/1.1" 304 - TS:0 WAS: TIME:491
23.243.33.194 xx.x.xxx.xxx:38901 - 59196 [15/Jul/2014:17:53:26 -0700] "GET /wps/myportal/!ut/p/a1/hY7LDoIwEEW_hQVbWkR5uGuMJhIiBBKFbkghvEylpC3w-4IaV4Kzmjk5c2cABjHALRmaisiGtYTOMzbTg-9Y0cX1NroeOhDZ5jXwQ8vYhtYkJJMAFwrBf_s3gNcV_SOsnHABrijLXu8mqM0MuwKYF2XBC671fMK1lJ3Yq1CF4zhqIq8Zo6JsuJBl3muMVyqM3vA0wzRgXBJ6RGcVfttf4TUTEsQLmaB7xPC-o4OHFOUJ8gm7DQ!!/dl5/d5/L2dBISEvZ0FBIS9nQSEh/pw/Z7_CO97SNJL211R90A86VPOR734F4/act/id=F0ZnR_HOe7QFB/p=bf_action=_gen_call_pbAction_goToCheckingsSearchTranHistory_shareName/p=checkingsShareDesc=71/266691184729/=/ HTTP/1.1" 302 - TS:0 WAS:backend_server:10029 TIME:267941
108.220.220.26 xx.x.xxx.xxx:53683 - Unauthenticated [15/Jul/2014:17:53:26 -0700] "GET /wps/portal HTTP/1.1" 200 191960 TS:0 WAS:backend_server:10053 TIME:468361

Sometimes Splunk singles out the events and/or groups them as seen above. I need to make them each event......Also I have noticed a - symbol once in a while before an IP......

What would be the regex that needs to be added to my props.conf?? Please advise.

Tags (2)
0 Karma
1 Solution

strive
Influencer

I think only IP based line breaking should be good enough for your logs

In your props.conf add

 LINE_BREAKER=([\r\n]+)\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}

You have to tweak the regex to include - condition.

View solution in original post

strive
Influencer

I think only IP based line breaking should be good enough for your logs

In your props.conf add

 LINE_BREAKER=([\r\n]+)\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}

You have to tweak the regex to include - condition.

dperry
Communicator

Currently this is what I have in Props.conf:

[web_access]
TIME_PREFIX = \d+.\d+.\d+.\d+\s+\d+.\d+.\d+.\d+:\d+\s+-\s+\d+\s+[
TIME_FORMAT = %d/%b/%Y:%H:%M:%S\s%z
MAX_TIMESTAMP_LOOKAHEAD = 65
SHOULD_LINEMERGE = False
LINE_BREAKER = ([\r\n]+)(\d+.\d+.\d+.\d+\s+\d+.\d+.\d+.\d+:\d+\s+-\s+\d+\s+[\d+\/\w+\/\d{4}:\d{2}:\d{2}:\d{2})

0 Karma

dperry
Communicator

Thanks, tweak the regex to include - condition? how is this done?

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to May Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars this month. This ...

They're back! Join the SplunkTrust and MVP at .conf24

With our highly anticipated annual conference, .conf, comes the fez-wearers you can trust! The SplunkTrust, as ...

Enterprise Security Content Update (ESCU) | New Releases

Last month, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...