Getting Data In

Splunk automatically splits a event into two, because of two dates

sre_sl
New Member

Hi All

I tried a solution suggested online for a similar issue, but it didn't fix the problem

The below extract from the log is a single event

 2019-03-26 12:03:28.753 +0000 INFO  [zzz] [yyy] [] [] [rrId:] []  Message
----------------------------
ID: 7
Response-Code: 200
Encoding: UTF-8
Content-Type: text/xml;charset=utf-8
Headers: {Connection=[close], Content-Type=[text/xml;charset=utf-8], Date=[Tue, 26 Mar 2019 12:03:28 GMT],
Show less

but it results like this

 2019-03-26 12:03:28.753 +0000 INFO  [zzz] [yyy] [] [] [rrId:] []  Message
 ----------------------------
 ID: 7
 Response-Code: 200
 Encoding: UTF-8
 Content-Type: text/xml;charset=utf-8

In 2 lines

Headers: {Connection=[close], Content-Type=[text/xml;charset=utf-8], Date=[Tue, 26 Mar 2019 12:03:28 GMT],
         Show less

We have a cluster environment so I updated the props here - opt/splunk/etc/master-apps/_cluster/local/props.conf with the below
[log4j]

MAX_TIMESTAMP_LOOKAHEAD = 19

I pushed the change to the peers and restarted all the indexers

Any thoughts to fix this issue please?

Thanks

0 Karma
1 Solution

somesoni2
SplunkTrust
SplunkTrust

Try this for your props.conf (sourcetype definition)

[log4j]
SHOULD_LINEMBER=false
LINE_BREAKER=([\r\n]+)(?=\d{4}-\d{2}-\d{2})
TIME_PREFIX=^
TIME_FORMAT=%Y-%m-%d %H:%M:%S.%3N %z
MAX_TIMESTAMP_LOOKAHEAD=29

View solution in original post

0 Karma

somesoni2
SplunkTrust
SplunkTrust

Try this for your props.conf (sourcetype definition)

[log4j]
SHOULD_LINEMBER=false
LINE_BREAKER=([\r\n]+)(?=\d{4}-\d{2}-\d{2})
TIME_PREFIX=^
TIME_FORMAT=%Y-%m-%d %H:%M:%S.%3N %z
MAX_TIMESTAMP_LOOKAHEAD=29
0 Karma

sre_sl
New Member

Thanks.

It works partially but creates another issue.

The below shows as one event

2019-04-08 09:51:11.791 +0000 INFO   INFO  [[zzz] [yyy] [] [] [rrId:] []  Message In
----------------------------
ID: 3
Response-Code: 200
Encoding: UTF-8
Content-Type: text/xml;charset=utf-8
Headers: {Connection=[close], Content-Type=[text/xml;charset=utf-8], Date=[Mon, 08 Apr 2019 09:51:11 GMT]

But what happens is the next message is also in the same event, which should be a separate event

2019-04-08 09:51:11.727 +0000 INFO [[zzz] [yyy] [] [] [rrId:] []  Message Out

ID: 3
Address: xxxx
Encoding: UTF-8
Http-Method: POST
Content-Type: text/xml
Headers: {Accept=[/], Connection=[close], SOAPAction=[""]}

2019-04-08 09:51:11.791 +0000 INFO INFO [[zzz] [yyy] [] [] [rrId:] []  Message In

ID: 3
Response-Code: 200
Encoding: UTF-8
Content-Type: text/xml;charset=utf-8
Headers: {Connection=[close], Content-Type=[text/xml;charset=utf-8], Date=[Mon, 08 Apr 2019 09:51:11 GMT]

0 Karma

somesoni2
SplunkTrust
SplunkTrust

Give this a try

[log4j]
 SHOULD_LINEMBER=false
 LINE_BREAKER=([\r\n]+)(?=\s*\d{4}-\d{2}-\d{2}\s*\d+)
 TIME_PREFIX=^
 TIME_FORMAT=%Y-%m-%d %H:%M:%S.%3N %z
 MAX_TIMESTAMP_LOOKAHEAD=29
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...