How do I configure Splunk to read events by timestamp?

[cj039165]

Hello

All our logging events start with a time stamp that looks like this: 00:00:23,746

The data in between the event can have carriage returns, along with different delimiters. For example data can contain * ~ @ ^ | < > …..etc.

How can I get Splunk to read the events by timestamp? I don’t want any of the data between the time stamps to cause issues.

[cj039165]

my props.conf in /opt/splunkforwarder/etc/apps/search/local/ looks like this. Splunk has been restarted. I'm still not seeing the events split by time stamps. Interesting, some are, some are not.

[hdx_payer_receive_logs]
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)(?=\d{2}\:\d{2}\:\d{2},\d{3}\s)
TIME_PREFIX = ^
TIME_FORMAT = %H:%M:%S,%N
MAX_TIMESTAMP_LOOKAHEAD = 13

[hdx_payer_send_logs]
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)(?=\d{2}\:\d{2}\:\d{2},\d{3}\s)
TIME_PREFIX = ^
TIME_FORMAT = %H:%M:%S,%N
MAX_TIMESTAMP_LOOKAHEAD = 13

[pradeepkumarg]

From the path, it looks like you have props.conf on a forwarder. Is it a heavy forwarder?
If it is an universal forwarder, then you have it in the wrong place. Typically you should have props.conf on your indexers.

[somesoni2]

the default install directory for universal forwarder is /opt/splunkforwarder, so it does look like a UF and the props.conf here will not do any good. You need that in your Indexer.

[cj039165]

Understood. I'll work moving it and see if it fixes the issue. Thanks.

[somesoni2]

Try this for your event processing setting (props.conf on the Indexer/Heavy Forwarder)

[yoursourcetype]
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)(?=\d{2}\:\d{2}\:\d{2},\d{3}\s)
TIME_PREFIX = ^
TIME_FORMAT = %H:%M:%S,%N
MAX_TIMESTAMP_LOOKAHEAD = 13

[lakshman239]

can you try to load that sample file directly to splunk (via add data) and point the time to 00:00:23, 746 and splunk should be able able to parse it and show you the props.conf applied. have you tried it?

if your events allways start with 00:00:x,yyy (timestamp), your regex can use starting line right? I am getting it?

[Jeremiah]

Please post a sanitized sample of your log data; without it, its difficult to make a recommendation.

[cj039165]

Here is a logging example:

15:34:43,309 DEBUG  [WebContainer : 3] --MIMEBoundary_813952806c0080beb138925fa27f2a4e4aec4e2b7937d8fe
Content-Type: application/xop+xml; charset=UTF-8; type="application/soap+xml"
Content-Transfer-Encoding: binary
Content-ID: <0.913952806c0080beb138925fa27f2a4e4aec4e2b7937d8fe@apache.org>

X12_271_Response_005010X279A1RealTime4a87d24e-c3d0-4165-b760-9a0c37ed00cd 07-27-2016 15:34:41+04:0000302EXC000182.2.0ISA*00*          *00*621REF    *ZZ*00302          *ZZ*EXC00018       *160727*1534*{*00501*067723665*0*T*^~GS*HB*00302*EXC00018*20160727*15344285*67723665*X*005010X279A1~ST*271*123235177*005010X279A1~BHT*0022*11*123240987*20160727*1934423~HL*1**20*1~NM1*PR*2*Excellus*****PI*302~PER*IC*BLUECARD ELIGIBILITY*TE*8006762583~HL*2*1*21*1~NM1*1P*2*HDX TEST PROVIDER*****XX*1234567893~HL*3*2*22*0~TRN*2*00000002765123235177*HDXMSGUTIL~NM1*IL*1*TEST*TEST****MI*ABC123456789~AAA*Y**72*C~DMG*D8*19730806~DTP*291*RD8*20160727-20160727~SE*14*123235177~GE*1*67723665~IEA*1*067723665~SuccessNone
--MIMEBoundary_813952806c0080beb138925fa27f2a4e4aec4e2b7937d8fe--