My propfs.conf file for my app looks like the following :
[bit_fuse_log]
pulldown_type = true
NO_BINARY_CHECK = true
TIME_PREFIX = ^
TIME_FORMAT = %H:%M:%S\,%3Q
MAX_TIMESTAMP_LOOKAHEAD = 12
SHOULD_LINEMERGE = false
BREAK_ONLY_BEFORE_DATE = true
TRUNACTE = 0
[bit_fuse_access_proxy]
SHOULD_LINEMERGE = false
NO_BINARY_CHECK = true
MAX_TIMESTAMP_LOOKAHEAD = 32
disabled = false
TRUNCATE = 0
MAX_EVENTS = 10240
and my log files look like the following (note that every line does not start with a blank space)
<texttexttext>
<text>
<text>texttexttext</text>
<text>text text text</text>
</text>
</texttexttext>
</text>
</text>
</text>
</texttext>
--------------------------------------
11:35:11,715 | INFO | 12345678-12345 | texttexttexttext | 107 - texttexttexttexttexttext | Outbound Message
---------------------------
ID: xxxxxxx
Address: http://url
Encoding: UTF-8
Http-Method: POST
Content-Type: text/xml
Headers: {Accept=[*/*], SOAPAction=[""]}
Payload: <texttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttext>
<texttexttext>
<texttexttexttexttexttexttexttexttexttexttexttext"texttexttexttexttexttexttexttexttexttext">
<texttext>
<text>text</text>
<text>text</text>
<text/>
<text/>
<text>text</text>
<text>001</text>
<text/>
<text>texttext</text>
<OPERATION>QueryCardDtlsLst</OPERATION>
<SOURCE_OPERATION/>
<SOURCE_USERID/>
[...]
--------------------------------------
08:20:22,972 | INFO | 12345678-234567890 | texttexttexttext | texttexttexttexttexttexttexttexttext | Outbound Message
---------------------------
ID: 1234
Response-Code: 200
Content-Type: application/json
Headers: {Content-Type=[application/json], Date=[Mon, 06 Jan 2020 01:20:22 GMT]}
Payload: {"texttexttext":{"texttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttext}
--------------------------------------
08:20:24,862 | INFO | 12345678-234567890 | texttexttexttexttexttext | blablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablabla
08:20:24,862 | INFO | 12345678-234567890 | texttexttexttexttexttext | blablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablabla
08:20:24,865 | INFO | 12345678-234567890 | texttexttexttexttexttext | blablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablabla
08:20:24,865 | WARN | 12345678-234567890 | texttexttexttexttexttext | blablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablabla
08:20:24,865 | INFO | 12345678-234567890 | texttexttexttexttexttext | blablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablabla
08:20:24,865 | INFO | 12345678-234567890 | texttexttexttexttexttext | blablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablabla
08:20:24,865 | INFO | 12345678-234567890 | texttexttexttexttexttext | blablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablabla
08:20:24,866 | INFO | 12345678-234567890 | texttexttexttexttexttext | blablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablabla
08:20:24,867 | INFO | 12345678-234567890 | texttexttexttexttexttext | blablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablabla
08:20:24,867 | INFO | 12345678-234567890 | texttexttexttexttexttext | blablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablabla
----------------------------
ID: 123456
Address: texttexttexttexttexttexttexttext
Encoding: texttext
Http-Method: POST
Content-Type: application/json
Headers: {Accept=[application/json], texttexttexttexttexttexttexttexttexttexttexttexttexttexttext}
Payload: {"startDate":"2020-01-01T00:00:00.000+0700","endDate":"2020-01-06T23:59:59.000+0700","pageNumber":1,"pageSize":300}
--------------------------------------
08:20:24,862 | INFO | 12345678-234567890 | texttexttexttexttexttext | blablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablabla
08:20:24,862 | INFO | 12345678-234567890 | texttexttexttexttexttext | blablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablabla
08:20:24,865 | INFO | 12345678-234567890 | texttexttexttexttexttext | blablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablabla
08:20:24,865 | WARN | 12345678-234567890 | texttexttexttexttexttext | blablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablablabla
<text>
I would like to break at every timestamp at the beginning of line (every arrow in the following picture).
I don't understand Splunk's behavior considering my props.conf file.
My issue is that Splunk sometimes breaks as expected and sometimes not. The most common behavior is :
I can't find a pattern for this behavior, do you have an idea ?
try:
SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n]+)\d\d:\d\d:\d\d,
NO_BINARY_CHECK=true
TIME_FORMAT=%H:%M:%S,%3N
TIME_PREFIX=^
MAX_TIMESTAMP_LOOKAHEAD=15