My props.conf is like:
BREAK_ONLY_BEFORE_DATE = true
TIME_PREFIX = GMT
TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%3N
MAX_DAYS_HENCE = 5
MAX_TIMESTAMP_LOOKAHEAD = 24
SHOULD_LINEMERGE = true
and my events is like this:
41785:11 INFO [machine] 150 GMT2016-11-16T22:31:07.330Z (18 ms) [uuid] 13683279 [firm] 9001 [sn] 866562 onRequestExpired: request id: 6353697407667535883
41785:11 INFO [machine] 150 GMT2016-11-16T22:31:07.330Z (18 ms) [uuid] 13683279 [firm] 9001 [sn] 866562 postApplicationDataEvent roomId BCAST-fs:582CDE21190C000D data: {"retractEvent":{"retractType":"BY_TIMER"}}
41785:11 INFO [machine] 150 GMT2016-11-16T22:31:07.689Z (59 ms) [uuid] 13683279 [firm] 9001 [sn] 866562 BCAST-fs:582CDE21190C000D processRetractEvent
41785:11 INFO [machine] 150 GMT2016-11-16T22:31:07.845Z (22 ms) [uuid] 13683279 [firm] 9001 [sn] 866562 scrape:
{
"requestId": "6353697450617208879",
"chatId": "BCAST-fs:582CDE21190C000D",
"operationTypeEnum": "EXPIRED",
"initiator": 13683279,
"capturer": 13683279,
"counterPartyUser": 0,
"counterPartyUserIdUrn": null,
"events": [
{
"idUrn": "urn:identity-ib-bloomberg-net:1:0:urn%3Afb-ib-bloomberg-net%3ABGEU%3Ain%3Df:uuid%3D13683279",
"content": "hi=5,\n",
"eventTypeEnum": "CHAT"
}
],
"ibdRequestId": "6353697407667535883",
"takerDealCode": "BGEU",
"makerDealCode": "QA01",
"text": "",
"pointX": 0,
"pointY": 0,
"height": 100,
"width": 100,
I would like to break the events with time. But they take all the above 4 events as one event.
How should I fix this?
Hi,
You can try this in the file props.conf:
SHOULD_LINEMERGE=true
NO_BINARY_CHECK=true
BREAK_ONLY_BEFORE=(\s\d\d\d\d\d:\d\d\sINFO)
Tks
Rodrigo Ribeiro
I have changed to this setting:
TZ=UTC
TRUNCATE = 0
BREAK_ONLY_BEFORE_DATE = false
BREAK_ONLY_BEFORE = ^\d+:\d+\s(INFO|ERROR|FATAL|WARN|DEBUG|TRACE)\s[machine]\s\d+\sGMT
TIME_PREFIX = ^\d+:\d+\s(INFO|ERROR|FATAL|WARN|DEBUG|TRACE)\s[machine]\s\d+\sGMT
TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%3N
MAX_DAYS_HENCE = 5
MAX_TIMESTAMP_LOOKAHEAD = 24
SHOULD_LINEMERGE = true
Now it only break here:
"dealTime": 1479323911
}
,
undefined
,
And it never break at something like 41785:11 INFO [machine] 150 GMT2016-11-16T22:31:07.330Z now .
Somebody knows why this happen? Thank you.
I changed to
TZ=UTC
TRUNCATE = 0
BREAK_ONLY_BEFORE_DATE = false
BREAK_ONLY_BEFORE = ^\d+:\d+\s(INFO|ERROR|FATAL|WARN|DEBUG|TRACE)\s[machine]\s\d+\sGMT
TIME_PREFIX = ^\d+:\d+\s(INFO|ERROR|FATAL|WARN|DEBUG|TRACE)\s[machine]\s\d+\sGMT
TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%3N
MAX_DAYS_HENCE = 5
MAX_TIMESTAMP_LOOKAHEAD = 24
SHOULD_LINEMERGE = true
But the problem has not be solved: now it only break at :
"dealTime": 1479481957
}
,
undefined
,
{
"rcodeResponse": 0
}
This is now is took as one event. and
41785:11 INFO [machine] 150 GMT2016-11-16T22:31:07.689Z (59 ms) [uuid] 13683279 [firm] 9001 [sn] 866562 BCAST-fs:582CDE21190C000D processRetractEvent
now is not taken as an event.
Hi,
You can try this in the file props.conf:
SHOULD_LINEMERGE=true
NO_BINARY_CHECK=true
BREAK_ONLY_BEFORE=(\s\d\d\d\d\d:\d\d\sINFO)
Tks
Rodrigo Ribeiro
Thank you Rodrigo,
Sometimes the begging of the event is 41785:11 ERROR [machine]
How I could express this after BREAK_ONLY_BEFORE? Thank you!
Is this correct? BREAK_ONLY_BEFORE=(\s\d\d\d\d\d:\d\d\s\d{1,5}\s[machine])
And what is NO_BINARY_CHECK=true?
This is no problem, I use the following site to test my regular expressions:
https://regex101.com/r/YNDBcR/1
So it should look something like this:
(\s\d\d\d\d\d:\d\d\s(INFO|ERROR))
Note: It is worth noting that this is not a rule, it can be improved.
This option (NO_BINARY_CHECK), according to the link:
http://docs.splunk.com/Documentation/Splunk/6.5.0/Admin/Propsconf
NO_BINARY_CHECK = [true|false]
* When set to true, Splunk processes binary files.
* Can only be used on the basis of [], or [source::],
not [host::].
* Defaults to false (binary files are ignored).
* This setting applies at input time, when data is first read by Splunk.
The setting is used on a Splunk system that has configured inputs
acquiring the data.
Tks
Rodrigo Ribeiro