Getting Data In

timestamp and line breaks

rewritex
Contributor

The timestamp and linebreaking doesn't seem to be working as expected. They are nagios/pnp4nagios logs.
I get a burst of events similar to the below data every few seconds/minutes and it seems the first line of each data burst is being recognized for the TIMET timestamp but all other events within that data burst aren't being handled correctly.

TIMET::1506034709 = timestamp in epoch time
DATATYPE:: = start/end of event

Data is sent in this format: DATATYPE::SERVICEPERFDATA\tTIMET::$TIMET$\tHOSTNAME::$HOSTNAME$\t

Here's the data:

DATATYPE::HOSTPERFDATA  TIMET::1506034709   HOSTNAME::host1 HOSTPERFDATA::time=0.000342s;;;0.000000;20.000000   HOSTCHECKCOMMAND::check_tcp!255.255.25.25!443   HOSTSTATE::UP   HOSTSTATETYPE::HARD HOSTOUTPUT::TCP OK - 0.000 second response time on 255.255.25.25 port 443   
DATATYPE::HOSTPERFDATA  TIMET::1506034713   HOSTNAME::host2 HOSTPERFDATA::time=0.000368s;;;0.000000;20.000000   HOSTCHECKCOMMAND::check_tcp!255.255.25.256!443  HOSTSTATE::UP   HOSTSTATETYPE::HARD HOSTOUTPUT::TCP OK - 0.000 second response time on 255.255.25.256 port 443

Here's the sourcetype config: - timestamp/linebreak

[nagios:core:perfdata]
event_breaks: (I've tried auto and every line)
BREAK_ONLY_BEFORE = ([\r\n]+)DATATYPE
SHOULD_LINEMERGE = true
TIME_FORMAT =  %s
TIME_PREFIX = TIMET::
lookahead 128
0 Karma
1 Solution

rewritex
Contributor

I had to modify the props.conf on the cluster/indexers and that seemed to get it working. I was in the SH mucking around with the props.conf
Thanks for the reminder.

$SPLUNK_HOME$/etc/master-apps/_cluster/local/props.conf

BREAK_ONLY_BEFORE=DATATYPE::
SHOULD_LINEMERGE = false
TIME_FORMAT = %s
TIME_PREFIX = TIMET::

View solution in original post

rewritex
Contributor

I had to modify the props.conf on the cluster/indexers and that seemed to get it working. I was in the SH mucking around with the props.conf
Thanks for the reminder.

$SPLUNK_HOME$/etc/master-apps/_cluster/local/props.conf

BREAK_ONLY_BEFORE=DATATYPE::
SHOULD_LINEMERGE = false
TIME_FORMAT = %s
TIME_PREFIX = TIMET::

s2_splunk
Splunk Employee
Splunk Employee

Ha, he/she who's never done that, speak up now or be silent forever! 🙂
Glad you got it working.

0 Karma

s2_splunk
Splunk Employee
Splunk Employee

Is there a line breaker in the source events at all? From your post there is, so standard line breaking (using CRLF) should work. If it doesn't, there is no line feed in the source.
You can try BREAK_ONLY_BEFORE=DATATYPE::
Unless you are dealing with multi-line events, set SHOULD_LINEMERGE=false
Line 7 in your props.conf above is not a valid setting, it should be MAX_TIMESTAMP_LOOKAHEAD=128.

Also, you configured that where parsing occurs (indexer, heavy forwarder), correct?

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...