All Apps and Add-ons

Events not appearing

chriswong187
Explorer

I configured the sourcetype to fml:log according to the details but no events appear.

When the sourcetype is changed to syslog events start to appear.

Switching back to fml:log stops events again. Any ideas?

1 Solution

chriswong187
Explorer

Hi all, after going through the splunkd.log and I saw a few entries that indicated the LINE_BREAKER entry in props.conf wasn't working right.

LineBreakingProcessor - Truncating line because limit of 99999 bytes has been exceeded with a line length >= 100063

I tried a few variations of LINE_BREAKER in props.conf and ultimately got it to work. I ended up changing

LINE_BREAKER = ([\n\r]+)date=\d{4}\-\d{2}\-\d{2}

To

LINE_BREAKER = ([\n\r]+)

I also reverted the previous changes I had tried with TRANSFORMS back to their default values and things still seemed to work with this new LINE_BREAKER. ¯\(ツ)

Thanks to all that replied with suggestions.

*edit: code blocks

View solution in original post

chriswong187
Explorer

Hi all, after going through the splunkd.log and I saw a few entries that indicated the LINE_BREAKER entry in props.conf wasn't working right.

LineBreakingProcessor - Truncating line because limit of 99999 bytes has been exceeded with a line length >= 100063

I tried a few variations of LINE_BREAKER in props.conf and ultimately got it to work. I ended up changing

LINE_BREAKER = ([\n\r]+)date=\d{4}\-\d{2}\-\d{2}

To

LINE_BREAKER = ([\n\r]+)

I also reverted the previous changes I had tried with TRANSFORMS back to their default values and things still seemed to work with this new LINE_BREAKER. ¯\(ツ)

Thanks to all that replied with suggestions.

*edit: code blocks

woodcock
Esteemed Legend

Come back and click Accept on your answer to close your question.

0 Karma

pruthvikrishnap
Contributor

Hi Chris,
one thing we can do is to go to Splunk UI

Settings > Sourcetypes > select syslog > advanced settings
Copy the settings and create a new sourcetype with the same settings and then use it.
It should have it resolved.

0 Karma

prakash007
Builder

@chriswong187 looks like you have to make some tweaks to bypass syslog(in built splunk sourcetype) parsing according to your requirements..

Check this out....
https://www.splunk.com/blog/2008/04/16/overriding-default-syslog-host-extraction.html

Check your props/transforms with btool..

$SPLUNK_HOME/bin/splunk cmd btool props list --debug syslog
$SPLUNK_HOME/bin/splunk cmd btool props list --debug fml:log
0 Karma

chriswong187
Explorer

Thanks for pointing me to the article. I took some steps to correct everything based on the article but it didn't seem to work, unless i overlooked something. Here's some info on what I found:

PS C:\Program Files\Splunk\bin> splunk cmd btool props list --debug syslog
C:\Program Files\Splunk\etc\system\default\props.conf            TRANSFORMS = syslog-host

PS C:\Program Files\Splunk\bin> splunk cmd btool props list --debug fml:log
C:\Program Files\Splunk\etc\apps\Splunk_TA_fortimail\local\props.conf   TRANSFORMS = 

I modified the fortimail props.conf to include this:

TRANSFORMS = fortimail

And included this in the transforms.conf:

[fortimail]
DEST_KEY = MetaData:Host
REGEX = .
FORMAT = host::host.fqdn host::host2.fqdn
0 Karma

prakash007
Builder

which splunk instance you have this configs deployed..?? heavy forwarder or indexer..??

0 Karma

chriswong187
Explorer

Indexer only, I have a single server deployment.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...