Getting Data In

Where to edit props.config for breaking log into multiple events?

itdeptPFS
New Member

I am using universal forwarders to move log data from remote servers to a centralized Splunk Light server. Where do I edit the props.config? On the remote server or on the centralized Splunk Light server? When I search for props.config, I am returned server files, which one should I choose? I am a surprised that Splunk doesn't automatically break these into events because my log file has a blank space between each event.

MessageType: INFO | ApplicationName: MicroSaleDataTransporter | ApplicationVersion: 1.3.0.0 | Message: Hourly...

MessageType: INFO | ApplicationName: MicroSaleDataTransporter | ApplicationVersion: 1.3.0.0 | Message: Successfully....

MessageType: INFO | ApplicationName: MicroSaleDataTransporter | ApplicationVersion: 1.3.0.0 | Message: File created...

MessageType: INFO | ApplicationName: MicroSaleDataTransporter | ApplicationVersion: 1.3.0.0 | Message: People Counter...

MessageType: INFO | ApplicationName: MicroSaleDataTransporter | ApplicationVersion: 1.3.0.0 | Message: Successfully...

Thanks,
Chris

0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

Edit the props.conf file wherever event parsing is done. This is usually your indexer, but could also be a heavy forwarder.

You probably have several different props.conf files. The one to edit is the one in the app the corresponds to the data you are indexing. Be sure to edit local/props.conf (create it if you need to) rather than default/props.conf.

Splunk expects events to be separated by line-end characters (\n on Linux, \r\n on Windows) and to have a timestamp. It's best, however, not to allow Splunk to make guesses about the format of your events. Instead, use props.conf to describe your data. At the very least, include the TIME_PREFIX, TIME_FORMAT, LINE_BREAKER, SHOULD_LINEMERGE, TRUNCATE, and MAX_TIMESTAMP_LOOKAHEAD attributes.

---
If this reply helps you, Karma would be appreciated.

View solution in original post

0 Karma

pruthvikrishnap
Contributor

Hi itdept,

Description by richgalloway is very details and will work, it should be something like this basing on your data.
[sourcetype]
SHOULD_LINEMERGE=true
NO_BINARY_CHECK=true
BREAK_ONLY_BEFORE=Regex
TIME_FORMAT=%H:%M:%S.%3N
TIME_PREFIX=^
MAX_TIMESTAMP_LOOKAHEAD=12

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Edit the props.conf file wherever event parsing is done. This is usually your indexer, but could also be a heavy forwarder.

You probably have several different props.conf files. The one to edit is the one in the app the corresponds to the data you are indexing. Be sure to edit local/props.conf (create it if you need to) rather than default/props.conf.

Splunk expects events to be separated by line-end characters (\n on Linux, \r\n on Windows) and to have a timestamp. It's best, however, not to allow Splunk to make guesses about the format of your events. Instead, use props.conf to describe your data. At the very least, include the TIME_PREFIX, TIME_FORMAT, LINE_BREAKER, SHOULD_LINEMERGE, TRUNCATE, and MAX_TIMESTAMP_LOOKAHEAD attributes.

---
If this reply helps you, Karma would be appreciated.
0 Karma

itdeptPFS
New Member

Thank you! I created the props.conf file on the indexer and it is working. I am still not exactly sure how this fowarder creates a SourceType name but I am finally getting my logs broken correctly.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

The sourcetype is specified in the forwarder's inputs.conf file.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...