Getting Data In

How to assign syslog file to specific index based on file name in transforms.conf?

dhavamanis
Builder

Can you please provide sample configuration for the below, We have multiple forwarding sources and they are using syslog-ng. how to assign index based on log filename at indexer side.

Example :

mynbc-syslog-2015-07-30.log pattern files to go "mynbc" index
msnbc-syslog-2015-07-29.log patter files to go "msnbc" index
bravotv-syslog-2015-08-01_1.log patter files to go "bravotv" index

0 Karma
1 Solution

martin_mueller
SplunkTrust
SplunkTrust

Filename-based transforms.conf use the source key from the metadata like this:

[send_to_index_by_source]
SOURCE_KEY = MetaData:Source
REGEX = ^source::/path/to/files/(\w+)-syslog
DEST_KEY = _MetaData:Index
FORMAT = $1

Reference that in props.conf with TRANSFORMS-foo = send_to_index_by_source.

View solution in original post

martin_mueller
SplunkTrust
SplunkTrust

Filename-based transforms.conf use the source key from the metadata like this:

[send_to_index_by_source]
SOURCE_KEY = MetaData:Source
REGEX = ^source::/path/to/files/(\w+)-syslog
DEST_KEY = _MetaData:Index
FORMAT = $1

Reference that in props.conf with TRANSFORMS-foo = send_to_index_by_source.

dhavamanis
Builder

Forwarder- outputs.conf

[tcpout:splunkssl]
server = splunk.abc.com:9997
compressed = true

[tcpout-server://splunk.abc.com:9997]
sslCertPath = $SPLUNK_HOME/etc/certs/forwarder.pem
sslPassword = password
sslRootCAPath = $SPLUNK_HOME/etc/certs/cacert.pem

0 Karma

dhavamanis
Builder

trying with the below config, seems its not working for me, Can you please correct this if anything wrong,

props.conf

[syslog]
TRANSFORMS-idx_routing = generic_idx_routing

transforms.conf (created index abc)

[generic_idx_routing]
SOURCE_KEY = MetaData:Host
REGEX = abc.xyz.\nbcu.com
DEST_KEY = _MetaData:Index
FORMAT = $1

inputs.conf

[splunktcp-ssl:9997]
compressed = true
sourcetype = syslog

[SSL]
password = password
requireClientCert = false
rootCA = $SPLUNK_HOME/etc/certs/cacert.pem
serverCert = $SPLUNK_HOME/etc/certs/splunknode.pem

0 Karma

dhavamanis
Builder

Can you please provide the sample REGEX while using the the file name pattern like this ${sitename}.${logname} ${Existing MSG}. we want to parse the sitename and assign to index name.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...