Can you please provide sample configuration for the below, We have multiple forwarding sources and they are using syslog-ng. how to assign index based on log filename at indexer side.
Example :
mynbc-syslog-2015-07-30.log pattern files to go "mynbc" index
msnbc-syslog-2015-07-29.log patter files to go "msnbc" index
bravotv-syslog-2015-08-01_1.log patter files to go "bravotv" index
Filename-based transforms.conf use the source
key from the metadata like this:
[send_to_index_by_source]
SOURCE_KEY = MetaData:Source
REGEX = ^source::/path/to/files/(\w+)-syslog
DEST_KEY = _MetaData:Index
FORMAT = $1
Reference that in props.conf with TRANSFORMS-foo = send_to_index_by_source
.
Filename-based transforms.conf use the source
key from the metadata like this:
[send_to_index_by_source]
SOURCE_KEY = MetaData:Source
REGEX = ^source::/path/to/files/(\w+)-syslog
DEST_KEY = _MetaData:Index
FORMAT = $1
Reference that in props.conf with TRANSFORMS-foo = send_to_index_by_source
.
Forwarder- outputs.conf
[tcpout:splunkssl]
server = splunk.abc.com:9997
compressed = true
[tcpout-server://splunk.abc.com:9997]
sslCertPath = $SPLUNK_HOME/etc/certs/forwarder.pem
sslPassword = password
sslRootCAPath = $SPLUNK_HOME/etc/certs/cacert.pem
trying with the below config, seems its not working for me, Can you please correct this if anything wrong,
props.conf
[syslog]
TRANSFORMS-idx_routing = generic_idx_routing
transforms.conf (created index abc)
[generic_idx_routing]
SOURCE_KEY = MetaData:Host
REGEX = abc.xyz.\nbcu.com
DEST_KEY = _MetaData:Index
FORMAT = $1
inputs.conf
[splunktcp-ssl:9997]
compressed = true
sourcetype = syslog
[SSL]
password = password
requireClientCert = false
rootCA = $SPLUNK_HOME/etc/certs/cacert.pem
serverCert = $SPLUNK_HOME/etc/certs/splunknode.pem
Can you please provide the sample REGEX while using the the file name pattern like this ${sitename}.${logname} ${Existing MSG}. we want to parse the sitename and assign to index name.