Getting Data In

transforms and props.conf splitting sourcetype not matching

dstark
Explorer

I'm trying to follow the pattern of matching a string and transforming the event into a new sourcetype. I'm using a sourcetype for syslog defined in inputs.conf; it is being read from logs.

/var/log/syslog contains events matching string "sqsd" that I would like to rewrite to a new sqsd sourcetype

I've found multiple answers posts about this topic but can't seem to determine why I'm not getting any data as the transformed sourcetype. Originally, I thought the problem was in my REGEX in transforms.conf, but if I set it to .* or remove it completely I still don't get results.

inputs.conf

[monitor:///var/log/syslog]
sourcetype=syslog
index = test
ignoreOlderThan = 24h

props.conf

[syslog]
TRANSFORMS-syslog = set_sqsd_sourcetype

transforms.conf

[set_sqsd_sourcetype]
REGEX = sqsd
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::sqsd

splunkd.log does not show any errors, so I don't think my conf files are invalid. Right now I am seeing all events as sourcetype=syslog.

0 Karma
1 Solution

Nicolo_Figiani
Path Finder

Hi,
try using the btool command to troubleshoot your issue. On the Splunk instance that should do the sourcetype rewrite (keep in mind that this happens at index time), do the following from the CLI:

splunk cmd btool props list syslog --debug

Thie above will lists all the specs of the "syslog" sourcetype. Make sure you're seeing the transformation you've defined "set_sqsd_sourcetype".

Do the same for the actual transformation:

splunk cmd btool transforms list set_sqsd_sourcetype --debug

Once again, make sure, the specs you have defined are listed.

Very likely, the configuration is not applied correctly or it is overridden by something else..the btool command should give you the exact snapshot of which configuration is enabled and it helped me hundreds of time (many times for the issue of mis-overwritten sourcetypes at index time).

Hope it help,
N

View solution in original post

0 Karma

somesoni2
SplunkTrust
SplunkTrust

Are you using Universal forwarder OR heavy forwarder to collect the data? If it's UF then your props.conf and transforms.conf should be on Indexer/next heavy forwarder in the flow. If it's HF then your props and transforms should be on Heavy Forwarder.

dstark
Explorer

Ah....I'm using a universal forwarder and Splunk Cloud. I found this link (search "im-struggling-with-how-i-should-be-doing-inputs-and-also-props-transforms-etc-stuff-within-splunk-cloud" as I can't post links) which explains how to modify props and transforms.conf using the GUI in Splunk Cloud. I'll give it a shot and post screenshots if I can get it working.

Thanks somesoni2!

0 Karma

dstark
Explorer

I've verified that I should be changing the Splunk Cloud instance with the link above. However, I'm still not seeing results with sourcetype=sqsd.

I'm not able to set the DEST_KEY = MetaData:Sourcetype using the GUI, maybe this is the problem?
http://imgur.com/a/jFy8B

0 Karma

somesoni2
SplunkTrust
SplunkTrust

That's probably it. You need Splunk's held to get this props and transforms deployed.

0 Karma

Nicolo_Figiani
Path Finder

Hi,
try using the btool command to troubleshoot your issue. On the Splunk instance that should do the sourcetype rewrite (keep in mind that this happens at index time), do the following from the CLI:

splunk cmd btool props list syslog --debug

Thie above will lists all the specs of the "syslog" sourcetype. Make sure you're seeing the transformation you've defined "set_sqsd_sourcetype".

Do the same for the actual transformation:

splunk cmd btool transforms list set_sqsd_sourcetype --debug

Once again, make sure, the specs you have defined are listed.

Very likely, the configuration is not applied correctly or it is overridden by something else..the btool command should give you the exact snapshot of which configuration is enabled and it helped me hundreds of time (many times for the issue of mis-overwritten sourcetypes at index time).

Hope it help,
N

0 Karma

dstark
Explorer

I had to contact Splunk support to apply these configs to my Splunk Cloud indexer. Thank you Nicolo_Figiani and somesoni2 for your help!

Nicolo_Figiani
Path Finder

I guess we know why now. You're configuring the source type override on your universal forwarder. However, it is not a full Splunk instance and It has not parsing capabilities. You should configure everything on the first FULL Splunk instance ( HFW or IDX ) of your chain.
Your configuration looks ok so, just place it on the right Splunk instance and everything should work.

In addition, the entry regarding the props.conf has a different name "syslog-host" whereas you're looking for "syslog" and it refers to /system/default/props.conf. This is not you're configuration so, be sure to deploy your props config as well..

Hope it helps and let me know..

dstark
Explorer

As far as I can tell btool is showing exactly what I expect. There's a unique TRANSFORMS for syslog in /etc/system/local and the default TRANFSORMS from /etc/system/default:

/opt/splunkforwarder/etc/system/default/props.conf TRANSFORMS = syslog-host
/opt/splunkforwarder/etc/system/local/props.conf TRANSFORMS-syslog = set_sqsd_sourcetype

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...