Getting Data In

How do you modify data values before indexing?

soumyacharya91
Path Finder

Hi All,

I want to remove more than 2 white spaces from event values at heavy forwarder before ingesting to indexer. Can anyone guide me with this change so that I can be able to fix the issue.

Current State :

field1="xxxxxx", field2="xxx                          ", field3="xxx    ", field4="x", field5="xxxx                                                                       ", field6="xxx  ", field7="xxx                                                                                            ", field8="xxxx                                                                                           ", field9="xxxxx                                                                                                                                                                                                                                   ", field10="xxxxx"

Required State
field1="xxxxxx", field2="xxx", field3="xxx", field4="x", field5="xxxx", field6="xxx", field7="xxx", field8="xxxx", field9="xxxxx", field10="xxxxx"

0 Karma
1 Solution

harsmarvania57
Ultra Champion

Hi @soumyacharya91,

If you have consistent same data in single sourcetype in that case you can implement below configuration in props.conf on Indexer or Heavy Forwarder whichever comes first from UF.

props.conf

[yoursourcetype]
SEDCMD-removewhitespace = s/\b(?:(\w+))\b=\"(?:(\w+).*?)\"/\1="\2"/g

Restart splunk on Indexer/Heavy Forwarder.

For your reference here is regex with sample data https://regex101.com/r/wf7DAH/1

View solution in original post

harsmarvania57
Ultra Champion

Hi @soumyacharya91,

If you have consistent same data in single sourcetype in that case you can implement below configuration in props.conf on Indexer or Heavy Forwarder whichever comes first from UF.

props.conf

[yoursourcetype]
SEDCMD-removewhitespace = s/\b(?:(\w+))\b=\"(?:(\w+).*?)\"/\1="\2"/g

Restart splunk on Indexer/Heavy Forwarder.

For your reference here is regex with sample data https://regex101.com/r/wf7DAH/1

soumyacharya91
Path Finder

It is coming from db input.

0 Karma

soumyacharya91
Path Finder

Thanks a lot It is now fixed. You deserve a Chocolate !!

0 Karma

harsmarvania57
Ultra Champion

Great it worked, chcolate pending 🙂

0 Karma

harsmarvania57
Ultra Champion

You can implement above configuration on server on which Splunk DB Connect is installed, here I am guessing that you have configured DB Input in DB Connect to pull data at certain interval from Database.

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...