Splunk Search

How to use SEDCMD to anonymize a field after automatic lookup from a CSV file at index-time?

joarsvensson
New Member

I want to do an automatic lookup from a CSV file on index time, and add new fields to the event. I got this working, but what if I want to anonymize the field used as lookup key afterwards?

Using this won't work since it seem to happen prior to the lookup runs:

props.conf

[default]
SEDCMD-anonymize = s/username=(......)/username=XXXXXX/g

Help appreciated!

0 Karma
1 Solution

woodcock
Esteemed Legend

It cannot be done without augmenting the data at Index-Time to include the lookup details. Lookups happen at Search-Time ALWAYS.

View solution in original post

0 Karma

koshyk
Super Champion

Hope Splunk enabled a similar option for "tokenisation" of certain fields at index time (eg credit card numbers for apple pay)

0 Karma

woodcock
Esteemed Legend

It cannot be done without augmenting the data at Index-Time to include the lookup details. Lookups happen at Search-Time ALWAYS.

0 Karma

joarsvensson
New Member

Thank you for clarifying! So I need to populate the data prior to indexing, in order for this to work.

0 Karma

woodcock
Esteemed Legend

Yes, think of it this way: any field created at Index-Time must be based off of a continuous string inside of the event itself (e.g. field X starts as position Y and ends at position Z) or in the meta-data for the event (e.g. source). This is how all Index-Time fields are defined and there is not (and probably never will be) any exception. Once I realized this, my thinking about fields became much more clear.

joarsvensson
New Member

Does no one have a solution or guidance to this? Help is very much appreciated!

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...