Splunk Search

How to write a regex to extract a field within a particular field?

sundarrajan
Path Finder

I am looking for a way to some how extract and mask some of important information that comes within logs. I don't have absolute permission to access "props.conf" and "transforms.conf" and hence i have to first search for the key word, extract the key field and then then mask it. The information that i wish to mask is password and login credentials which is part of field .
USER_CREDENTIAL, field is of the format \"userID\":\"ABC1234\" ,\"password\":\"abcd345\", \"email\":\"abcd@gmail.com\",\"country\":\"AAAA\"
i tried the following format to extract "password", rex field=USER_CREDENTIAL "(?\w+\D+\w+[^\]) but it extract both userID and password as PASSWORD,
I am also looking for an easy way to mask the same, earlier i tried of masking the details but i was left with masking both userID and password, basesearch | rex field=USER_CREDENTIAL mode=sed "s/(\w+)(\D+)(\w+)(\w+)/1\\":\\"XXXXXXX/2", but the masking is not efficient when the format in userID changes. Hence thought of extracting the field and then masking the same.
Please do suggest if there could be any easy way to get the masking done.

0 Karma
1 Solution

somesoni2
SplunkTrust
SplunkTrust

Give this a try. First two lines are to generate sample data, replace that with your search. First rex extracts fields and second rex masks the sensitive information

| gentimes start=-1 | eval USER_CREDETIAL="\\\"userID\\\":\"ABC1234\\\" ,\\\"password\\\":\\\"abcd345\\\", \\\"email\\\":\\\"abcd@gmail.com\\\",\\\"country\\\":\\\"AAAA\\\"" | table USER_CREDETIAL 
| rex field=USER_CREDETIAL "userID([^\"]+\"){2}(?<userID>[^\\\]+).+password([^\"]+\"){2}(?<password>[^\\\]+)" 
| rex field=USER_CREDETIAL mode=sed "s/(userID[^\"]+\"[^\"]+\")([^\\\]+)(.+password[^\"]+\"[^\"]+\")([^\\\]+)/\1XXXXX\3XXXX/"

View solution in original post

0 Karma

somesoni2
SplunkTrust
SplunkTrust

Give this a try. First two lines are to generate sample data, replace that with your search. First rex extracts fields and second rex masks the sensitive information

| gentimes start=-1 | eval USER_CREDETIAL="\\\"userID\\\":\"ABC1234\\\" ,\\\"password\\\":\\\"abcd345\\\", \\\"email\\\":\\\"abcd@gmail.com\\\",\\\"country\\\":\\\"AAAA\\\"" | table USER_CREDETIAL 
| rex field=USER_CREDETIAL "userID([^\"]+\"){2}(?<userID>[^\\\]+).+password([^\"]+\"){2}(?<password>[^\\\]+)" 
| rex field=USER_CREDETIAL mode=sed "s/(userID[^\"]+\"[^\"]+\")([^\\\]+)(.+password[^\"]+\"[^\"]+\")([^\\\]+)/\1XXXXX\3XXXX/"
0 Karma

yannK
Splunk Employee
Splunk Employee

Assuming your event is like :

  \"userID\":\"ABC1234\" ,\"password\":\"abcd345\"

For the sed replacement, have you tried to use a broader condition, by example the double quote character as string limit for your values ?

mysearch | rex mod=sed "s/\"userID\":\"([^\"]*)\"/\"userID\":\"XXXX\"/g"

mysearch | rex mod=sed "s/\"password\":\"([^\"]*)\"/\"password\":\"XXXX\"/g"

It will of course fail if your user or password do contains doublequotes in it.

then you can merge in one sed, or use 2 sed commands.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...