Getting Data In

Sanitize Incoming Data (remove Passwords)

aattinello
Engager

Hello,
I am looking for a sanitize my incoming data. My customers sometimes pass GET parameters instead of POST parameters, which is normally fine. However in some cases they pass their password into our API as a GET parameter which then appears in plain text in my webserver log. When i send this data to Splunk I would like to match that password and replace with a string like FILTERED. The tool I am using to log this data has no way to scrub that data while preserving the other get parameters, so I was hoping Splunk was able to.

So some of my sample lines looks like this
10.213.172.3 [02/May/2014:16:31:07 -0400] 31249 "GET /endPoint/?action=login&loginUsername=test&loginOrganization=Test&loginPassword=superTest HTTP/1.1" 200 570 4243 "Zend_Http_Client" "-" -

10.213.172.3 [02/May/2014:16:31:16 -0400] 187498 "POST /endpoint/other/otherPage.html?loginUsername=test&loginPassword=superTest&loginOrganization=Test HTTP/1.1" 200 1573 708 "Zend_Http_Client" "en-US,en;q=0.8" 6E1182505E7B71DAA4340E831A53F440.node1

I am looking to match this parameter (up until the first space or &)
&loginPassword=((.*&)|(\S+))
And replace that with something like
&loginPassword=FILTERED

So those 2 examples would end up indexed as
10.213.172.3 [02/May/2014:16:31:07 -0400] 31249 "GET /endPoint/?action=login&loginUsername=test&loginOrganization=Test&loginPassword=FILTERED HTTP/1.1" 200 570 4243 "Zend_Http_Client" "-" -

10.213.172.3 [02/May/2014:16:31:16 -0400] 187498 "POST /endpoint/other/otherPage.html?loginUsername=test&loginPassword=FILTERED&loginOrganization=Test HTTP/1.1" 200 1573 708 "Zend_Http_Client" "en-US,en;q=0.8" 6E1182505E7B71DAA4340E831A53F440.node1

0 Karma
1 Solution

gfreitas
Builder

Hi aattinello,

I know you can mask sensitive data using props.conf and transforms.conf.

In props.conf:

[source::\\yoursource.log]
TRANSFORMS-password = password_mask

And in transforms.conf:

[password_mask]
DEST_KEY = _raw
REGEX = (.*loginPassword=)\d\s
FORMAT = $FILTERED$

I don't understand a lot of regex, but maybe you can modify it using some online checker.

Hope this helps!

View solution in original post

gfreitas
Builder

Hi aattinello,

I know you can mask sensitive data using props.conf and transforms.conf.

In props.conf:

[source::\\yoursource.log]
TRANSFORMS-password = password_mask

And in transforms.conf:

[password_mask]
DEST_KEY = _raw
REGEX = (.*loginPassword=)\d\s
FORMAT = $FILTERED$

I don't understand a lot of regex, but maybe you can modify it using some online checker.

Hope this helps!

aattinello
Engager

Yes, that is what i was looking for, thank you very much.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...