Getting Data In

Masking a data at search time

tmuthuk
Path Finder

There is log which we have n index has sensitive information like Date of birth /SSN /Credit card

I want to mask the sensitive information at execution time . Here is the sample log

...................
ns6:Date_Time2013-05-13T01:00:09/ns6:Date_Time
ns6:UpdateSourceLY/ns6:UpdateSource
ns7:Person
ns7:FirstNameTest/ns7:FirstName
ns7:LastNameuser/ns7:LastName
ns7:DateOfBirth1984-03-03/ns7:DateOfBirth
......................

I want to mask the data of birth as below

ns7:DateOfBirthXXXX-XX-XX/ns7:DateOfBirth

The transforms.conf i used is

REGEX = (?m)^(.):DateOfBirth>\d{4}-d{2}-d{2}(.)$
FORMAT = $1:DateOfBirth>####-##-##$2
DEST_KEY = _raw

But it is not working . Can anyone please help on this ?

0 Karma

isoutamo
SplunkTrust
SplunkTrust

Hi

try to add a next definitions on search node's props.conf
--8<---
[your-source-type]
EXTRACT-DateOfBirth = \d{4}-\d{2}-\d{2}
EVAL-DateOfBirth = if(isnull(DateOfBirth),NULL,"####-##-##")
EVAL-_raw = replace(_raw,"\d{4}-\d{2}-\d{2}","####-##-##")
--8<---

And to metadata/default.meta
---8<--
[props/your-source-type/EXTRACT-DateOfBirth]
access = read : [ * ], write : [ admin ]
export = system
--8<--

These are working on our client's environment when we have to mask some other fields.

And if there are situation where you have to mask several fields on _raw then you should use next one on props.conf

--8<---
EVAL-_raw = replace(_raw, "(field1|field2|field3)=[([^]]+)]","\1=[####]")
EVAL-field1 = if(isnull(field1),NULL,"[####]")
EVAL-field2 = if(isnull(field2),NULL,"[####]")
EVAL-field3 = if(isnull(field3),NULL,"[####]")
--8<--

and of course to default.meta should be added all those definitions so no-one cannot update those by "accidentally".

where input is
field1=[asa],field2=[asdaa] etc.

regards Ismo

splunkreal
Motivator

Hi, looks nice, thanks.

* If this helps, please upvote or accept solution 🙂 *
0 Karma

martin_mueller
SplunkTrust
SplunkTrust

The key issue here appears to be that you're defining a mask at index time, not at search time. See http://docs.splunk.com/Documentation/Splunk/5.0.2/Data/Anonymizedatausingconfigurationfiles#Through_... for reference.

0 Karma

vkakani60
Path Finder

what if the data is coming from dbconnect query ?

0 Karma

MuS
SplunkTrust
SplunkTrust

Hi tmuthuk

your regex does not match the dateofbirth, d{4} looks for four d's and not four digits. I've played around at RegExr and found that this regex

\d{4}-\d{2}-\d{2}(?!T)

will match the dateofbirth in your provided log text.

cheers,
MuS

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...