Splunk Search

Masking Sensitive Data

dperry
Communicator

Ok Splunkers......

I have 1 search-head, 2 indexers, 1 Deployment server

Here is the event (sourcetype=mysourcetype)that I want to mask out the CC number:

2014-06-01 00:01:34 W3SVC1 10.0.99.120 GET /Disputes/BackToMYDomain.aspx ID=183481&ClaimNum=05/31-1370&DType=DMC&DClass=Debit%20Fraud&DeptExt=1234&**Card=1234567891234567**&SubmitDate=5/31/2014&samedayptr=N&CrdHolder=JOHN+DOE&TotClaim=150.00&Action=REVPC80MyDomain\mmouse10.XX.XX.XXMozilla/4.0+compatible;+MSIE+7.0;+Windows+NT+6.1;+WOW64;+Trident/5.0;+SLCC2;+.NET+CLR+2.0.50727;+.NET+CLR+3.5.30729;+.NET+CLR+3.0.30729;+Media+Center+PC+6.0;+.NET4.0C;+.NET4.0E;+InfoPath.3) 200 0 0

I have the following props & Transforms in my Splunk_HOME/etc/system/local/directory:

props.conf:
[sourcetype]
TRANSFORMS-1card = cc_num_anon

Transforms.conf:
[cc_num_anon]
REGEX = (.*CARD=)\d{12}(\d{4}.*)
DEST_KEY = _raw
FORMAT = $1xxxxxxxxxxxx$2

I ran the debug mode, reloaded the forwarders....but Im still seeing the card information.....Oh I forgot to mention that the variable changes from Card/CARD within the event.

0 Karma
1 Solution

martin_mueller
SplunkTrust
SplunkTrust

Try this props.conf-only solution that honours the variable case:

[sourcetype]
SEDCMD-cc = s/(?i)(card=)\d{12}(\d{4})/\1xxxxxxxxxxxx\2/g

This should also run considerably faster because a leading .* in a regex will eat your server's soul.

View solution in original post

martin_mueller
SplunkTrust
SplunkTrust

Try this props.conf-only solution that honours the variable case:

[sourcetype]
SEDCMD-cc = s/(?i)(card=)\d{12}(\d{4})/\1xxxxxxxxxxxx\2/g

This should also run considerably faster because a leading .* in a regex will eat your server's soul.

martin_mueller
SplunkTrust
SplunkTrust

This config belongs on the indexers. If you have heavy forwarders that perform parsing then you will need it there as well.

The second event is several hours older, it was indexed before the SEDCMD was applied. I'm sure of this because there is $2 at the end, and there is no dollar sign used in the SEDCMD.

0 Karma

dperry
Communicator

So Im having a different scenario since I added the SEDCMD to the indexer local props.conf:

Right output: (IT WORKS)

2015-01-31 03:58:10 W3SVC1 10.XX.XX.XX GET /Disputes/BackTohost.aspx ID=222888&ShID=70&
Choice=Fraud&CARD=xxxxxxxxxxxx5144&DType=DEBIT&ACode=W&SCode=G&AmtFr=&AmtTo=&DtFr=01/29/2015&DtTo=01/30/2015&CCIssued=&
Action=History 80 MyDomain\mmouse 10.XX.XX.XX Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.1;+WOW64;+Trident/5.0;+SLCC2;
+.NET+CLR+2.0.50727;+.NET+CLR+3.5.30729;+.NET+CLR+3.0.30729;+Media+Center+PC+6.0;+.NET4.0C;+.NET4.0E;+InfoPath.3) 200 0 0

Then I get a different output from the same sourcetype: (Its not changing the Card=, but adding an end output)

2015-01-30 20:33:52 W3SVC1 10.XX.XX.XX GET /Disputes/BackTohost.aspx ID=222796&
ShID=ALL%20SHARES&Choice=Fraud&CARD=1234567891234567&DType=DEBIT&ACode=W&SCode=G&AmtFr=&AmtTo=&DtFr=&DtTo=&
CCIssued=&Action=History 80 MyDoamin\mmouse 10.XX.XX.XX Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.1;+WOW64;+Trident/5.0;
+SLCC2;+.NET+CLR+2.0.50727;+.NET+CLR+3.5.30729;+.NET+CLR+3.0.30729;+Media+Center+PC+6.0;+.NET4.0C;+.NET4.0E) 200 0 0xxxxxxxxxxxx$2

The the second event it applying the numbers 0xxxxxxxxxxxx$2

Is this because the event is different in word count?

0 Karma

dperry
Communicator

Silly question...when you say add to the props.conf...this is on the indexers, correct?

Indexer
Splunk_HOME/etc/system/local

OR

The deployment server in Deployment apps (the app) local folder.

0 Karma

chanfoli
Builder

Normally, such masking will happen at parse time on the indexer, so just to be clear, these configs are on the indexers, correct?

A few things strike me as off about your regex:

REGEX = (.CARD=)d{12}(d{4}.)

I am not sure about the dots at the beginning and end, case is not matching your data either, also the character types don't have backslashes. So, to me this looks a little better:

REGEX = (Card=)\d{12}(\d{4})

Also note that this will only apply to newly indexed data once the transform is in place.

0 Karma

chanfoli
Builder

See updated answer.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...