Ok Splunkers......
I have 1 search-head, 2 indexers, 1 Deployment server
Here is the event (sourcetype=mysourcetype)that I want to mask out the CC number:
2014-06-01 00:01:34 W3SVC1 10.0.99.120 GET /Disputes/BackToMYDomain.aspx ID=183481&ClaimNum=05/31-1370&DType=DMC&DClass=Debit%20Fraud&DeptExt=1234&**Card=1234567891234567**&SubmitDate=5/31/2014&samedayptr=N&CrdHolder=JOHN+DOE&TotClaim=150.00&Action=REVPC80MyDomain\mmouse10.XX.XX.XXMozilla/4.0+compatible;+MSIE+7.0;+Windows+NT+6.1;+WOW64;+Trident/5.0;+SLCC2;+.NET+CLR+2.0.50727;+.NET+CLR+3.5.30729;+.NET+CLR+3.0.30729;+Media+Center+PC+6.0;+.NET4.0C;+.NET4.0E;+InfoPath.3) 200 0 0
I have the following props & Transforms in my Splunk_HOME/etc/system/local/directory:
props.conf:
[sourcetype]
TRANSFORMS-1card = cc_num_anon
Transforms.conf:
[cc_num_anon]
REGEX = (.*CARD=)\d{12}(\d{4}.*)
DEST_KEY = _raw
FORMAT = $1xxxxxxxxxxxx$2
I ran the debug mode, reloaded the forwarders....but Im still seeing the card information.....Oh I forgot to mention that the variable changes from Card/CARD within the event.
Try this props.conf-only solution that honours the variable case:
[sourcetype]
SEDCMD-cc = s/(?i)(card=)\d{12}(\d{4})/\1xxxxxxxxxxxx\2/g
This should also run considerably faster because a leading .*
in a regex will eat your server's soul.
Try this props.conf-only solution that honours the variable case:
[sourcetype]
SEDCMD-cc = s/(?i)(card=)\d{12}(\d{4})/\1xxxxxxxxxxxx\2/g
This should also run considerably faster because a leading .*
in a regex will eat your server's soul.
This config belongs on the indexers. If you have heavy forwarders that perform parsing then you will need it there as well.
The second event is several hours older, it was indexed before the SEDCMD was applied. I'm sure of this because there is $2
at the end, and there is no dollar sign used in the SEDCMD.
So Im having a different scenario since I added the SEDCMD to the indexer local props.conf:
Right output: (IT WORKS)
2015-01-31 03:58:10 W3SVC1 10.XX.XX.XX GET /Disputes/BackTohost.aspx ID=222888&ShID=70&
Choice=Fraud&CARD=xxxxxxxxxxxx5144&DType=DEBIT&ACode=W&SCode=G&AmtFr=&AmtTo=&DtFr=01/29/2015&DtTo=01/30/2015&CCIssued=&
Action=History 80 MyDomain\mmouse 10.XX.XX.XX Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.1;+WOW64;+Trident/5.0;+SLCC2;
+.NET+CLR+2.0.50727;+.NET+CLR+3.5.30729;+.NET+CLR+3.0.30729;+Media+Center+PC+6.0;+.NET4.0C;+.NET4.0E;+InfoPath.3) 200 0 0
Then I get a different output from the same sourcetype: (Its not changing the Card=, but adding an end output)
2015-01-30 20:33:52 W3SVC1 10.XX.XX.XX GET /Disputes/BackTohost.aspx ID=222796&
ShID=ALL%20SHARES&Choice=Fraud&CARD=1234567891234567&DType=DEBIT&ACode=W&SCode=G&AmtFr=&AmtTo=&DtFr=&DtTo=&
CCIssued=&Action=History 80 MyDoamin\mmouse 10.XX.XX.XX Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.1;+WOW64;+Trident/5.0;
+SLCC2;+.NET+CLR+2.0.50727;+.NET+CLR+3.5.30729;+.NET+CLR+3.0.30729;+Media+Center+PC+6.0;+.NET4.0C;+.NET4.0E) 200 0 0xxxxxxxxxxxx$2
The the second event it applying the numbers 0xxxxxxxxxxxx$2
Is this because the event is different in word count?
Silly question...when you say add to the props.conf...this is on the indexers, correct?
Indexer
Splunk_HOME/etc/system/local
OR
The deployment server in Deployment apps (the app) local folder.
Normally, such masking will happen at parse time on the indexer, so just to be clear, these configs are on the indexers, correct?
A few things strike me as off about your regex:
REGEX = (.CARD=)d{12}(d{4}.)
I am not sure about the dots at the beginning and end, case is not matching your data either, also the character types don't have backslashes. So, to me this looks a little better:
REGEX = (Card=)\d{12}(\d{4})
Also note that this will only apply to newly indexed data once the transform is in place.
See updated answer.