Been trying to mask data before indexing into indexer using heavy forwarders. below is the log sample and data am trying to mask
JSESSIONID=SD1SL10FF3ADFF3" to JSESSIONID=#######FF3ADFF3"
189.222.1.46 - - [24/Jul/2014:11:27:00] "GET /flower_store/product.screen?product_id=RP-SN-01 HTTP/1.1" 200 10897 "http://mystore.splunk.com/flower_store/category.screen?category_id=BALLOONS&JSESSIONID=SD1SL10FF3ADF..." "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.10) Gecko/20070223 CentOS/1.5.0.10-0.1.el4.centos Firefox/1.5.0.10" 527 3006
10.2.91.38 - - [24/Jul/2014:11:28:00] "POST /flower_store/j_signon_check HTTP/1.1" 302 309
"http://mystore.splunk.com/flower_store/enter_order_information.screen&JSESSIONID=SD1SL10FF3ADFF3" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.10) Gecko/20070223 CentOS/1.5.0.10-0.1.el4.centos Firefox/1.5.0.10" 3441 2576
192.0.1.38 - - [24/Jul/2014:11:28:15] "GET /flower_store/images/cat3.gif HTTP/1.1" 200 5024 "http://mystore.splunk.com/flower_store/item.screen?item_id=EST-21&JSESSIONID=SD1SL10FF3ADFF3" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.10) Gecko/20070223 CentOS/1.5.0.10-0.1.el4.centos Firefox/1.5.0.10" 4323 3071
below is my props.conf and transforms.conf:
[access_common]
TRANSFORMS-anonymize = session-anonymizer
[session-anonymizer]
REGEX = (?m)^(.)JSESSIONID=\w{2}\d\w{2}\d{2}(\w+.)
FORMAT = $1JSESSIONID=#######$2
DEST_KEY = _raw
Kindly advice....i do not see and changes in fields after applying this configuration
This works:
| makeresults
| eval raw="189.222.1.46 - - [24/Jul/2014:11:27:00] \"GET /flower_store/product.screen?product_id=RP-SN-01 HTTP/1.1\" 200 10897 \"http://mystore.splunk.com/flower_store/category.screen?category_id=BALLOONS&JSESSIONID=SD1SL10FF3ADFF3\" \"Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.10) Gecko/20070223 CentOS/1.5.0.10-0.1.el4.centos Firefox/1.5.0.10\" 527 3006:::10.2.91.38 - - [24/Jul/2014:11:28:00] \"POST /flower_store/j_signon_check HTTP/1.1\" 302 309
\"http://mystore.splunk.com/flower_store/enter_order_information.screen&JSESSIONID=SD1SL10FF3ADFF3\" \"Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.10) Gecko/20070223 CentOS/1.5.0.10-0.1.el4.centos Firefox/1.5.0.10\" 3441 2576:::192.0.1.38 - - [24/Jul/2014:11:28:15] \"GET /flower_store/images/cat3.gif HTTP/1.1\" 200 5024 \"http://mystore.splunk.com/flower_store/item.screen?item_id=EST-21&JSESSIONID=SD1SL10FF3ADFF3\" \"Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.10) Gecko/20070223 CentOS/1.5.0.10-0.1.el4.centos Firefox/1.5.0.10\" 4323 3071"
| makemv delim=":::" raw
| mvexpand raw
| rename raw AS _raw
| rex mode=sed "s/JSESSIONID=\w{2}\d\w{2}\d{2}(\w+.)/JSESSIONID=#######\1/"
So you can use this:
SEDCMD-session-anonymizer = s/JSESSIONID=\w{2}\d\w{2}\d{2}(\w+.)/JSESSIONID=#######\1/
All you need in this case is the following line in your props.conf
file (no transforms.conf
config needed) for the sourcetype:
SEDCMD-hidesessionid = s/JSESSIONID=\w{2}\d\w{2}\d{2}/JSESSIONID=#######/
Hi @Venkat_16,
Please use below configuration in transforms.conf on Heavy forwarder.
[session-anonymizer]
REGEX = (?m)^(.*)JSESSIONID=.*((?=\"\s\").*)$
FORMAT = $1JSESSIONID=#######$2
DEST_KEY = _raw
After changing above configuration please reload configuration using https://<HF FQDN>:8000/debug/refresh
OR restart splunk on heavy forwarder.
I hope this helps.
Thanks,
Harshil
Thanks for your answer @harsmarvania57. But it didnt work 😞
[access_common]
TRANSFORMS-anonymize = session-anonymizer
[session-anonymizer]
REGEX = (?m)^(.)JSESSIONID=\w{2}\d\w{2}\d{2}(\w+.)
FORMAT = $1JSESSIONID=#######$2
DEST_KEY = _raw
i restarted heavy forwarder also....logs are getting indexed with out any masking
Looks like you are still using your old configuration, please use configuration which I have provided.
sorry it was copy paste error:
[session-anonymizer]
REGEX = (?m)^(.)JSESSIONID=.((?=\"\s\").*)$
FORMAT = $1JSESSIONID=#######$2
DEST_KEY = _raw
is the one am using
Still this configuration is wrong, you can see my regex has (?m)^(.*)
and configuration which you pasted starts with (?m)^(.)
Thanks alot for helping patiently but still it didnt work:
[access_common]
TRANSFORMS-anonymize = session-anonymizer
[session-anonymizer]
REGEX = (?m)^(.)JSESSIONID=.((?=\"\s\").*)$
FORMAT = $1JSESSIONID=#######$2
DEST_KEY = _raw
Please refer this https://regex101.com/r/CftIqK/1, regex is working perfectly fine. Can you please provide your configuration in Code Sample
format because your REGEX is still wrong, you can see 101010 button when you type comment/answer please use that and paste your transforms.conf configuration.