Getting Data In

Can you help props.conf to break the event and mask the data?

vrmandadi
Builder

I have the below sample event

{"timestamp": 1553559218742, "message": "(0133108c-4f5c-11e9-82ca-1b5bad0211a1) Method request path: {serverId=s-2f9b4670b10148058, username=mike}", "ingestionTime": 1553559219615}, {"timestamp": 1553559218742, "message": "(0133108c-4f5c-11e9-82ca-1b5bad0211a1) Method request query string: {}", "ingestionTime": 1553559219615},{"timestamp": 1553559218743, "message": "(0133108c-4f5c-11e9-82ca-1b5bad0211a1) Endpoint request body after transformations: {\n  \"username\": \"mike\",\n  \"password\": \"mike\",\n  \"serverId\": \"s-2f9b4670b10148058\"\n}\n", "ingestionTime": 1553559219615}, {"timestamp": 1553559218743, "message": "(0133108c-4f5c-11e9-82ca-1b5bad0211a1) Sending request to https://lambda.us-east-1.amazonaws.com/2015-03-31/functions/arn:aws:lambda:us-east-1:114086737623:fu..., "ingestionTime": 1553559219615}
  1. I am trying to break the event after the ingestion time, like below

{"timestamp": 1553559218742, "message": "(0133108c-4f5c-11e9-82ca-1b5bad0211a1) Method request path: {serverId=s-2f9b4670b10148058, username=mike}", "ingestionTime": 1553559219615}
2. In some of the events, the user name has a password in it, and I want to mask that one .Below is the sample event:

{"timestamp": 1553559218743, "message": "(0133108c-4f5c-11e9-82ca-1b5bad0211a1) Endpoint request body after transformations: {\n \"username\": \"mike\",\n **\"password\": \"mike\"**,\n \"serverId\": \"s-2f9b4670b10148058\"\n}\n", "ingestionTime": 1553559219615}

Thanks In advance !

0 Karma
1 Solution

somesoni2
Revered Legend

Give this a try

[YourSourcetype]
SHOULD_LINEMERGE=false
LINE_BREAKER=(\{\"timestamp\")
TIME_PREFIX=^\:\s+
TIME_FORMAT = %s%N
# line breaker actually remove timestamp, so putting it back
SEDCMD-header=s/^/{"timemstamp"/
#masking password
SEDCMD-passwordmask=s/(\"password[^\"]+\"[^\"]+)(\"[^\"]+\")/\1"##MASKED##"/

View solution in original post

somesoni2
Revered Legend

Give this a try

[YourSourcetype]
SHOULD_LINEMERGE=false
LINE_BREAKER=(\{\"timestamp\")
TIME_PREFIX=^\:\s+
TIME_FORMAT = %s%N
# line breaker actually remove timestamp, so putting it back
SEDCMD-header=s/^/{"timemstamp"/
#masking password
SEDCMD-passwordmask=s/(\"password[^\"]+\"[^\"]+)(\"[^\"]+\")/\1"##MASKED##"/
Get Updates on the Splunk Community!

Stay Connected: Your Guide to May Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars this month. This ...

They're back! Join the SplunkTrust and MVP at .conf24

With our highly anticipated annual conference, .conf, comes the fez-wearers you can trust! The SplunkTrust, as ...

Enterprise Security Content Update (ESCU) | New Releases

Last month, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...