Getting Data In

Can you help props.conf to break the event and mask the data?

vrmandadi
Builder

I have the below sample event

{"timestamp": 1553559218742, "message": "(0133108c-4f5c-11e9-82ca-1b5bad0211a1) Method request path: {serverId=s-2f9b4670b10148058, username=mike}", "ingestionTime": 1553559219615}, {"timestamp": 1553559218742, "message": "(0133108c-4f5c-11e9-82ca-1b5bad0211a1) Method request query string: {}", "ingestionTime": 1553559219615},{"timestamp": 1553559218743, "message": "(0133108c-4f5c-11e9-82ca-1b5bad0211a1) Endpoint request body after transformations: {\n  \"username\": \"mike\",\n  \"password\": \"mike\",\n  \"serverId\": \"s-2f9b4670b10148058\"\n}\n", "ingestionTime": 1553559219615}, {"timestamp": 1553559218743, "message": "(0133108c-4f5c-11e9-82ca-1b5bad0211a1) Sending request to https://lambda.us-east-1.amazonaws.com/2015-03-31/functions/arn:aws:lambda:us-east-1:114086737623:fu..., "ingestionTime": 1553559219615}
  1. I am trying to break the event after the ingestion time, like below

{"timestamp": 1553559218742, "message": "(0133108c-4f5c-11e9-82ca-1b5bad0211a1) Method request path: {serverId=s-2f9b4670b10148058, username=mike}", "ingestionTime": 1553559219615}
2. In some of the events, the user name has a password in it, and I want to mask that one .Below is the sample event:

{"timestamp": 1553559218743, "message": "(0133108c-4f5c-11e9-82ca-1b5bad0211a1) Endpoint request body after transformations: {\n \"username\": \"mike\",\n **\"password\": \"mike\"**,\n \"serverId\": \"s-2f9b4670b10148058\"\n}\n", "ingestionTime": 1553559219615}

Thanks In advance !

0 Karma
1 Solution

somesoni2
Revered Legend

Give this a try

[YourSourcetype]
SHOULD_LINEMERGE=false
LINE_BREAKER=(\{\"timestamp\")
TIME_PREFIX=^\:\s+
TIME_FORMAT = %s%N
# line breaker actually remove timestamp, so putting it back
SEDCMD-header=s/^/{"timemstamp"/
#masking password
SEDCMD-passwordmask=s/(\"password[^\"]+\"[^\"]+)(\"[^\"]+\")/\1"##MASKED##"/

View solution in original post

somesoni2
Revered Legend

Give this a try

[YourSourcetype]
SHOULD_LINEMERGE=false
LINE_BREAKER=(\{\"timestamp\")
TIME_PREFIX=^\:\s+
TIME_FORMAT = %s%N
# line breaker actually remove timestamp, so putting it back
SEDCMD-header=s/^/{"timemstamp"/
#masking password
SEDCMD-passwordmask=s/(\"password[^\"]+\"[^\"]+)(\"[^\"]+\")/\1"##MASKED##"/
Get Updates on the Splunk Community!

Join Us for Splunk University and Get Your Bootcamp Game On!

If you know, you know! Splunk University is the vibe this summer so register today for bootcamps galore ...

.conf24 | Learning Tracks for Security, Observability, Platform, and Developers!

.conf24 is taking place at The Venetian in Las Vegas from June 11 - 14. Continue reading to learn about the ...

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...