Splunk Search

Filter event data using conditional regex

anshul0915
Explorer

HI All,

Below is my raw event data .
{"FormatVersion":"1.1","StartTime":"2017-09-22T01:11:38.565Z","EndTime":"2017-09-22T01:11:39.468Z","EventType":"Login","Result":"Success","UserId":"dmorand","TerminalId":"172.16.3.85","SessionId":"RCIcAM1DxUYmG7WMDMkEuQXyGTpOqcBMtyrGOPpFUPU=","LoginUri":"/login-auth/saml","EventSource":"Platform","ServerHostname":"fe02.hbc.stage.us-west-2.orionsaas"}

I want event indexing like below condition.
1. IF "EventType":"Login" and "LoginUri":"/login-auth/saml" the index those event. means we need to discard those event in which event type = login and login uri != /login-auth/saml
2. if "EventType":"Login" and "LoginUri" is not present then index those event
3. If in logs event type not equal to login then index those events also

please help for making this regex .

0 Karma
1 Solution

jkat54
SplunkTrust
SplunkTrust

Props.conf

 [sourcetypeName]
 KV_MODE=json
 INDEXED_EXTRACTIONS=true
 TRANSFORMS-toNull=toNull

Transforms.conf

 [toNull]
 REGEX= .*^((?!EventType":"Login".*LoginUri":"\/login-auth\/saml).*)
 DEST_KEY=queue
 FORMAT=nullQueue

View solution in original post

0 Karma

jkat54
SplunkTrust
SplunkTrust

Props.conf

 [sourcetypeName]
 KV_MODE=json
 INDEXED_EXTRACTIONS=true
 TRANSFORMS-toNull=toNull

Transforms.conf

 [toNull]
 REGEX= .*^((?!EventType":"Login".*LoginUri":"\/login-auth\/saml).*)
 DEST_KEY=queue
 FORMAT=nullQueue
0 Karma

jkat54
SplunkTrust
SplunkTrust

Needs to be on the forwarder(s) and indexer(s). Will only apply to new data that is indexed after the settings are in placeZ

0 Karma

anshul0915
Explorer

Why on both places configuration required . On only indexer will it not work??

0 Karma

jkat54
SplunkTrust
SplunkTrust

INDEXED_EXTRACTIONS has to be on the forwarder. The queue routing happens on first full Splunk instance (heavy forwarder or indexer).

0 Karma

Sukisen1981
Champion

what does the last point 'if "EventType":"Login" not equals to Login index those event' mean?

0 Karma

anshul0915
Explorer

If in logs event type not equal to login then index those events also

0 Karma

Sukisen1981
Champion

one last question - do you want to extract them in the conf files before getting indexed or you want to write a post search regex to extract ONLY these events using regex for this particular use case?

0 Karma

anshul0915
Explorer

Want in conf file before getting indexed

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...