Hi all,
I am looking at using the Proofpoint Protection Server TA for Splunk, and having set it up, I am having some difficulty with field extraction in that the app is not doing what I expect.
2016-08-14T08:00:01.774397+01:00 dc1-pro-prp03 filter_instance1[7090]: rprt s=24sr7mgd5h mod=session cmd=disconnect module= rule= action= helo=<redacted_host> msgs=1 rcpts=1 routes=allow_relay,default_inbound,internalnet,outbound duration=0.264 elapsed=0.547
I was hoping that Splunk would extract s=24sr7mgd5h
as a field named s and a value of 24sr7mgd5h
. This would then allow me to run transaction commands and get useful session data from the devices.
I see that this answer https://answers.splunk.com/answers/86461/search-proofpoint-logs.html shows a Splunk user using the s field in their transaction. I am wondering if they have done some Splunk magic to make this happen.
I have found that adding the following will give me what I need, but I am hoping to avoid having to have this for all searches:
| extract pairdelim=" ",kvdelim="=\,"
If anyone can help me with their Splunk Ninja skills, I would be very much appreciative!
In general Splunk behavior, s field will be extracted. When an app or Add-on would like to have their own custom field extractions in order to avoid unexpected fields populated by Splunk's auto field extraction. In such case, they use "KV_MODE = none" in props.conf. You can change it to KV_MODE = auto and see how it works.
However, for more detail of the reason why the developer of the add-on disabled or potential issue, I recommend to contact author of the add-on.
Proofpoint now has a beta app that will allow you report on and visualze your Proofpoint Protection Server and TAP data! Check out the new app here:
https://splunkbase.splunk.com/app/3727/#/details
Be sure to follow the instructions listed in the details to get all the needed TA's etc that the app needs to work correctly.
The latest TA (1.06) has this functionality now. Search using: message_session_id=(your session ID you want results from)
This is a field extraction for session ID and will automatically group all of the logs into one transaction result from your query if you format the query like this example:
message_session_id=2b1wdr84m3 | transaction maxpause=3s
Hello world,
Due to naming collisions, KV_MODE was turned off. It is best to create your own field extractions as needed for now.
For the scenario you are working with now, you can accomplish it like this:
Go To:
Fields » Field extractions
Click New to create a new Field Extraction
Fill in these details:
Destination app: TA_pps
Name: sid
Apply to: sourcetype
Named: pps_filter_log
Type: Inline
Extraction/Transform: \s+s=(?P[^ ]+)
***** you need to add in < sid >(without any spaces) between P and [ above.... It is not allowing me to past this in without changing the formatting. *******
You may need to adjust the permissions depending on your setup. Make it read/write and available to all apps if you are not sure.
Run a search like this to see all the processing details for a session ID:
sid="254qq8142p"|transaction maxpause=2s
Hi @cumbers
I noticed you upvoted the answer by @Masa. If his answer solved your issue, don't forget to resolve the post by clicking "Accept" directly below his answer. Thanks!
Patrick
Yes indeed, apologies. I was on a dodgy cell connection, and was unable to click the button. All done now 🙂
In general Splunk behavior, s field will be extracted. When an app or Add-on would like to have their own custom field extractions in order to avoid unexpected fields populated by Splunk's auto field extraction. In such case, they use "KV_MODE = none" in props.conf. You can change it to KV_MODE = auto and see how it works.
However, for more detail of the reason why the developer of the add-on disabled or potential issue, I recommend to contact author of the add-on.
Thank you! I am going to ask the author (ProofPoint) why they did this, as I can't see a good reason. I'll post back here once I have an answer!