Hi all,
We're running 6.2 with Splunk Add-on for Cisco ESA (https://apps.splunk.com/app/1761/) installed.
Can't use transaction to co-relate events using simple commands like this (this commands works in Splunk training lab exercise :):
sourcetype=cisco_esa | transaction mid, icid, dcid
Output events don't get grouped in single event. Instead, individual lines are returned.
If I disable the app, only MID and ICID fields are defined.
Would anyone please help? Sorry for the newbie question.
Thanks and regards,
/ST Wong
For people who are looking to create a single line event from Cisco ESA data here is a search I have found to work.
If you want to collect the data into a summary index add | collect index=email_summary
to the end of the search changing the index name.
sourcetype="cisco:esa:textmail"
| eventstats values(src) AS cs_ip BY icid
| eventstats values(dest) AS ss_ip BY dcid
| eval recipient_count=mvcount(recipient)
| stats values(internal_message_id) AS tmpMID
values(icid) AS icid
values(sender) AS sender
values(recipient) AS recipient
values(message_size) AS message_size
values(av_verdict) as av_verdict
values(response) AS response
values(message_subject) as message_subject
values(cs_ip) AS cs_ip
values(ss_ip) AS ss_ip
values(dcid) AS dcid BY internal_message_id
| eval recipient_count=mvcount(recipient)
| eval mid=tmpMID
| mvexpand mid
| eventstats values(tmpMID) AS tmp BY mid
| eval t=mvjoin(tmp, " ")
| rex field=sender "@(?.*)"
| stats values(sender) AS sender
values(sender_domain) AS sender_domain
values(recipient) AS recipient
values(message_subject) as subject
values(av_verdict) as av_verdict
max(message_size) AS message_size
max(recipient_count) AS recipient_count
values(internal_message_id) AS internal_message_id
values(dcid) AS dcid
values(response) AS response
values(cs_ip) AS cs_ip
values(ss_ip) AS ss_ip BY icid
| search icid!=0
Do you have to use transaction? For instance, are you able to use stats command to do a similar thing?
sourcetype=cisco_esa | stats count min(_time) as begin max(_time) as end by mid, icid, dcid | eval duration=end-begin
If you want other fields persisted through that stats, you have to add them into the line as multivalue fields (i.e. "values(myField) as myField" or "list(myField2) as myField2"). You also have to choose which time field you want to reassign back to _time if you want something plotted on a timechart (i.e. "eval _time=end" for upper end reporting).
Same problem.. I installed this add-on on my test box.. and I was hoping that now all the fields would appear to query such as MID, ICID etc, and then use transaction command to query and group an entire email transaction just like Ironport does it when you perform message tracking on the ironport itself and it groups all the events for a single email transaction nicely. This is proving to be very difficult. I even tried using the regex's that were included in this app for example for MID they have regex as MID (?\d+), I tried rex field=_raw "MID\s(?\d+)" in a query and that doesn't even work right. So.. frustrated.
MID is extracted as internal_message_id - you should use this field when grouping logs back together via transaction or stats.
Please note that the issue of having to group logs for a given mail back together stems from the fact that the Ironport ESA add-on does handle this when Splunk is configured to receive text mail logs via rsyslog.
This is a non-issue when Splunk is configured to pick up text mail logs via a monitored input (i.e. Ironport is configured to scp/ftp logs to a monitored location), as all logs for a given mail are already grouped together.
Although the field extractions defined in the Ironport ESA add-on do work correctly in both ingestion scenarios, an unfortunate side effect of ingesting logs via rsyslog is that the Email CIM datamodel does not get populated correctly.
I consider this to be a bug with the add-on and have filed a support case to have the documentation updated to note the issue.
Hi,
no idea what's wrong, but I'd be happy to try to help... can you send me an anonymized data sample?