All Apps and Add-ons

transaction not working for Cisco ESA data

stwong
Communicator

Hi all,

We're running 6.2 with Splunk Add-on for Cisco ESA (https://apps.splunk.com/app/1761/) installed.

Can't use transaction to co-relate events using simple commands like this (this commands works in Splunk training lab exercise :):

sourcetype=cisco_esa | transaction mid, icid, dcid

Output events don't get grouped in single event. Instead, individual lines are returned.
If I disable the app, only MID and ICID fields are defined.

Would anyone please help? Sorry for the newbie question.

Thanks and regards,
/ST Wong

dperre_splunk
Splunk Employee
Splunk Employee

For people who are looking to create a single line event from Cisco ESA data here is a search I have found to work.
If you want to collect the data into a summary index add | collect index=email_summary to the end of the search changing the index name.

sourcetype="cisco:esa:textmail"
| eventstats values(src) AS cs_ip BY icid
| eventstats values(dest) AS ss_ip BY dcid
| eval recipient_count=mvcount(recipient)
| stats values(internal_message_id) AS tmpMID
values(icid) AS icid
values(sender) AS sender
values(recipient) AS recipient
values(message_size) AS message_size
values(av_verdict) as av_verdict
values(response) AS response
values(message_subject) as message_subject
values(cs_ip) AS cs_ip
values(ss_ip) AS ss_ip
values(dcid) AS dcid BY internal_message_id
| eval recipient_count=mvcount(recipient)
| eval mid=tmpMID
| mvexpand mid
| eventstats values(tmpMID) AS tmp BY mid
| eval t=mvjoin(tmp, " ")
| rex field=sender "@(?.*)"
| stats values(sender) AS sender
values(sender_domain) AS sender_domain
values(recipient) AS recipient
values(message_subject) as subject
values(av_verdict) as av_verdict
max(message_size) AS message_size
max(recipient_count) AS recipient_count
values(internal_message_id) AS internal_message_id
values(dcid) AS dcid
values(response) AS response
values(cs_ip) AS cs_ip
values(ss_ip) AS ss_ip BY icid
| search icid!=0

0 Karma

jeremiahc4
Builder

Do you have to use transaction? For instance, are you able to use stats command to do a similar thing?

sourcetype=cisco_esa | stats count min(_time) as begin max(_time) as end by mid, icid, dcid | eval duration=end-begin

If you want other fields persisted through that stats, you have to add them into the line as multivalue fields (i.e. "values(myField) as myField" or "list(myField2) as myField2"). You also have to choose which time field you want to reassign back to _time if you want something plotted on a timechart (i.e. "eval _time=end" for upper end reporting).

0 Karma

mendesjo
Path Finder

Same problem.. I installed this add-on on my test box.. and I was hoping that now all the fields would appear to query such as MID, ICID etc, and then use transaction command to query and group an entire email transaction just like Ironport does it when you perform message tracking on the ironport itself and it groups all the events for a single email transaction nicely. This is proving to be very difficult. I even tried using the regex's that were included in this app for example for MID they have regex as MID (?\d+), I tried rex field=_raw "MID\s(?\d+)" in a query and that doesn't even work right. So.. frustrated.

0 Karma

S146547
Engager

MID is extracted as internal_message_id - you should use this field when grouping logs back together via transaction or stats.

Please note that the issue of having to group logs for a given mail back together stems from the fact that the Ironport ESA add-on does handle this when Splunk is configured to receive text mail logs via rsyslog.

This is a non-issue when Splunk is configured to pick up text mail logs via a monitored input (i.e. Ironport is configured to scp/ftp logs to a monitored location), as all logs for a given mail are already grouped together.

Although the field extractions defined in the Ironport ESA add-on do work correctly in both ingestion scenarios, an unfortunate side effect of ingesting logs via rsyslog is that the Email CIM datamodel does not get populated correctly.

I consider this to be a bug with the add-on and have filed a support case to have the documentation updated to note the issue.

0 Karma

jcoates_splunk
Splunk Employee
Splunk Employee

Hi,

no idea what's wrong, but I'd be happy to try to help... can you send me an anonymized data sample?

Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...