Splunk Enterprise Security

How is the index=threat_activity filled up with data in splunk Enterprise Security (ES)? How can I add an additional field to it?

btiggemann
Path Finder

We have got squid proxy logs that are compared with the threat lists in splunk ES.
It works fine, but on the list on splunk ES Advanced Threat - Threatlist Activity - Threat Activity Details we only see ip addresses in the dest field.

alt text

In the log events of squid I also have the URL, which is much more human readable.
What I want is to add the field uri_host also to my data in the index=threat_activity.

It looks like the index is filled by the a saved search: Threat - Source And Destination Matches - Threat Gen
The data looks like:

11/27/2015 14:15:00 +0100, search_name="Threat - Source And Destination Matches - Threat Gen", search_now=1448630100.000, info_min_time=1448622000.000, info_max_time=1448630100.000, info_search_time=1448630114.038, dest="xxx.xxx.xx.xxx", orig_sourcetype="cisco:asa", src="yyy.yyy.yyy.yyy", threat_collection=ip_intel, threat_collection_key="emerging_threats_ip_blocklist|43.229.52.0/22", threat_key=emerging_threats_ip_blocklist, threat_match_field=src, threat_match_value="43.229.53.53"

The search looks like this:

| src_dest_tstats("allowed") | truncate_domain_dedup(src) | truncate_domain_dedup(dest) | threatintel_multilookup(src) | threatintel_multilookup(dest) | search threat_collection_key=* | fields - count | zipexpand_threat_matches | fields sourcetype,src,dest,threat*

I tried to add just | fields sourcetype,src,dest, uri_host, threat* but this is not working.

Does anybody have a description of this macros? Or where can I find them to adjust them?

1 Solution

ssuresh
Explorer

Hello,

You need to understand how this search works. This correlation rule is running against all the data source and matching the fields against the Source types. In squid proxy sourcetype, extract the field called dest for the URL. then you threat list activity will match the dest filed to URL. Hope this should work.

View solution in original post

0 Karma

saravanan90
Contributor

Paste the query in search box and press Ctrl+shift+E. You will get to know the full query, modify the query based on your requirement.
"Threat - Source And Destination Matches - Threat Gen" check logs from datamodels of Network_Traffic, Web & IDS.

0 Karma

renjujacob88
Path Finder

If you want to capture the domain field values in the threat activity dashboard, you need toa create a saved search ( say for example "Threat - URL squid Matches - Threat Gen".

It would be good if you have a datamodel for squid or you can go with normal index command. Please find the query below.

The query will look for the squid domains, followed by comparing with the lookup to see if there's a hit.

| tstats prestats=true local=false values(sourcetype) as sourcetype,values(squid.src),values(squid.dest) from datamodel=squid by squid.domain | eval url='squid.domain' | eval threat_match_field="squid.domain" | eval url=if(isnull(url),'squid.domain',url) | eval threat_match_field=if(isnull(threat_match_field),"url",threat_match_field) | stats values(sourcetype) as sourcetype,values(squid.src) as src,values(squid.dest) as dest by url,threat_match_field | lookup update=true ip_intel domain as url OUTPUTNEW

Hope this works

0 Karma

ssuresh
Explorer

Hello,

You need to understand how this search works. This correlation rule is running against all the data source and matching the fields against the Source types. In squid proxy sourcetype, extract the field called dest for the URL. then you threat list activity will match the dest filed to URL. Hope this should work.

0 Karma

btiggemann
Path Finder

Yes, that's it. I already fixed it last year.

0 Karma

esix_splunk
Splunk Employee
Splunk Employee

You can find the macro in the GUI under Settings -> Advanced Search -> Search Macros. You can dissect them from there.

0 Karma

renjujacob88
Path Finder

If you want to capture the domain field values in the threat activity dashboard, you need to write a search driven lookup ( say for example "Threat - URL squid Matches - Threat Gen".

It would be good if you have a datamodel for squid or you can go with normal index command. Please find the query below.

The query will look for the squid domains, followed by comparing with the lookup to see if hit.

| tstats prestats=true local=false values(sourcetype) as sourcetype,values(squid.src),values(squid.dest) from datamodel=squid by squid.domain | eval url='squid.domain' | eval threat_match_field="squid.domain" | eval url=if(isnull(url),'squid.domain',url) | eval threat_match_field=if(isnull(threat_match_field),"url",threat_match_field) | stats values(sourcetype) as sourcetype,values(squid.src) as src,values(squid.dest) as dest by url,threat_match_field | lookup update=true ip_intel domain as url OUTPUTNEW

Hope this works

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...