Splunk Search

Importing a CSV file as a threat intelligence lookup list, how can I test that this file is being parsed against my data?

santorof
Path Finder

I just completed importing a CSV file as a threat intelligence lookup list. I followed Splunk documentation (6.2) and one of the details for threats by IP address is that IPs and descriptions are required. The file I was given only had IP addresses, so I changed under parsing options to only look at the first column for IPs with nothing defining descriptions. My question is, how can I test that this threat intelligence document is being parsed against my data and would files only with IPs still function without a description field?

0 Karma

sheamus69
Communicator

This is how I would do it (assuming that an existing threat intel field exists, called the-threat-intel-list.csv):

  1. Upload "Update.csv" into search app
  2. Change permissions on Update to App level
  3. Once App viewable, you will be able to delete Update once done
  4. Craft Splunk search to load in previous threat list, then append on the Update file (assuming updates go on the end) - this will NOT make any permanent changes at this point, merely display the output : | inputlookup the-threat-list.csv | inputlookup append=t Update.csv 4a. Dedup on certain columns, if desired (quoted only if special characters, such as periods or spaces, in field names)
  5. Inspect search results to ensure columns match up properly (generally not a problem, in Splunk, beware if updating a CSV from python)
  6. When confident list is as desired, write out to file: | inputlookup the-threat-list.csv | inputlookup append=t Update.csv | dedup [field to dedup if desired] | outputlookup the-threat-list.csv
  7. Inspect file was written as intended: | inputlookup the-threat-list.csv
0 Karma

rashid47010
Communicator

HI Ninjas,

I upload custom threat intelligence. file_name, description,url
the threat activity detected notable is triggering successfully against custom threat intelligence that I have upload previously.
Now I want to remove those threat intelligence feeds. that threat activity detected should not trigger against that custom feeds.

0 Karma

santorof
Path Finder

I did the above but have more questions. The threat list(the-threat-list) does show all the IP's I want to filter out by. So my logic is comparing these ip's in the IP field to a field called IP in my sourcetype firewall for example(checking for hits manually). Would I use the diff command to compare the IP field from the-threat-list to my sourcetype firewall field=IP for any matches? Just not sure how to go about it. I also tried to do the threatlookup command with files that are being pulled down from a url but got no results.

0 Karma

woodcock
Esteemed Legend

Have you seen the GetWatchList app?

https://splunkbase.splunk.com/app/635/

0 Karma

santorof
Path Finder

I looked at it but was not looking to add any apps that weren't built by splunk or splunk supported. Would I be able to install this app on my local machine and extract the information I would need to compare my data against specific threat lists?

0 Karma

woodcock
Esteemed Legend

This anti-open-source bias is silly and severely limiting, but it is your system.

0 Karma

woodcock
Esteemed Legend

Yes, that is the whole point; you give it the URL where it can get the threatlist and it does the rest.

0 Karma

santorof
Path Finder

Sorry I did not make it clear. I have a local copy of splunk with no data I use as my test environment. The real data and infrastructure is what I don't want to install the app on. If it was xml or dashboard logic I could extract it from my local version but because its a command I dont believe it will help me with the real data.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...