Splunk Enterprise Security

ES - Threat Intelligence - FS-ISAC Feeds

ajhsjahdpgjhapi
Engager

Attempting to ingest feeds from FS-ISAC into ES.
I can see in splunk that a file is created:
2018-06-19 17:01:28,107 INFO pid=23553 tid=MainThread file=stix_parser.py:preprocess:154 | msg="Finished parsing STIX documents" filename="/opt/splunk/etc/apps/SA-ThreatIntelligence/local/data/threat_intel/fsisac_filehash_TAXII_filehash_2018-06-19T17-01-22.135143.xml" success="0" failed="0"

ls -lah /opt/splunk/etc/apps/SA-ThreatIntelligence/local/data/threat_intel/
total 12K
drwx--x---. 2 splunk splunk 4.0K Jun 19 16:56 .
drwx--x---. 3 splunk splunk 25 Oct 17 2016 ..
-rw-------. 1 splunk splunk 483 Jun 19 16:56 fsisac_all_high_TAXII_all_high_2018-06-19T16-56-22.004935.xml
-rw-------. 1 splunk splunk 483 Jun 19 16:56 fsisac_filehash_TAXII_filehash_2018-06-19T16-56-21.863297.xml

Below are the contents of the file:
2018-06-19T21:01:06.060327+00:00

2018-06-19T21:01:06.101416+00:00

Whats strange is the file is quickly deleted and ever poll, splunk re-creates the file, then deletes it again. I never see any of the threat intelligence, I've disabled all other feeds in an attempt to get this to work and I don't see anything on the "Threat Intelligence > Threat Activity" dashboard.

I've:
1. Created multiple feeds on analysis.fsisac[dot]com
2. Created multiple Threat Intelligence Downloads in an attempt to get data from any of them (see inputs below):

I don't see any errors associated with feeds.

Status of fsisac threatintel_internal_logs:

eventtype=threatintel_internal_logs fsisac | stats count by status
status count
TAXII feed polling starting 5450
continuing 5450
retrieved_checkpoint_data 5300
Retrieved document from TAXII feed 4307
no_checkpoint_data 150
Detected updated threatlist stanzas - ALL lookup gen searches will be executed 5

inputs for fsisac:

[threatlist://fsisac]
description = FS-ISAC threat intel
index = _internal
initial_delay = 300
interval = 300
max_age = -1y
post_args = collection="Default" earliest="-1y" taxii_username="redacted" taxii_password="redacted" cert_file="redacted.crt" key_file="redacted.key"
retries = 3
retry_interval = 60
source = ModularInput:Threatlist
sourcetype = ModularInput:Threatlist
target = threatlist
timeout = 30
type = taxii
url = https://analysis.fsisac.com/
weight = 50

[threatlist://fsisac_2]
delim_regex = ,
description = FS-ISAC threat intel
ignore_regex = (^#|^\s*$)
index = _internal
initial_delay = 300
interval = 300
max_age = -1y
post_args = taxii_username="redacted" taxii_password="redacted" cert_file="redacted.crt" key_file="redacted.key"
retries = 3
retry_interval = 60
skip_header_lines = 0
source = fsisac
sourcetype = ModularInput:Threatlist
target = threatlist
timeout = 30
type = taxii
url = https://analysis.fsisac.com/taxii-discovery-service
weight = 50

[threatlist://fsisac_all_high]
delim_regex = ,
description = FS-ISAC threat intel
ignore_regex = (^#|^\s*$)
index = _internal
initial_delay = 300
interval = 300
post_args = collection="all_high" taxii_username="redacted" taxii_password="redacted" cert_file="redacted.crt" key_file="redacted.key"
retries = 3
retry_interval = 60
skip_header_lines = 0
source = fsisac_filehash
sourcetype = ModularInput:Threatlist
target = threatlist
timeout = 30
type = taxii
url = https://analysis.fsisac.com/
weight = 50

[threatlist://fsisac_filehash]
delim_regex = ,
description = FS-ISAC threat intel
disabled = 0
ignore_regex = (^#|^\s*$)
index = _internal
initial_delay = 300
interval = 300
post_args = collection="filehash" taxii_username="redacted" taxii_password="redacted" cert_file="redacted.crt" key_file="redacted.key"
retries = 3
retry_interval = 60
skip_header_lines = 0
source = fsisac_2
sourcetype = ModularInput:Threatlist
target = threatlist
timeout = 30
type = taxii
url = https://analysis.fsisac.com/
weight = 50

pappjr
Path Finder

These settings worked for me:

NOTE: Make sure you put your .crt and .key file issued by FS-ISAC in the auth folder of the app directory you create the input inside of (e.g. /opt/splunk/etc/apps/DA-ESS-ThreatIngelligence/auth). You can verify you're connecting successfully by reviewing the threat intel download logs (index=_internal sourcetype=threatintel:download)

[threatlist://fs-isac-default]
delim_regex = ,
description = FS-ISAC system.Default feed
ignore_regex = (^#|^\s*$)
interval = 43200
is_threatintel = 1
max_age = -30d
post_args = collection="system.Default" earliest="-1y" taxii_username="<your_provided_username>" taxii_password="<your_password>" cert_file="<your_cert.crt>" key_file="<your_key.key>"
retries = 3
retry_interval = 60
sinkhole = 0
skip_header_lines = 0
timeout = 30
type = taxii
url = https://analysis.fsisac.com/taxii-discovery-service
weight = 1
0 Karma

stevenbukovic
Explorer

These generally worked for me as well, but I would note that some of the stanza's are invalid if you are not on a more current version of Splunk/ES combo. I would recommend starting out without sinkhole and is_threatintel otherwise the taxii polling won't even start. I found this out by restarting the splunk service and paying attention to the error streams that show up in the startup output. Errors will look like the following:

Invalid key in stanza
[threatlist://fs-isac-default] in
/opt/splunk/etc/apps/SA-ThreatIntelligence/local/inputs.conf, line 47: is_threatintel (value: 1).
Invalid key in stanza
[threatlist://fs-isac-default] in
/opt/splunk/etc/apps/SA-ThreatIntelligence/local/inputs.conf, line 52: sinkhole (value:0).

I overlooked the original note on .crt and .key files need to be placed in the auth/ folder within the app.

I found the following conf talk from 2017 as well which may help fill in some details for folks. Though it doesn't mention ISAC data specifically could be a good primer for others.

https://conf.splunk.com/files/2017/slides/enterprise-security-biology-dissecting-the-splunk-enterpri...

0 Karma

stevenbukovic
Explorer

Any updates on this thread?

0 Karma

mad4wknds
Path Finder

"What's strange is the file is quickly deleted and ever poll, Splunk re-creates the file, then deletes it again. I never see any of the threat intelligence."

Under Threat Intelligence Management you can remove the sinkhole policy that deletes the files

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...