Hello,
Working on a threatq list which takes more than 1min to be generated, I was always looping in splunk with :
status="threat list download failed after multiple retries"
And I have discovered that in the Intelligence Download Settings
, the field Timeout
is a lie, as it is not used for real in the code behind.
In the script /opt/splunk/etc/apps/SA-ThreatIntelligence/bin/threatlist.py
the value is set to a variable timeout
self.DEFAULT_TIMEOUT_INTERVAL = 30
(......)
IntegerField("timeout", "Timeout interval", "Time before regarding a download attempt as failed, in seconds. [Defaults to {0}]".format(self.DEFAULT_TIMEOUT_INTERVAL), required_on_create=True, required_on_edit=True),
But after the call of /opt/splunk/etc/apps/SA-Utils/lib/SolnCommon/protocols.py
we have :
_timeout = 30 # The timeout for queries conducted by this handler.
(......)
def set_options(self, *args, **kwargs):
valid_keys = ['app', 'debug', 'owner', 'proxy_port',
'proxy_server', 'proxy_user', 'proxy_password',
'site_user', 'site_password', 'user_agent']
(......)
try:
response = urllib2.urlopen(request, timeout=self._timeout)
So at the end the http timeout is always to 30s max whatever you will set
¯\(ツ)/¯
I post my crap and dirty solution in case...
Edit /opt/splunk/etc/apps/SA-ThreatIntelligence/bin/threatlist.py
look for the part :
handler = handler_cls(self._logger, self._input_config.session_key, **handler_args)
temp_checkpoint_filehandle = None
and add just after it
handler._timeout = handler_args.get('timeout')
The bug is still there.
As the code was a bit reworked now the solution is to :
Edit /opt/splunk/etc/apps/SA-ThreatIntelligence/bin/threatlist.py
and add at line 497
handler._timeout = handler_args.get('timeout')