All Apps and Add-ons

Is there a timeout in the TA-dmarc add-on for Splunk that would cause the process to die before its finished processing the messages?

swharper79
Engager

We have a mailbox with a large number of emails we're attempting to ingest into Splunk (over 150,000). Before any data is ingested the process is timing out (connection reset by peer). Is there a timeout in the TA that would cause the process to die before its finished processing the messages?

2018-08-21 14:16:40,062 ERROR pid=21172 tid=MainThread file=base_modinput.py:log_error:307 | Get error when collecting events.
Traceback (most recent call last):
  File "/opt/splunk/etc/apps/TA-dmarc/bin/ta_dmarc/modinput_wrapper/base_modinput.py", line 127, in stream_events
    self.collect_events(ew)
  File "/opt/splunk/etc/apps/TA-dmarc/bin/dmarc_imap.py", line 88, in collect_events
    input_module.collect_events(self, ew)
  File "/opt/splunk/etc/apps/TA-dmarc/bin/input_module_dmarc_imap.py", line 48, in collect_events
    filelist = i2d.process_incoming()
  File "/opt/splunk/etc/apps/TA-dmarc/bin/dmarc/imap2dir.py", line 218, in process_incoming
    response = self.get_dmarc_message_bodies(new_messages)
  File "/opt/splunk/etc/apps/TA-dmarc/bin/dmarc/imap2dir.py", line 88, in get_dmarc_message_bodies
    response = self.server.fetch(messages, ['RFC822'])
  File "/opt/splunk/etc/apps/TA-dmarc/bin/imapclient/imapclient.py", line 971, in fetch
    tag = self._imap._command(*args)
  File "/opt/splunk/lib/python2.7/imaplib.py", line 872, in _command
    raise self.abort('socket error: %s' % val)
abort: socket error: [Errno 104] Connection reset by peer
Collapse
2018-08-21 14:16:40,059 DEBUG pid=21172 tid=MainThread file=base_modinput.py:log_debug:286 | Success deleting temporary directory /tmp/tmpIxrlSr
2018-08-21 14:16:39,906 INFO pid=21172 tid=MainThread file=base_modinput.py:log_info:293 | Start processing 154245 new messages of 154245 on 10.168.16.246
2018-08-21 14:16:39,905 DEBUG pid=21172 tid=MainThread file=base_modinput.py:log_debug:286 | filter_seen_messages: uids new       set([20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55,.......
0 Karma

jorritf
Path Finder

Thanks for the report, I'll look into it when I have time somewhere next week.
Can you create an issue in the Github tracker?

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...