All Apps and Add-ons

eStreamer logs flow for a minute and then stop. Client complains of being "out of memory".

att35
Builder

Hi,

We have recently started facing some issues with the eStreamer client app on our Splunk indexer(CentOS). eStreamer starts with Splunk, makes the initial connection and logs flow for a minute or two. Then it just stops.

In Splunk, we only see client_check logs.

alt text

And following on the eStreamer process side: ( That is something new. Never saw that before. Not sure what would cause the process to go into that state)

root     44724  0.1  0.0 238572 66044 ?        S    18:06   0:01 /usr/bin/perl /opt/splunk/etc/apps/eStreamer/bin/estreamer_client.pl -d -c /opt/splunk/etc/apps/eStreamer/local/estreamer.conf -l /opt/splunk/etc/apps/eStreamer/log/estreamer.log

root     44756  0.0  0.0      0     0 ?        Z    18:06   0:01 [estreamer_clien] <defunct>

After enabling debug logging, we see the following:

SFPkcs12 : Processing /opt/splunk/etc/apps/eStreamer/bin/X.X.X.X_29.pkcs12
SFPkcs12 : Writing ./server.crt
SFPkcs12 : Writing ./server.key
May 15 18:06:45 [44724] Retrieving metadata from file ./metadata.dat
May 15 18:06:46 [44724] Migrating prior version bookmark (if necessary)
May 15 18:06:46 [44724] Starting processing for other
May 15 18:06:46 [44724] Building connection parameters for all other events
May 15 18:06:46 [44724] Connecting to X.X.X.X port 8302
May 15 18:06:46 [44756] Starting processing for rna
May 15 18:06:46 [44756] Building connection parameters for RNA
May 15 18:06:46 [44756] Connecting to X.X.X.X port 8302
May 15 18:06:46 [44724] Starting bookmark is 1494615753
May 15 18:06:46 [44724] Requesting Event Stream
May 15 18:06:46 [44724] Entering Event Loop
May 15 18:06:46 [44756] Starting bookmark is 1494615600
May 15 18:06:46 [44756] Requesting Event Stream
May 15 18:06:46 [44756] Entering Event Loop
Out of memory!
May 15 18:07:17 [44724] Waiting for more logs
May 15 18:07:47 [44724] Waiting for more logs
May 15 18:08:17 [44724] Waiting for more logs
May 15 18:08:47 [44724] Waiting for more logs
May 15 18:09:17 [44724] Waiting for more logs

Not sure why it would suddenly start complaining for "out of memory". Nothing has changed for Server or Splunk Enterprise. Also, this is a physical server with 500 G memory.

Is this some problem on the Splunk server or an issue with the SF Defense center?

Thanks,

~ Abhi

0 Karma

douglashurd
Builder

A new Splunk Firepower solution is now available if you are using Firepower version 6.x. You can download the new eStreamer eNcore for Splunk and the separately installable dashboard from the two links below:

eStreamer eNcore
https://splunkbase.splunk.com/app/3662/

eNcore Dashboard
https://splunkbase.splunk.com/app/3663/

It is free to use and well documented but if you would like to purchase a TAC Support service so that you can obtain installation and configuration assistance and troubleshooting you can order the software from Cisco (support obligatory with this purchase). The Product Identifier is: FP-SPLUNK-SW-K9.

Regardless of whether you take up the support option or not, updated versions will be made available to all free of charge and posted on Splunkbase as well as Cisco Downloads.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...