Where does Cisco eStreamer for Splunk app need to be installed in a distributed search environment?

[lbur]

We have a distributed search environment with 3 Search Heads and 6 Indexers. All of them are linux. Does anyone know where the eStreamer app needs to be installed.

Does it get installed on both the Indexer, so that the app will obtain the logs from Defense Center and put them into the eStreamer index, and on the Search Head, so that users have access to the dashboards? Or is it possible this app only works on a Search Head / Indexer combination, and not in a distributed search environment?

Thank you.

[douglashurd]

A new Splunk Firepower solution is now available if you are using Firepower version 6.x. You can download the new eStreamer eNcore for Splunk and the separately installable dashboard from the two links below:

eStreamer eNcore
https://splunkbase.splunk.com/app/3662/

eNcore Dashboard
https://splunkbase.splunk.com/app/3663/

It is free to use and well documented but if you would like to purchase a TAC Support service so that you can obtain installation and configuration assistance and troubleshooting you can order the software from Cisco (support obligatory with this purchase). The Product Identifier is: FP-SPLUNK-SW-K9.

Regardless of whether you take up the support option or not, updated versions will be made available to all free of charge and posted on Splunkbase as well as Cisco Downloads.

[hopnscotch]

eStreamer Installation Tips for Distributed Splunk

I have a distributed environment with an Enterprise Security (ES) search head (SH), an ad-hoc search head and separate indexers. It wasn't too difficult to get eStreamer going but there are some confusing points that aren't well documented, so I'll share my experience and tips.

For this example, let say we'll will be running the collection script from an indexer, and we'll want to access the app on the ad-hoc SH as well as the ES SH.

First off, the estreamer for Splunk app contains a perl script which is how it collects data. I read that it needed to be installed on an indexer, but now that I see how it works, I think you could install it anywhere (SH or heavy forwarder), as long as you are forwarding your SH events to your indexers.

At the end of this, you will have installed 'eStreamer for Splunk' on the indexers, ad-hoc SH and ES SH (you only need it on one indexer, but this will ensure the estramer index is created on both), and 'Splunk Add-on for Cisco FireSIGHT' on the ES SH.

Indexer:

You'll need to get the perl dependencies installed (in this case on the indexer), and there is one that CAN cause you a headache if you use CPAN. Use the OS package manager for the SSL packages: yum install openssl-devel perl-Net-SSLeay

Install these first themn the rest of the perl modules should install fine with CPAN.

Install the eStreamer for Splunk app. To test, run the bin/estreamer_client.pl and make sure there are no errors (you should see usage instructions when it runs).

On the indexer, go to the eStreamer app from the web interface to set it up. (There's also a 'config_nogui.sh' if you have disabled web or are otherwise inclined.) It requires the IP, port and full cert path (this should be provided to you by the sourcefire admin). Once these values are set, uncheck the "disabled" check box and click 'Save'.

Hopefully at this point you start to see events on the estreamer status dashboard (we're still on the indexer at thsi point, but this is just for verification). The logs are collected in the eStreamer app directory under log/.

Ad-hoc SH:

Now, you obviously don't want people to access the indexer for the app, so you go to the ad-hoc SH and install the eStreamer app. I don't think you need to install any of the perl dependencies here, as we will not be using those scripts on the SH. When you access the eStreamer app you'll see that it wants you to setup the collection script in order to see the app.

To get around this, create the app.conf file under local/ and add these lines:

[install]
state = enabled
is_configured = 1

It should now bypass the config screen here on your SH.

Now you will see that on the eStreamer Status dashboard the client says it's in an error state. This is because you are not running the client on the SH and this dashboard is picking up the status of all of this scripts (client_check.py) output in your environment. To get around this, just change the dashboard panel to only include the host that you have your collection script running on (in this case, your indexer). (Add 'host=' just before the '|eval ..' in the search string.) More thoroughly, you should disable the script input $SPLUNK_HOME/etc/apps/eStreamer/bin/client_check.py on the SH and any other place you are using the app but not running the client.

ES SH:

Now, if you have Enterprise Security (ES) running, there's another app that helps with all of the eventtyping and tagging needed for ES to bring these events in, Splunk Add-on for Cisco FireSIGHT. You need to install this on the ES SH. The issue now is that the sourcetypes may not be in sync between this app and the eStreamer app. It depends on the version of Sourcefire you have. In my case it required the FireSIGHT app to see cisco:sourcefire as the sourcetype, so I changed it (local/inputs.conf on the indexer). But then the eStreamer app stopped working because the sourcetype in that app is set to look for sourcetype=estreamer. So again, it depends on your version, but for me I moved over all of the conf files that contained sourcetype=estreamer to the local directory, and changed it to my new sourcetype. I did it on both the indexer and the SH but I believe these are only search time configs so you should only need to do this on the SH (anywhere you run the eStreamer app).

Then everything worked.

I hope this helps others get by some of the obstacles for this setup. It's not the most straightforward process in a distributed environment, nor when the collection script is tied to the app this way this is.

[BansodeSantosh]

Thanks @hopnscotch for the detailed explanation. But I have a doubt that in cluster environment all peer should have the same configuration file.
If you are pushing the application from Master node then it will apply on all nodes.

So how this is feasible to install eStreamer supplication only on one indexer. I am totally confused about the architecture....can you please help me to clear it.

Thanks @hopnscotch.

[pil321]

You may want to take a look at this:

http://answers.splunk.com/answers/179891/why-are-no-results-displaying-in-cisco-estreamer-f.html