Hi,
I am currently processing syslog events, using the hfw. This feed is pretty busy - hundreds of files, and I'm being asked to forward all the data to Hadoop. How will this affect the forwarder? These events are critical within Splunk, and I don't want any delay to be added. I'm not sure of the need for real-time here, so my suggestion is going to be Hadoop Connect.
Hi,
indexing the data using Splunk Enterprise Core in combination with a HFW should not be influenced if you export the data off the indexer using Hadoop Connect.
Remember, Hadoop Connect will run a search and then export the result/data to Hadoop.
Maybe the Hadoop Data Roll feature is a better option if you want to archive buckets instead of exporting files.
https://docs.splunk.com/Documentation/Splunk/latest/Indexer/ArchivingindexestoHadoop
HTH,
Holger
thousands of events...