Hi MicTech, OpenShift v3 is based on Kubernetes and includes the same default logging layer (Fluentd). Fluentd can send messages to Splunk with some community-built plugins that will need to be configured for sending to either the Splunk API, HTTP Event Collector, or TCP receiver. Documentation for the latest version (3) of OpenShift about logging, and configuring the specific output can be found here: https://docs.openshift.com/enterprise/3.1/install_config/aggregate_logging.html Treasure Data also offers Fluentd Enterprise that has a supported Splunk plugins for the HTTP Event Collector and over TCP. More information can be found on https://fluentd.treasuredata.com or email me at a @ treasuredata.com Thanks, Anurag ... View more

Hi Dhavamanis, Fluentd is one of the preferred logging layers of Kubernetes, and using Fluentd is preferred for Kubernetes data routing to Splunk, Elasticsearch, Kafka, Amazon S3, etc. Using a Kubernetes Daemon Set you can deploy a Fluentd node inside of every Kubernetes Node and have the configuration to then route stdout, stderr, etc. data into Elasticsearch, Splunk, etc. Additionally, Fluentd has additional capabilities to append information about the Kubernetes Pod, Namespace, Node. Documentation of Kubernetes Daemon Set: https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/ Documentation on Fluentd Daemon Set:http://docs.fluentd.org/v0.12/articles/kubernetes-fluentd Documentation on Fluentd-Elasticsearch Daemon Set:http://docs.fluentd.org/v0.12/articles/kubernetes-fluentd#logging-to-elasticsearch If you want a supported Splunk plugin and a Kubernetes -> Splunk DaemonSet , Fluentd Enterprise offers SLA support for sending data to Splunk Enterprise and Splunk Cloud. If you want more information you can email me at A@ Treasuredata.com and find more information here: https://fluentd.treasuredata.com Thanks, Anurag ... View more

Hi Lisaac, For number 2, Fluentd offers a great alternative to HEC as it allows you to bifurcate/copy/process/transform/parse the data at the node level and then send to the specific data store you need. For example, you can take all warning, error, critical priority Syslog messages and send those to Splunk while sending the lower priority + warning, error, criticial messages to HDFS, Amazon S3, etc. This allows you to reduce the volume that hits the Splunk indexers. Additionally, Fluentd offers an enterprise version that offers SLA based support for each of the outputs you mentioned (Splunk Enterprise, HDFS, Kafka, Amazin S3) - if you are interested email me at a@treasuredata dot com. More information can be found at https://fluentd.treasuredata.com Thanks, Anurag ... View more

Hi vya998, Thanks for using Fluentd! The first bit is that the Splunk API plugin that you referenced is deprecated, and you should switch to sending messages over TCP or through the Splunk HTTP Event Collector. Additionally, I see that your configuration for translating and parsing data is being done on the Splunk indexer side. I would recommend translating those configurations over to Fluentd to distribute that compute layer to the endpoints so Splunk can focus on search. Fluentd has the ability to do most of the common translation on the node side including nginx, apache2, syslog [RFC 3624 and 5424], etc. Additionally, if you are interested in the Fluentd Enterprise Splunk TCP and HTTP Event Collector plugin and help in optimizing parsing and transformation logic you can email me at A at TreasureData dot com. More info for https://fluentd.treasuredata.com Thanks, Anurag ... View more

You can also use Fluentd Enterprise to bifurcate data to separate indexers. You can use message content, message type, hostname, etc. to route the messages between the indexers. https://fluentd.treasuredata.com/splunk-optimize ... View more