Splunk Search

Where do I configure inputs.conf for Splunk DB Connect 1.1.5 in a search head pooling environment?

bohrasaurabh
Communicator

We faced HTTP 401 issues with Java Bridge for DB Connect 1.1.6, so I downgraded it to 1.1.5 and the bridge started right away.

We have a Search Head Pooling environment and I followed the Documentation to configure DB Connect and verified that I can see the database configured on all servers.

http://docs.splunk.com/Documentation/DBX/1.1.5/DeployDBX/Setupsearchheadpooling

Per documentation "While you can install and configure Splunk DB Connect on a single search head, in a pooling environment, the app state is written to shared storage and is visible to all search heads. "

I want to know where should I place my inputs.conf based on the above statement?

When I put it in $SPLUNK_HOME/etc/system/local/ , I get below message

2015-01-22 12:32:49.281 monsch1:INFO:Scheduler - No database input configured. No need to start dbmon scheduler.
2015-01-22 12:32:49.281 monsch1:INFO:Scheduler - Shutting down database inputs...
2015-01-22 12:32:49.282 monsch1:INFO:ExecutionContext - Execution finished in duration=615 ms
2015-01-22 12:37:48.448 dbx6340:INFO:Splunkd - Splunkd REST Keep-alive successful for user splunk-system-user
2015-01-22 12:37:48.448 dbx6340:INFO:ExecutionContext - Execution finished in duration=13 ms

The btool in the inputs finds my stanza prefix.

0 Karma
1 Solution

bohrasaurabh
Communicator

Based on inputs from Splunk PS and testing, in the SHP environment the inputs.conf for DBConnect should be on $SPLUNK_HOME/etc /system/local area on ONLY 1 SH.

When I had asked this question, I had already put inputs.conf with just dbmon-tail in the local area and restarted splunk, however the inputs.conf was not getting read and the above messages were getting logged. After several retries, when this did not work, I added another input - with dbmon-dump. Once I did that and restarted splunk, suddenly inputs.conf in system/local area started getting read and DBconnect started working - HOW -WHY, I don't know.

I had opened a ticket with support to validate this, however per them inputs.conf for DB connect outside pooled area is not supported but they were also unable to tell me how to configure DB Connect based on documentation statement: "While you can install and configure Splunk DB Connect on a single search head, in a pooling environment, the app state is written to shared storage and is visible to all search heads. ". and kept referring to the Architecture statement:

`About search head pooling and dbmon-tail

We do not recommend using dbmon-tail inputs in a search head pooling environment. In a search head pooling environment, each search head has its own persistent storage that keeps track of the last rising column. This can cause Splunk to index different values for each search head.

We recommend instead that you use a dedicated heavy forwarder with DB Connect installed, to forward data to Splunk indexers.`

Anyways, at this point DB connect has been working for us over a month as expected in the SHP environment and I have not seen any performance issue on any of the SH's.

View solution in original post

bohrasaurabh
Communicator

Based on inputs from Splunk PS and testing, in the SHP environment the inputs.conf for DBConnect should be on $SPLUNK_HOME/etc /system/local area on ONLY 1 SH.

When I had asked this question, I had already put inputs.conf with just dbmon-tail in the local area and restarted splunk, however the inputs.conf was not getting read and the above messages were getting logged. After several retries, when this did not work, I added another input - with dbmon-dump. Once I did that and restarted splunk, suddenly inputs.conf in system/local area started getting read and DBconnect started working - HOW -WHY, I don't know.

I had opened a ticket with support to validate this, however per them inputs.conf for DB connect outside pooled area is not supported but they were also unable to tell me how to configure DB Connect based on documentation statement: "While you can install and configure Splunk DB Connect on a single search head, in a pooling environment, the app state is written to shared storage and is visible to all search heads. ". and kept referring to the Architecture statement:

`About search head pooling and dbmon-tail

We do not recommend using dbmon-tail inputs in a search head pooling environment. In a search head pooling environment, each search head has its own persistent storage that keeps track of the last rising column. This can cause Splunk to index different values for each search head.

We recommend instead that you use a dedicated heavy forwarder with DB Connect installed, to forward data to Splunk indexers.`

Anyways, at this point DB connect has been working for us over a month as expected in the SHP environment and I have not seen any performance issue on any of the SH's.

abhijitmishra
Explorer

Place it in ~SearhHeadPool/etc/apps/dbx/local

0 Karma
Get Updates on the Splunk Community!

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...