Getting Data In

Need deployment server to assign hostnames dynamically via inputs.conf

Jaci
Splunk Employee
Splunk Employee

I have a deployment server app with a single inputs.conf file.

[tcp://localhost:9997]
sourcetype = tcp-raw
index = pp-dev

A windows .NET application writes to this port with XML data. Splunk is indexing the data correctly, although the host is showing up as 127.0.0.1. I have this app deployed on about 50 hosts via the deployment server. Is there a way for the Splunk to show the hostname vs. 127.0.0.1 on the indexer?

I am looking for a way to dynamically assign the name. If I have to hardcode the name, then this defeats the benefits of the deployment server.

Tags (2)
1 Solution

Lowell
Super Champion

Great question. I doubt it's possible. The deployment process seem to be pretty limited, IMHO.

WARNING: UGLY UGLY HACK...

I guess you could do a hack with an input script...

Make sure your existing inputs.conf is in the default directory. Then write a simple python script to check for the existence of the local/inputs.conf in your app. If it does not already exist, then your app should create it with the following template:

[tcp:://localhost:9997]
host = <host to be dynamically filled in by script>

After the local/inputs.conf file is written, it should issue a splunk restart command so that the local entry takes effect. When splunk startup up again, the script will run again, but this time since local/inputs.conf exists, it will not make any changes or restart splunkd. (You really don't want a recursive restart loop. That would be bad, especially on 50 machines).

You would probably want to schedule this script to run like once every 31536000 seconds (every year); so that it will only effectively run once a splunkd restart.

When you redeploy your app, obviously your local/input.conf will be wiped out. This shouldn't be a problem though because splunkd will have to restart once (oh yeah, make sure you have restartSplunkd=true in your deployment config). So when splunkd restarts with the newly deployed app, since local/inputs.conf will be missing again, the file will be written and splunkd will be restarted again, but this time with your proper host entry...

Wow! That's ugly. not as bad as it could be, definitely doable, but certainly not pretty.

Do you have any python experience? I could probably whip up a prototype if you want. I know there have been times where I've wanted a feature like this before.


UPDATE... I went ahead and wrote a quick python script that should get the job done (I haven't actually tested it, so there could be issues.)

""" hack_local_inputs.py:  Simple hack to write out a local/inputs.conf file
for this app with a hardcoded host value.
"""

import os
import socket
from subprocess import call

APP_NAME = "my_app_name"
SPLUNK_HOME = os.evniron["SPLUNK_HOME"]
HOSTNAME = socket.gethostname()

conf_file = os.path.join(SPLUNK_HOME, "etc", "apps", APP_NAME, "local", "inputs.conf")

if os.path.exists(conf_file):
    # This is for debugging this script
    print 'Nothing to do here, conf file already exists...  conf="%s"' % conf_file
else:
    stream = open(conf_file, "w")
    stream.write("[tcp:://localhost:9997]\n")
    stream.write("host = %s\n" % HOSTNAME)
    stream.write("disabled = 0\n")
    stream.close()
    # This is for debugging this script
    print 'Finished Hardcoding host hack!  splunk_app=%s new_host=%s conf="%s"' \
            % (APP_NAME, HOSTNAME, conf_file)
    splunk_exe = os.path.join(SPLUNK_HOME, "bin", "splunk")
    call([splunk_exe, "restart", "splunkd"])

Then, in your default/inputs.conf add something like this:

[script::$SPLUNK_HOME/etc/apps/my_app_name/bin/hack_local_inputs.py]
sourcetype = hack_local_inputs
interval = 31536000

View solution in original post

Lowell
Super Champion

Great question. I doubt it's possible. The deployment process seem to be pretty limited, IMHO.

WARNING: UGLY UGLY HACK...

I guess you could do a hack with an input script...

Make sure your existing inputs.conf is in the default directory. Then write a simple python script to check for the existence of the local/inputs.conf in your app. If it does not already exist, then your app should create it with the following template:

[tcp:://localhost:9997]
host = <host to be dynamically filled in by script>

After the local/inputs.conf file is written, it should issue a splunk restart command so that the local entry takes effect. When splunk startup up again, the script will run again, but this time since local/inputs.conf exists, it will not make any changes or restart splunkd. (You really don't want a recursive restart loop. That would be bad, especially on 50 machines).

You would probably want to schedule this script to run like once every 31536000 seconds (every year); so that it will only effectively run once a splunkd restart.

When you redeploy your app, obviously your local/input.conf will be wiped out. This shouldn't be a problem though because splunkd will have to restart once (oh yeah, make sure you have restartSplunkd=true in your deployment config). So when splunkd restarts with the newly deployed app, since local/inputs.conf will be missing again, the file will be written and splunkd will be restarted again, but this time with your proper host entry...

Wow! That's ugly. not as bad as it could be, definitely doable, but certainly not pretty.

Do you have any python experience? I could probably whip up a prototype if you want. I know there have been times where I've wanted a feature like this before.


UPDATE... I went ahead and wrote a quick python script that should get the job done (I haven't actually tested it, so there could be issues.)

""" hack_local_inputs.py:  Simple hack to write out a local/inputs.conf file
for this app with a hardcoded host value.
"""

import os
import socket
from subprocess import call

APP_NAME = "my_app_name"
SPLUNK_HOME = os.evniron["SPLUNK_HOME"]
HOSTNAME = socket.gethostname()

conf_file = os.path.join(SPLUNK_HOME, "etc", "apps", APP_NAME, "local", "inputs.conf")

if os.path.exists(conf_file):
    # This is for debugging this script
    print 'Nothing to do here, conf file already exists...  conf="%s"' % conf_file
else:
    stream = open(conf_file, "w")
    stream.write("[tcp:://localhost:9997]\n")
    stream.write("host = %s\n" % HOSTNAME)
    stream.write("disabled = 0\n")
    stream.close()
    # This is for debugging this script
    print 'Finished Hardcoding host hack!  splunk_app=%s new_host=%s conf="%s"' \
            % (APP_NAME, HOSTNAME, conf_file)
    splunk_exe = os.path.join(SPLUNK_HOME, "bin", "splunk")
    call([splunk_exe, "restart", "splunkd"])

Then, in your default/inputs.conf add something like this:

[script::$SPLUNK_HOME/etc/apps/my_app_name/bin/hack_local_inputs.py]
sourcetype = hack_local_inputs
interval = 31536000

Jaci
Splunk Employee
Splunk Employee

Thank you for the answer and the python script.

0 Karma
Get Updates on the Splunk Community!

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...