Setting Hostname with TCP input

[jbarteet]

Hi, Everyone,

I'm working with one of our developers who wants to send logs into splunk via a TCP input. I cooked up an inputs.conf listening to port 1138 that I've tested using netcat like this:

nc localhost 1138 < /tmp/tcp-test-input

And I put some random events in /tmp/tcp-test-input that start each line with hostname and time stamp.

It mostly works. The data arrives in my splunk indexer, and the time stamp extraction is correct, but the host extraction winds up being that of the load balancer in front of my indexers. ( I have 14 indexers )

My inputs.conf is like this:

[default]

host = fqdn-of-my-host

[tcp://127.0.0.1:1138]

host = fqdn-of-my-host

disabled = false

sourcetype = athena

I tried a few other things, like connection_host = dns for example, but no love.

Do I need to create a props.conf entry for my 'athena' sourcetype and denote something there?

Thanks! ( THX? )

[jbarteet]

Hi, lquinn,

Thank you so much for your response. Unfortunately, this isn't working for me. I was trying to get these ( props.conf and transforms.conf ) to work on the host that employs the forwarder. (v4.2.1)

Do I have to put these on the indexer? 'No' I hope. I have a number of hosts that I would like to forward TCP information from.

I tried to apply the settings in props.conf with source in addition to sourcetype, but I can't make the needle move. Fiddle!

As an aside, I've been sending data from my forwarders to my indexers through a load balancer for some time. We have our netscaler set up to make the connections 'sticky' and it's been working for some time now. I'm wondering now what sort of problems you've encountered with this architecture?

[lguinn2]

If your connections are sticky, then it may work fine. You just have to know what you are doing - since the universal forwarders don't parse the events but just send blocks of an data stream.

[lguinn2]

First - I hope that you are only putting the load balancer in front of this network input to your Splunk indexers. You should never have a load balancer between your Splunk forwarders and your Splunk indexers. If you are doing this, let's discuss under a separate topic - I don't want to derail your actual question.

Second - yes, you can fix the host name. But you will have to use props.conf and transforms.conf to do it as the events are parsed. You can't do it at input time with inputs.conf. (But nice try.) Here is what you need in props.conf and transforms.conf:

props.conf

[athena]
TRANFORMS-h1=set-host-name
SHOULD_LINEMERGE=false

transforms.conf

[set-host-name]
DEST_KEY = MetaData:Host
REGEX =^(\w+)\s
FORMAT = host::$1

The REGEX assumes that the host name is the first "word" in the event, followed by whitespace. (A word in regex contains only letters, digits, and underscores; no periods allowed.) Set your regular expression accordingly. Note that this is an index-time field extraction, which is appropriate for the host name. You should not use a search-time field extraction here.

Also, I specified that the settings apply to all data of the athena sourcetype, but you could specify the transformation based on source or host instead... And I specified that these are single-line events, which will make the parsing a little faster; you don't have to specify that, but I figured that you might as well if you had to create props.conf anyway.

Update: where do the .conf files go?

Answer: wherever you are parsing the data.

If you are using a universal forwarder to collect this information - you must put the props.conf and transforms.conf on the indexer. UFs do not parse data - they merely collect it and send it.

If you are using a heavy forwarder, you must put the props.conf and transforms.conf on the heavy forwarder.

If you put the props.conf on the indexer, it will apply the transformation to all events of sourcetype athena, regardless of which host sent the data. Hopefully this will not be a problem.

Actually, if I were setting up a network input into a forwarder, I would probably use a heavy forwarder. I would also prefer that server to be simply a Spunk server / data collection point - not a normal production server with extra Splunk duties thrown on top...