Getting Data In

How to use a portion of the hostname in your inputs.conf monitor path?

joesrepsolc
Communicator

Got a bunch of logs to pickup from different machines. Evidently each machine has a share to the other machines, so I need to only pickup the log in the directory matching with the actual host name. How can I get the last 4 digits from the host name, and use that in my inputs.conf to pickup only the log files in that subdirectory? Tried different combinations of host_segment but not getting anything that works.

Server Name (Log Path)
abcserver1001 (/opt/log/AppName_1001/server.log)
abcserver1002 (/opt/log/AppName_1002/server.log)
abcserver1003 (/opt/log/AppName_1003/server.log)
abcserver1004 (/opt/log/AppName_1004/server.log)
abcserver1005 (/opt/log/AppName_1005/server.log)

Thanks in advance!!! Joe

0 Karma

mloyola_splunk
Splunk Employee
Splunk Employee

Have you tried the host_segment = 3?
If you only want the last four digits you will need to use transforms and props in indexer or HF so you can reassign and extract the host field.

Are you doing this by Splunk UF -> Splunk Indexer ?

sample tranforms.conf
[overridinghost]
REGEX = AppName_(?<host>[0-9]+)
FORMAT = host::$1
DEST_KEY = MetaData:Host

props.conf
[sourcetype or the source]
TRANSFORMS-serverhost=overridinghost

0 Karma

joesrepsolc
Communicator

Not using heavy forwarders for this input (just sending from UF to index cluster) so can implement on the indexer cluster. Still not understanding how changes on that tier would effect the inputs.conf of the monitor.

0 Karma

joesrepsolc
Communicator

Doing this transform on the Indexer tier would be AFTER the monitor has collected the data. If the path to the correct log is wrong in the monitor, i don't see how that could work.

I need to set the monitor line of the inputs.conf with some intelligence, to only get the last 4 digits of the host, then use that value in the monitor line.

Or are you saying to include the props.conf and transforms.conf with the app I push to the forwarder? My understanding is that the props/transforms portion comes in AFTER the monitor of the inputs.conf... that is too late for what I'm trying to do.

0 Karma

woodcock
Esteemed Legend

The easiest will be to use host_segment=3 in inputs.conf and then use a traditional method of host override (google it) or possibly even INGEST_EVAL-host=replace(host, "^[^_]+_", "abcserver") on the Indexers. The first part in inputs.conf (for server abcserver1001) is like this:

[monitor:///opt/log/AppName_*/server.log]
index = middleware
host_segment = 3
sourcetype = jboss:server:log
disabled = 0
0 Karma

joesrepsolc
Communicator

Still not able to figure out these method(s).

INPUTS.CONF (for server abcserver1001)
[monitor:///opt/log/AppName_(how to dynamically set here based on last 4 of server name)/server.log]
index = middleware
sourcetype = jboss:server:log
disabled = 0

woodcock
Esteemed Legend

See my updated answer.

0 Karma

joesrepsolc
Communicator

Checking this method out now (I googled it!).

0 Karma
Get Updates on the Splunk Community!

Introducing Splunk Enterprise 9.2

WATCH HERE! Watch this Tech Talk to learn about the latest features and enhancements shipped in the new Splunk ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...