Create sourcetype based on hostname

[Lord_Middleton]

I am trying to create a new source type for my esxi hosts based on their hostname.
Their hostnames are vmXX (where XX is the server number)
I was wondering what the best way would be to create a sourcetype that will group all of them.

I am currently using syslog for esxi5.0 hosts.

Thanks!

[hexx]

In all likelihood, you'll want to use an index-time transformation to extract the string of your choosing from the host field and use it to rewrite the sourcetype field.

As an example, the following transformation will assign the "esxi_hosts" sourcetype to any event that has a value of "vm[0-9][0-9]" for the host field:

[rewrite_sourcetype_from_host]
SOURCE_KEY = MetaData:Host
REGEX = vm\d\d
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::esxi_hosts

For more details, make sure to check the spec and examples of transforms.conf.

[lguinn2]

In inputs.conf, you are probably specifying a network input - a port where Splunk is listening for the syslog data. Your inputs.conf stanza could be:

[udp://:1047]
sourcetype=esxi_syslog

- if your esxi data is coming in over UDP port 1047, for example. This will work great if port 1047 is only being used for esxi data. If you are doing this, then stop reading here. Congratulations, you are done. And I do strongly recommend that you do not mix data on a single network port, partly for this reason.

But if you do use the port for multiple kinds of data, not just esxi, then you will have to figure out how to separate it.

in props.conf, figure out a stanza that will identify the data. This may work:

[host::vm*]
sourcetype=esxi_syslog

Note that this will set the sourcetype for all data that comes from hosts whose names start with vm. Again, if this works for you, you are done!

Otherwise, you will have to go one step further and edit your sourcetype using transforms.conf. I am going to assume, for now, that you don't need to go there. If either of the other suggestions work, that's better.

In any case, changing the sourcetype in your configuration files will only affect NEW data. Data that has already been indexed will NOT be changed. To change existing data, you need to clean the events from the index, and then re-index the existing data. This may be problematic - do you have a copy of the existing data stored somewhere, or did it go directly across the network to Splunk? Also, when you reindex the data, it will run through your license meter a second time; this may not be an issue for you, but it might be if you are close to your daily license volume already.

You may be able to get around the problem of existing data by using sourcetype aliasing. Maybe.

Hope this helps!

[lguinn2]

Yikes! You shouldn't edit files in the default directories. Do it in splunk/etc/system/local instead. If props.conf doesn't already exist, just create it.

Or, if your input is defined in splunk/etc/apps/search/local, put the props.conf there instead...

[Lord_Middleton]

Aha!
I do not have a good mental image of how splunk operates... I edited the props.conf in splunk/etc/system/default

[lguinn2]

Which props.conf? There are so many to choose from 🙂

Hopefully this is in the props.conf that is in the same directory as the inputs.conf that defines your syslog input.

[Lord_Middleton]

Hello,
I am trying out your [host::vm*]
sourcetype=esxi_syslog and it doesn't seem like it wants to create a new sourcetype... I am guessing I am missing something as I tried the other users' suggestion and that didn't like me either.

What I did was add in the [host::vm*]... to props.conf, saved, restarted, and hoped. Am I missing something?

[Lord_Middleton]

I am using splunk as a syslog listener, they get sent out directly from the esx hosts to the splunk server (no forwarder)

[Lowell]

Please provide some additional details. Are you using splunk as a syslog listener, or a sylog services like syslogd or syslog-ng to write events to local files?