Casing and FQDNs means triple host entries

[CMSchelin]

I have quite a few hosts that are listed multiple times in Splunk. For an example:

HOST01
host01
host01.fqdn

I'm relatively new to Splunk. What are my options for merging those entries?

[DalJeanis]

We feel your pain. Also, please note, in some types of events we tend to end up with the IP address standing in place of a missing hostname, so you can't grab everything to the left of the period until after you've checked that the entire hostname isn't of the format of an IP. Usually, we end up using upper() and some rexes to make sure we are getting the best guess at the short host.

| rex field=myhostname "^(?<shost>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}|[^.\]*)"
| eval shost=upper(coalesce(shost, myhostname, "((MISSING))" ))

The above grabs an IP address if it starts out the field (imprecise IP-grabbing rex for demo purposes only) then grabs everything to the left of the first period... which if there is no period, means the whole field. The only failure in the rex is if the first character is an actual period, in which case the eval/coalesce puts the invalid hostname back into place before translating the whole thing to upper case.

General notes - 1) searches are case-insensitive, but field values are case-sensitive. 2) Field aliasing is your friend.

[woodcock · Accepted solution]

The best thing to do is to have each forwarder report (self-identify) the way that you prefer. This is controlled in server.conf on the forwarder so read the documentation on that. This effects NEW data but old data will stay jumbled so you can do something like this before you aggregate your data with stats or whatever:

... | rex field=host mode=sed "s/\..*$//" | eval host=lower(host) ...