Getting Data In

Casing and FQDNs means triple host entries

CMSchelin
Path Finder

I have quite a few hosts that are listed multiple times in Splunk. For an example:

HOST01
host01
host01.fqdn

I'm relatively new to Splunk. What are my options for merging those entries?

1 Solution

woodcock
Esteemed Legend

The best thing to do is to have each forwarder report (self-identify) the way that you prefer. This is controlled in server.conf on the forwarder so read the documentation on that. This effects NEW data but old data will stay jumbled so you can do something like this before you aggregate your data with stats or whatever:

... | rex field=host mode=sed "s/\..*$//" | eval host=lower(host) ...

View solution in original post

DalJeanis
SplunkTrust
SplunkTrust

We feel your pain. Also, please note, in some types of events we tend to end up with the IP address standing in place of a missing hostname, so you can't grab everything to the left of the period until after you've checked that the entire hostname isn't of the format of an IP. Usually, we end up using upper() and some rexes to make sure we are getting the best guess at the short host.

| rex field=myhostname "^(?<shost>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}|[^.\]*)"
| eval shost=upper(coalesce(shost, myhostname, "((MISSING))" ))

The above grabs an IP address if it starts out the field (imprecise IP-grabbing rex for demo purposes only) then grabs everything to the left of the first period... which if there is no period, means the whole field. The only failure in the rex is if the first character is an actual period, in which case the eval/coalesce puts the invalid hostname back into place before translating the whole thing to upper case.

General notes - 1) searches are case-insensitive, but field values are case-sensitive. 2) Field aliasing is your friend.

0 Karma

woodcock
Esteemed Legend

The best thing to do is to have each forwarder report (self-identify) the way that you prefer. This is controlled in server.conf on the forwarder so read the documentation on that. This effects NEW data but old data will stay jumbled so you can do something like this before you aggregate your data with stats or whatever:

... | rex field=host mode=sed "s/\..*$//" | eval host=lower(host) ...
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...