Can Splunk check for hosts sending data against a lookup file?

[a212830]

Hi,

Is it possible to have Splunk check for hosts sending data against a lookup file? I have a customer that maintains a file on a web server, which I can get via wget, and use as a lookup. I then want to validate that all of these hosts has sent data to Splunk in the last day.

Is that possible?

[MuS · Accepted solution]

Hi a212830,

sure; get this file by wget once, setup your lookup once and afterwards use a cron job or similar to do the update of the csv.

Then use something like this, to get the entries from the lookup table first, filter it based on which host you are seeing in the system logs. Let's say your lookup table is called "lookup.csv", the relevant logs have sourcetype="systemlogs" and that the field "host" exists in those log events. In that case, this search should get you going:

| inputlookup lookup.csv | search NOT [search sourcetype="systemlogs" | dedup host | fields host]

hope this helps or gets you starting to find the result you're looking for

cheers, MuS

[a212830]

Thanks. I knew how to get the file into Splunk, my question was more around the search to determine which hosts hadn't reported in. This looks like what I want.

Thanks!

[martin_mueller]

There's at least three Splunk-based ways to get this file into your instance.

First, you could define a scripted lookup that looks into the file at runtime - always giving you the most up-to-date data, but requiring a connection all the time.
Second, you could define an input using the REST Modular Input app that loads the file at a given interval into an index of your choice. Then you can use that data to do any query you like, including look for events from each host listed in the file. http://apps.splunk.com/app/1546/
Third, you could define your wget call in a small bash script that gets run by Splunk as a scripted input, similar to the modular input above.

I'm sure there are more ways, and you can of course have a cron job outside of Splunk load the file at regular intervals as well.

[MuS]

dammit, too slow again 🙂