Getting Data In

NOOB - run a query in batch with varying input

ranshe
New Member

Hi,
A total Splunk NOOB, so please go easy.

Anyway, every night I receive a bunch of numbers and I want Splunk to dig through my logs and tell me if these numbers are not there, or if they are tell me the info associated with these numbers.
How do I go about achieving that?

Thanks,

Tags (1)
0 Karma
1 Solution

Ayn
Legend

This can certainly be done. The learning curve of implementing it might become a bit steep if you're totally new to Splunk though. 🙂

You should be able to achieve what you want using a combination of

  1. A subsearch (http://docs.splunk.com/Documentation/Splunk/latest/User/HowSubsearchesWork)
  2. A lookup (http://docs.splunk.com/Documentation/Splunk/latest/User/CreateAndConfigureFieldLookups)

A subsearch works a lot like backticks in UNIX if you're familiar with them. It runs first of all and passes on its results as arguments to the main search. A lookup can be either static, a CSV file that Splunk reads from, or dynamic, in which case Splunk calls a script and uses its results.

The idea in your case would be to have a lookup CSV file containing all the numbers you want to monitor, and then pass those numbers to the main search which checks if the numbers are there or not. Then, create an alert (http://docs.splunk.com/Documentation/Splunk/4.2.4/User/SchedulingSavedSearches) that triggers if you're getting results back.

My best advice from here would be to go read up on the mechanisms involved before diving into this. Your final search command might look something like this:

sourcetype="yoursourcetypewiththenumbers" [| inputlookup numbers.csv | fields number]

Here, I set the sourcetype to just a dummy sourcetype - I don't know how you filter on the exact logs that you want to check these numbers for. I also assumed that the numbers exist in a field called "numbers" in both the CSV file and in the logs - change this to whatever fieldname you're using. Hope this helps with getting you started.

View solution in original post

Ayn
Legend

This can certainly be done. The learning curve of implementing it might become a bit steep if you're totally new to Splunk though. 🙂

You should be able to achieve what you want using a combination of

  1. A subsearch (http://docs.splunk.com/Documentation/Splunk/latest/User/HowSubsearchesWork)
  2. A lookup (http://docs.splunk.com/Documentation/Splunk/latest/User/CreateAndConfigureFieldLookups)

A subsearch works a lot like backticks in UNIX if you're familiar with them. It runs first of all and passes on its results as arguments to the main search. A lookup can be either static, a CSV file that Splunk reads from, or dynamic, in which case Splunk calls a script and uses its results.

The idea in your case would be to have a lookup CSV file containing all the numbers you want to monitor, and then pass those numbers to the main search which checks if the numbers are there or not. Then, create an alert (http://docs.splunk.com/Documentation/Splunk/4.2.4/User/SchedulingSavedSearches) that triggers if you're getting results back.

My best advice from here would be to go read up on the mechanisms involved before diving into this. Your final search command might look something like this:

sourcetype="yoursourcetypewiththenumbers" [| inputlookup numbers.csv | fields number]

Here, I set the sourcetype to just a dummy sourcetype - I don't know how you filter on the exact logs that you want to check these numbers for. I also assumed that the numbers exist in a field called "numbers" in both the CSV file and in the logs - change this to whatever fieldname you're using. Hope this helps with getting you started.

ranshe
New Member

Wow, lots of great info - highly appreciated!
Now the "homework" is on me - I'll play around and see how to use all this info.

Thanks.

0 Karma

ranshe
New Member

Thanks lguinn.
My question is really about the "You need to run a search, and then save it and schedule it to run every night" part - how do I do that?
Also, is there a way for me to dump a file with my numbers and have Splunk read it and act on it?

0 Karma

lguinn2
Legend

It would be helpful if you could give us some examples of your data (fake the ip addresses, etc. if needed).

You need to run a search, and then save it and schedule it to run every night. Schedule it to run at a time after you receive the numbers. Also, set it up to alert you via email "always" and it will send you a report every night that shows if anything matched.

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...