Splunk Search

Compare two sets of data from different times

sranga
Path Finder

Hi

I am at a loss on how to approach this problem. Lets say we have the following data:

Input 1: Contains list of non-numeric Ids of data that should be picked up by a processing module 30 days from now
Input 2: Contains list of non-numeric Ids of data that the processing module picked up today

1) Is there anyway to show the diff between the list of today (Input 2) compared with what we "expected" this list to be 30 days back (Input 1)?
2) If the inputs contain a "count" instead of the actual list of data, would it be an easier comparison? I would still be interested in getting the Ids that didn't get picked up today though

Would I be able to use the "diff" command for this? Thanks for your help.

Tags (3)
1 Solution

Stephen_Sorkin
Splunk Employee
Splunk Employee

diff isn't the right way to approach it since it compares two individual events. set diff can be used for this but is often hard to use. Let's assume that you have two searches that identify the events, and we'll call them <search_1> and <search_2>.

Your base search would then be:

<search_1>
| eval input_type="baseline"
| append [search <search_2> | eval input_type="today"]
| chart count by id input_type

Now to answer your question, you'd add to this search: | search baseline>0 today=0.

The problem is marginally easier if you already have a field to distinguish the two types of events, as you'd just search (<search_1>) OR (<search_2>) | chart count by id distinguishing_field.

View solution in original post

Stephen_Sorkin
Splunk Employee
Splunk Employee

diff isn't the right way to approach it since it compares two individual events. set diff can be used for this but is often hard to use. Let's assume that you have two searches that identify the events, and we'll call them <search_1> and <search_2>.

Your base search would then be:

<search_1>
| eval input_type="baseline"
| append [search <search_2> | eval input_type="today"]
| chart count by id input_type

Now to answer your question, you'd add to this search: | search baseline>0 today=0.

The problem is marginally easier if you already have a field to distinguish the two types of events, as you'd just search (<search_1>) OR (<search_2>) | chart count by id distinguishing_field.

Stephen_Sorkin
Splunk Employee
Splunk Employee

Yes, you are correct. I've edited my answer.

0 Karma

sranga
Path Finder

Thanks. In the search command you specified in your answer | append [search <search_1>. Should that be search_2 instead of search_1? Input-1 & Input-2 would be two separate searches.

0 Karma

Stephen_Sorkin
Splunk Employee
Splunk Employee

By search_1, search_2, I mean input 1, input 2 from your statement. If they're the same input, let me know and I'll add to my answer to handle that case.

0 Karma

sranga
Path Finder

Thanks. Would that be search_2 in the "append" command? If I summary index the searches, would that be able to provide the distinguishing_field?

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...