Splunk Search

Is there a way to have different timescale for lookups than the actual search?

abhijitp
Path Finder

Hi,

I am looking for a solution for this problem. I have implemented Lookup tables based on time and they are working fine.

The issue I am having is that if the lookup table entry falls outside the search timescale it returns with no results. I wanted to see if there a way to stretch the lookup timeline more than the actual search.

I know everyone likes the actual search string, but in this case it will not help.

eg. I have a lookup table with an entry made say 24 hours ago as shown below.
Unit 111 Version 1.1 Time 1:31pm 5/5/2016

If I search using lookup for Versions in the last 4 hours, it will not show the above entry.
However, If i stretch it to last 7 days, it shows up properly.

So here is my question (i could not find any documentation around it).

I need a search than runs on the last 24 hours continuously but it should incorporate lookup entries for the Unit Versions from the last known entry which could be 1 week or even 30 days in the past.

Sorry for all text. I am really hoping this is possible.

Please help.

Thanks a lot,
Abhi

Tags (1)
0 Karma

woodcock
Esteemed Legend

Do it all in SPL like this:

Your Base Search Here | appendpipe [|inputlookup YourLookupDefinitionHere | dedup YourTimeFieldHere] | stats values(*) AS * by YourLookupKeyFieldHere

This solution assumes that the most-recent/latest/highest time values are at the top of the lookup file; if they are at the bottom, you must add | reverse before | dedup.

0 Karma

jensonthottian
Contributor

Have a search query running lets say every hour and appending the results to an existing lookup table with time added - | inputlookup append=true

. Now since you have a time field in the table in your search string put filters for filtering using this time field.

0 Karma

abhijitp
Path Finder

Hi jensonthottian,

I already have an hourly query that appends the lookup table "only if" any information changes from the previous record. In this case, since there will be not update (except time) the newer entry will be ignored. This needs to be done, otherwise there will be an entry every hour with the updated time in the lookup table which is not helpful and risky as the size will keep on growing.

The lookup table should have a more bigger timescale than the actual search criteria.

0 Karma

jensonthottian
Contributor

why not after doing an append add this to the existing lookup file with command "|outputlookup csvfilename.csv "

this will create a new CSV whenever the search runs, so you dont need a big timescale to be searched ~1 hour. I was running into a similiar issue and I solved by using this technique.

If this works please accept the answer,.

0 Karma

abhijitp
Path Finder

I am already putting the output of the hourly searches into a csv file.

0 Karma

jensonthottian
Contributor

I thought that was your issue, the csv file having an older date. I am sorry can you provide more details on what exactly you are running into.

0 Karma

abhijitp
Path Finder

Thanks for your help.

I need a way for the lookup table which is stored in a csv file to go back in time more than the actual search to capture the latest information about a unit.

eg. I have a lookup table with an entry made say 24 hours ago as shown below.
Unit 111 Version 1.1 Time 1:31pm 5/5/2016

If I search using lookup for Versions in the last 4 hours, it will not show the above entry.
However, If i stretch it to last 7 days, it shows up properly.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...