Splunk Search

Is dynamic query construction possble in Splunk without using any SDKs ?

Arun_N_007
Communicator

I need to modify the query of saved search based on some conditions. Is it possible using only Splunk query language?

Tags (3)
0 Karma

woodcock
Esteemed Legend

This is crazy but you could use the ouputlookup command to use the KV store as a registry of sorts like this:

Write your initial lookup :

index=* | eval HardCodedKey= 0 | stats first(HardCodedKey) AS HardCodedKey BY host | dedup HardCodedKey | eval NextSearchString="Your Inital or Default Search Here" | stats count by HardCodedKey, NextSearchString| outputlookup MyLookup

Then your scheduled search could do something like this (and also something like above, to refresh the registry for the next run):

index=* | eval HardCodedKey= 0 | stats first(HardCodedKey) AS HardCodedKey BY host | dedup HardCodedKey | lookup MyLookup HardCodedKey OUTPUT NextSearchString | map search="Some Search Stuff $NextSearchString$"
0 Karma

Arun_N_007
Communicator

Hi Woodcock,

Any other way to achieve this use case other than using map..

Any equivalent to map without 10K limit???

0 Karma

Arun_N_007
Communicator

Thank you woodcock,
For showing unknown approach 🙂

But still map got limit of 10K (due to Sub Search limit) :)..

I know that if am using SDKs I can accomplish inside commands but I don't have SDK.

0 Karma

woodcock
Esteemed Legend
0 Karma

Arun_N_007
Communicator

Hmm I tried that you cant return a macro name with parameters using return statement.

0 Karma

woodcock
Esteemed Legend

What I meant was that maybe you could abandon your current approach and start over using a macro-based approach.

0 Karma

Arun_N_007
Communicator

Hi,

Use case is like,

  1. I need to calculate the average completion time of Jobs.
  2. Period is configurable.
  3. I have list of Jobs in Lookup for which i need to caluclate averages.
  4. In that list i have different type of Jobs for which calculation Logic (Due to time format and all...) differs.
  5. Since Splunk got limit on transformation commands like stats,eventstats(50K) i need to make sure that scaling should be done based on limit.
  6. I should be able to design buckets dynamically using queries. And Average query should run for each buckets without any manual intervention.
0 Karma

acharlieh
Influencer

Based on what sort of conditions and what sort of modifications do you need to make? Query replacement tokens and subsearches might be helpful here but a complete answer requires more detail as to what you have and are trying to accomplish. The docs on savedsearch gives a hint at using string substitution for replacement tokens

0 Karma

Arun_N_007
Communicator

For Example,

I want to have a single saved search query which will get executed for every 1hr and it will be processing the particular batch of records.
And for each batch query logic should be modifed dynamically.

Can we accomplish this using custom command? Can we invoke another query inside custom command without using Splunk Python SDK?

0 Karma

acharlieh
Influencer

How do you identify "the particular batch of records" to process? Is it just events from the last hour or is that variable each hour? Are you processing multiple batches each hour? By "batch query logic should be modified dynamically" What determines the dynamics of this query? Something in how the search is launched? Something in the data of the batch? Something else?

Another potential thought is a combination of gentimes and map but without understanding what your goal is and what the data looks like it's still just a guess in the dark.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...