Splunk Dev

Events Being Passed to Custom Commands More Than Once

cschmidt0121
Path Finder

I've been working with custom commands for a while, and I've noticed some weird behavior regarding how events are passed into commands. It is my understanding that regardless of whether a command is declared to be streaming or not, Splunk may choose to send events to the command in chunks instead of one large set. So what I would expect to see is something like this:

For a total of 15,000 events piped into custom command "cmd"...

Splunk sends cmd events 0-5000
cmd completes and outputs results

Splunk sends cmd events 5001-10000
cmd completes and outputs results

Splunk sends cmd events 10001-15000
cmd completes and outputs results

However, by having my command log the events it receives, I instead see something that resembles this:

Splunk sends cmd events 0-5000
cmd completes and outputs results

Splunk sends cmd events 0-10000
cmd completes and outputs results (these results OVERWRITE previous results!)

Splunk sends cmd events 0-15000
cmd completes and outputs results (these results OVERWRITE previous results!)

So events 0-5000 end up being passed to the command 3 times, and events 5001-10000 are passed twice. Is this intended behavior? If so, can anyone explain why?

0 Karma

kchen_splunk
Splunk Employee
Splunk Employee

Setting "run_in_preview = false" in commands.conf will solve the problme

0 Karma

dart
Splunk Employee
Splunk Employee

This is, as far as I know the intended behaviour for preview functionality in Splunk.

Overwriting previous results makes sense to me - for example if you were implemented a command that added some global state to each event (like eventstats) then the more complete input would be correct, and the partial one doesn't.

I'd suggest also that you have a look at the new python SDK examples which make writing custom search commands easier

cschmidt0121
Path Finder

Ooooh. I see. Thank you very much for the response.

0 Karma

dart
Splunk Employee
Splunk Employee

It's so we can display the preview results - it's pretty handy for seeing if you're going to get the right data.

0 Karma

cschmidt0121
Path Finder

Yeah that certainly makes sense. I'm still confused as to why Splunk runs the command multiple times in the first place, though, if it's just going to discard the results of the first two runs.

Thanks for the link to the new SDK examples; I'm looking over them now.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...