All Apps and Add-ons

How do you change how much to poll in a single query?

rmoss84
Explorer

I have made a copy of the nwsdk_query and put it in local and changed the max_meta to a much higher number, but it still only polls 25,000 events at a time. I then thought maybe it was reading the default file, so I changed that to a different number, but it still only polls at 25,000 events. We are currently operating at 35,000k EPS, so only polling 25k every 5 seconds isn't going to work. We've had this running now for 3 days, and it still hasn't even caught up to the 1st day we started on. Any help would be much appreciated.

0 Karma

rataide
Path Finder

Hi,

I'm not sure I completely follow your issue, the value is for meta keys per call not "events/sessions" if there's no value being read from the config file for some reason the code actually defaults to 500k meta pieces per call so at 25,000 events it would be an average of 20 keys per event which if you only have 20 keys select would be about right. You should be able to increase that value to close to about 1.5M more than that the REST endpoint wouldn't really be able to handle.

That aside, I don't think the app will ever cope with the rates you are describing of 35,000k EPS can you give me a bit more background on what you are trying to do? Please feel free to contact me directly via the e-mail on the app files.

Hope this helps!

Thank you,

Rui

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...