Dashboards & Visualizations

NodeJS JavaScript SDK real-time results are delayed

mik_cox
Explorer

Hey all,

I've set up a real-time search using NodeJS and the JavaScript SDK as outlined in the example at:

https://github.com/splunk/splunk-sdk-javascript/blob/master/examples/node/helloworld/search_realtime...

I've set the earliest_time: 'rt-7s' and latest_time: 'rt' and am polling the job's preview regularly for updates.

The problem is that the results I'm getting back are all delayed by about 60 seconds... which seems to defeat the purpose of a real-time search. Am I doing something obviously wrong here, or might there be something else going on with the JavaScript SDK that's causing this delay? (results still come in at the correct rate, but they're always delayed by around a minute)

Thanks in advance!

0 Karma
1 Solution

mik_cox
Explorer

Based on @sjohnson's answer, @sk4l's comments, and some research of my own, I've sort of figured out what's going on in my environment:

  1. The setting indexed_realtime_use_by_default in limits.conf was true.
  2. In addition, there is another property indexed_realtime_disk_sync_delay which was not set, which means that it was using the default delay of 60 seconds (which is why I saw the 60 second delay I mentioned in the original post).
  3. The delay time mentioned above is the amount of delay for indexed real-time searches (to make sure the data gets fully processed and indexed and synced to disk). That means that I could still do non-indexed real-time searches with no delays. However, for an indexed search, there will be the delay. It's been pointed out to me that there's a significant performance hit for performing non-indexed real-time searches though, so there's a trade-off there between the performance hit and the delay.

My use case was to see data over the past couple of seconds as it comes in, so doing a non-indexed search could be an acceptable solution in my case.

View solution in original post

0 Karma

mik_cox
Explorer

Based on @sjohnson's answer, @sk4l's comments, and some research of my own, I've sort of figured out what's going on in my environment:

  1. The setting indexed_realtime_use_by_default in limits.conf was true.
  2. In addition, there is another property indexed_realtime_disk_sync_delay which was not set, which means that it was using the default delay of 60 seconds (which is why I saw the 60 second delay I mentioned in the original post).
  3. The delay time mentioned above is the amount of delay for indexed real-time searches (to make sure the data gets fully processed and indexed and synced to disk). That means that I could still do non-indexed real-time searches with no delays. However, for an indexed search, there will be the delay. It's been pointed out to me that there's a significant performance hit for performing non-indexed real-time searches though, so there's a trade-off there between the performance hit and the delay.

My use case was to see data over the past couple of seconds as it comes in, so doing a non-indexed search could be an acceptable solution in my case.

0 Karma

sjohnson_splunk
Splunk Employee
Splunk Employee

There is a setting in limits.conf:

indexed_realtime_use_by_default =
* Should we use the indexedRealtime mode by default
* Precedence: SearchHead
* Defaults to false

This might have the effect of delaying your "real time" search results if it is enabled on your search head.

0 Karma

sk4l
Explorer

It's disabled by default, so I doubt this is the OP's problem. And in my case, enabling it makes the delay even longer! But that's somewhat normal I think, since this gets you results only after they've been indexed.

The problem is that the real-time search isn't actually real-time!

0 Karma

sk4l
Explorer

I am also experiencing the same problem. My results are delayed approximately 30 seconds. Does anyone have a solution?

0 Karma

mik_cox
Explorer

As a temporary workaround, I'm no longer using a realtime search and am instead doing a one-off query of the last 5 seconds and re-querying every 5 seconds. This feels "hacky", but it's a band-aid solution until this gets sorted out in the JavaScript SDK.

sk4l
Explorer

I've also settled for this way for now. My company is going to submit a support request today or tomorrow, so I'll post back if there's a solution.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...