Splunk Search

Splunk search causes browser to crash

mlevsh
Builder

One of the searches by our user caused his browser to crash.
"index=oseventlog OR index=activedir OR index=oseventlog_pc bfrisc" for 1 day time period
where bfrisc is an user

Search seems to be able to run, finalize, but eventually goes to "Aw, snap!" page with "Reload" button.
1. User could run the same search for longer time periods , but for other users
2. Clicking on "Reload" button didn't help
3. Cleaning browser cache didn't help
3. Switching to a different browser didn't solve the issue
4. Decreasing the time period didn't help
5. I was able to recreate the same issue on different search head

Any advices in which direction to troubleshoot?

Thank you!

Tags (1)
0 Karma

woodcock
Esteemed Legend

Are you sure that it is the browser or Splunk? What usually happens is that a user runs a poorly constructed/framed search that causes an outrageous amount of RAM to be consumed and the OOMKiller comes by and kills Splunk or the Browser for taking up all the RAM available.

0 Karma

DalJeanis
Legend

If the issue is too much data, then one solution would be to prune the data, for example by using the |fields command to limit the data to that which is actually required, rather than extracting and returning all fields.

The real question here is "what is that user trying to find out, and what is the most effective way of getting that information for them?"

If the active directory index events are too chatty to be useful, then that index should be eliminated from the search. On the other hand, if some subset of the information is useful and needs to be retrieved, then a narrowly-tailored search needs to be developed that gives the information that is essential to the role/function doing the search, and no more.

0 Karma

burwell
SplunkTrust
SplunkTrust

Hi. Can you look for errors in _internal index at the time of search?

0 Karma

mlevsh
Builder

@burwell, didn't find any

0 Karma

burwell
SplunkTrust
SplunkTrust

Hi. I recall seeing this error when using the machine learning toolkit when there was some kind of graphics with too many data points. It was in Chrome for me.

I quit the browser and ran the query again and didn't get the error.

From what I can tell, this is Chrome crashing.

https://stackoverflow.com/questions/1728483/how-to-get-more-info-when-the-aw-snap-screen-shows-up-in...

Can you try with a different browser?

0 Karma

mlevsh
Builder

@burwell, seems you are right. In our case it seems that the search selects active directory events, each event has too many data (listing practically all company users) causing this issue. Browser hits memory limits. I wonder what can be a work around that.
Tried in IE - a little better but still the same issue

0 Karma

burwell
SplunkTrust
SplunkTrust

Hi @mlevsh . So it's definitely a bug. Nothing you should query in Splunk should make the browser crash.

So are you just looking at all the events and not aggregating them with stats count? Could you try that?

0 Karma

burwell
SplunkTrust
SplunkTrust

@mlevsh did you work around this issue?

0 Karma
Get Updates on the Splunk Community!

More Ways To Control Your Costs With Archived Metrics | Register for Tech Talk

Tuesday, May 14, 2024  |  11AM PT / 2PM ET Register to Attend Join us for this Tech Talk and learn how to ...

.conf24 | Personalize your .conf experience with Learning Paths!

Personalize your .conf24 Experience Learning paths allow you to level up your skill sets and dive deeper ...

Threat Hunting Unlocked: How to Uplevel Your Threat Hunting With the PEAK Framework ...

WATCH NOWAs AI starts tackling low level alerts, it's more critical than ever to uplevel your threat hunting ...