Monitoring Splunk

How to calculate the memory Splunk consumes while running a query?

pramit46
Contributor

guys,

How can I find out how much memory does a Splunk Query consume?

0 Karma
1 Solution

rsennett_splunk
Splunk Employee
Splunk Employee

A quick and dirty way to do this, would be to steal the "Top Memory Consuming Searches" from the SOS app.
(This is a handy app that you should have installed anyway. http://apps.splunk.com/app/748/ Everyone should use Splunk on Splunk (SOS)

Under the Resource Usage Menu, choose "CPU/Memory". The last panel on the bottom of the dashboard is the one you want.

Hover your mouse over the lower left hand corner of the panel and you'll see a tiny magnifying glass... click it.
It will open the search in a new window.
This search has a lot of stuff in there that you don't need if you are pinpointing one particular search... but rather than pulling it apart... you can insert the SID from the search in question.

Run it in another tab, click "Job Inspector" and get the SID (it's right at the top). Find the following line in the search:

| search sid=* sid!="subsearch*" search!=typeahead* search!="|history*"]

Be careful, because this is the end of an append

Right after | search sid=*

insert your SID so it looks like this:

| search sid=* sid="YOURSIDHERE" sid!="subsearch*" search!=typeahead* search!="|history*"]


That will create a lovely report showing you just the stats for that one particular search.



Again... rather than breaking it... I just added the "Search FOR this thing" with all the "Search for NOT this stuff"

Of course... next thing to do is pick it apart and learn what it's doing. 🙂

With Splunk... the answer is always "YES!". It just might require more regex than you're prepared for!

View solution in original post

rsennett_splunk
Splunk Employee
Splunk Employee

A quick and dirty way to do this, would be to steal the "Top Memory Consuming Searches" from the SOS app.
(This is a handy app that you should have installed anyway. http://apps.splunk.com/app/748/ Everyone should use Splunk on Splunk (SOS)

Under the Resource Usage Menu, choose "CPU/Memory". The last panel on the bottom of the dashboard is the one you want.

Hover your mouse over the lower left hand corner of the panel and you'll see a tiny magnifying glass... click it.
It will open the search in a new window.
This search has a lot of stuff in there that you don't need if you are pinpointing one particular search... but rather than pulling it apart... you can insert the SID from the search in question.

Run it in another tab, click "Job Inspector" and get the SID (it's right at the top). Find the following line in the search:

| search sid=* sid!="subsearch*" search!=typeahead* search!="|history*"]

Be careful, because this is the end of an append

Right after | search sid=*

insert your SID so it looks like this:

| search sid=* sid="YOURSIDHERE" sid!="subsearch*" search!=typeahead* search!="|history*"]


That will create a lovely report showing you just the stats for that one particular search.



Again... rather than breaking it... I just added the "Search FOR this thing" with all the "Search for NOT this stuff"

Of course... next thing to do is pick it apart and learn what it's doing. 🙂

With Splunk... the answer is always "YES!". It just might require more regex than you're prepared for!

rsennett_splunk
Splunk Employee
Splunk Employee

Oh! I didn't see that! 🙂

With Splunk... the answer is always "YES!". It just might require more regex than you're prepared for!
0 Karma

pramit46
Contributor

@rsennett_splunk, I also found that in the job inspector page it shows the memory space the query had consumed.

Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...