Monitoring Splunk

deployment monitor MB received SLOOOOOOWWWW

mcbradford
Contributor

If I run the All Sourcetypes dashboard, the MB received panel for the past 24 hours, the panel takes just over nine minutes to complete. I studied the search and it is made up of three macros that form this search:

index="_internal" source="*license_usage.lo*" type!=*Summary | eval lastReceived = _time | rename s as source st as mysourcetype h as host b as bytes o as originator | eval my_splunk_server = splunk_server | fields lastReceived source mysourcetype host bytes pool originator my_splunk_server source | bin _time span=10m | stats sum(bytes) as bytes max(lastReceived) as lastReceived by mysourcetype _time pool host | eval kb = bytes/1024 | eval mb = kb/1024 |timechart minspan=10m bins=200 sum(mb) as mbytes by mysourcetype

If I run this search manually the results are returned within 1 minute.

Any idea about what is going on????

0 Karma

mkinsley_splunk
Splunk Employee
Splunk Employee

The macro powering the search in the "MB Recevied" Panel is:

sourcetype_metrics_timechart

You can see what is consuming all the time in your search by inspecting the job. Here is what you'll want to do:

  1. Reload the "All Sourcetypes" panel and go on a small coffee break ( not too long of a break or the job details will get cleaned up).

  2. Click on the "Jobs" link in the upper right corner

  3. Click "Inspect" on the entry for "sourcetype_metrics_timechart"

It sounds like the search might not be using Report Acceleration correctly. Do you see a message indicating that sumaries are being used?

It would look something like the following:

DEBUG: [my.host.name] Using summaries for search, 

If Search summaries are being used, then you may have run into a bug in the core product with search acceleration. In that case, I would recommend opening a support case.

araitz
Splunk Employee
Splunk Employee

Not enough information. Can you open a support case please?

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...