Splunk Search

License Usage past 30 days don't work

ddarmand
Communicator

There is no results found when i use this dashboard in splunk 6.0 but the first one (today) is working.

How can i fix that ?

Thanks you.

Tags (2)
1 Solution

hexx
Splunk Employee
Splunk Employee

A lack of results in the panels of the "Last 30 days" panel of the License Usage Report View indicates that the License Master instance on which this page is viewed is unable to find events from its own $SPLUNK_HOME/var/log/splunk/license_usage.log file when searching.

This typically has one of two causes:

  • The License Master is configured to forward its events to the indexers (this is a best practice) but it has not been configured to be a search-head. This can be simply remedied by adding all indexers to whom the License Master is forwarding events to as search peers.
  • The License Master is not reading (and therefore, indexing) events from its own $SPLUNK_HOME/var/log/splunk directory. This can happen if the the [monitor://$SPLUNK_HOME/var/log/splunk] default data input is disabled for some reason.

View solution in original post

damonmanni
Path Finder

Running either of your cmds you suggest above fail for me with:
Error in 'foreach' command: arguments must contain at least one field specifier

Is there a typo or part of cmd left off your post above?

I am running splunk v6.4.1

0 Karma

shahzadarif
Path Finder

I've been looking at this issue again today and I think Licence master isn't forwarding data to the indexers. How did I come to this conclusion? If I run this command for "Previous 30 days" licence history I get nothing and this is the built-in command:
index=_internal source=*license_usage.log type="RolloverSummary" earliest=-30d@d | eval _time=_time - 43200 | bin _time span=1d | stats latest(b) AS b by slave, pool, _time | timechart span=1d sum(b) AS "volume" fixedrange=false | join type=outer _time [search index=_internal source=*license_usage.log type="RolloverSummary" earliest=-30d@d | eval _time=_time - 43200 | bin _time span=1d | stats latest(stacksz) AS "stack size" by _time] | fields - _timediff | foreach

However if I modify it by adding splunk_server=local then it works and gives me data for last 30 days.

index=_internal splunk_server=local source=*license_usage.log type="RolloverSummary" earliest=-30d@d | eval _time=_time - 43200 | bin _time span=1d | stats latest(b) AS b by slave, pool, _time | timechart span=1d sum(b) AS "volume" fixedrange=false | join type=outer _time [search index=_internal source=*license_usage.log type="RolloverSummary" earliest=-30d@d | eval _time=_time - 43200 | bin _time span=1d | stats latest(stacksz) AS "stack size" by _time] | fields - _timediff | foreach

I'm assuming when I go to Previous 30 days tab, it searches the index=_internal on indexers (we've got 3 right now) but it doesn't find anything specific to licencing because its not forwarding anything to those indexers? If that's the case how could I forward its internal logs to the indexers. As per the above answer its a best practice solution.
Thanks for all your help so far!

0 Karma

shahzadarif
Path Finder

Happy New Year everyone.
I hope someone would be able to provide me an answer.

0 Karma

stevepraz
Path Finder

I was having a similar issue but I think I figured it out. So, my setup was a license server/DMC server. My last 30 days license reports weren't working. I did a bunch of digging and found your notes as well. It was working in a similar setup in non-prod. By chance I looked at my distsearch.conf and noticed that my DMC/license server was categorized as an indexer (it was not setup to forward the data to the other indexers) while in prod that was not the case.

I went into the DMC roles in prod and made my DMC/License server also have the indexer role and my reports started working again.

0 Karma

ddarmand
Communicator

it was because the forwarder was on 5.4 and the main splunk on 6.0

0 Karma

hexx
Splunk Employee
Splunk Employee

A lack of results in the panels of the "Last 30 days" panel of the License Usage Report View indicates that the License Master instance on which this page is viewed is unable to find events from its own $SPLUNK_HOME/var/log/splunk/license_usage.log file when searching.

This typically has one of two causes:

  • The License Master is configured to forward its events to the indexers (this is a best practice) but it has not been configured to be a search-head. This can be simply remedied by adding all indexers to whom the License Master is forwarding events to as search peers.
  • The License Master is not reading (and therefore, indexing) events from its own $SPLUNK_HOME/var/log/splunk directory. This can happen if the the [monitor://$SPLUNK_HOME/var/log/splunk] default data input is disabled for some reason.

hexx
Splunk Employee
Splunk Employee

Try to run:
$SPLUNK_HOME/bin/splunk cmd btool inputs list --debug
...or use the S.o.S Configuration File Viewer to check your effective inputs.conf settings.

ricercar
Explorer

This is a great possible answer, but how do you check if the default data input is disabled?

0 Karma
Get Updates on the Splunk Community!

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...