So we have this query:
index=_internal type=Usage st!=splunk_metrics earliest=-1d@d latest=-0d@d | bucket _time span=1d | stats sum(eval(b/1024/1024/1024)) as GB by _time
Its been running on Splunk for years for us, producing some info about how much is being indexed per day .. we upgraded to Splunk 6.6 and it seems like it doesn't work anymore.
I don't see the field "type" anymore
Does anyone know if they changed this in this new version?
The problem was that for some reason when we upgrades, the inputs.conf changed the hostname of our licensing server (very odd) so once we fixed that it all worked correctly.
The problem was that for some reason when we upgrades, the inputs.conf changed the hostname of our licensing server (very odd) so once we fixed that it all worked correctly.
Before Splunk 6.5.x, Splunk used to report license data in a single log file license_usage.log
. It used to differentiate frequent license usage vs daily rollover summary via field type
that you used in the search above. Starting 6.5.x, the license rollover summary logs have been moved a dedicated log file called license_usage_summary.log
(so all logs with type=RolloverSummary
), thus the field type
is removed. See below links for brief details on both the files (and other internal log files) in Splunk.
This is interesting. When I go on our licensing server and look at license_usage.log, I still see a Type=Usage being logged as of a few mins ago.
We are currently experiencing some unusual SSL error connecting to our licensing server when we run our script so I suspect that may be part of the issue that our original query isn't working:
index=_internal type=Usage st!=splunk_metrics earliest=-1d@d latest=-0d@d | bucket _time span=1d | stats sum(eval(b/1024/1024/1024)) as GB by _time
It is looking for a log file with type=Usage which only exists in the license_usage.log on the licensing manager which cannot be accessed. When I change it to type=* ( and remove st!=splunk metrics which seems like an artifact), I get these types:
Message - License usage logging not available for slave licensing instances
RolloverSummary
SlaveWarnSummary
This seems to correlate with what you are saying (sort of) and also is retrieving license_usage.log files from slave licensing instances which do not have the type=Usage field (hence why the original query got no results).
We will work on getting the SSL error resolved then go from there. Thanks for the info.
Hi @EricLloyd79
Can you give this search a try
index=_internal source=*license_usage.log type=Usage earliest=-1d@d latest=-0d@d | bucket _time span=1d | stats sum(eval(b/1024/1024/1024)) as GB by _time
Thanks
I tried that search and got no results.
This search works to find specifically by host:
index="_internal" source="*metrics.log" group="per_host_thruput" | chart sum(kb) by series | sort - sum(kb)
Now, I was able to get results when I took type=Usage out:
index=_internal source=*license_usage.log earliest=-1d@d latest=-0d@d | bucket _time span=1d | stats sum(eval(b/1024/1024/1024)) as GB by _time
But I am beginning to suspect it has to do with, for some reason, we are unable to access license_usage.log on our licensing server.
We get this message:
LicenseUsage - type=Message - License usage logging not available for slave licensing instances, please see license_usage.log on license master=https://10.10.x.x:8089 for usage breakdown
I can see the license_usage.log file in our licensing server via CLI but when I run this query it can't seem to find it. We recently upgraded to 6.6 but I doubt that would have anything to do with it.