Splunk Search

Is the "type" field removed from Splunk metrics in 6.6?

EricLloyd79
Builder

So we have this query:

index=_internal type=Usage  st!=splunk_metrics earliest=-1d@d latest=-0d@d  | bucket _time span=1d | stats sum(eval(b/1024/1024/1024)) as GB by _time

Its been running on Splunk for years for us, producing some info about how much is being indexed per day .. we upgraded to Splunk 6.6 and it seems like it doesn't work anymore.
I don't see the field "type" anymore
Does anyone know if they changed this in this new version?

0 Karma
1 Solution

EricLloyd79
Builder

The problem was that for some reason when we upgrades, the inputs.conf changed the hostname of our licensing server (very odd) so once we fixed that it all worked correctly.

View solution in original post

0 Karma

EricLloyd79
Builder

The problem was that for some reason when we upgrades, the inputs.conf changed the hostname of our licensing server (very odd) so once we fixed that it all worked correctly.

0 Karma

somesoni2
SplunkTrust
SplunkTrust

Before Splunk 6.5.x, Splunk used to report license data in a single log file license_usage.log. It used to differentiate frequent license usage vs daily rollover summary via field type that you used in the search above. Starting 6.5.x, the license rollover summary logs have been moved a dedicated log file called license_usage_summary.log (so all logs with type=RolloverSummary), thus the field type is removed. See below links for brief details on both the files (and other internal log files) in Splunk.

https://docs.splunk.com/Documentation/Splunk/6.6.0/Troubleshooting/WhatSplunklogsaboutitself#Interna...

0 Karma

EricLloyd79
Builder

This is interesting. When I go on our licensing server and look at license_usage.log, I still see a Type=Usage being logged as of a few mins ago.

We are currently experiencing some unusual SSL error connecting to our licensing server when we run our script so I suspect that may be part of the issue that our original query isn't working:
index=_internal type=Usage st!=splunk_metrics earliest=-1d@d latest=-0d@d | bucket _time span=1d | stats sum(eval(b/1024/1024/1024)) as GB by _time

It is looking for a log file with type=Usage which only exists in the license_usage.log on the licensing manager which cannot be accessed. When I change it to type=* ( and remove st!=splunk metrics which seems like an artifact), I get these types:

Message - License usage logging not available for slave licensing instances
RolloverSummary
SlaveWarnSummary

This seems to correlate with what you are saying (sort of) and also is retrieving license_usage.log files from slave licensing instances which do not have the type=Usage field (hence why the original query got no results).

We will work on getting the SSL error resolved then go from there. Thanks for the info.

0 Karma

PowerPacked
Builder

Hi @EricLloyd79

Can you give this search a try

index=_internal source=*license_usage.log type=Usage earliest=-1d@d latest=-0d@d | bucket _time span=1d | stats sum(eval(b/1024/1024/1024)) as GB by _time

Thanks

0 Karma

EricLloyd79
Builder

I tried that search and got no results.

This search works to find specifically by host:
index="_internal" source="*metrics.log" group="per_host_thruput" | chart sum(kb) by series | sort - sum(kb)

Now, I was able to get results when I took type=Usage out:
index=_internal source=*license_usage.log earliest=-1d@d latest=-0d@d | bucket _time span=1d | stats sum(eval(b/1024/1024/1024)) as GB by _time

But I am beginning to suspect it has to do with, for some reason, we are unable to access license_usage.log on our licensing server.
We get this message:

LicenseUsage - type=Message - License usage logging not available for slave licensing instances, please see license_usage.log on license master=https://10.10.x.x:8089 for usage breakdown

I can see the license_usage.log file in our licensing server via CLI but when I run this query it can't seem to find it. We recently upgraded to 6.6 but I doubt that would have anything to do with it.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...