Splunk Search

How can I break a yearly total into months?

pipegrep
Path Finder

I'm using this simple search to get indexing volume by host.

index="_internal" source="*metrics.log" group="per_host_thruput" | chart sum(kb) by series | sort + series

This sorts by hostname which is what I want;

HOST     sum(kb)
hosta    1234567                     
hostb    2345678
hostc    3456789
...

...but my ultimate goal is actually to break the total down by month:

HOST   January   February   March   ...
hosta   1234567   1234567   1234567   ...                    
hostb   2345678   1234567   1234567   ...
hostc   3456789   1234567   1234567   ...
...

I've looked at timechart and span options but I feel I may be running into a more fundamental problem with my approach.

Tags (1)
1 Solution

MuS
Legend

Hi pipegrep,

how about this:

index="_internal" source="*metrics.log" group="per_host_thruput" | chart sum(kb) by series, date_month | sort + series

cheers, MuS

View solution in original post

MuS
Legend

Hi pipegrep,

how about this:

index="_internal" source="*metrics.log" group="per_host_thruput" | chart sum(kb) by series, date_month | sort + series

cheers, MuS

yannK
Splunk Employee
Splunk Employee

Remember that the metrics contains only the top10 of each series, so if you have more than 10 indexes or hosts, the sum will not match your actual license usage,

0 Karma

Lucas_K
Motivator

How can you get it to show absolutely EVERYTHING?

If I have for example ~10,000 hosts and I put in

limits.conf (on my intermediate forwarder which passes all traffic).
[metrics]
maxseries = 100000

This should have easily covered everything, but it doesn't.

My figures for each of the different metrics groups do not match up.
http://answers.splunk.com/answers/177618/inconsistent-data-rates-reported-between-index-int.html

0 Karma

MuS
Legend

Take a look at the license_usage.log on your license master. From the docs http://docs.splunk.com/Documentation/Splunk/6.2.1/Troubleshooting/WhatSplunklogsaboutitself

license_usage.log
Indexed volume in bytes per pool, index, source, sourcetype, and host. Starting in 4.2, license_usage.log is available only on a Splunk license master.
0 Karma

pipegrep
Path Finder

That was simpler than I expected, thanks. Now I need to figure out why I only get data for this month and last, but that's another issue. Thanks.
*EDIT: Looks like it's a 30 day lookup only and that's why.

0 Karma

MuS
Legend

Nope, the default retention of the index _internal is 30days. This can be changed in indexes.conf on your splunk server see the docs http://docs.splunk.com/Documentation/Splunk/6.2.0/admin/Indexesconf

pipegrep
Path Finder

Yes!

Get Updates on the Splunk Community!

Introducing Splunk Enterprise 9.2

WATCH HERE! Watch this Tech Talk to learn about the latest features and enhancements shipped in the new Splunk ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...