Installation

How to display license consumed by an index over 24 hour period?

Mohsin123
Path Finder

Hi,

i am trying to display top 10 license consumed by an index over 24 hours split over 2 hours each . i have a doubt :

does sum(kb) split by series(index) in thruput_group of metrics.log same as sum(size_bytes) per index using eventcount command : |eventcount summarize=false report_size=true $tokApp$ $tokIndex$

I'm confused because my total license consumed per index over 24 hours is not matching according to 2 outputs.. below is my code:

index=_internal  source=*metrics.log group=per_index_thruput earliest=-24h@h  NOT (series=_* OR series=*summary)   | timechart span=2h sum(eval(kb/1024/1024)) as License_Used_GB by series limit=10 useother=f usenull=f|sort -License_Used_GB

2nd one is :

|eventcount summarize=false report_size=true
|stats sum(size_bytes) AS size_bytes  by  index
|eval size_bytes_GB=size_bytes/1024/1024/1024
|eval  size_bytes_GB= round(size_bytes_GB,3)
|rename size_bytes_GB as "Total License occupied by an appilcation by index (Gigs)"  

Kindly help . Total license occupied should match right ? or it can not match also because i am using different sources ? please note: metrics.log results less size than eventcount

Labels (2)
0 Karma
1 Solution

inventsekar
Ultra Champion

please note: metrics.log results less size than eventcount -
-- how different the metrics log results and event count result?

metrics.log is measuring the thruput of data being actually being indexed by Splunk, as a measure of how well your input and indexing pipelines are performing. The metrics.log file itself is indeed indexed to the _internal index because you can run a splunk search and have it show up.

However, this data and the other data indexed by Splunk about Splunk in _internal and _introspection and a few other indexes, does not actually count toward your license. Additionally data that is indexed by Splunk out of summarization queries run against other Splunk data and written into Summary Indexes is additionally not counted toward your license, however it is possible to configure your Splunk Server(s) to have inputs of their own and pick up data that isn't about Splunk itself, thus would actually count toward your license.

To figure out actual license impact (instead of performance metrics) you'll want to look on your license master, there should be a search called the "License Usage Data Cube" which helps build breakdowns and the License Usage Report View which will let you see the actual license impact against various indexes and hosts. (You should read the documentation page because there is squashing behavior that could take place in the data sent to the license master from each indexer.

View solution in original post

0 Karma

inventsekar
Ultra Champion

please note: metrics.log results less size than eventcount -
-- how different the metrics log results and event count result?

metrics.log is measuring the thruput of data being actually being indexed by Splunk, as a measure of how well your input and indexing pipelines are performing. The metrics.log file itself is indeed indexed to the _internal index because you can run a splunk search and have it show up.

However, this data and the other data indexed by Splunk about Splunk in _internal and _introspection and a few other indexes, does not actually count toward your license. Additionally data that is indexed by Splunk out of summarization queries run against other Splunk data and written into Summary Indexes is additionally not counted toward your license, however it is possible to configure your Splunk Server(s) to have inputs of their own and pick up data that isn't about Splunk itself, thus would actually count toward your license.

To figure out actual license impact (instead of performance metrics) you'll want to look on your license master, there should be a search called the "License Usage Data Cube" which helps build breakdowns and the License Usage Report View which will let you see the actual license impact against various indexes and hosts. (You should read the documentation page because there is squashing behavior that could take place in the data sent to the license master from each indexer.

0 Karma

Mohsin123
Path Finder

Thanks @inventsekar .
Could you please tell me one thing ..
in my license master, i can see the license quota and license used for an env , say prod env ..
does that license include only the apps or includes all summary, internal,introspection ?

0 Karma

inventsekar
Ultra Champion

summary, internal,introspection - all these three are not counted toward your license, it will not be reported on the license master.
you will see only all apps indexed data

0 Karma

Mohsin123
Path Finder

thanku so so so much ! this is what i wanted ....
well, in our license manster , for our dev pool / prod poo / qa pool the license used is like very less as compared to my query for fetching license usage using eventcount/metrics.log

say, LM shows dev pool consuming 100gb whereas my dashboard in the prod search head shows 400gb ! thankyou also for giving a clear understanding on metrics.log. Noted !

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...