Reporting

How do you calculate the log size per day for a specific sourcetype or source?

kteng2024
Path Finder

Hi,

Can i please know how to calculate the log size per day for a specific source or a sourcetype reporting to splunk.

0 Karma

niketn
Legend

@kteng2024, several options have been given to your for calculating log size per day by sourcetype. Is your issue resolved?

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

nikita_p
Contributor

Hi @kteng2024,
You can try below query;-
| dbinspect index=xyz | fields - size | eval date_s=strftime(startEpoch,"%d/%m/%y") | eval date_e=strftime(endEpoch,"%d/%m/%y") | stats count sum(sizeOnDiskMB) AS size sum(eventCount) AS eventcount by date_e,path|eval sizeinGB=round(size/1024,2) | fields - size​

0 Karma

niketn
Legend

Check out Monitoring Console within Splunk for License Usage

Also check out Meta Woot app from Splunkbase

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

mayurr98
Super Champion

hey @kteng2024

If you want to calculate log size per day for a specific sourcetype try below:

index=_internal [`set_local_host`] source=*license_usage.log* type="Usage" | eval h=if(len(h)=0 OR isnull(h),"(SQUASHED)",h) | eval s=if(len(s)=0 OR isnull(s),"(SQUASHED)",s) | eval idx=if(len(idx)=0 OR isnull(idx),"(UNKNOWN)",idx) | bin _time span=1d | stats sum(b) as b by _time, pool, s, st, h, idx   | timechart span=1d sum(b) AS volumeB by st fixedrange=false  | join type=outer _time [search index=_internal [`set_local_host`] source=*license_usage.log* type="RolloverSummary" earliest=-30d@d | eval _time=_time - 43200 | bin _time span=1d | stats latest(stacksz) AS "stack size" by _time] | fields - _timediff  | foreach * [eval <<FIELD>>=round('<<FIELD>>'/1024/1024/1024, 3)]

If you want to calculate log size per day for a specific source try below:

index=_internal [`set_local_host`] source=*license_usage.log* type="Usage" | eval h=if(len(h)=0 OR isnull(h),"(SQUASHED)",h) | eval s=if(len(s)=0 OR isnull(s),"(SQUASHED)",s) | eval idx=if(len(idx)=0 OR isnull(idx),"(UNKNOWN)",idx) | bin _time span=1d | stats sum(b) as b by _time, pool, s, st, h, idx   | timechart span=1d sum(b) AS volumeB by s fixedrange=false  | join type=outer _time [search index=_internal [`set_local_host`] source=*license_usage.log* type="RolloverSummary" earliest=-30d@d | eval _time=_time - 43200 | bin _time span=1d | stats latest(stacksz) AS "stack size" by _time] | fields - _timediff  | foreach * [eval <<FIELD>>=round('<<FIELD>>'/1024/1024/1024, 3)]

Let me know if it helps you!

0 Karma

richgalloway
SplunkTrust
SplunkTrust

This should get you started.

index=foo source=bar | bin span=1d _time | stats sum(eval(len(_raw))) as TotalSize by _time
---
If this reply helps you, Karma would be appreciated.

anthonymelita
Contributor

Hi,

This functionality is built in to the License Usage report accessible from either the License settings page and choose from the Split By droplist, or the Monitoring conosle > Indexing > License Usage.
You can easily open any of the prebuilt panels in search and modify the query to suit your needs.

index=_internal [`set_local_host`] source=*license_usage.log* type="Usage" 
| eval h=if(len(h)=0 OR isnull(h),"(SQUASHED)",h) 
| eval s=if(len(s)=0 OR isnull(s),"(SQUASHED)",s) 
| eval idx=if(len(idx)=0 OR isnull(idx),"(UNKNOWN)",idx) 
| bin _time span=1d 
| stats sum(b) as b by _time, s, st, h, idx   
| timechart span=1d sum(b) AS volumeB by st fixedrange=false  
| fields - _timediff  
| foreach * [eval <<FIELD>>=round('<<FIELD>>'/1024/1024, 2)]

Take note of the stats line. It has the optional parameters you want. Modify the timechart line below that with the specific split you're looking for. s=source, st=sourcetype, h=host, idx=index

0 Karma

lux209
New Member

Hello,

I know it is an very old post but it is close to what I'm looking for.

I'm trying to extract the log volume per source type, the below query is working fine but it groups all "small" source types in an "other" column. I can't find how to show all sourcetypes in the result ?

index=_internal (host=*.*splunk*.* NOT host=sh*.*splunk*.*) source=*license_usage.log* type="Usage"
| eval h=if(len(h)=0 OR isnull(h),"(SQUASHED)",h)
| eval s=if(len(s)=0 OR isnull(s),"(SQUASHED)",s)
| eval idx=if(len(idx)=0 OR isnull(idx),"(UNKNOWN)",idx)
| timechart span=1d eval(round((sum(b)/pow(2,30)),3)) AS Volume by st
| append
[| search (index=summary source="splunk-entitlements")
| bin _time span=1d
| stats max(ingest_license) as license by _time]
| stats values(*) as * by _time
| rename license as "license limit"
| fields - volume

0 Karma

BABATELO
New Member

It appears to be Splunk default behavior to roll up data in that manner, I believe you may need to remove the empty (NULL) -if any, and OTHER field values from the display.
Use:
useother=f
usenull=f

in your SPL.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...