How to write a search to check the amount of data indexed by my app each day for a certain time range?

[sushmitha_mj]

Hi,

I want to a graph to check the amount of data indexed by my app on each day for a certain time period. I have multiple csv files in the source, one host and an index ( not default, I created this ). I used

index="abc" | eval MB=kb/1024 | search group="per_sourcetype_thruput" | timechart span=1d sum(MB) by series 

and

index="abc" group="per_index_thruput" | timechart per_second(kb) as " kbps" by series 

but they seem to return no results.

[lguinn2 · Accepted solution]

The groups that you are looking for are not in the "abc" index - they are in the _internal index. Also, you can get the "per index" through-put by index, or the "per sourcetype" through-put by sourcetype - but you can't get the "per index" through-put by sourcetype!

index=_internal sourcetype=splunkd group=per_sourcetype_thruput
| eval MB=kb/1024 | timechart span=1d sum(MB) by series

index=_internal sourcetype=splunkd group=per_index_thruput series="abc"
|timechart per_second(kb) as " kbps"

HTH

[sushmitha_mj]

@Iguinn
Thank you...
Both worked, and I had an option of entering the series on the first one as well. As the graph itself was multiple series lines.

[sushmitha_mj]

@Iguinn
I have a doubt... When I added the query "index=_internal sourcetype=splunkd group=per_host_thruput | timechart per_second(kb) by series" to the dashboard made a default time range of 365 days, I get a different graph and when I run it for All time it is different. Although the data is only there starting march. Any idea why?

[lguinn2]

Well, if you run the same search twice, you will get different results usually, as the data is always changing.
But, there is also the problem of the first/final "grouping" of the data - if the data is only partial days, it can look weird. And different timeranges can certainly make the graph look different.

I would probably set the search time range explicitly to something like earliest=-30d@d latest=@d to avoid the problem of partial buckets. Or, if you want to be able to pick the timerange each time you run the report, do this in the timechart command:

timechart fixedrange=f partial=f per_second(kb) as " kbps"

This will restrict the graph to only the part of the timerange with valid data and also remove any partial buckets.