Query to check license violation

[pdash]

I need to write a query to check for license violation and the day it happened and then send an email alert with some details like how much is the indexed volume and which splunk indexer had the violation. Can some one help me with this query?

[ma7859]

Here is a query which checks every 30 mins the license limit and compare it with the daily indexing limit.
Basically it first calculates the daily ideal index rate for 30 mins based on the daily limit (in my case its 30 GB) and then it calculates last 30 mins indexed data volume.

This query gonna show following columns if last_30_Mins_indexed_Rate exceeds Ideal_Indexed_Rate :

indexed_Data_Volume_Consumed, Current_Time, Hours_left, Ideal_Indexed_Rate, Indexed_Data_Volume_Left, last_30_Mins_indexed_Rate

Query:

index="_internal" source="*metrics.log" per_index_thruput series="main" earliest=-30m | stats sum(eval(kb/1024/1024)) as last_30_Mins_index | eval hour=tonumber(strftime(now(), "%H")) | eval min=tonumber(strftime(now(), "%M")) | eval timeslots=(24-hour)*2 - round(min/30)
| map search="search index="_internal" source="*metrics.log" per_index_thruput series="main" earliest=@d
| eval GB_Indexed = kb/1024/1024
| stats sum(GB_Indexed) as indexed_Data_Volume_Consumed | eval Indexed_Data_Volume_Left=round((30- indexed_Data_Volume_Consumed),2)| eval Ideal_Indexed_Rate=Indexed_Data_Volume_Left/($timeslots$)| eval Current_Time = now()
| convert ctime(Current_Time)
| eval last_30_Mins_indexed_Rate=$last_30_Mins_index$ |eval Hours_left=round($timeslots$/2,2)
|where ((last_30_Mins_indexed_Rate>Ideal_Indexed_Rate) AND (Hours_left<=20 OR last_30_Mins_indexed_Rate>1)) OR ((Hours_left<=2) AND (Ideal_Indexed_Rate<.5))"

Please reply if this helps.

[DaveSavage]

pdash, most of what you need is in here - Splunk has a rich doco repository and this subject for sure isn't new 😉
http://docs.splunk.com/Documentation/Splunk/5.0.1/Admin/Aboutlicenseviolations

Note especially the use of the _internal index for drill down and alerting. If I see a canned version, I'll post it...but its very easy to interrogate.
Br
Dave

[DaveSavage]

The plug-in Splunk on Splunk (SoS) is also a good reference base if you have call / control over your environment, or can influence such?

[DaveSavage]

sorry - and just to be clear - your problem is and isn't as to which indexed volume has transgressed per se...if you are over, then you are over across the board. Your fix / remediation may be as to which channel is being overly chatty...but if that's 'main' e.g your mainstream indexes and inputs which is our usual experience, then 'which' becomes academic. 'Why', capacity planning, host etc will probably be of more interest to you?

[DaveSavage]

Er - I'm assuming you know a lot about the basics...which maybe isn't fair or true. Your Q said a query to note the dets of a violation. Usually you wouldn't need such as the information is freely available under the Manager tab. I've assumed you needed an alert / incident instead. Splunk is also going to give you plenty of warnings through the banners.
Tbh this may be overkill if you are running dashboards real-time / all the time..because sure as heck it will ping you. If you have some kinda passive system then yes, I can see your different needs....thinking email or SMS.