All,
I was hoping to build a few alerts based on the information that I see in the Activity >> Jobs dashboard in Splunk 6.2.2.
A couple simple things
1) Alert when there are 50+ jobs running
2) alert if a job is running more than an hour
3) See searches per hour by user
Any example searches of app you can recommend to get me started?
Try something like this:
1) Alert when there are 50+ jobs running
| rest /services/search/jobs|stats count |eval triggerer=(if( count>50,"ok","no"))|where triggerer="ok"|table triggerer
2) alert if a job is running more than an hour
| rest /services/search/jobs |where runDuration>1600
3) See searches per hour by user (i'm not yet able to let you see searches per hour )
| rest /services/search/jobs|chart values(custom.search) by author
Have you looked into the Splunk on Splunk app?
https://splunkbase.splunk.com/app/748/
This should provide a very considerable head start on your searching performance and more!