Splunk Search

Count of API calls over X time_taken, only if average time_taken is over a threshold

pzhou07920
Explorer

Hi,

I currently have a query that returns the a chart of API's whose calls average over a specific time limit (unique per API). I would then like to be able to display the count of calls over X seconds time_taken ONLY if that API had an average time_taken over X seconds.

Would I be correct in thinking that I should make my first search a sub search and then search on that to find the counts of timed out APIs?

Here is my current search for the APIs with average time_taken over a limit.

index=mykplan_main cs_uri_stem="AAA" OR cs_uri_stem="BBB"

| eval URI=cs_uri_stem
| eval URI = lower(URI)
| stats avg(eval(time_taken*.001)) as avg_duration by URI
| eval avg_duration=round(avg_duration,2)
| eval alert=if((avg_duration > 3 AND URI="AAA") OR (avg_duration > 1 AND URI="BBB") ,"alert", "ignore")
| where alert="alert"
| fields - alert

Tags (1)
0 Karma

somesoni2
Revered Legend

How about this? YOu're already fetching avg time for a URI. You can just then get the count based on avg_duration as you're already filtering for avg_duration>threshold.

index=mykplan_main cs_uri_stem="AAA" OR cs_uri_stem="BBB" 
| eval URI=cs_uri_stem 
| eval URI = lower(URI) 
| stats avg(eval(time_taken*.001)) as avg_duration by URI 
| eval avg_duration=round(avg_duration,2) 
| eval alert=if((avg_duration > 3 AND URI="AAA") OR (avg_duration > 1 AND URI="BBB") ,"alert", "ignore") 
| where alert="alert" 
| chart count by avg_duration
0 Karma

pzhou07920
Explorer

Doing this only returns a count of 1 for every result, I think it counts the avg duration or something. I preferably want to only get the count of API calls over the same avg_duration threshold I used but total count of events by URI should be okay too.

0 Karma

micahkemp
Champion

You should be able to just add count to your stats command.

index=mykplan_main cs_uri_stem="AAA" OR cs_uri_stem="BBB" 
| eval URI=cs_uri_stem 
| eval URI = lower(URI) 
| stats avg(eval(time_taken*.001)) as avg_duration, count by URI 
| eval avg_duration=round(avg_duration,2) 
| eval alert=if((avg_duration > 3 AND URI="AAA") OR (avg_duration > 1 AND URI="BBB") ,"alert", "ignore") 
| where alert="alert" 
| fields - alert
0 Karma

rekhan
New Member

Hi ,

I am new to Splunk... We are pulling our logs from cloudwatch into splunk.

I want to create a dashboard to show the number of API calls by response - 2xx, 4xx and 5xx.

Basically I want to count the numbers of alerts for each response type for the API call.

Can you please help.

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...