Hi, I'm trying to create a report of the endpoints of our API that are not being called. I know how to get a list of the endpoints that are being called, something like:
search ApiRequest
| rex field=_raw "Finished call to (?<endpoint>(GET|POST|PUT|DELETE) [^\s\?]+).*"
| stats values(endpoint)
but I don't know how to make a diff between a list of all the endpoints and that result.
Thanks!
You would need maintain a list of endpoints somewhere/somehow and then use that result with current search to compare and see which are endpoints are being reported. One option could be to maintain a lookup table files which you generate at some interval, say daily, which contains list of all endpoints. They you compare that lookup data against your search like this:
search ApiRequest
| rex field=_raw "Finished call to (?<endpoint>(GET|POST|PUT|DELETE) [^\s\?]+).*"
| stats count by endpoint | append [| inputlookup yourendpointlookup.csv | tabel endpoint | eval count=0]
| stats max(count) as count by endpoint | where count=0
This should give you list of endpoints where there was no data in your search.
If you can't get a lookup table option done and assuming that all endpoints report at least once a week, you can try something like this (assuming your current search's time range is today, if not, then you need to adjust the relative_time function value accordingly)
search ApiRequest earliest=-7d
| rex field=_raw "Finished call to (?<endpoint>(GET|POST|PUT|DELETE) [^\s\?]+).*"
| eval period=if(_time>=relative_time(now(),"@d"),2,1)
| stats max(Period) as period by endpoint | where period=1
You would need maintain a list of endpoints somewhere/somehow and then use that result with current search to compare and see which are endpoints are being reported. One option could be to maintain a lookup table files which you generate at some interval, say daily, which contains list of all endpoints. They you compare that lookup data against your search like this:
search ApiRequest
| rex field=_raw "Finished call to (?<endpoint>(GET|POST|PUT|DELETE) [^\s\?]+).*"
| stats count by endpoint | append [| inputlookup yourendpointlookup.csv | tabel endpoint | eval count=0]
| stats max(count) as count by endpoint | where count=0
This should give you list of endpoints where there was no data in your search.
If you can't get a lookup table option done and assuming that all endpoints report at least once a week, you can try something like this (assuming your current search's time range is today, if not, then you need to adjust the relative_time function value accordingly)
search ApiRequest earliest=-7d
| rex field=_raw "Finished call to (?<endpoint>(GET|POST|PUT|DELETE) [^\s\?]+).*"
| eval period=if(_time>=relative_time(now(),"@d"),2,1)
| stats max(Period) as period by endpoint | where period=1
Awesome! Thank you very much!