I am attempting to find out the elapsed time between two log statements as a percentage of the duration of the full request in my service logs. Here's what I have which does not work:
index=service_public service=my-service | transaction correlation_id | join correlation_id [search index=service_public service=my-service | transaction correlation_id startsWith=eval(message="Sent request to other service") endsWith=eval(message="Response received from other service") | rename duration AS duration_other_service]
correlation_id is a UUID unique to a user request, but shared across services (passed via headers). I am trying to use a join to make duration_other_service a field on each transaction in the original query, so I can do something like eval other_service_dur_perc = duration_other_service / duration
.
For some reason, this join returns no results. Do joins not work on transactions? I know the join works, because this returns results:
index=service_public | join correlation_id [search index=service_public message="Response received from other service"]
I have also tried using append
and stats func by correlation_id
, but I can't figure out what func
should be.
Try something like this
index=service_public service=my-service | transaction correlation_id | table correlation_id duration|append [search index=service_public service=my-service | transaction correlation_id startsWith=eval(message="Sent request to other service") endsWith=eval(message="Response received from other service") | table correlation_id duration| rename duration AS duration_other_service] | stats values(*) as * by correlation_id eval other_service_dur_perc = round(duration_other_service*100/duration,2)