Is it possible to create one graph that has lines for each field and a timechart average for each field as additional lines?

[pcorchary]

So, I have a simple search

index="prod1" source="/opt/apps/logs/my.log"  Performance Timing foobar adapter

resulting in data like below. I already have in my app the field extraction for 'foobar_call' and 'foobar_call_duration', so I don't need separate regex in the search to extract them.

What I would like to do is have one graph with multiple lines, one for each type (getfoobarPersona, validateCredential, getfoobarIdentity, etc - there are about 20 API calls here possible), and have a timechart avg() for each one as a line on the graph. Is this possible?

[c0040731][2016-10-25 00:18:04,198][foo][DEBUG][PerformanceMonitoringServiceImpl.java:48][] ,Performance Timing,com.ep.foobar.adapter.foobarAdapter,getfoobarPersona,104    
[c0040481][2016-10-25 00:18:03,895][foo][DEBUG][PerformanceMonitoringServiceImpl.java:48][] ,Performance Timing,com.ep.foobar.adapter.foobarAdapter,validateCredential,155  
[c0040731][2016-10-25 00:18:03,758][foo][DEBUG][PerformanceMonitoringServiceImpl.java:48][] ,Performance Timing,com.ep.foobar.adapter.foobarAdapter,getfoobarPersona,81 
[c0040481][2016-10-25 00:18:03,722][foo][DEBUG][PerformanceMonitoringServiceImpl.java:48][] ,Performance Timing,com.ep.foobar.adapter.foobarAdapter,getfoobarPersona,90 
[c0040731][2016-10-25 00:18:03,653][foo][DEBUG][PerformanceMonitoringServiceImpl.java:48][] ,Performance Timing,com.ep.foobar.adapter.foobarAdapter,getfoobarIdentity,100   
[c0040481][2016-10-25 00:18:03,539][foo][DEBUG][PerformanceMonitoringServiceImpl.java:48][] ,Performance Timing,com.ep.foobar.adapter.foobarAdapter,getfoobarPersona,90 
[c0040481][2016-10-25 00:18:03,449][foo][DEBUG][PerformanceMonitoringServiceImpl.java:48][] ,Performance Timing,com.ep.foobar.adapter.foobarAdapter,getfoobarPersona,94 
[c0040481][2016-10-25 00:18:03,416][foo][DEBUG][PerformanceMonitoringServiceImpl.java:48][] ,Performance Timing,com.ep.foobar.adapter.foobarAdapter,getfoobarIdentity,194   
[c0040481][2016-10-25 00:18:03,327][foo][DEBUG][PerformanceMonitoringServiceImpl.java:48][] ,Performance Timing,com.ep.foobar.adapter.foobarAdapter,getfoobarIdentity,106   
[c0040731][2016-10-25 00:18:02,960][foo][DEBUG][PerformanceMonitoringServiceImpl.java:48][] ,Performance Timing,com.ep.foobar.adapter.foobarAdapter,getfoobarPersona,103    
[c0040450][2016-10-25 00:18:02,851][foo][DEBUG][PerformanceMonitoringServiceImpl.java:48][] ,Performance Timing,com.ep.foobar.adapter.foobarAdapter,getfoobarPersona,92 
[c0040450][2016-10-25 00:18:02,720][foo][DEBUG][PerformanceMonitoringServiceImpl.java:48][] ,Performance Timing,com.ep.foobar.adapter.foobarAdapter,getfoobarIdentity,90    
[c0040763][2016-10-25 00:18:02,584][foo][DEBUG][PerformanceMonitoringServiceImpl.java:48][] ,Performance Timing,com.ep.foobar.adapter.foobarAdapter,getfoobarPersona,98

[somesoni2 · Accepted solution]

Try something like this (you would need to fix the span of the timechart, I';m assuming the span as 1h here)

index="prod1" source="/opt/apps/logs/my.log"  Performance Timing foobar adapter | bucket span=1h _time | stats avg(foobar_call_duration) as avgduration by _time foobar_call | appendpipe [| stats avg(avgduration) as avgduration by _time | eval foobar_call="All_Svc_Average" ] | timechart span=1h avg(avgduration) by foobar_call limit=0

You'll get one line for avg foobar_call_duration for each Service (each value of field foobar_call) and one additional line for avg of all Service for that time bucket.

[pcorchary]

Wow! That's bloody awesome! Thank you so much!

Side question ... is there a way in the search terms to tell the graph to be line type instead of the default?

[gokadroid]

Depending on, when you are charting the values which one out of max or min you want to take as the response time values (given timechart will automatically aggregate results) you can try this:

| timechart max(foobar_call_duration) as callRespTime, avg(foobar_call_duration) as callAvgResp by foobar_call

Or

| timechart min(foobar_call_duration) as callRespTime, avg(foobar_call_duration) as callAvgResp by foobar_call

Or

| timechart values(foobar_call_duration) as callRespTime, avg(foobar_call_duration) as callAvgResp by foobar_call

Whichever suits your need