Thanks for getting back to me, I couldn't quite get that to work but it did start me down another path that works. So basically although I only have the 'correlation id' at the start and end I create a field and artificially populate it with the correlation id.
Logic is
- If line has a correlation id do nothing, use it
- If line has no correlation id and thread is the same as previous thread, use the previous correlation id
- If line has no correlation id and thread is NOT the same as previous thread, ignore it
Seems to be working reasonably well so far, basic query is ~
eval t=time | search host=|
sort thread, host, _time |
eval myCid=coalesce(correlationid,prevCorrelationid,t+""+thread+"unknown") |
streamstats current=f window=1 global=f last(myCid) as prevCid |
streamstats current=f window=1 global=f last(thread) as prevThread |
eval myCid=if(thread=prevThread,coalesce(correlationid,prevCid,t+""+thread+"unknown"),coalesce(correlationid,t+""+thread+"_unknown"))
... View more