What I want to achieve is to extract surrounding log lines by thread ID, which is found on the line where the keyword is seen.
A sample snippet of my log looks like
2018-01-01 01:00:01,000 [Module thread#2] info - action starts
2018-01-01 01:00:01,010 [Module thread#3] info - unrelated log
2018-01-01 01:00:01,011 [Module thread#2] info - look for user some.one@email.com
2018-01-01 01:00:01,012 [Module thread#4] info - other unrelated log
2018-01-01 01:00:01,030 [Module thread#2] info - found record in DB
2018-01-01 01:00:01,034 [Module thread#2] info - checking user permission
2018-01-01 01:00:01,034 [Module thread#3] info - unrelated log
2018-01-01 01:00:01,040 [Module thread#2] error - user has no permission for this action
...
2018-01-01 01:00:15,123 [Module thread#2] info - another action starts reusing thread ID #2
...
When I search for some.one@email.com and get _time (2018-01-01 01:00:01,011) and thread ID (Module thread#2) using rex, I want to extract everything associated with thread#2 within a range of +/- 30-second window. But I don't want to extract the last line in the example above, which is part of another set of logs for a different action. So expected result should only contain line 1, 3, 5, 6 and 8.
One way I know is to get _time and thread ID and use them in a subsequent map search on the same log again with earliest=_time-30 and latest=_time+30 (roughly like this). But it would be tricky to get rid of the last line which falls into the window but is unrelated.
When googling around, someone mentioned transaction. So I tried to use transaction with maxspan=30s maxpause=5s. But it seems only returning lines with the keyword.
Can someone please suggest an efficient and accurate way to achieve this?
... View more