Hi,
I have this search:
eventtype=mlc sourcetype=murex_log4j source=launchermxmlc.mxres.log | stats earliest(_time) as start | appendcols [ search eventtype=mlc sourcetype=sun_jvm service_name=mxmlc.gc.log | table relative_time after_gc max_gc ] | eventstats max(start) as start_time | eval _time = start_time + relative_time | fields - start - start_time - relative_time | timechart fixedrange=false bins=2000 max(after_gc), max(max_gc)
The idea is that the GC log doesn't have a time stamp, only the relative time since the start of the log, so I need to get the start time from a different log before I can make the timechart. This works perfectly when I set span=x
, but not with bins=x
. When I set bins=2000, all the data goes into a single bucket, which doesn't make any sense to me. I need it to work with bins because there may be days worth of data, so span=1s would give too many points to display.
Can anyone help please?
Interesting. Can you replace the timechart
command with just table _time after_gc max_gc
, rerun, export those rows and either paste them here or put them on pastebin? Sounds like some interesting behavior in timechart, quite possibly a bug, and I'd like to reproduce.
Sure, this is the result
_time after_gc max_gc
2016-05-09 10:56:11.265 242241 478464
2016-05-09 10:54:59.594 241847 467712
2016-05-09 10:53:48.860 245289 456448
2016-05-09 10:53:08.290 212081 452608
2016-05-09 10:52:42.023 187632 454656
...
For 33 rows
I think this search may have once made sense, but that it has lost it's way recently. I'm going to walk through it pipe by pipe.
In particular, I think the problem is that all the search language before the timechart command can only ever produce a single row and a single _time value for timechart to chart.
Here are the first two commands.
eventtype=mlc sourcetype=murex_log4j source=launchermxmlc.mxres.log
| stats earliest(_time) as start
As always when a stats has no group-by field, this will produce only a single output row. that row has one field named "start".
the next command will add two fields to that single row, but it will not add any rows.
| appendcols [ search eventtype=mlc sourcetype=sun_jvm service_name=mxmlc.gc.log | table relative_time after_gc max_gc ]
The next command is a strange-looking way to create a new field called "start_time" and set it to the value of "start". If there were ever more than one row here, it would make more sense, so I suspect at one point the earlier clauses were quite different.
| eventstats max(start) as start_time
Then there follow an eval and a fields clause, neither of which can ever create more than our one row. So when the timechart clause receives its input rows, there will only be one value coming in, and only one value of "_time".
Hi, thanks very much for giving a reply, but one point I would make is that the query does work when I set span=1s. And if I take off the timechart command altogether, leaving me with this query:
eventtype=mlc sourcetype=murex_log4j source=launchermxmlc.mxres.log | stats earliest(_time) as start | appendcols [ search eventtype=mlc sourcetype=sun_jvm service_name=mxmlc.gc.log | table relative_time after_gc max_gc ] | eventstats max(start) as start_time | eval _time = start_time + relative_time | fields - start - start_time - relative_time
I end up with 33 rows of results; the appendcols has provided the extra rows.
Oh right. Sorry I forgot that appendcols will happily append extra rows if the base search has fewer rows. It's a strange way to do that. By way of apology, here is your search refactored as a disjunction without the appendcols:
eventtype=mlc (sourcetype=murex_log4j source=launchermxmlc.mxres.log ) OR ( sourcetype=sun_jvm service_name=mxmlc.gc.log )
| eval log4j_start=if(sourcetype="murex_log4j",_time,null())
| fields log4j_start relative_time after_gc max_gc
| eventstats min(log4j_start) as start_time
| eval _time = start_time + relative_time
| fields - start - start_time - relative_time
| timechart fixedrange=false bins=2000 max(after_gc), max(max_gc)
Cool, I've never seen it done like that - thanks!
Hi @johnraftery
If the answer by @sideview solved your question, please don't forget to resolve the post by clicking "Accept" directly below his answer. Also, be sure to upvote his answer and/or comment that helped you out.
Actually this particular answer+comment thread didn't resolve his issue - That seems to be a bug or at least a strange convention in timechart. I just refactored his search a bit as a side matter.