Here is the query I am currently using. The idea is to find the duration of the event within the subsearch then run a join in order to retain the _time aspect of the BEGIN messages for use as the starting point in the timeline visualization. The query is set to produce a timeline visualization for whatever "abcd" is. It works fine for general use but runs into a problem in a specific situation.
index=default source=mysql-pipe sourcetype=pipeline_logs AND run_id="abcd" AND msg_type!="COMPLETE"
| join type=left message [search index=geniachip source=mysql-pipe sourcetype=pipeline_logs AND run-id="abcd"
| eval time=strptime(time_at,"%Y-%m-%d %H:%M:%S")
| chart values(time) by message, msg_type
| eval TimeDiff = (COMPLETE-BEGIN) * 1000]
| fillnull value=0 TimeDiff
| table _time message TimeDiff
The table output looks like this (when adding COMPLETE and BEGIN to the table part of the query)
_time message COMPLETE BEGIN duration
2016-12-11 14:35:16 bank02 per-node-processing 1481537943.000000 1481510388.000000 15271000.000
In certain cases the event contained in the message occurs twice during the run "abcd" meaning that there are two timestamps as shown below. If it occurs twice it means that either both times it completed successfully or one failed to complete while the other didn't. In this situation the eval duration does not function correctly and the column doesn't contain values.
_time message COMPLETE BEGIN duration
2016-12-11 14:35:16 bank02 per-node-processing 1481537943.000000 1481510388.000000
1481524516.000000
2016-12-11 10:42:49 bank03 per-node-conversion 1481524291.000000 1481537710.000000 1481510569.000000 1481524996.000000
Is there a way to break up the two timestamps and essentially create two different events? Building off the previous table it would look like this instead
_time message COMPLETE BEGIN duration
2016-12-11 14:35:16 bank02 per-node-processing 1481537943.000000 1481510388.000000 15271000.000
2016-12-11 10:39:48 bank02 per-node-processing 1481510447.000000
2016-12-11 10:42:49 bank03 per-node-conversion 1481524291.000000 1481537710.000000 34442000.000 2016-12-11 10:43:52 bank03 per-node-conversion 1481510569.000000 1481524996.000000 34694000.000
Have a look at mvexpand example1 here which might not be exactly what you need but will definitely create a new event for every value within the multivalued field. Try something like:
your base query
| mvexpand BEGIN
| table the values here
This will create one row in table for each of the values the multivalue field BEGIN has, keeping all the others intact.