I've got data that looks (functionally) like this:
Event 1 contains String-A Field-X
Event 2 contains String-B Field-X Field-Y
Event 3 contains String-C Field-Y
I'm trying to correlate these three events together. I want to create a single transaction so I can calculate the duration between Event 1 and Event 2.
After some trial and error I came up with this query:
String-A OR (String-B Field-Y=*) | transaction Field-X | append [search String-C] | transaction Field-Y
That works, but it seems really ugly to me, and I'm wondering if I'm missing some better way to do this. Ideally I'd also like to be able to find examples of Event-1 with no matching Event-3, but I think I can do that with transaction as well, using keepevicted=1.
Any suggestions for a better path to my desired results?
Edit:
Ok, maybe I boiled down my example too much. Here's the original data:
Event 1:
2,51216,117,117,626,1284525367018,4172,6876,17:reqid=-1284474701,0:,0:,0,(49|A136:Msg: type = 39 BRMWroteMsg: backupid=us-pghdbp1.ariba.com_1284523604, KBytes written=3444, End of Data=1, KBytes per sec=20935.5, copy=0|)
This is a Symantec NetBackup job completing. The key pieces of Event 1 are "End of Data=1" (There were lots of records before this with that value as 0), and the backupid. I'm extracting the backupid as a field.
Event 2:
0,51216,111,111,60899,1284525448888,7992,7172,0:,203:
ImageCopyExt_Record => {
MasterServerName => hubmaster
BackupID => us-pghdbp1.ariba.com_1284523604
CopyNumber => 2
JobID => 2698
LifecycleSourceTag => 1
},33:ImageCatalogImpl::updateImageCopy,1
This is Symantec scheduling the data in this backupid as part of a replication job between storage arrays. They may be multiple of these entries, for multiple backup objects all grouped together in one replication. The key elements I've identified are "ImageCopyExt_Record", the backupid (same as event 1) and the jobid (will match event 3). Other events exist with the search string and the backupid but without a jobid.
Event 3:
2,51216,116,116,5,1284526119527,6788,7860,10:jobid=2701,0:,0:,0,(14|A4:NONE|A10:SLP_HUBSLP|A3:Dup|A1:0|A50:the requested operation was successfully completed|)
This is Symantec logging that the replication completed. Here we see the matching jobid from event 2, 'A3:Dup' means this is about a replication event, and the string says completed.
So the query I had built was:
("End of Data=1" OR (ImageCopyExt_Record netbackup_jobid=*)) | transaction netbackup_backupid | append [search index=backups A3:Dup completed] | transaction netbackup_jobid
I'm grabbing event1 and event 2 and transactioning them on the backupid, then append to that the event3 records and transactioning again. I'm sure there is a better way.
... View more