How do I 'chain' events together where the common value is in different fields?

[johntobin]

Trying to solve a problem about ‘chaining’ events together.

Here’s a set of typical log lines - (in real life, A1, B2, C2, etc, etc, are all really long, random, unique token strings, not values that are actually related):

request token: proof_token=A1 new_token=A2
request token: proof_token=B1 new_token=B2
request token: proof_token=A2 new_token=A3
request token: proof_token=C1 new_token=C2
request token: proof_token=A3 new_token=A4
request token: proof_token=C2 new_token=C3

If I group them, I get three groups, in that these groups of events all 'chain' together (not because they share the same A, B, and C values in their tokens):

request token: proof_token=A1 new_token=A2
request token: proof_token=A2 new_token=A3
request token: proof_token=A3 new_token=A4

request token: proof_token=B1 new_token=B2

request token: proof_token=C1 new_token=C2
request token: proof_token=C2 new_token=C3

I want to be able to create and report on these groupings of events, ‘chaining’ them together, in Splunk. The first group count of events is 3, the second one is 1, the third one is 2. I would like to add those up, divide by 3, and get an average of the number of times someone is using their first token and keeping the ‘chain’ of token requests going (in this case, the average is 2).

Is there a way to do this? Even the transaction command doesn’t seem to do the trick here? Is there a stats or streamstats trick I can use?

Any help appreciated. Thanks.

[johntobin · Accepted solution]

We resolved the issue using a result on this answer. Essentially we joined the two linking fields together into a new temporary field using eval, with a delimiter between the two, and then used makemv to put those two fields back into a single multi value field (splitting over the delimiter we picked). This meant we had all the 'joining' or 'chaining' fields in a single field value, so we could simply run the transaction command over that field.

There may be other or better ways to do this. I was wondering about a new command, like transaction, but which takes multiple fields. This made me read the doc for the transaction command again and it looks like it does take multiple fields. In fact, in reading the answer that I link to, I see there's an update that specifies just this. Probably a change to the transaction command in the past fixed this.

Perhaps this wasn't an issue after all, and this 'solution' is overkill.

[ramdaspr]

Sorry, I dont have an answer but you could use this answer as a start to recursively link fields