Extract particular field value and assign to a variable to be used for further searches

[jeni]

Hi

I am trying to do the following.

I have to prepare a report which contains the TransactionId, servername, some other fields values for a transaction with encoutered a particular error.

i am able to get the part of transaction which has the error message, but not able to get teh complete trace of that transaction from which i have to get some other values for my report.

Could anyone help me in this regard. Thanks

---

for example: to capture all transactions and comeplete details of the transactions that has errored with error message "xyz"

i tried to search on error message "xyz" and for example in 5th step if it has errored - when i tried to search on the error message it will give me only the 5th part - but from that i can get the transaction id (thread id ) as it remains the same for the complete transaction (all teh steps). using rex command i got the transaction.

Now how i need use this transactionid to get the complete trace of that particular transaction from the beginning instead of getting the error message part alone.

[Lowell · Accepted solution]

There are two approaches that you could take. These have a very different performance implication if you have lots of events, but if you don't then either one should be pretty fast.

Transaction search:

sourcetype=my_source_type ... | transaction fields="host,TransactionId" | search "my-xyz-error-message"

This approach will first group all of your events into individual transaction events based on unique combinations of host and TranscationId. You may also want to look into the other transaction options as well, such as maxspan which is helpful to limit the overall time range of your transactions, which is especially important if TransactionId repeats.

You can make this approach work better if you can filter out unwanted messages in your base search. For example, if you have many known-unhelpful messages (such as debug messages or startup messages that can be ignored) you can save resources (and process a larger number of transactions) if you can filtering out a large number of unwanted messages.

Use a sub-search:

sourcetype=my_source_type [ search sourcetype=my_source_type ... "my-xyz-error-message" | fields + host, TransactionId | fields - _* | format ]

This approach will take the inner-search (This is a "subsearch" which is looking for any TransactionId and host combination that has your error message) and builds a dynamic search for the combinations that are known to have this error. (Note there is a 100 event limit on the on sub-searches which may prevent this from being a valid option for you.)

With this search, events will be show individually. If you prefer the combined transaction look you can do that here too. Simply stick ` | transaction fields="host,TransactionId" at the end of this search.

---

For other tips and tricks, I recommend the following:

• Browse around on this (answers.splunk.com) site and the docs.
• Check out the commands shown on the Search command cheat sheet

[Lowell]

Yes, there are probably other ways to do this too; you could write your own custom search command if you really wanted to; however these are probably the two best approaches to use in this situation. You may also be able to pull some stuff together grouping values together using stats, but only if TransactionId doesn't repeat. As for other tips, I'll stick some more stuff in my answer; which is easier to read than putting it into comments.

[Lowell]

If you have more than 100 results in your sub-search (the one in square brackets) then the results will be truncated. Therefore I suggest you use the first search approach or use a smaller time window.

[jeni]

When i was doing my search and my search critera has 222 results.
During search i got this error message...

Any one please explain why the error message is thrown

[subsearch]: The format command is using only the first 100 (of 130) results

[jeni]

Thank you so much Lowell for your tips and help.

I tried something like this and i got the output i needed.

---

[ search "my_error_message" | fields + TransactionId | fields - _* | format ] | search my_other_operation | sort _time | xmlkv | fields + TransactionId, FieldB, FieldC | stats values(*) by TransactionId

i got the fields "TransactionId, FieldB, FieldC" grouped by TransactionId.

---

but also i request to let me know if there is any other way and more tips.

Once again thanks

[Lowell]

Also an example of the search that you are using would be helpful. (Please use the "edit" link to add that info to your question.)

[Stephen_Sorkin]

Could you include a small data sample?