Splunk Search

Finding an Error in the middle of a Transaction

tyronetv
Communicator

I have a unique ID (RID) for a each client click. A single click can execute between 3 and, sometimes, over 100 log entries. Sometimes, in the middle of all of that is a situation where an error occurs.

I currently have a search that generates a list of actions by userid/RID but the report creates every line as an "INFO" response (I guess because the first and last line of the transaction are always INFO log level). What I need to do is set the corresponding report to acknowledge the transaction as an ERROR level based upon the ERROR existing someone in the middle of the transaction.

How would I do that?

2014-02-04 05:58:14,473 INFO [ndler-HTTPThreadGroup-289] RID=1391522013732-458538

2014-02-04 05:58:14,462 ERROR [ndler-HTTPThreadGroup-289] RID=1391522013732-458538

2014-02-04 05:53:33,776 DEBUG [ndler-HTTPThreadGroup-289] RID=1391522013732-458538

2014-02-04 05:53:33,760 DEBUG [ndler-HTTPThreadGroup-289] RID=1391522013732-458538

2014-02-04 05:53:33,760 INFO [ndler-HTTPThreadGroup-289] RID=1391522013732-458538

The above is for a client search request. The request timed out after 5 minutes. When I report on this transaction as a summary of actions taken by the client I currently report on the action and the duration. I would like to include the "Error_Level" as a column with a drill down to the actual transaction(RID) experiencing the error. But first, I just want to be able to identify it as an 'error' transaction.

Tags (2)
0 Karma
1 Solution

somesoni2
Revered Legend

I am assuming the log_level is the field containing INFO/ERROR/DEBUG, try this.

..your base search..| ..your transaction command on RID..| eval failFlag=mvfind(log_level,"ERROR") | eval HasError=if(isnull(failFlag),"No","Yes") |...your other reporting command including HasError field..

View solution in original post

somesoni2
Revered Legend

I am assuming the log_level is the field containing INFO/ERROR/DEBUG, try this.

..your base search..| ..your transaction command on RID..| eval failFlag=mvfind(log_level,"ERROR") | eval HasError=if(isnull(failFlag),"No","Yes") |...your other reporting command including HasError field..

kristian_kolb
Ultra Champion

Maybe not exactly what you're asking for, but a general optimization tip, especially if there are a large amount of transactions, and only a few of them has errors.

Instead of creating transactions and then looking for possible ERRORs, it might be better to do it the other way round:

index=blah sourcetype=bleh [index=blah sourcetype=bleh ERROR | dedup RID | fields + RID] | transaction RID

The subsearch will find the RIDs where there are ERRORs, and the outer search will build transactions based on them.

0 Karma

tyronetv
Communicator

Yeah. I understand this as a solution but more often then not (and shame on me training them as such) the users are used to seeing a sequential listing of the user's activity and I've been asked to mark the one that Error'd as part of the report. So, I find the user, then their RID's, and then generate a report. I need the report to show the ERROR on the line item of the users activity.

0 Karma
Get Updates on the Splunk Community!

Splunk APM: New Product Features + Community Office Hours Recap!

Howdy Splunk Community! Over the past few months, we’ve had a lot going on in the world of Splunk Application ...

Index This | Forward, I’m heavy; backward, I’m not. What am I?

April 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

A Guide To Cloud Migration Success

As enterprises’ rapid expansion to the cloud continues, IT leaders are continuously looking for ways to focus ...