i have the following failures in the logs that i need to capture and show as timechart count by the type of errors , in a single dashboard .
Need help with framing the Query
UploadFile : Processing failed:
UploadFile : screen_error='Metadata file Transfer failed for'
UploadFile : status='failed', details='Metadata FTP failed. There is an orphan PDF on the system,
Caused by: java.lang.IllegalStateException: failed to connect
Caused by: java.lang.IllegalStateException: failed to create SFTP Session
SFTPServiceImpl : Failed to send file:
P.S ;- Just starting with splunk and having difficulty understanding splunk regular expressions . Need some links to interactive tutorials.
Hi yannK
i do not have a strong base search , so i would like to try the first option .
But what do you mean the ? in the rex expression
| rex "screen_error='(?Metadata file Transfer failed for)'"
| rex "Caused by: java.lang.IllegalStateException: (?failed to connect)"
Can you help me with some examples ?
The ( ) are the matching group, the fieldname between greaterthan/lessthan after the question mark, is a way to name the field to extract on the fly.
see http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Rex for examples.
Do you want to extract the "type of error" from the event strings directly ?
In that case, you can show us the final expected results.
Or are you trying to normalize the type of errors ?
Or do you want a simple overall count of errors for all those possible messages ?
in this case, a simple list of OR conditions can be enough.
Example of regex , plus normalization for :UploadFile : screen_error='Metadata file Transfer failed for'
You can do
index=myindex source=mysource "screen_error" OR "java.lang.IllegalStateException"
| rex "screen_error='(?<errors_string>Metadata file Transfer failed for)'"
| rex "Caused by: java.lang.IllegalStateException: (?<error_string>failed to connect)"
| eval error_type=case(
match(error_string,"Metadata"),"Metadata error",
match(error_string,"connect"),"Connection error",
1=1, "unknown error")
| timechart count by error_type
You can add more case options, and more rex extractions.
Remember that the rex will be successively extracted, to is a string match several rex, the last one will prevail.
PS : If you have a strong base search. You could use the "case" directly on the _raw field.
index=myindex source=mysource "screen_error" OR "java.lang.IllegalStateException"
| eval error_type=case(
match(_raw,"Metadata"),"Metadata error",
match(_raw,"failed to connect"),"Connection error",
1=1, "unknown error")
| timechart count by error_type