Splunk Search

How do I remove unique IDs from error logs and stacktraces in a search in order to group and count similar errors?

adseros
Engager

Hi all,

I want to count similar errors and stacktraces in order to prioritize them.

I have a search that works in most cases:

index=ix_dis_appl_p loglevel="ERROR" | rex "ERROR](?<actionab>.*)" | stats dc(SessionID) by actionab

Some error lines have unique id's which makes counting similar errors hard. Example 1:

n.r.g.m.a.b.s.m.ProcessSpecificMessageBusiness Unexpected service fault for OrderIdentification [orderId=1-HRT5HT5, unitOfWorkId=1-Q4W06K0]. Returning empty resultset.

The values like 1-HRT5HT5 & 1-Q4W06K0 make every error unique. All these error lines are in the results with a count of 1.

Example 2: A stack trace that has ids on the first line :

. Returning empty resultset. nl.abcd.gict.mcv.access.mcvsrvc.retrieve_process_specific_texts.client.exception.NoResultsException: Application fault: Error retrieving order 1-NSFV1RA-1-NSHGNZC at

This part makes every error unique: 1-NSFV1RA-1-NSHGNZC

How can I remove the id's in order to get a count with results as below:

n.r.g.m.a.b.s.m.ProcessSpecificMessageBusiness Unexpected service fault for OrderIdentification [orderId=, unitOfWorkId=]. Returning empty resultset. | 4185
. Returning empty resultset. nl.abcd.gict.mcv.access.mcvsrvc.retrieve_process_specific_texts.client.exception.NoResultsException: Application fault: Error retrieving order at| 2585

I have been trying with rex, sed but I can't make anything work out. Hope someone can help.

0 Karma
1 Solution

sundareshr
Legend

If I understand right, you want to remove the "unique id" from _raw, right? This rex command will do it

| rex mode=sed field=_raw "s/(\[[^\]]+\])//g" | rex mode=sed field=_raw "s/(\d-\w+-\d-\w+)//g"

View solution in original post

sundareshr
Legend

If I understand right, you want to remove the "unique id" from _raw, right? This rex command will do it

| rex mode=sed field=_raw "s/(\[[^\]]+\])//g" | rex mode=sed field=_raw "s/(\d-\w+-\d-\w+)//g"

adseros
Engager

Thanks sundareshr! It works fine.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...