Splunk Search

How to extract an unmapped field in the stats table from logs?

Nidd
Path Finder

I am trying to extract the value of an unmapped field from logs. I have logs where the status could either be ERROR or FATAL.

This way:

Type 1:

 [2/13/18 22:54:45:880 MST] 00000ea2 SystemOut     O ERROR: CustomerServices - [Log_CustomerService_FailedToInvokeMDMServer:] Failed to invoke MDM Server <?xml version="1.0" encoding="UTF-8"?>
    ... 36 lines omitted ...
                    <ErrorMessage>The following is required: Language or Locale</ErrorMessage>
                    <ErrorType>FVERR</ErrorType>
    ... 44 lines omitted ...
                    <ErrorMessage>The following is required: Language or Locale</ErrorMessage>
                    <ErrorType>FVERR</ErrorType>
    Show all 108 lines
    host = p01aml214 source = /env/IBM/logs/Custom01/SystemOut.log sourcetype = MDM:PROD:LOG

Type 2:

[2/13/18 22:54:45:990 MST] 00000ea2 SystemOut     O FATAL: CustomerServices - [Log_CustomerService_FailedToInvokeMDMServer:] Failed to invoke MDM Server <?xml version="1.0" encoding="UTF-8"?>
... 36 lines omitted ...
                <ErrorMessage>The following is required: Language or Locale</ErrorMessage>
                <ErrorType>FVERR</ErrorType>
... 44 lines omitted ...
                <ErrorMessage>The following is required: Language or Locale</ErrorMessage>
                <ErrorType>FVERR</ErrorType>
Show all 108 lines
host = p01aml214 source = /env/IBM/logs/Custom01/SystemOut.log sourcetype = MDM:PROD:LOG

I need the status table like:

source                                status   count
/env/IBM/logs/Custom01/SystemOut.log    ERROR   2
/env/IBM/logs/Custom01/SystemOut.log    FATAL   3

The count being the number of logs having ERROR/FATAL scenario.

I have no idea how to retrieve this unmapped field. Could someone please help?

Thanks.

0 Karma
1 Solution

FrankVl
Ultra Champion

Assuming those particular strings don't occur anywhere else in these events, you could try:

...your search... | rex "(?<status>ERROR|FATAL):" | stats count by source,status

You can make it a bit more robust by adding some more context around the capture group to make sure it doesn't capture occurences of these strings from elsewhere in the event if needed.

View solution in original post

FrankVl
Ultra Champion

Assuming those particular strings don't occur anywhere else in these events, you could try:

...your search... | rex "(?<status>ERROR|FATAL):" | stats count by source,status

You can make it a bit more robust by adding some more context around the capture group to make sure it doesn't capture occurences of these strings from elsewhere in the event if needed.

Nidd
Path Finder

Clear ! Thanks a lot @FrankVI.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...