Splunk Search

How to create a base search that retains multiple regex fields?

kmaron
Motivator

I have a dashboard that is built from 3 different searches. They all come from the same data so I would like to turn them into a base search for the page. However, each one of them has a different regex to pull out a field and I can't figure out how to combine them without losing those fields.

These are my current searches:

index=foo source=bar host=PRD* CIWEB AND Error 
 | rex field=_raw "CIWEB\.(?<PluginName>.*?Plugin)"              
 | timechart span=1h count(PluginName) by PluginName

index=foo source=bar host=PRD* CIWEB AND Error 
 | rex field=_raw "\sE\_(?<ErrorType>.*?):"  
 | timechart span=1h count(ErrorType) by ErrorType

    index=foo source=bar host=PRD* CIWEB AND Error 
 | rex field=_raw "\.(?<ExceptionName>\w*?Exception)" 
 | timechart span=1h count(ExceptionName) by ExceptionName
1 Solution

woodcock
Esteemed Legend

Your base search is this:

index=foo source=bar host=PRD* CIWEB AND Error 
| rex "CIWEB\.(?<PluginName>.*?Plugin)"              
| rex "\sE\_(?<ErrorType>.*?):"  
| rex "\.(?<ExceptionName>\w*?Exception)" 
| multireport
   [ timechart span=1h count(PluginName)    BY PluginName    | untable _time PluginName    count]
   [ timechart span=1h count(ErrorType)     BY ErrorType     | untable _time Errortype     count]
   [ timechart span=1h count(ExceptionName) BY ExceptionName | untable _time ExceptionName count]

Then you make each post-process one of these:

fields _time PluginName count | xyseries _time PluginName count

OR:

fields _time Errortype count | xyseries _time Errortype count

OR:

fields _time ExceptionName count | xyseries _time ExceptionName count

View solution in original post

woodcock
Esteemed Legend

Your base search is this:

index=foo source=bar host=PRD* CIWEB AND Error 
| rex "CIWEB\.(?<PluginName>.*?Plugin)"              
| rex "\sE\_(?<ErrorType>.*?):"  
| rex "\.(?<ExceptionName>\w*?Exception)" 
| multireport
   [ timechart span=1h count(PluginName)    BY PluginName    | untable _time PluginName    count]
   [ timechart span=1h count(ErrorType)     BY ErrorType     | untable _time Errortype     count]
   [ timechart span=1h count(ExceptionName) BY ExceptionName | untable _time ExceptionName count]

Then you make each post-process one of these:

fields _time PluginName count | xyseries _time PluginName count

OR:

fields _time Errortype count | xyseries _time Errortype count

OR:

fields _time ExceptionName count | xyseries _time ExceptionName count

gemrose
Explorer

Hello,
Looking into the solution I am facing an issue when I do base-search. When I use the regex in SPL code and when it gets converted to xml. My code is not working. If I change my xml code I get unvalidated tag . Is there a way to get it working in base search

Example:

IN SPL code:

| rex field="log.mess" ".*\"Category\":\"(?<Category>[^\"]+)"

In xml:

| rex field="log.mess" ".*\"Category\":\"(?&amp;lt;Category&amp;gt;[^\"]+)"
Tags (1)
0 Karma

somesoni2
SplunkTrust
SplunkTrust

I didn't knew of multireport command, don't see in the documentation as well. Thanks

0 Karma

cmerriman
Super Champion

multireport isn't in documentation. I've brought it up to the documentation team and there is a ticket with them and the engineers. it isn't fully tested out on every aspect of how it works yet.

0 Karma

somesoni2
SplunkTrust
SplunkTrust

Based on my brief testing, it runs all those timecharts (or any other aggregation command that you put in) one by one and appends the results together, making it ideal for base searches.

0 Karma

cmerriman
Super Champion

right. I sent @cpride_splunk my use case for it after conf last year. mine was to basically to create summation rows based on different fields. for instance:

| multireport
    [ stats count by PluginName]
    [ stats count by ErrorType PluginName]
    [ stats count by ExceptionName ErrorType PluginName]

would add summary row counts for each by statement. not necessarily for this data, but something similar to how i used it.

0 Karma

kmaron
Motivator

This worked perfectly! Thank you woodcock!

0 Karma

somesoni2
SplunkTrust
SplunkTrust

Do all events have those fields that you're extracting?

0 Karma

cmerriman
Super Champion

you're base search could just be:

index=foo source=bar host=PRD* CIWEB AND Error 
  | rex field=_raw "CIWEB\.(?<PluginName>.*?Plugin)"       
 | rex field=_raw "\sE\_(?<ErrorType>.*?):
| rex field=_raw "\.(?<ExceptionName>\w*?Exception)" 

with each panel having a query:

 | timechart span=1h count(PluginName) by PluginName

 | timechart span=1h count(ErrorType) by ErrorType

| timechart span=1h count(ExceptionName) by ExceptionName

kmaron
Motivator

I got that far but I thought a base search had to have a stats function? As soon as I add that it breaks.

0 Karma

woodcock
Esteemed Legend

You are correct; see my answer.

0 Karma

cmerriman
Super Champion

try adding |table *

0 Karma

somesoni2
SplunkTrust
SplunkTrust

Or better (keep only what you need)

| table _time PluginName ErrorType ExceptionName
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...