Alerting

Generate alert listing broken Field extractions

splunk_zen
Builder

We get fed quite a wide range of formats.
This makes it hard for us to keep track of log formatting changes.

From time to time we get mission critical search alerts which rely on Field extractions

stopping to work to work because a line format has changed.

Those searches do start generating:

error in 'eval' command: The expression is malformed

so I'll be grateful if someone suggests a search picking on that,

but is it possible to limit it to Field extractions/scheduled alerts ?
We'll inevitably get spammed everytime a user makes a typo/is testing a new search otherwise.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

If your critical logs each have a known regex-testable format you could define alerts that fire if events are indexed that don't match their respective regex.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

Field extractions in and of themselves don't produce errors when the events don't match... they just don't match, it's normal behaviour.

If your fields are all required you could create an alert for events that don't have all fields, ie must have a changed format.

splunk_zen
Builder

Thank you Martin, but wouldn't that double the maintenance overhead?
We'd essentially have to clone the Field extrations expresssions and define search alerts.
Isn't it possible to define a single search alert which would catch all Field extractions erroring out ?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...