Alerting

Generate alert listing broken Field extractions

splunk_zen
Builder

We get fed quite a wide range of formats.
This makes it hard for us to keep track of log formatting changes.

From time to time we get mission critical search alerts which rely on Field extractions

stopping to work to work because a line format has changed.

Those searches do start generating:

error in 'eval' command: The expression is malformed

so I'll be grateful if someone suggests a search picking on that,

but is it possible to limit it to Field extractions/scheduled alerts ?
We'll inevitably get spammed everytime a user makes a typo/is testing a new search otherwise.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

If your critical logs each have a known regex-testable format you could define alerts that fire if events are indexed that don't match their respective regex.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

Field extractions in and of themselves don't produce errors when the events don't match... they just don't match, it's normal behaviour.

If your fields are all required you could create an alert for events that don't have all fields, ie must have a changed format.

splunk_zen
Builder

Thank you Martin, but wouldn't that double the maintenance overhead?
We'd essentially have to clone the Field extrations expresssions and define search alerts.
Isn't it possible to define a single search alert which would catch all Field extractions erroring out ?

0 Karma
Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Built-in Service Level Objectives Management to Bridge the Gap Between Service & ...

Wednesday, May 29, 2024  |  11AM PST / 2PM ESTRegister now and join us to learn more about how you can ...

Get Your Exclusive Splunk Certified Cybersecurity Defense Engineer Certification at ...

We’re excited to announce a new Splunk certification exam being released at .conf24! If you’re headed to Vegas ...