Knowledge Management

Configure Splunk to collect data around a spesific occurrence

avitallange
Explorer

Hi,

Is it possible to configure Splunk so that if an error trace occurs, it will start collecting info traces around the error?
For example: error trace occurs at 1:00PM. at 1:00 PM Splunk will start to collect info traces since 12:45 PM till 1:15 PM.
Errors and info can be saved in distinct files.
I am asking because the info traces are too noisy and we would like to collect them according specific needs.

Thanks,
Avital

Tags (1)
0 Karma

avital
Explorer

The concern is about unnecessarily consume your licence

0 Karma

mbenwell
Communicator

Is the concern about noise because it will unnecessarily consume your licence?

Or are you more interested in how to construct a search query to do the above?

0 Karma

Gilberto_Castil
Splunk Employee
Splunk Employee

Nope. This is not something that can be done on demand. The main reason that a monitor input is designed to read data from a file or files and keep telemetry on the read position in that file. This is what allows the monitor input to perpetually tail a file. This type of operation is done without any analysis of the data being read. It is just a blind consumption based on the read position on the file.

It is important to understand that for a monitor input to read a specific chunk of data, there must be a record of the seek pointer to where the data starts. To accomplish the rest of the task, it would also be required to know where in the file the data ends. However, this is not how the read functionality is designed. Each file is read from the beginning to the end and the data is entered into the input pipeline accordingly.

There are some absolutely mind-altering reads on the fishbucket in the blogs. If you need to dig more on the process, you may want to do a bit more digging starting here.


The only advice that may be applicable here is to have your application create rotational logs for shorter periods of time… say one hour. The train of thought is to have an alert which will trigger the activation of a monitor input which will capture only the active log. The activation can be done using the Deployment Server and a Splunk CLI command through a scripted alert, remote SSH, or whatever technical telepathy that suits your skills.

You would also need to know when to shut that special monitor off.

This path, however, will require you to test the procedure before you go and change the way your application creates its logs.

billford
Path Finder

I think you could do this with an alert, script and the sdk. I don't think there's any native way to do it but I'd be thrilled to be wrong.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...