Getting Data In

How to restrict Folder Ingestion based on timestamp?

bteele
New Member

We're collecting text files into Splunk from numerous servers, but due to the growing size of the parent directory, it's causing CPU to be pegged, as Splunk seems to be continuously reading all of the files.

How can I modify the input to only check for items in today or yesterday's folders? It's a parent directory, followed by dated folders for each day, which contains anywhere from one to dozens of uniquely named files.

The current input is as follows:

[monitor://C:\Windows\Logs\(ParentFolder)]
index = (indexName)
disabled = 0
0 Karma
1 Solution

tiagofbmm
Influencer

Hi

Think of exploring that parameter on the monitor stanza

ignoreOlderThan = <nonnegative integer>[s|m|h|d]
* The monitor input will compare the modification time on files it encounters
  with the current time.  If the time elapsed since the modification time
  is greater than this setting, it will be placed on the ignore list.
* Files placed on the ignore list will not be checked again for any
  reason until the Splunk software restarts, or the file monitoring subsystem
  is reconfigured.  This is true even if the file becomes newer again at a
  later time.

View solution in original post

0 Karma

tiagofbmm
Influencer

Hi

Think of exploring that parameter on the monitor stanza

ignoreOlderThan = <nonnegative integer>[s|m|h|d]
* The monitor input will compare the modification time on files it encounters
  with the current time.  If the time elapsed since the modification time
  is greater than this setting, it will be placed on the ignore list.
* Files placed on the ignore list will not be checked again for any
  reason until the Splunk software restarts, or the file monitoring subsystem
  is reconfigured.  This is true even if the file becomes newer again at a
  later time.
0 Karma

bteele
New Member

Thanks! This worked for me.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...