Getting Data In

How do you limit the amount of time that splunk looks into the past for log data?

Whitefoc
Explorer

I am trying to limit the windows event logs being pulled to 180 days instead of all logs from two years ago?

Tags (2)
1 Solution

jbsplunk
Splunk Employee
Splunk Employee

The setting your looking for is MAX_DAYS_AGO in props.conf:

http://www.splunk.com/base/Documentation/latest/admin/Propsconf

MAX_DAYS_AGO = <integer>
* Specifies the maximum number of days past, from the current date, that an extracted date
  can be valid.
* For example, if MAX_DAYS_AGO = 10, Splunk ignores dates that are older than 10 days ago.
* Defaults to 2000 (days).
* IMPORTANT: If your data is older than 2000 days, increase this setting.

So if in your props.conf in your $SPLUNK_HOME/etc/system/local/ folder you put something like this, where the stanza is the source/sourcetype/host, it should do the trick:

[stanza]
MAX_DAYS_AGO = 180

View solution in original post

sideview
SplunkTrust
SplunkTrust

I think there's a big ambiguity here.

1) Are you trying to get Splunk to not even index those files?

-- the answer here would be MAX_DAYS_AGO

2) Or are you trying to get Splunk to index them but never assign a timestamp older than 180 days?

-- im actually not sure if this is possible

3) Or do you want the data to get thrown out of the index after 180 days?

-- you could do this by setting a really tight retention policy on your index(es) http://www.splunk.com/base/Documentation/latest/admin/HowSplunkstoresindexes

4) Or do you just want the searches to be faster and you only ever search on the most recent 180 days anyway?

-- The answer here is basically just to set a narrower time range when you search. If the TimeRangePicker module is saying 'All time', then your searches will take much longer to complete than if it's selected to only search the "last 60 minutes". Because time range effects your search speed so dramatically, it is important to always pay attention to the time range controls. Particularly when all you care about is the first page of events anyway.

And depending on what kind of app this is and what kind of views you're using or what your use case is, you might also want to change the default time ranges involved.

Another trick if you're just searching raw events, is to just tack " | head 100" onto the end of your search. This will make Splunk only return the most recent 100 events for the search.

sideview
SplunkTrust
SplunkTrust

MAX_DAYS_AGO will be an imperfect solution at best if you just want to make your searches faster. You're much better off simply restricting the time range of your searches and reports.

0 Karma

Whitefoc
Explorer

Okay. My answer is number 4.

0 Karma

jbsplunk
Splunk Employee
Splunk Employee

The setting your looking for is MAX_DAYS_AGO in props.conf:

http://www.splunk.com/base/Documentation/latest/admin/Propsconf

MAX_DAYS_AGO = <integer>
* Specifies the maximum number of days past, from the current date, that an extracted date
  can be valid.
* For example, if MAX_DAYS_AGO = 10, Splunk ignores dates that are older than 10 days ago.
* Defaults to 2000 (days).
* IMPORTANT: If your data is older than 2000 days, increase this setting.

So if in your props.conf in your $SPLUNK_HOME/etc/system/local/ folder you put something like this, where the stanza is the source/sourcetype/host, it should do the trick:

[stanza]
MAX_DAYS_AGO = 180

jbsplunk
Splunk Employee
Splunk Employee

You may need to create one, but it usually goes into $SPLUNK_HOME/etc/system/local/ or $SPLUNK_HOME/apps/search/local/

0 Karma

Whitefoc
Explorer

Where is the props.conf location?

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...