Getting Data In

2 easy questions about indexes.conf

tkwaller
Builder

Somehow our default time changed from 30 days to ~6 years and going though indexes.conf in $SPLUNKHOME/etc/system/local and it seems that none of the index stanza contain a setting for frozenTimePeriodInSecs so it defaulted to ~6 years. SO I went though and added the line for frozenTimePeriodInSecs = 2592000 to freeze after 30 days.

My questions are:
1. This will delete/drop data older than 30 day correct?
2. Is there any other impact to doing so?

0 Karma
1 Solution

vasanthmss
Motivator

Hi tkwaller,

1. yes. in default data will be deleted. if you want you can configure to keep older buckets.
2. I guess there will not be any impact until you don't want to see the data more than 30 days old.

Here are few points about index.

The different stages of an index may all have a specific location; this is how you can spread your data on different volumes.

1. homePath location for the Hot and Warm buckets
2. Hot (intensive read and write, this is where the indexing occurs)
3. Warm (mostly read, and optimization)
4. coldPath location for the Cold buckets (moved once, then read, used for searches only)
5. thawedPath location for Thawed buckets (used only if you want to re-import frozen buckets)
6. There is no Frozen location defined in Splunk, because the default action is to delete them.

Check this post, https://wiki.splunk.com/Deploy:BucketRotationAndRetention

Question for you, What do you mean Somehow our default time changed from 30 days to ~6 years?
Are you saying default time meaning search time in the GUI or particular index's retention policy?

Thanks,
V

V

View solution in original post

vasanthmss
Motivator

Hi tkwaller,

1. yes. in default data will be deleted. if you want you can configure to keep older buckets.
2. I guess there will not be any impact until you don't want to see the data more than 30 days old.

Here are few points about index.

The different stages of an index may all have a specific location; this is how you can spread your data on different volumes.

1. homePath location for the Hot and Warm buckets
2. Hot (intensive read and write, this is where the indexing occurs)
3. Warm (mostly read, and optimization)
4. coldPath location for the Cold buckets (moved once, then read, used for searches only)
5. thawedPath location for Thawed buckets (used only if you want to re-import frozen buckets)
6. There is no Frozen location defined in Splunk, because the default action is to delete them.

Check this post, https://wiki.splunk.com/Deploy:BucketRotationAndRetention

Question for you, What do you mean Somehow our default time changed from 30 days to ~6 years?
Are you saying default time meaning search time in the GUI or particular index's retention policy?

Thanks,
V

V

tkwaller
Builder

Hello
By default time changed I mean within indexes.conf, specifically frozenTimePeriodInSecs, meaning someone probably change the conf file.

I went through all of that documentation prior to posing the question. I was really just looking if maybe I missed something I didn't think about. I put the config in a virtual environment to test it and it seems to have fixed most of my issues.

I do however have 1 question:
In the DMC under Index Detail: Instance
It tells you data age vs frozen age. I have many indexes that say something like 94/30. I have all indexes set to frozenTimePeriodInSecs = 2592000 why would data age be over still?

0 Karma

somesoni2
SplunkTrust
SplunkTrust

The retention period is applied at the data bucket level, not at event level. A data bucket is deleted/ rollever to frozen when the latest event in the bucket is older than retention period. So, for some sourcetypes, you may still see older data available as the corresponding bucket's latest event is not older than retention period.

Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...