Deployment Architecture

Edit hot/warm/cold data retentions

MAShawky
Explorer

Hello
I want to add below configuration to specific indexer
Hot/Warm/Cold Data retention 6 months 1.75TB
Frozen Data retention 6 months

configuration is

[myindex]
coldPath = $path\colddb
enableDataIntegrityControl = 0
enableTsidxReduction = 0
homePath = $path\db
maxTotalDataSizeMB = 1835008
thawedPath = $path\thaweddb
maxDataSize = 1835008
frozenTimePeriodInSecs = 15780000

but when am trying to add anew index i got error like below
The following issues were found with submitted configuration: stanza=myindexparameter=maxDataSize Value supplied='1835008' is illegal; default='750'

0 Karma

hettervik
Builder

Hi,

From the doc on indexes.conf

maxDataSize = <positive integer>|auto|auto_high_volume
* The maximum size in MB for a hot DB to reach before a roll to warm is
  triggered.
* Specifying "auto" or "auto_high_volume" will cause Splunk to autotune this
  parameter (recommended).
* You should use "auto_high_volume" for high-volume indexes (such as the
  main index); otherwise, use "auto".  A "high volume index" would typically
  be considered one that gets over 10GB of data per day.
* Defaults to "auto", which sets the size to 750MB.
* "auto_high_volume" sets the size to 10GB on 64-bit, and 1GB on 32-bit
  systems.
* Although the maximum value you can set this is 1048576 MB, which
  corresponds to 1 TB, a reasonable number ranges anywhere from 100 to
  50000.  Before proceeding with any higher value, please seek approval of
  Splunk Support.*
* If you specify an invalid number or string, maxDataSize will be auto
  tuned.
* NOTE: The maximum size of your warm buckets may slightly exceed
  'maxDataSize', due to post-processing and timing issues with the rolling
  policy.

Note the max size on maxDataSize is 1048576. maxTotalDataSizeMB and frozenTimePeriodInSecs are the parameters you need to control the data size of your index. See more on archiving data (rolling to frozen) here:

http://docs.splunk.com/Documentation/Splunk/7.0.0/Indexer/Automatearchiving

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...