- Splunk Answers
- :
- Splunk Administration
- :
- Deployment Architecture
- :
- How much data I can store in Splunk for historical...
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
How much data I can store in Splunk for historical searches
When I am searching in Splunk for past one month's data, I am getting on 10 days of data. I guess rest of the data is being purged and my current configuration is storing only 8 days of data to search.
Can you please let me know where to look for setting custom values for number of days data being stored and volume that should be stored?
Please advice.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
how about keeping 1 petabytes?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
You need to understand (and this is a good reference source, plus the doco) how Splunk uses buckets, their relevance and map to your business needs for searching from Hot to cold and, ultimately, frozen. Check out the license model, 'all about indexing', compression, bucket use...and THEN ask about storage.
Br - sounds like you may be on an interesting journey!
Dave
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Andy - if you need to store 1PB or 1k TB then the Deployment Manual, http://docs.splunk.com/Documentation/Splunk/5.0.1/Installation/CapacityplanningforalargerSplunkdeplo... and others are your friend - as should be Splunk or a Splunk Partner to discuss strategy. Seriously don't miss out on that conversation because it will save you time, effort and money.
For sure you will not be buying a 10gig license / day (indexing rate) 😉
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
are you talking about indexed data ? because by default splunk keep data for around 6years and up to 500GB per index,but will stop indexing if not enough disk space.
You can modify this in indexes.conf and here more info:
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I would add to this - the default size of the index is 500GB. When the index fills, Splunk rolls off the oldest data. If you are adding about 50 GB/day to the index, it will only hold about 10 days of data.
If you have the disk space, you can simply make the index size bigger - that will fix your immediate problem. And yes, you can make your index enormous (petabytes are not uncommon) if you have the disk space. Splunk doesn't care. However, some of the other answers are good too - if you have that much data, you should think carefully about how to manage it.
Introducing the 2024 SplunkTrust!
Introducing the 2024 Splunk MVPs!
Splunk Custom Visualizations App End of Life
