Deployment Architecture

Can we use object storage for storing data for cold buckets( as cold storage)

prachisaxena
Explorer

Hello,

I am looking for cold storage options for Splunk of longer term data rentention.

Can we use object storage for it ?
Has anyone tried testing this earlier?

Splunk version is 7.1.0 with ITSI.

Tags (1)
0 Karma

kzschach
New Member

Yes you can use Western Digital ActiveScale Object Storage https://blog.westerndigital.com/splunk-smartstore-supercharge-new-splunk-architecture/

0 Karma

prachisaxena
Explorer

Hi deepashri_123, thanks for replying .. yes I have gone through these answers. Now a days the Object storage vendors provide some type of connectors such as for NFS (some details below). I wanted to know if someone has tested this or would it work with Splunk since it may be just transparent to Splunk.

Scale-out File and Object Storage
• Amazon S3-compatible REST API with support for Microsoft Active Directory, AWS IAM, AWS Signature v2 and v4
• Scality HTTP REST API
• Scale-out NFS v3 with support for Kerberos Authentication, quotas.
• Scale-out SMB 2.0, 3.0
• Linux FUSE plus data compatibility with REST
• S3/NFS portability
• Scalable peer-to-peer RING architecture, with a native object storage core
• Integrated Scale-out File-System (SOFS) with POSIX semantics

0 Karma

mkamal18
New Member

Hello,

I am increasing the retention delay today by using the frozenTimePeriodInSecs option in indexes.conf.
if you want 6 months for example: you convert 6 months in seconds and you add a coldb and a homedb. Splunk will do the rest in order to dispatch the buckets in the cold and the warm directories. You can also change the MaxbucketSizeinMB to differentiate between hot, warm and cold buckets

0 Karma

prachisaxena
Explorer

@mkamal18 Thank you so much for replying .. I am looking more on the hardware required for cold storage rather than the configuration to move data between buckets.

0 Karma

deepashri_123
Motivator
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...