Getting Data In

Storage question

attgjh1
Communicator

Where does the indexed data gets stored?
Do they somehow get reduplicated when the logs are indexed?

i.e. I'm planning to use splunk to monitor up to 5gig of data daily. All the logs are sent to a server which a forwarder will forward to PC to index the data and run searches.

From the scenario above, is it necessary to set it up this way? or i can do everything via web splunk from any PC while the Indexer can be directly located with the server?

Tags (2)
0 Karma
1 Solution

sdaniels
Splunk Employee
Splunk Employee

The indexes will get stored on your indexer server. In your case a combined Indexer/Search Head. Forwarders will send the data from the servers to the indexer and it will be stored there. See the docs for details.

http://docs.splunk.com/Documentation/Splunk/latest/admin/HowSplunkstoresindexes

Architecture: http://docs.splunk.com/Documentation/Splunk/latest/installation/Splunksarchitectureandwhatgetsinstal...

View solution in original post

sdaniels
Splunk Employee
Splunk Employee

The indexes will get stored on your indexer server. In your case a combined Indexer/Search Head. Forwarders will send the data from the servers to the indexer and it will be stored there. See the docs for details.

http://docs.splunk.com/Documentation/Splunk/latest/admin/HowSplunkstoresindexes

Architecture: http://docs.splunk.com/Documentation/Splunk/latest/installation/Splunksarchitectureandwhatgetsinstal...

sdaniels
Splunk Employee
Splunk Employee

Typically it is compressed at 50% - 60% of the raw data but depends on the data and if you are using Summary indexing. So you'll store around 2.5+ GB per day based on the raw data of 5 GB per day.

0 Karma

attgjh1
Communicator

thanks for the fast reply. 😃

so technically, if im testing data on my PC now, for every 5 gig of data i index from the harddisk, another 5gig or so worth of indexed data will also be created and stored on the indexer, which is taking up my harddisk space?

just confused with this part as i need to factor in server space required to consider acquiring splunk 🙂

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...