In our environment, we have one application that uses 32 blades. On each of those 32 blades, we are looking to install splunk, and have them all configured to monitor the same directories and all forwarding data to one central repository.
Instead of manually installing and configuring splunk on each server's own filesystem, I was wondering if there is some way to use a generic install of splunk on an NFS mount that is shared between all the servers. The problem I know we'd face is that splunk does it's logging and configuration in the same directory as the installation, so this wouldn't work with 32 servers sharing the same configuration directory.
What I was wondering is if there a way to specify a separate local directory that is not at all tied in with the binaries needed to run splunk? This directory would house any local configurations and logs, which has a very small disk footprint and can just be on each server's local filesystem.
Has this been done before? How are other users installing splunk on very distributed environments?
... View more