Deployment Architecture

search head clustering

brod_geico
Path Finder

I need plan for a new infra splunk setup.

1TB/day of log volume. The log volume can go up to 2TB/day.

Number of concurrent users : >50

Number of concurrent searches: > 100

Product will be deployed on: VMs/physical

Search heads:3
/opt/splunk --250GB
Indexers:
6 with /splunk/logs 3TB to save hot/warm data

Few questions:

if my replication factor is 3 so im assuming i need 3 nodes for search head clustering.
what is the recommended size of file system need on /opt/splunk on each search head for 1TB data.
here im not using indxer clustering so 3-4TB SAN is fine to keep my logs like hot/warm/frozen data.
how we setup load balance on cluster for global url.

I have already read all splunk URL"s so expecting straight answers from folks.

who ever had cluster setup please post your recommendations.

0 Karma

jnicholsenernoc
Path Finder

We have had good results using an Amazon Elastic Load Balancer with sticky sessions enabled out in front of the search head cluster.

I'd suggest looking at an existing search head to determine what type of storage footprint you really need. It really depends on the saved searches, apps installed, user workload.

0 Karma

brod_geico
Path Finder

Thanks i was thinking same way

0 Karma

lguinn2
Legend

You need at least 3 nodes for search head clustering, regardless; that is the required minimum. You aren't storing any of the data on the search heads. However, what you are storing is

  • search artifacts: the logs and results from the many searches - size depends on how many searches and how long you store the results
  • configuration files: settings for reports, apps, etc.; this will probably be smaller

So I don't think anyone can say exactly how much storage you need on the search heads. The "dedicated search head recommendation" of

2 x 300GB, 10,000 RPM SAS hard disks, configured in RAID 1

is a good starting point.

You will need a load-balancer that can provide "layer 7" load-balancing (also known as "session sticky"). The actual loading balancing setup is not done in Splunk at all - it must be set up on the load balancer. I can't help you with that.

Personally, I would not use a Virtual Machine for either a search head or an indexer for an environment of this size. You should probably review the Splunk Capacity Planning manual. I would also check out the Search head clustering information. My apologies if you have already read all of this, but I wasn't sure what "all splunk URLs" meant.

Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...