What are all the major considerations we need to have while designing the syslog servers. How can we decide the server counts for syslog servers?
This will be dependent upon many factors, some of the following you need to consider:
1) How many devices
2) Volume of events from devices
3) Velocity of events from devices
4) Retention requirements of log files on disk
5) DR / HA requirements for syslog server(s)
6) Network topologies of devices in relation to syslog server
There are other considerations out there also I'm sure. Consulting with rsyslog or syslog-ng sizing guides would be useful. From the Splunk side, virtual machines are more typical and these usually look like 8 cores / 8gb RAM and accommodating storage. Mileage varies, and typically VM's are better as you can allocate more compute as required and replicate the machines easily across data centers.
What is the data velocity and how much data do you plan on working with?
I am planning for 100GB/day