I just installed splunk on RedHat enterprise 5, and want to know how to monitor remote linux servers from RedHat server
There are a number ways of that you can get data from remote systems into your Splunk instance.
You can install a forwarding agent on each of the machines that you wish to collect data from that will send events to the central indexer. More information on data cloning and routing can be found here: http://docs.splunk.com/Documentation/Splunk/5.0/Deploy/Aboutforwardingandreceivingdata
You can configure syslog/syslog-ng to forward event data to a central Splunk index. You will then configure Splunk to listen on the specified UDP (syslog) or TCP (syslog-ng) port. More information on configuring Splunk to listen on a network port can be found here: http://docs.splunk.com/Documentation/Splunk/5.0/Data/Monitornetworkports
You can write a small script that collects the various files from your remote systems and feeds them into your index using our scripted input method. More information on scripted inputs can be found here: http://docs.splunk.com/Documentation/Splunk/5.0/AdvancedDev/ScriptedInputsIntro
Since UDP is unreliable, would it be best just to rsync remote logs to the local instance on an interval?
There are a number ways of that you can get data from remote systems into your Splunk instance.
You can install a forwarding agent on each of the machines that you wish to collect data from that will send events to the central indexer. More information on data cloning and routing can be found here: http://docs.splunk.com/Documentation/Splunk/5.0/Deploy/Aboutforwardingandreceivingdata
You can configure syslog/syslog-ng to forward event data to a central Splunk index. You will then configure Splunk to listen on the specified UDP (syslog) or TCP (syslog-ng) port. More information on configuring Splunk to listen on a network port can be found here: http://docs.splunk.com/Documentation/Splunk/5.0/Data/Monitornetworkports
You can write a small script that collects the various files from your remote systems and feeds them into your index using our scripted input method. More information on scripted inputs can be found here: http://docs.splunk.com/Documentation/Splunk/5.0/AdvancedDev/ScriptedInputsIntro