better field discovery with SUF

[mikefoti]

I was under the impression that using SUF to forward events would some hope provide more automatically discovered fields. But when I compare events forwarded from a proxy server using SUF vs Snare, I see no additonal fields being autimatically available. Do I need to provide any info to the SUF, so it knows what type fo logs it's forwarding?

[Ayn]

The Universal Forwarder doesn't extract any fields itself. Most field extractions in Splunk are done at search-time, so all field "discovery" is done on the central indexer, not the Universal Forwarders. So, any work you want to do should be done on the indexer. What fields are you missing? Perhaps there is an app for the type of logs you are reading into Splunk?

[Ayn]

OK. I don't know the exact log format, but perhaps it is similar to that used by the IronPort WSA or Squid. There are apps for both those.

[mikefoti]

Thank you Ayn. The source of the proxy logs is Microsoft's TMG proxy, the new version of ISA.

[Ayn]

No. As all these field extractions take place on the indexer, syslog vs UF makes no difference. The only case where it would make a difference would be if the UF outputs events in a format that is different from the syslog agent's format (such as UF vs Snare for Windows event logs).

What kind of proxy logs are we talking about?

[mikefoti]

I had hoped I would see fields that match the header of the log file being monitored/forwarded by SUF, for example Client IP, Desination IP, Result Code, etc.

So do I understand correctly that SUF offers no advantages over syslog? Let me re-phrase that... I know SUF uses TCP rather than UDP and that is an advantage. But I was hoping that migrating from a SYSLOG forwarder to SUF would reduce the time req'd to manually extract fields. So my question is.... with respect to not having to manually extract fields, does SUF offer any advantage over syslog?