We have a remote windows ftp server that splunk is pulling logs off and I am not able to get it to recognize the remote host or ip as searchable fields.
The raw data looks like this
(110680) 11/21/2013 16:01:07 PM - acfXXX (10.180.48.145)> TYPE I
(110680) 11/21/2013 16:01:07 PM - acfXXX (10.180.48.145)> 200 Type set to I
(110680) 11/21/2013 16:01:07 PM - acfXXX (10.180.48.145)> PORT 10,180,48,145,19,136
(110680) 11/21/2013 16:01:07 PM - acfXXX (10.180.48.145)> 200 Port command successful
(110680) 11/21/2013 16:01:07 PM - acfXXX (10.180.48.145)> STOR /../../
(110680) 11/21/2013 16:01:07 PM - acfXXX (10.180.48.145)> 150 Opening data channel for file transfer.
(110680) 11/21/2013 16:01:08 PM - acfXXX (10.180.48.145)> 226 Transfer OK
(110680) 11/21/2013 16:01:08 PM - acfXXX (10.180.48.145)> TYPE I
(110680) 11/21/2013 16:01:08 PM - acfXXX (10.180.48.145)> 200 Type set to I
(110680) 11/21/2013 16:01:08 PM - acfXXX (10.180.48.145)> PORT 10,180,48,145,4,2
The local.conf on the windows server looks like this.
[monitor://C:\Program Files\FileZilla Server\Logs]
host = ftp01
index = hosted
sourcetype = ftp_filezilla
_whitelist = (?i).*\.log$
[monitor://C:\Program Files\FileZilla Server\Logs]
host = ftp01
index = hosted
sourcetype = ftp_filezilla
_whitelist = (?i).*\.log$
[WinEventLog:Application]
disabled = 1
[WinEventLog:Security]
disabled = 1
[WinEventLog:System]
disabled = 1
The host name acfXXX is not at all similar every time and I would need to configure it to strip it out by row number ie in linux awk '{print $6}'
(110680) 11/21/2013 16:01:08 PM - acfXXX (10.180.48.145)> 226 Transfer OK
Looking at the sample data, something like this should let you extract the two fields on the fly:
... | rex "-\s+(?<remote_host>\S+)\s+\((?<remote_ip>(\d+\.?)+)\)"
Once you're happy with the extraction you can move the expression to props.conf / Settings -> Fields -> Field Extractions to have the fields searchable every time.
(110680) 11/21/2013 16:01:08 PM - acfXXX (10.180.48.145)> 226 Transfer OK
Looking at the sample data, something like this should let you extract the two fields on the fly:
... | rex "-\s+(?<remote_host>\S+)\s+\((?<remote_ip>(\d+\.?)+)\)"
Once you're happy with the extraction you can move the expression to props.conf / Settings -> Fields -> Field Extractions to have the fields searchable every time.