Hi,
I am processing Bluecoat logs on a heavy forwarder. I'm trying to set up some fields using FIELDALIAS, but they are not appearing. I have the following on my heavy forwarder. Should they be somewhere else?
Props.conf
[source::tcp:1918]
MAX_TIMESTAMP_LOOKAHEAD = 19
SHOULD_LINEMERGE = false
REPORT-main = BlueCoatTCP_HTTPLogs
TRANSFORMS-tcp = nullPound
TIME_FORMAT = %Y-%m-%d %T
FIELDALIAS-srcip = c_ip AS src_ip
FIELDALIAS-cs_host = cs_host AS dest_host
FIELDALIAS-protocol = cs_uri_scheme AS protocol
FIELDALIAS-dest_port = cs_uri_port AS dest_port
FIELDALIAS-src_user = cs_username AS src_user
FIELDALIAS-s_ip = s_ip AS dvc_ip
FIELDALIAS-http_response = sc_status AS http_response
FIELDALIAS-bytes_out = cs_bytes AS bytes_out
FIELDALIAS-bytes_in = sc_bytes AS bytes_in
FIELDALIAS-r_ip = r_ip AS resolved_ip
FIELDALIAS-r_port = r_port AS resolved_port
FIELDALIAS-s_computername = s_computername AS proxy_server
pulldown_type = true
TZ = UTC
KV_MODE = none
Transforms.conf
[BlueCoatTCP_HTTPLogs]
DELIMS = " "
FIELDS = "date", "time", "s-computername", "cs-username", "c-ip", "sc-filter-res
ult", "cs-method", "cs-uri", "sc-status", "cs(Referer)", "cs-categories", "cs-au
th-group", "x-exception-id", "s-action", "cs(User-Agent)", "time-taken", "sc-byt
es", "cs-bytes", "cs-uri-scheme", "cs-host", "cs-uri-port", "cs-uri-path", "cs-u
ri-query", "cs-uri-extension", "rs(Content-Type)", "r-ip", "r-port"
A good resource I use for identifying where configuration items should be set: http://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings%3F
FIELDALIAS is a search time operation, so it needs to go on the search head.
However, if you are using bluecoat proxy, there is already an add-on for that: https://splunkbase.splunk.com/app/2758/
Split your props.conf into two, one goes to Heavy forwarder, other on Search Head
Heavy forwarder : Props.conf
[source::tcp:1918]
MAX_TIMESTAMP_LOOKAHEAD = 19
SHOULD_LINEMERGE = false
TRANSFORMS-tcp = nullPound
TIME_FORMAT = %Y-%m-%d %T
pulldown_type = true
TZ = UTC
KV_MODE = none
Heavy Forwarder : Transforms.conf
[nullPound]
<<I thing this is the one you missed to mention>>
Search Head: Props.conf
[source::tcp:1918]
REPORT-main = BlueCoatTCP_HTTPLogs
FIELDALIAS-srcip = c_ip AS src_ip
FIELDALIAS-cs_host = cs_host AS dest_host
FIELDALIAS-protocol = cs_uri_scheme AS protocol
FIELDALIAS-dest_port = cs_uri_port AS dest_port
FIELDALIAS-src_user = cs_username AS src_user
FIELDALIAS-s_ip = s_ip AS dvc_ip
FIELDALIAS-http_response = sc_status AS http_response
FIELDALIAS-bytes_out = cs_bytes AS bytes_out
FIELDALIAS-bytes_in = sc_bytes AS bytes_in
FIELDALIAS-r_ip = r_ip AS resolved_ip
FIELDALIAS-r_port = r_port AS resolved_port
FIELDALIAS-s_computername = s_computername AS proxy_server
Search Head: Transforms.conf
[BlueCoatTCP_HTTPLogs]
DELIMS = " "
FIELDS = "date", "time", "s-computername", "cs-username", "c-ip", "sc-filter-res
ult", "cs-method", "cs-uri", "sc-status", "cs(Referer)", "cs-categories", "cs-au
th-group", "x-exception-id", "s-action", "cs(User-Agent)", "time-taken", "sc-byt
es", "cs-bytes", "cs-uri-scheme", "cs-host", "cs-uri-port", "cs-uri-path", "cs-u
ri-query", "cs-uri-extension", "rs(Content-Type)", "r-ip", "r-port"
What is the empty nullpound for in the transforms.conf?
In your original props.conf, there is a line for a transform but there is no definition provided in transforms. I just added a placeholder.
TRANSFORMS-tcp = nullPound