Hi,
I'm developing an app with CIM support and W3C log format.
My props is configured to change field names to CIM fields, but it's not working.
How can I use W3C extractions and CIM Field_Names?
Thanks.
---- props.conf ----
[myapp]
...
INDEXED_EXTRACTIONS = W3C
...
FIELD_NAMES = CIM_FIELDS
---- log example ----
2014-10-15 06:11:25 14.162.88.85 GET /origin-carrinho.mysite.com.br/Site/BemVindo.aspx?token=A81CA3C962D542eb8433CE58CD0BB99D&ReturnUrl=http%3A%2F%2Fwww.mysite.com.br%2F 302 1721 1 "http://www.mysite.com.br/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.124 Safari/537.36" "-" "NKCR_11244||"
2014-10-15 06:11:31 14.162.88.85 GET /origin-carrinho.mysite.com.br/Site/BemVindo.aspx?token=A81CA3C962D542eb8433CE58CD0BB99D&ReturnUrl=http%3A%2F%2Fwww.mysite.com.br%2F 302 1721 2 "http://www.mysite.com.br/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.124 Safari/537.36" "-" "NKCR_11244||"
2014-10-15 06:15:57 180.76.6.46 GET /origin-carrinho.mysite.com.br/Site/Carrinho.aspx?idSku=259515 200 15616 595 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" "-" "NKCR_11244||"
My reading of the settings in the props.conf documentation is that FIELD_NAMES are only used if the file doesn't seem to have a header, which it does.
When I've seen this implemented for other data sources, FIELD_ALIAS is the most common means of mapping the "native" fields to the Common Information Model. You could also use DELIMS and FIELDS to capture space-separated fields in with the right names the first time, but then these are no longer indexed extractions, they are done at search time.
I also implemented via FIELD_ALIAS but maybe there is a better approach.
FIELDALIAS-item = dest_ip AS dest src_ip AS src cs-ip AS src_ip cs-method AS http_method cs-uri AS uri_path sc-status AS status sc-bytes AS bytes time-taken AS duration cs(Referer) AS http_referrer cs(User-Agent) AS http_user-agent cs(Cookie) AS cookie x-wafinfo AS signature signature AS rule
It was to represent a list. The actual fields are:
FIELD_NAMES = date,time,src_ip,http_method,uri_path,status,bytes,duration,http_referrer,http_user-agent,cookie,signature
Source: http://docs.splunk.com/Documentation/CIM/latest/User/Web
Did you use the literal string "CIM_FIELDS" in your FIELD_NAMES configuration attribute, or was that simply to represent a list?