All Apps and Add-ons

Why is my props.conf configuration not changing W3C extractions to CIM field names?

pedromvieira
Communicator

Hi,

I'm developing an app with CIM support and W3C log format.
My props is configured to change field names to CIM fields, but it's not working.
How can I use W3C extractions and CIM Field_Names?

Thanks.

---- props.conf ----
[myapp]
...
INDEXED_EXTRACTIONS = W3C
...
FIELD_NAMES = CIM_FIELDS

---- log example ----

Fields: date time cs-ip cs-method cs-uri sc-status sc-bytes time-taken cs(Referer) cs(User-Agent) cs(Cookie) x-wafinfo

2014-10-15 06:11:25 14.162.88.85 GET /origin-carrinho.mysite.com.br/Site/BemVindo.aspx?token=A81CA3C962D542eb8433CE58CD0BB99D&ReturnUrl=http%3A%2F%2Fwww.mysite.com.br%2F 302 1721 1 "http://www.mysite.com.br/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.124 Safari/537.36" "-" "NKCR_11244||"
2014-10-15 06:11:31 14.162.88.85 GET /origin-carrinho.mysite.com.br/Site/BemVindo.aspx?token=A81CA3C962D542eb8433CE58CD0BB99D&ReturnUrl=http%3A%2F%2Fwww.mysite.com.br%2F 302 1721 2 "http://www.mysite.com.br/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.124 Safari/537.36" "-" "NKCR_11244||"
2014-10-15 06:15:57 180.76.6.46 GET /origin-carrinho.mysite.com.br/Site/Carrinho.aspx?idSku=259515 200 15616 595 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" "-" "NKCR_11244||"

0 Karma

sowings
Splunk Employee
Splunk Employee

My reading of the settings in the props.conf documentation is that FIELD_NAMES are only used if the file doesn't seem to have a header, which it does.

When I've seen this implemented for other data sources, FIELD_ALIAS is the most common means of mapping the "native" fields to the Common Information Model. You could also use DELIMS and FIELDS to capture space-separated fields in with the right names the first time, but then these are no longer indexed extractions, they are done at search time.

pedromvieira
Communicator

I also implemented via FIELD_ALIAS but maybe there is a better approach.

FIELDALIAS-item = dest_ip AS dest src_ip AS src cs-ip AS src_ip cs-method AS http_method cs-uri AS uri_path sc-status AS status sc-bytes AS bytes time-taken AS duration cs(Referer) AS http_referrer cs(User-Agent) AS http_user-agent cs(Cookie) AS cookie x-wafinfo AS signature signature AS rule

0 Karma

pedromvieira
Communicator

It was to represent a list. The actual fields are:

FIELD_NAMES = date,time,src_ip,http_method,uri_path,status,bytes,duration,http_referrer,http_user-agent,cookie,signature

Source: http://docs.splunk.com/Documentation/CIM/latest/User/Web

0 Karma

sowings
Splunk Employee
Splunk Employee

Did you use the literal string "CIM_FIELDS" in your FIELD_NAMES configuration attribute, or was that simply to represent a list?

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...