All Apps and Add-ons

Why is my props.conf configuration not changing W3C extractions to CIM field names?

pedromvieira
Communicator

Hi,

I'm developing an app with CIM support and W3C log format.
My props is configured to change field names to CIM fields, but it's not working.
How can I use W3C extractions and CIM Field_Names?

Thanks.

---- props.conf ----
[myapp]
...
INDEXED_EXTRACTIONS = W3C
...
FIELD_NAMES = CIM_FIELDS

---- log example ----

Fields: date time cs-ip cs-method cs-uri sc-status sc-bytes time-taken cs(Referer) cs(User-Agent) cs(Cookie) x-wafinfo

2014-10-15 06:11:25 14.162.88.85 GET /origin-carrinho.mysite.com.br/Site/BemVindo.aspx?token=A81CA3C962D542eb8433CE58CD0BB99D&ReturnUrl=http%3A%2F%2Fwww.mysite.com.br%2F 302 1721 1 "http://www.mysite.com.br/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.124 Safari/537.36" "-" "NKCR_11244||"
2014-10-15 06:11:31 14.162.88.85 GET /origin-carrinho.mysite.com.br/Site/BemVindo.aspx?token=A81CA3C962D542eb8433CE58CD0BB99D&ReturnUrl=http%3A%2F%2Fwww.mysite.com.br%2F 302 1721 2 "http://www.mysite.com.br/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.124 Safari/537.36" "-" "NKCR_11244||"
2014-10-15 06:15:57 180.76.6.46 GET /origin-carrinho.mysite.com.br/Site/Carrinho.aspx?idSku=259515 200 15616 595 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" "-" "NKCR_11244||"

0 Karma

sowings
Splunk Employee
Splunk Employee

My reading of the settings in the props.conf documentation is that FIELD_NAMES are only used if the file doesn't seem to have a header, which it does.

When I've seen this implemented for other data sources, FIELD_ALIAS is the most common means of mapping the "native" fields to the Common Information Model. You could also use DELIMS and FIELDS to capture space-separated fields in with the right names the first time, but then these are no longer indexed extractions, they are done at search time.

pedromvieira
Communicator

I also implemented via FIELD_ALIAS but maybe there is a better approach.

FIELDALIAS-item = dest_ip AS dest src_ip AS src cs-ip AS src_ip cs-method AS http_method cs-uri AS uri_path sc-status AS status sc-bytes AS bytes time-taken AS duration cs(Referer) AS http_referrer cs(User-Agent) AS http_user-agent cs(Cookie) AS cookie x-wafinfo AS signature signature AS rule

0 Karma

pedromvieira
Communicator

It was to represent a list. The actual fields are:

FIELD_NAMES = date,time,src_ip,http_method,uri_path,status,bytes,duration,http_referrer,http_user-agent,cookie,signature

Source: http://docs.splunk.com/Documentation/CIM/latest/User/Web

0 Karma

sowings
Splunk Employee
Splunk Employee

Did you use the literal string "CIM_FIELDS" in your FIELD_NAMES configuration attribute, or was that simply to represent a list?

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...