Splunk Search

Problems with props.conf and regex

dap263
New Member

I have a tab delimited log file that looks like:

#Fields: time   Data    LoginID ContextID       
"2011-02-20 21:38:59"   /opt/opensso_domain1/opensso/log/       "cn=dsameuser,ou=DSAME Users,dc=xxx,dc=xxx,dc=com"      66bee9c28495081c
"2011-02-20 21:38:59"   "Login Success|module_instance|Application"     "cn=dsameuser,ou=DSAME Users,dc=xxx,dc=xxx,dc=com"      b65b9f90eff3192b01

I created a deployed app, and assigned the sourcetype to be "openam_log" in inputs.conf
Then I tried to do an inline field extraction in props.conf

[openam_log]
EXTRACT-openam_date = "(?<openam_time>[^\t]+)"

I verified the regex in Search with the command:

sourcetype=openam_log | rex field=_raw "(?<openam_time>[^\t]+)"

That worked in Search, however the field still doesn't show up in Manager > Fields > Extracted Fields in the UI.

0 Karma
1 Solution

Lamar
Splunk Employee
Splunk Employee

You might want to try to use DELIMS as your separator.

props.conf

[openam_log]
REPORT-openam_fields = openam_fields

transforms.conf

[openam_fields]
DELIMS = "\t"
FIELDS = openam_time, openam_data, openam_ldap, openam_id

View solution in original post

dap263
New Member

The entire issue was related to placing props.conf and transforms.conf in

/opt/splunk/etc/deployment-apps/openam_new

Instead of

/opt/splunk/etc/apps/app_name/default

Thank you!

0 Karma

_d_
Splunk Employee
Splunk Employee

dap263,

Do you have a .meta file? If yes, can you paste its contents here? Otherwise, create a file called default.meta in your metadata directory in your app and paste this in there (openam_new/metadata/default.meta):

[]
access = read : [ * ], write : [ admin ]
[eventtypes]
export = system
[props]
export = system
[transforms]
export = system

Hope this helps.

> please upvote and accept answer if you find it useful - thanks!

0 Karma

dap263
New Member

Thank you for fast response! I had tested something similar before without success. The example above still did not work. Is there a way to troubleshoot from the command line or in the splunk logs to see where/how its failing?

splunk3:/opt/splunk/etc/deployment-apps/openam_new/default # /opt/splunk/bin/splunk cmd btool --app=search props list
[splunk_web_service]
EXTRACT-useragent = userAgent=(?P<browser>[^ (]+)
[splunkd]
EXTRACT-fields = (?i)^(?:[^ ]* ){2}(?:[+\-]\d+ )?(?P<log_level>[^ ]*)\s+(?P<component>[^ ]+) - (?P<message>.+)

Also, I don't think that the props file is being read properly.

0 Karma

Lamar
Splunk Employee
Splunk Employee

You might want to try to use DELIMS as your separator.

props.conf

[openam_log]
REPORT-openam_fields = openam_fields

transforms.conf

[openam_fields]
DELIMS = "\t"
FIELDS = openam_time, openam_data, openam_ldap, openam_id

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...