Hey,
I am having a look at transforms.conf and props.conf configuration files and wondering about the following question:
How do you do a field extraction from the source field (i.e. field=source) at search time? (An example of this would be great to see)
Thanks in advance for your help.
To add to what I said above:
The source field displays a file path when you run a search in Splunk. There is a word within that file path which I want Splunk to extract and place in a field called TheWord. How can I achieve this using props.conf and/or transforms.conf?
You just need to use SOURCE_KEY in the transform.
props.conf:
[source-extract]
REPORT-sourcefield = source-transform
transform.conf:
[source-transform]
SOURCE_KEY = source
REGEX = .......
FORMAT = fieldname=$1
You can set up the just the props.conf to extract fields at search time.
I'm going to use my environment as an example. Our Apache logs are pipe | delimited. So I have the following in my props.conf:
[prod_apache_logs]
EXTRACT- = (?P[\d]+.[\d]+.[\d]+.[\d]+)\|[(?P[\d]{2}\/[\w]{3}\/[\d]{4}):(?P[\d]{2}:[\d]{2}:[\d]{2})\s-[\d]+]\|(?P[\d]+)\|\"(?P[\S]+)\s(?P\/[\S]+)\s(?P.)\"\|(?P[\d]+)\|(?P[\W\d]+)\|(?P.)\|(?P.*)
prod_apache_logs = sourcetype of my apache logs.
Easiest way to test this is to use the "rex" command in the search. IE, I would use something like this:
sourcetype="prod_apache_logs" | rex "(?P[\d]+.[\d]+.[\d]+.[\d]+)\|[(?P[\d]{2}\/[\w]{3}\/[\d]{4}):(?P[\d]{2}:[\d]{2}:[\d]{2})\s-[\d]+]\|(?P[\d]+)\|\"(?P[\S]+)\s(?P\/[\S]+)\s(?P.)\"\|(?P[\d]+)\|(?P[\W\d]+)\|(?P.)\|(?P.*)"
Hope this helps!
Have a look at http://answers.splunk.com/questions/6511/field-extraction-mystery for an example source
If you provide an example of your source, I'll write up the exact command.
In my case, my source is something like this: /opt/http/logs/www.mysite.com-access.log.
Using ltrim / rtrim, I extract out the /opt/http/logs and the -access.log to leave me with www.mysite.com:
..| eval web_site=ltrim(source,"/opt/httpd/logs") | eval web_site=rtrim(web_site,"-access.log")
How would I do it using eval and ltrim/rtrim?
Okay, I just read your update. I'm not sure how you would do it in in props.conf or transforms.conf.. I know you can do it via the standard search using eval and ltrim / rtrim..
I forgot to say thanks for your help. I added more to my question above which will hopefully confirm what I am seeking
The data that I want to extract is not in _raw but it is in source. Therefore I don't believe this will work. I tried this and it did not work as expected.