Hi, I am using splunk 5.0.3 but found fields can't be extracted automatically on the splunk UI. To test, I loaded the sample csv file and use the customized sourcetype test_csv_log defined in props.conf. However, the fields like c1, c2, etc defined in transforms.conf are not auto-discovered by splunk. I am wondering if I miss anything? P.S. I did select verbose mode when doing the search......
Thanks!
sample.csv
07/19/2013 08:18:16:369 EDT, john,car, note,king,queen
07/19/2013 12:53:16:369 EDT, ws,ed,rf,tg,yh,uj
in props.conf
[test_csv_log]
TZ = 'America/New_York'
NO_BINARY_CHECK = 1
pulldown_type = 1
REPORT-r15 = test_csv_fields
in transforms.conf
[test_csv_fields]
DELIMS = ","
FIELDS = c1,c2,c3,c4,c5,c6,c7,c8,c8,c9,c10
It is interesting that this works well by following the standard procedure (as you have done). For reference and comparison, here is the configuration for this test.
#inputs.conf
[monitor:///answers/7-31-2013/1/data]
disabled = false
index = test
sourcetype = answers-1375288490
#props.conf
[answers-1375288490]
REPORT-r15 = csv_fields_1375288490
#transforms.conf
[csv_fields_1375288490]
DELIMS = ","
FIELDS = c1,c2,c3,c4,c5,c6,c7,c8,c8,c9,c10
Here is what we see in SplunkWeb.
At this point, I will venture say there is something not clicking right in your test setup. Can you also post your inputs.conf?
Assuming that you are _not able to see the data displayed, the same can be accomplished in the UI with the following:
sourcetype="answers-1375288490" | rex "EDT,\s+(?<message>.+)" | rex field=message max_match=0 "(?<c>\w+)(?:,|$)"
And, these are the results. Note the field "C" is available.
Or, you may also try this:
sourcetype="answers-1375288490" | rex "EDT,\s+(?<c>.+)" | makemv delim="," c
Surely you will agree that your objective is possible in a number of ways. Let's get back to your test and compare.
--gc
It is interesting that this works well by following the standard procedure (as you have done). For reference and comparison, here is the configuration for this test.
#inputs.conf
[monitor:///answers/7-31-2013/1/data]
disabled = false
index = test
sourcetype = answers-1375288490
#props.conf
[answers-1375288490]
REPORT-r15 = csv_fields_1375288490
#transforms.conf
[csv_fields_1375288490]
DELIMS = ","
FIELDS = c1,c2,c3,c4,c5,c6,c7,c8,c8,c9,c10
Here is what we see in SplunkWeb.
At this point, I will venture say there is something not clicking right in your test setup. Can you also post your inputs.conf?
Assuming that you are _not able to see the data displayed, the same can be accomplished in the UI with the following:
sourcetype="answers-1375288490" | rex "EDT,\s+(?<message>.+)" | rex field=message max_match=0 "(?<c>\w+)(?:,|$)"
And, these are the results. Note the field "C" is available.
Or, you may also try this:
sourcetype="answers-1375288490" | rex "EDT,\s+(?<c>.+)" | makemv delim="," c
Surely you will agree that your objective is possible in a number of ways. Let's get back to your test and compare.
--gc
Hey Gilberto. This problem still persists for me (see my comment under the question with props.conf and transforms.conf snippets). I am able to see the field when I query for it explicitly in splunk web with rex, but not otherwise. Note that the log data was all imported with command-line oneshot calls like this:
splunk add oneshot logfile -index main -sourcetype mysrctype -host myhost
...so there is not inputs.conf segment. Can you spot a problem with my configuration that might explain this?
This is very useful. Thank you very much!
I am having this same issue. In transforms.conf I have:
[myfield-mv]
REGEX = (?P
MV_ADD = true
SOURCE_KEY = myinputfield
...and in props.conf I have:
REPORT-myfield = myfield-mv
...but myfield does not appear among the "interesting fields" in searches from the we interface. However, if I search like this:
* | rex field=myinputfield "(?P
...i do see myfield in the "interesting fields". Help!
you sample csv has variable colum length?