I am importing a CSV with around 100 fields. When importing, I see the review screen and it shows all of the fields and values in the file in a nicely formatted table. Once I submit and go to the search are with the new data source select, I only see around 20 fields in the "interesting fields" list and another 11 in the "11 more fields" link. Also, when I am typing in the search bar, it is autocompleting with fields that are missing, but when I submit the search, it finds no records.
How can I choose what fields I see in that search list? is there a limit to the number of fields the CSV can have?
Might be a restriction with the new INDEXED_EXTRACTIONS=csv
feature. Do file a case with Splunk Support to be certain.
After more testing, there appears to be a 50 field limit. Does that sound right? With the testing data, I number the fields 1-100 and it had every field from 1-50 and then stopped.
There is a bug related to a large number of fields at http://answers.splunk.com/answers/129773/advice-for-when-you-have-more-than-100-automatically-extrac... but it shouldn't lead to you only seeing 30ish fields.
I can't paste in any sample data here because the limit is to short, but I just generated a bunch of sample data through Mockaroo with 100 fields and 3000 rows with only colors as the content and I still had the same problem. It only listed some of the fields in the interesting fields column.
If most fields are very rare then check they're not being filtered from that view. Click "All Fields" or the "11 more fields" link and see if changing the Coverage changes the number of fields you see.
If that doesn't change anything, post some sample data - maybe there's some oddities in there that stop fields from being extracted.
No, all fields are not present in all events, they are only present part of the time. My props.conf looks like this:
(#)your settings
NO_BINARY_CHECK=1
(#)set by detected source type
INDEXED_EXTRACTIONS=csv
KV_MODE=none
SHOULD_LINEMERGE=false
pulldown_type=true
If you're only seeing around 30 fields in total then you're not hitting any potential field count limits.
Are all fields present in all events?
What's in props.conf under that sourcetype's stanza?