Adding a field extraction causes other fields to disappear

[agthurber]

I have come across a problem where the fields i have defined in my transforms.conf for a csv file are disappearing from the available fields list (on the left of the search results) after I create a new Field Extraction from the GUI. But only for the APP that I create the field extraction in, if i go back to the search app all of the fields defined in my transforms.conf file are available. I have reproduced this problem a couple times and am quite confused why adding a field extraction could stop these other fields from showing up in the App they are associated with.

In this case I have a list of fields that the CSV file provides, but i want to chop up a couple of the defined fields into smaller pieces. So i did a field extraction from the GUI. The log is from a mail server. I have a field defined for the sending address (orig) and I am trying to extract a client ID from that address (orig_client). here is the regex the field extractor came up with,

(?i)^[^\.]*\.(?P<orig_client>[^@]*)(?=@) 

this regex does work to find the values I am looking for, but after saving it something appears to break and it hides most, but not all, of the other fields that are defined in the transforms.conf file. I had previously added a couple other field extractions prior to this one and they worked, but adding this one caused the issue both times I saw this happen. I looked in the conf files for anything that could be overwriting the fields that should be available but have not been able to find anything out of the ordinary. Please help, this is a very strange issue and does not behave how I would expect it should.

How do you force splunk to display the fields you want when they appear to not exist in the list of available fields?

Thanks, Arlen

[woodcock]

Here are 2 different ways to make sure fields with no/null value will be updated to have the value "UNK":

1: Use "coalesce" on individual fiels:

sourcetype=MyEvents | MyField=coalesce(MyField, "UNK") | ByField=coalesce(ByField, "UNK") | stats count count(eval(MyField!=MyValue)) AS MyValueCount BY ByField 

2: Use "fillnull" to cover all fields:

sourcetype=MyEvents | fillnull value="UNK" | stats count count(eval(MyField!=MyValue)) AS MyValueCount BY ByField 

[agthurber]

lets forget this is a specific incident, I have seen this behavior many times in splunk, but this was the only time i could draw a correlation as to why the fields disappeared. here is a much more general question...

Why do fields sometimes not display when you would expect them to?