How come only one of many fields is not extracting?

[shayhibah]

I have a weird behavior in my environment.

When I get new data, I parse them using my regex (= as delimiter between key-value and | as delimiter between pairs).

In addition, I create mapping for them in order to support CIM fields (fields alias).

When I see my logs in Search app, all of the records contain the field "action" but — the problem is that I can't find this field on my left side (interesting fields) even if I choose 100% coverage (all other fields are fine).

I tired to look at my props.conf & transforms.conf files, but there is no special behavior for this field.

Can someone advise me how to deal with it?

Thanks

[sudosplunk]

Hi,

Just so I understand this correctly, you checked on the "action" field by going into "All Fields" tab on the left side, correct. Because it works for me. (see the screenshot below)

If it doesn't work this way, try tweaking ui-pref.conf under your user ($SPLUNK_HOME/etc/users/) directory on search head (do this on all search heads if you're using search head cluster). HTH!

ui-prefs.conf:

[search]
display.events.fields = ["source","sourcetype","host","index","version"] 

[mstjohn_splunk]

hi @shayhibah

Did the answer below solve your problem? If so, please resolve this post by approving it! If your problem is still not solved, keep us updated so that someone else can help ya. Thanks for posting!

[vishaltaneja070]

Hello @shayhibah

I think the issue seems to be with regex only .

What I can suggest you here, you can directly extract the filed directly using Splunk WEB UI using regex (On search screen, extract more fields). and assign it the name action. That stanza will be automatically added to props.conf and transforms.conf

[shayhibah]

hi @vishaltaneja07011993
using rex in my search screen works but after refreshing the page - it turns back into its bad behavior.

I just want to emphasize that I did use only field extraction as part of my source type transform (have not done it using the WEB UI directly). why only this field behave differently?

Just to give more info:

query:
index IN (main) sourcetype="my_log" AND severity IN (Medium,High,Critical)

transformation function:
([a-zA-Z0-9_]+)=([^|]+)

raw log:
time=1540422044|hostname=LS|severity=Critical|action=Detect|ifdir=inbound

Interesting fields:
hostname
ifdir
severity
time

Update:
In addition - I MUST MENTION THAT I HAVE MULTIPLE LOGS FROM THE SAME LOG FILE I SENT. THERE ARE RECORDS THAT THE ACTION FIELD IS EXTRACTED AND THERE ARE RECORD THAT THE ACTION FIELD IS NOT EXTRACTED (ALTHOUGH ITS IN THE RAW LOG).

[vishaltaneja070]

Hello @shayhibah
What I suggest you here, if you are using the latest version of Splunk. Then there is a Data Quality dashboard, just check there is any data issue occurred.

That can give you better insight.

Thanks!

[shayhibah]

@vishaltaneja07011993

Unfortunately could not find something there..