Splunk Search

Why is foreach with wildcards not picking up all fields in my search?

_jgpm_
Communicator

I'm not exactly sure why this isn't working. I couldn't find it in the documentation. I'm on 6.4.3.

basic search | table abc*abc def* |

This creates a table populated by fields that fit the wildcards. This would be an example table.

abc1abc|abc2abc|abc3abc|def1|def2|def3
sample1|sample2|sample3|123 |2345|null

Then I use foreach * [eval <<FIELD>>=1 ]

I'm doing this to just test what foreach is being applied to.

The table turns into:

abc1abc|abc2abc|abc3abc|def1|def2|def3
sample1|   1   |   1   |123 |2345|null

I have no idea why the * is only being applied to some subset of the fields. Is this somehow a carryover from the original table definition?

Thanks!

0 Karma
1 Solution

Flynt
Splunk Employee
Splunk Employee

Odd this seems to work fine for me but then I'm just using junk data.

|stats count|eval  abc1abc="sample1",cabc2abc="sample2", abc3abc="sample3", def1=123, def2=123, def3=""|fields - count| foreach * [eval <<FIELD>>=1 ]

On the offchance it's a naming issue, does adding "s work?

|stats count|eval  abc1abc="sample1",cabc2abc="sample2", abc3abc="sample3", def1=123, def2=123, def3=""|fields - count| foreach * [eval "<<FIELD>>"=1 ]

View solution in original post

Flynt
Splunk Employee
Splunk Employee

Odd this seems to work fine for me but then I'm just using junk data.

|stats count|eval  abc1abc="sample1",cabc2abc="sample2", abc3abc="sample3", def1=123, def2=123, def3=""|fields - count| foreach * [eval <<FIELD>>=1 ]

On the offchance it's a naming issue, does adding "s work?

|stats count|eval  abc1abc="sample1",cabc2abc="sample2", abc3abc="sample3", def1=123, def2=123, def3=""|fields - count| foreach * [eval "<<FIELD>>"=1 ]

somesoni2
SplunkTrust
SplunkTrust

+1 on using double quotes around field name.

0 Karma

_jgpm_
Communicator

I'm not sure if I can replicate it here. My base search has 460K+ events. Suffice to say, I have 6 fields: 5 in the format abc*def and 1 in xyz_*.

This is a near cut & paste index=index1 tag=tag1 | table abc*def xyz_* | foreach * [ eval <>_flag=1 ]

the first field abcFOXdef,abcDOGdef have 8 fields that are full, rest are empty. abcCATdef, abcBIRDdef,abcCROWdef all have thousands of entries. xyz_STAR has 12 events.

Only abcCATdef, abcBIRDdef,abcCROWdef had _flag=1 fields created.

0 Karma

somesoni2
SplunkTrust
SplunkTrust

The foreach will be applied to all the events, for each field, so not sure number of events will make a difference here. If you're not getting a <<FIELD>>_flag field created for a field with null values, then try using a fillnull command before foreach. Something like this

basic search | table abc*abc def* | fillnull value="null" abc*abc def* | foreach * [eval "<<FIELD>>_flag"=1 ]

OR

basic search | table abc*abc def* | fillnull value="null" abc*abc def* | foreach abc*abc def* [eval "<<FIELD>>_flag"=1 ]

_jgpm_
Communicator

fillnull definitely forced the FIELD_flag operation to occur for all fields. I used just 'fillnull value=""' and it worked. However, running fillnull on 460K x 6 fields caused my query to go from 38 secs to 56 secs. #firstworldproblems. I'm just using my laptop though. I will have to clean up some logic, but thank you for solving the issue!

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...