Search all fields except ____

[smileyge]

I have a log with say 50 fields.Is there a way to search all fields except one of them? Something like

... |search MyTerm donotMatch(ThisField=MyTerm)

Note, I am familiar with "NOT ThisField=MyTerm". That excludes a row that has MyTerm both in ThisField and SomeOtherField, but I'm interested in finding the ones in SomeOtherField. Looking for something more elegant than a really long search string of this form:

... | field_A="MyTerm" OR field_B="MyTerm" OR field_C="MyTerm"

[kristian_kolb · Accepted solution]

This worked for a test to detect when the string "404" exists in a web log, but not when it only exists in the status field.

sourcetype=access_combined 404 | rex "(?<myterm>\b404\b)" max_match=0 | eval mycount=mvcount(myterm) | where mycount==1 AND status!=404 OR mycount>1

Adjust as needed.

/k

[smileyge]

This is very close, I think this will meet the need.

[lukejadamec]

I'm confused. Earlier you said you DID want other fields in the event that contained ThisField, which is what the fields - command did. But now your saying you don't want other fields in the same event that contain ThisField.

[somesoni2]

If you can put all the field names you want to compare (excluding ThisField), then your can do something like this to just match all other fields (and don't care about content of ThisField)

   your base search [|inputlookup FieldNamesToMatch.csv | eval query="searchTerm=".fieldName| table query | format ]

where FieldNamesToMatch.csv is lookup file containing 49 entries with field as 'fieldName'. your search will expand like

your base search ((SearchTerm=fieldName1) OR SearchTerm=fieldName2.....)

[smileyge]

I like this solution except for having to make a CSV file and would have accepted it had it not been for the one I did accept.

[linu1988]

okay then if your fields are extracted then can you do the below?

search |eval ThisField=if(ThisField="MyTerm","dummy",ThisField)|table ...|where condition..

[smileyge]

I tried using the fields - command. The trouble is in the order. If you search MyTerm on all fields, and then eliminate that field, it still returns events that had MyTerm in ThisField, but ThisField is no longer in the set. If I flip the order, and pipe it back to search, it seems to still search the raw events or possibly the _raw field (which I tried also minusing out) and teh search still wants to match ThisField somehow. I even tried an experiment to do Table on just one field I was interested in, then piped back out to search and it searched _raw still apparently.

[lukejadamec]

If you're trying to get rid of ThisField regardless of the value (what you said to start your question), then you can use the fields - ThisField command.

|search MyTerm |fields - ThisField| ...

[smileyge]

I believe where has the same issue as the plain search. Can you give an example of how you would do this with where?

[smileyge]

addressed in the question. It will exclude all events that have MyTerm in that field, regardless of whether the term is in another field or not. I'm looking for MyTerm in any of the other fields and it could be contained in both.

[linu1988]

will where work? how a filter for some column is filter other column data?

[martin_mueller]

What's wrong with this?

MyTerm NOT ThisField=MyTerm