Little strange issue I got...
I ingest files into an index. I want to add a yes/no field to my events, based on if the same event is present in a previous file.
Lets say, I have events with fields likes this:
keyfield1, keyfield2, miscfield1 ... miscfieldN
And I want to do something likes this:
index=myindex source=inputfile1 | eval flag=if([index=myindex source=inputfile2 keyfield1=<value of keyfield1 in the event> AND keyfield2=<value of keyfield2 in the event>],"true","false")
I know the syntax of the if statement is not true, but hope it can articulate my problem.
This should something best suited for collections or lookups/
Try this:
index=myindex source=inputfile1 OR source=inputfile2 | eventstats dc(source) AS DC1 by keyfield1 | eventstats dc(source) AS DC2 by keyfield2 | where source=inputfile1 | eval flag=if((DC1>1 AND DC2>1), "true", "false") | fields - DC1 DC2
Will there be any common key between data in two sources for comparison?