I want to find entries added to a sourcetype today, that haven't been seen in the last N days. I've tried search A | fields + want | fields - ignore | search NOT [search B | fields + want | fields - ignore] and that doesn't seem to work. I still see results that are in search B.
Note: fields want | fields - ignore
is redundant. Because if you narrow the field list to just the want
field, then there is no ignore
field to remove.
Also, when you're using subsearches it can be helpful to look at the search inspector "Actions > Inspect Search". You'll see that just putting NOT in front of a subsearch will not NOT the entire subsearch set. I think it will only NOT the first parenthetical term. Possibly wrapping the entire subsearch in parens will do it. Otherwise you'll have to play around with the format
command if you want to take this approach.
A subsearch might be the way to go. However for fun here's an approach that doesn't use subsearches. This shows URL's in access data that were accessed previously in the time range, but not accessed today.
sourcetype=access_combined | eval happened_today=if(relative_time(now(),"@d")<=_time,1,0) | eval happened_before=if(relative_time(now(),"@d")>_time,1,0) | streamstats sum(happened_today) as happened_today sum(happened_before) as happened_before by url | stats count by url happened_today happened_before | search happened_before>0 happened_today=0
Note: fields want | fields - ignore
is redundant. Because if you narrow the field list to just the want
field, then there is no ignore
field to remove.
Also, when you're using subsearches it can be helpful to look at the search inspector "Actions > Inspect Search". You'll see that just putting NOT in front of a subsearch will not NOT the entire subsearch set. I think it will only NOT the first parenthetical term. Possibly wrapping the entire subsearch in parens will do it. Otherwise you'll have to play around with the format
command if you want to take this approach.
A subsearch might be the way to go. However for fun here's an approach that doesn't use subsearches. This shows URL's in access data that were accessed previously in the time range, but not accessed today.
sourcetype=access_combined | eval happened_today=if(relative_time(now(),"@d")<=_time,1,0) | eval happened_before=if(relative_time(now(),"@d")>_time,1,0) | streamstats sum(happened_today) as happened_today sum(happened_before) as happened_before by url | stats count by url happened_today happened_before | search happened_before>0 happened_today=0
Ah got it. yea "fields foo" will leave all the fields that begin with underscores untouched. that's why you often see "fields foo | fields - _*"
fields - _raw otherwise I get the raw record in the output when I just want the values for the field of interest. However, using stats this isn't an issue.