So unfortunately, about three and a half hours after asking this (which was already about four hours into my search for a solution), I came up with something that works. I highly doubt this is as optimized as it could be - in fact it's probably awful - but it's the only way I could figure out how to do this...
Note that I determined that for the needs of my application, we don't actually have to look at all history, we only need to check over the past 31 days...
index="main" earliest=-31d latest=now | spath type | search type=request | spath "request.client_token" | search "request.client_token"!="" | stats values("request.remote_address") as all_addresses by "request.client_token" | eval all_count=mvcount(all_addresses) | join "request.client_token" type=outer [search index="main" earliest=-31d latest=-5m | spath type | search type=request | spath "request.client_token" | search "request.client_token"!="" | stats values("request.remote_address") as old_addresses by "request.client_token" | eval old_count=mvcount(old_addresses)] | fillnull value=0 all_count old_count | eval new_ip_count=(all_count - old_count) | where new_ip_count > 0 AND all_count > 1
both searches are based on search index="main" earliest=-31d latest=<<>> | spath type | search type=request| spath "request.client_token" | search "request.client_token"!="" | stats values("request.remote_address") as old_addresses by "request.client_token" ; this finds events which:
are in the "main" search index
are within a specified time range: earliest=-31d latest=<<>>
have a type field: spath type
are of type request: search type=request
have a client_token that isn't empty/null: spath "request.client_token" | search "request.client_token"!=""
collects statistics of remote addresses grouped by client token: stats values("request.remote_address") as old_addresses by "request.client_token"
the first search looks at events from 31 days ago to now, and puts the remote addresses in an all_addresses field; this represents ALL remote IPs seen for each token in the last 31 days (which should encompass the default max_ttl of 30 days, so it should get all data for all non-expired tokens).
we add a count of the distinct addresses to the results: all_count=mvcount(all_addresses)
we then perform an outer join on the client_token field, with another search run with the same parameters/filters but against records that are more than 5 minutes old
we end up with data that tells us, for each client_token seen in the last 31 days, what IPs it came from over all that time ( all_addresses ), and what IPs it came from prior to the last 5 minutes ( old_addresses ), and the count of each ( all_count and old_count , respectively)
The mvcount() function returns null for a field with no values, but we want to do math with this, so we convert the nulls to zero in those fields: fillnull value=0 all_count old_count
the total IP count minus the old IP count becomes the count of new IPs seen in the last 5 minutes: eval new_ip_count=(all_count - old_count)
we use where new_ip_count > 0 to limit the results to tokens which came from new IP addresses in the last 5 minutes
At this point, we'd get results for any tokens that are more than 5 minutes old and have been used by a new IP within the last 5 minutes, including tokens that have been created in the last 5 minutes and used from multiple IPs. But we also get any tokens that have been created in the last 5 minutes and have only been used once. We fix this by appending AND all_count > 1 to the end.
... View more