How to correlate two searches that don't share a common field?

[nterry]

So I am trying to correlate two searches with one another. Unfortunately, I don't have any common fields between the two searches, but I can derive the host field and eval to store it... Here is the search so far:

index=an_index sourcetype=a_sourcetype level=ERROR | rex field=_raw "Application (?<app_sys_name>.*) environments .* timed out waiting to reach a status in .* and health .*"
| append [search index=a_differnet_index sourcetype=a_different_index "/health" | rex field=_raw "HTTP/1.1\" (?<code>\d\d\d) \d+"]
| eval hst=app_sys_name + "-" + deploy_version
| streamstats current=f window=1 last(hst) as hst

So, I get a bunch of results and I want a way to filter out the hosts where host != hst*. The only event that generates a hst field, however, is the single result from the outer search.... I need to use the hst field in the appended results... I thought streamstats would work at the end by copying the field from the last event to the current event, but it didn't....

Can anyone help me?

[sideview]

I think your problem lies in the interpretation of how last() works, and in particular how it works in streamstats.

You have to think of streamstats working through the result rows, row by row starting from the first row to the last row. first(foo) will tell it to grab the first value it sees for that field and never get another value. last(foo) will likewise tell it to use the last value it has seen as it has been going through.

However, streamstats never goes backwards. So it cannot in principle modify an earlier row using data that it has seen in a later row.

So, I think if you're intending to get a set of rows from the first search, tack on another set of rows with append, construct a "hst" field basically only on that second set of rows, and then copy out that hst field onto all the earlier rows, well you may want eventstats instead of streamstats.

eventstats is similar, except it makes a pass through all of the rows, calculates one overall value for last(foo) and first(bar) and avg(baz), and then uses those values on all rows.

Note that both eventstats and streamstats take a "by someField" clause which can be extremely powerful. File that away for a rainy day.