Find closest events in 2 indexes

[vgarmash]

Hi.

We have distributed production environment with IHS as a HTTP server (3 hosts). Access logs from those hosts joined into index="app-prd-web". Then we have J2EE application deployed to cluster of Websphere Application Servers (9 hosts) with all application logs joined to index="application-prd". We seeing some FileNotFoundException errors in the index="application-prd" and we want to know what URL was used on the web when this happened.

To do that we would like to search in index="app-prd-web" for the closest preceding event to the timestamp of the FileNotFoundException in index="application-prd".

I couldn't find the proper example in Splunk documentation to do that. I have 2 separate searches but I can't figure out the way to join them.

Query 1: index="applicaiton-prd" AND FileNotFoundException

Query 2: index="app-prd-web" AND 200 AND http://*.do

I understand that I need to use transaction or subsearch... Could you please help me?

[martin_mueller]

Here's what you may be looking for:

index="applicaiton-prd" FileNotFoundException | localize timebefore=5s timeafter=1s | map search="search earliest=$starttime$ latest=$endtime$  index="app-prd-web" 200 http://*.do";

That will run a search for every FNFE with a timerange spanning five seconds before to one second after the exception, looking into the other type of data.

A thought: If 200 stands for the HTTP status code and you have that extracted as a field, consider using status=200 instead to boost readability.