There is a post regarding Nested searches which got me thinking about a problem i've been having. I have a very heterogenous collection of log data which spans data ranging from apache logs to tomcat logs to homegrown logs. The issue i'm having is how to pipe the output of subsearch into another search across a different sourcetype. Or would it be better to use transaction to auto-magically find the correlated data i'm looking for? Here's what i'm trying to do.
I "can" search my apache access logs to find errors we've thrown to our clients (error=blah) sourcetype=access. I want to then search our proprietary application logs to find details for same error (code=blah) sourcetype=events. Finally because there may be multiple sourcetype=events which match error code=blah I want to find which of the code=blah generated the error which was displayed in sourcetype=access (error=blah). Just to throw a further wrench into the works I want to further deep dive and search sourcetype=log4j for any other details I can use to get a picture of this events leading up to sourcetype=access (error=blah). I don't provide an error field for sourcetype=log4j because there isn't one:D.
I have three searches I use to solve the above problem but for the sake of my sanity i'm asking if I can nest these searches or somehow pipe the results into another search.
find errors i care about in the sourcetype=access -
sourcetype=access [ search sourcetype="access" Error="532" | fields client_ip_logformat_token,date_hour,date_minute,date_second ] | stats count by post_authid,date_hour,date_minute,date_second
manually take the post_authid date_hour, date_minute and (date_second +/- 1) second values and search event_log
sourcetype=event_log UserID=(numeric value) date_hour=(hr) date_minute=(min) date_second=(second-1)
(it would be cool to create a form searchable dashboard, but i'll work on that later.)
finally search the tomcat logs
sourcetype=log4j authid=(numeric value) date_hour=(hr) date_minute=(min) date_second=(second-1)
I've tried searches along these lines, but I have no clue what i'm looking at:
(sourcetype=event_log OR sourcetype=access) 532 | eval code=Error | transaction code maxspan=1s
code is the field extraction for sourcetype=event_log - Error is the field extraction for sourcetype=access if figured "tie-ing" them together would be me the result I wanted... Nope.
I've been recently trying in vain to use pipe subsearch results into a new search across sourcetypes... No love.
Thanks in advance,
Deeboh
... View more