Splunk Search

Need help combining 2 separate searches on different log files and show the results on a single timechart as 2 separate lines

pdpsplunk100
Path Finder

Hi - I'm having trouble in combining 2 separate searches and displaying the results on a single visualization (timechart). search 1 searches for events in the access log and search 2 searches for events in another log file (process log). Both log files have a field called "responseTime". In the process log the responseTime field represents "Processing Time". I'm able to run the searches independently and plot the timechart but when I combine them using join I don't get back any results. If I use append I get back results only from search 2. I am looking for 2 lines on the same timechart - one that represents processing time (from process log) and the other would be response time (from access log).

search1: index=xxx source=/x/x/x/localhost_access_log* field=value1
search2: index=xxx source=/x/x/x/process_log field=value2

Using join I tried the following queries: (No results)

search1 | join [search2 | rename responseTime as processTime] | timechart avg(responseTime), avg(processTime)

(and)

search1 | timechart avg(responseTime) | join [search2 | timechart avg(responseTime)]

Using append: (got back results only from search2)

search1 | append [search2 | rename responseTime as processTime] | timechart avg(responseTime), avg(processTime)
Tags (1)
0 Karma
1 Solution

pdpsplunk100
Path Finder

This did the trick.

index=xxx (source=/x/x/x/process_log AND request_param=xxx) | rename responseTime as processTime | timechart avg(processTime) as "Processing Time" | join _time [search index=xxx (source=/x/x/x/localhost_access_log* AND request_param=yyy) | timechart avg(responseTime) as "Response Time"]

View solution in original post

0 Karma

pdpsplunk100
Path Finder

This did the trick.

index=xxx (source=/x/x/x/process_log AND request_param=xxx) | rename responseTime as processTime | timechart avg(processTime) as "Processing Time" | join _time [search index=xxx (source=/x/x/x/localhost_access_log* AND request_param=yyy) | timechart avg(responseTime) as "Response Time"]
0 Karma

renjith_nair
SplunkTrust
SplunkTrust

Try this

index=xxx (source=/x/x/x/localhost_access_log* OR source=/x/x/x/process_log )|timechart avg(responseTime) as responseTime, avg(processTime) as processTime
Happy Splunking!
0 Karma

pdpsplunk100
Path Finder

I get how you use the OR to search both sources. But the field name is "responseTime" in both the log files. I would need to rename "responseTime" in the process_log to "processTime" before the timechart command. Where/How do I place the rename in your suggested query because I need to tell Splunk that the rename applies only to the process_log source.

0 Karma

sundareshr
Legend

Try this

index=xxx (source=/x/x/x/localhost_access_log* OR source=/x/x/x/process_log )| timechart avg(eval(match(source, "access") as  responseTime, avg(eval(match(source, "process") as processTime
0 Karma

pdpsplunk100
Path Finder

Tried using match but that does not seem to work. I get eval expression is malformed and timechart syntax is incorrect.

I was able to get what I wanted using the query below (using join). I'm still looking for ways to optimize the query as join is expensive but this does the job for now.

index=xxx (source=/x/x/x/process_log AND request_param=xxx) | rename responseTime as processTime | timechart avg(processTime) as "Processing Time" | join _time [search index=xxx (source=/x/x/x/localhost_access_log* AND request_param=yyy) | timechart avg(responseTime) as "Response Time"]

Thanks for your guidance folks.

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...