Splunk Search

merge 2 logs into one with at most one value for each field

shayhibah
Path Finder

Hi,

I would like to combine 2 logs (or more) as the following:

log #1:
time=1563281015|name=sh_lab|a=1|b=2|c=3|

log#2:
time=1563281010|a=2|d=4|

Output should be:
time=1563281015|name=sh_lab|a=1|b=2|c=3|d=4|

(time is taken from the first log since it is the latest log,d is taken from the second log since it does not part of the latest log, all other fields are taken from the first log since it is the latest log).

Is it possible to do it in Splunk?

Tags (1)
0 Karma

chinmoya
Communicator

@shayhibah :

Appendcols might give you what you desire. below is something i tried with your data set.

| makeresults | eval time=1563281015, name="sh_lab", a=1, b=2, c=3
| appendcols [makeresults | eval time=1563281010,a=2,d=4 | table d]

Kindly know that there is a high possibility you might not get correct results.

Because it will correlate the first event of log one with the 1st event of log two, and so on.

It will not care about the time and fields of the events or when events came.

0 Karma

shayhibah
Path Finder

any idea??

0 Karma

niketn
Legend

@shayhibah how have you indexed the two logs? Are they having same/different metadata like index, sourcetype, source? Best approach would be to write you search to pull all required data from index. Then you can apply correlation afterwards (as per your example does not seem like there is any correlation between the two logs).

Please provide more information for the community to assist you better. What is your current SPL? How are you correlating log 1 data with log 2? Will name field serve as the key to correlate and find unique events in each log? How many fields can these logs have?

Based on your explanation so far, you can try something like the following:

index=blah sourcetype IN ("sourcetypeForLog1","sourceTypeForLog2") source="path\log*"
| fields name a,b,c,d,
| stats min(_time) as _time last(*) as * by name
____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...