Splunk Search

Join 2 events with same "source"

henryyiu2degree
Engager

I want to join the nmap scanning results. The common field is the source "nmapscan_1.gnmap" while other scans will have a different source name.

Event 1

# Nmap 5.51 scan initiated Tue Dec 11 10:54:16 2017 as: nmap -A -T4 -oA scan_192.168.1.0_24_20171219 192.168.1.0/24
host =nmapserver   source =nmapscan_1.gnmap   sourcetype =nmap

Event 2

# Nmap done at Tue Dec 11 12:20:04 2017 -- 256 IP addresses (81 hosts up) scanned in 5147.70 seconds
host =nmapserver   source =nmapscan_1.gnmap   sourcetype =nmap

I want to get the following results:

Time, Subnet, #Hosts up, Duration
Tue Dec 11 10:54:16 2017, 192.168.1.0/24, 81, 5147.70

Could I use the "source" as a common field and how to do this? Thanks.

Tags (2)
0 Karma
1 Solution

micahkemp
Champion

If each file monitored represents a different scan then you can definitely use source for transaction.

Try this:

| rex "scan_(?<network>([0-9]{1,3}\.){3}[0-9]{1,3})_(?<masklen>[0-9]+)_"
| eval subnet=network."/".masklen
| rex "\((?<hosts_up>[0-9]+) hosts up\)"
| transaction startswith=eval(isnotnull(subnet)) endswith=eval(isnotnull(hosts_up)) source
| table _time subnet hosts_up duration

View solution in original post

0 Karma

micahkemp
Champion

If each file monitored represents a different scan then you can definitely use source for transaction.

Try this:

| rex "scan_(?<network>([0-9]{1,3}\.){3}[0-9]{1,3})_(?<masklen>[0-9]+)_"
| eval subnet=network."/".masklen
| rex "\((?<hosts_up>[0-9]+) hosts up\)"
| transaction startswith=eval(isnotnull(subnet)) endswith=eval(isnotnull(hosts_up)) source
| table _time subnet hosts_up duration
0 Karma

henryyiu2degree
Engager

This works perfectly, thanks for the help.

I have changed a little to use the last subnet as that one is generated by nmap:

| rex field=_raw ".*scan\sinitiated\s.*\s(?<subnet>\d+.\d+.\d+.\d+\/\d\d)"
| rex field=_raw "\((?<hosts_up>\d+)\shosts\sup\)\sscanned\sin\s(?<duration>\d+).\d\d\sseconds"
| transaction startswith=eval(isnotnull(subnet)) endswith=eval(isnotnull(hosts_up)) source
| table _time, subnet, hosts_up, duration
0 Karma

mayurr98
Super Champion

hey @henryyiu2degrees

No, you cannot use the source as a common field. As in that particular source field, there might be plenty of events so essentially there is nothing common as such in these particular two events. Also, the source is metadata field it is not something that is coming from logs.
So if you want to correlate these two events, then you need to have something common in the raw logs i.e. host_ip OR username userid which is a unique identifier only for those particular two events.

I hope this explanation helps you understand these things!

0 Karma

mayurr98
Super Champion

Also try this

index=index1  OR index=index2  (source=source1 OR source=source2)
| rex field=_raw "gtid\((?P<trans_id>[^\)]+)"
| stats values(_raw) as raw_event by trans_id

Let me know what you get

0 Karma

henryyiu2degree
Engager

Thanks @mayurr98 for your comment.

I afraid there is no common fields in the raw logs but could I join "scan initiated" and the next "Nmap done" together?

0 Karma

mayurr98
Super Champion

I don’t think but you can try below search

index=your_index | transaction startswith=“scan initiated” endswith=“Nmap done”

After running this search see if like events are getting correlated although it is not recommended to use this command unless you have common field but just check !

Let me know if this works!

0 Karma

henryyiu2degree
Engager

I have tried this and work as well, but the others answer is more accurate.
Appreciate your help, thanks a lot.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...