Splunk Search

Why splunk is not comparing all values from 2 different fileds?

chandana204
Communicator

Hi,

I want to compare two fields in a certain timerange. I am working on 2 fields, those are process_ip and transfer_ip. These process and transfer fields has 100,000 values, process_ip count is greater than transfer_ip count most of the values are same except 1 to 20 process_ip values are not matching with transfer_ip values. I wanted to list out those 1-20 process_ip values. I have written below code to compare two fields:

index="index1" process="Process" OR transfer="transfer" 
| rex "(?P[a-zA-Z0-9.]+)" 
| rex "(hd)\s+(?P[a-zA-Z0-9.]+)" 
| streamstats count by ip, ip_P 
| stats values(ip_P) AS process_ip, values(ip) AS transfer_ip 
| mvexpand process_ip 
| fillnull value=NULL transfer_ip 
| eval file_types_process=if(process_ip!=transfer_ip, process_ip, NULL) 
| where ip_process!="NULL" 
| table ip_process 

it's not effecting properly on large amount of data. When I mentioned unmatched process_ip values in the search then it's listing properly as non_matched process_ip with transfer_ip in the table ip_process. I tried running this code on 3 modes(Verbose/Smart/Fast) but no luck.

Why Splunk is not able to compare two fields accurately on large amount of data?

Please let me know if i did any mistake in the code or missed anything.

Thanks,
Chandana

0 Karma
1 Solution

somesoni2
SplunkTrust
SplunkTrust

Assuming field process_ip and transfer_ip is already extracted (for events with process="Process" and transfer="transfer" respectively), try something like this

 index="index1" process="Process" OR transfer="transfer" 
| eval common_ip=coalesce(process_ip,transfer_ip)
| eval from=if(process="Process","Process","Transfer")
| stats values(from) as from by common_ip
| where mvcount(from)=1 AND from="Process"

Above should give your all process_ip values which are not available as transfer_ip.

View solution in original post

somesoni2
SplunkTrust
SplunkTrust

Assuming field process_ip and transfer_ip is already extracted (for events with process="Process" and transfer="transfer" respectively), try something like this

 index="index1" process="Process" OR transfer="transfer" 
| eval common_ip=coalesce(process_ip,transfer_ip)
| eval from=if(process="Process","Process","Transfer")
| stats values(from) as from by common_ip
| where mvcount(from)=1 AND from="Process"

Above should give your all process_ip values which are not available as transfer_ip.

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...