Hi,
I want to compare two fields in a certain timerange. I am working on 2 fields, those are process_ip and transfer_ip. These process and transfer fields has 100,000 values, process_ip count is greater than transfer_ip count most of the values are same except 1 to 20 process_ip values are not matching with transfer_ip values. I wanted to list out those 1-20 process_ip values. I have written below code to compare two fields:
index="index1" process="Process" OR transfer="transfer"
| rex "(?P[a-zA-Z0-9.]+)"
| rex "(hd)\s+(?P[a-zA-Z0-9.]+)"
| streamstats count by ip, ip_P
| stats values(ip_P) AS process_ip, values(ip) AS transfer_ip
| mvexpand process_ip
| fillnull value=NULL transfer_ip
| eval file_types_process=if(process_ip!=transfer_ip, process_ip, NULL)
| where ip_process!="NULL"
| table ip_process
it's not effecting properly on large amount of data. When I mentioned unmatched process_ip values in the search then it's listing properly as non_matched process_ip with transfer_ip in the table ip_process. I tried running this code on 3 modes(Verbose/Smart/Fast) but no luck.
Why Splunk is not able to compare two fields accurately on large amount of data?
Please let me know if i did any mistake in the code or missed anything.
Thanks,
Chandana
Assuming field process_ip and transfer_ip is already extracted (for events with process="Process" and transfer="transfer" respectively), try something like this
index="index1" process="Process" OR transfer="transfer"
| eval common_ip=coalesce(process_ip,transfer_ip)
| eval from=if(process="Process","Process","Transfer")
| stats values(from) as from by common_ip
| where mvcount(from)=1 AND from="Process"
Above should give your all process_ip values which are not available as transfer_ip.
Assuming field process_ip and transfer_ip is already extracted (for events with process="Process" and transfer="transfer" respectively), try something like this
index="index1" process="Process" OR transfer="transfer"
| eval common_ip=coalesce(process_ip,transfer_ip)
| eval from=if(process="Process","Process","Transfer")
| stats values(from) as from by common_ip
| where mvcount(from)=1 AND from="Process"
Above should give your all process_ip values which are not available as transfer_ip.