Getting Data In

ARIN Rest API external lookup

asucrews
Path Finder

Hello,

This is my first time creating a external lookup, and I think am missing something. The error I am getting is "Error in 'lookup' command: Could not find all of the specified lookup fields in the lookup table." Can someone please review and let me know what i am missing.

/opt/splunk/etc/apps/soc/local/transforms.conf

[arinrestapi]
 external_cmd = arinRestAPI.py src_ip
 fields_list = abuseemail,company

/opt/splunk/etc/apps/soc/bin/arinRestAPI.py

import csv
import json
import sys
import requests

def abuseEMail(ip):
     try:
         ipUrl = 'https://whois.arin.net/rest/ip/' + ip +'.json'
         r = requests.get(ipUrl)
         org = r.json()
         orgUrl = org['net']['orgRef']['$'] + '/pocs.json'
         r = requests.get(orgUrl)
         poc = r.json()
         abuseUrl = poc['pocs']['pocLinkRef'][2]['$'] + '.json'
         r = requests.get(abuseUrl)
         abuse = r.json()
         return abuse['poc']['emails']['email']['$']
     except:
         return ''

 def company(ip):
     try:
         ipUrl = 'https://whois.arin.net/rest/ip/' + ip +'.json'
         r = requests.get(ipUrl)
         org = r.json()
         orgUrl = org['net']['orgRef']['$'] + '/pocs.json'
         r = requests.get(orgUrl)
         poc = r.json()
         abuseUrl = poc['pocs']['pocLinkRef'][2]['$'] + '.json'
         r = requests.get(abuseUrl)
         abuse = r.json()
         return abuse['poc']['company']['$']
     except:
         return ''


 def main():
     if len(sys.argv) != 2:
         print "Usage: python arinRestAPI.py [ip field]"
         sys.exit(1)

     ipfield = sys.argv[1]

     infile = sys.stdin
     outfile = sys.stdout

     r = csv.DictReader(infile)
     header = r.fieldnames

     w = csv.DictWriter(outfile, fieldnames=r.fieldnames)
     w.writeheader()

     for result in r:
         if result[ipfield]:
         # only ip was provided, add host
             result[abuseEMail] = abuseEMail(result[ipfield])
             result[company] = copmany(result[ipfield])

 main()
0 Karma
1 Solution

asucrews
Path Finder

fix it, well sort of , changed it to one input field to get one output field. not sure if there way to to one input to many outpu fields.

View solution in original post

0 Karma

asucrews
Path Finder

fix it, well sort of , changed it to one input field to get one output field. not sure if there way to to one input to many outpu fields.

0 Karma

youngsuh
Contributor

could you provide step by step procedure?  We need to do same for us.  Thanks in advance for you help.

0 Karma

manish20171
New Member

Can you share the final working version? I am still getting error.

0 Karma

asucrews
Path Finder

I am making procgess but no running in this error "Script for lookup table 'arinrestapi' returned error code 1. Results may be incorrect." I still missing something but I don't understand python enoght or I don't understand what Splunk is doing.

    [arinrestapi]
     allow_caching = 0
     case_sensitive_match = 0
     external_cmd = arinrestapi.py src_ip
     fields_list = src_ip,abuseemail,company

    import csv
     import json
     import sys
     import requests

     def abuseEMail(ip):
         try:
             ipUrl = 'https://whois.arin.net/rest/ip/' + ip +'.json'
             r = requests.get(ipUrl)
             org = r.json()
             orgUrl = org['net']['orgRef']['$'] + '/pocs.json'
             r = requests.get(orgUrl)
             poc = r.json()
             abuseUrl = poc['pocs']['pocLinkRef'][2]['$'] + '.json'
             r = requests.get(abuseUrl)
             abuse = r.json()
             return abuse['poc']['emails']['email']['$']
         except:
             return ''

     def company(ip):
         try:
             ipUrl = 'https://whois.arin.net/rest/ip/' + ip +'.json'
             r = requests.get(ipUrl)
             org = r.json()
             orgUrl = org['net']['orgRef']['$'] + '/pocs.json'
             r = requests.get(orgUrl)
             poc = r.json()
             abuseUrl = poc['pocs']['pocLinkRef'][2]['$'] + '.json'
             r = requests.get(abuseUrl)
             abuse = r.json()
             return abuse['poc']['companyName']['$']
         except:
             return ''

     def main():
         if len(sys.argv) != 2:
             print "Usage: python arinRestAPI.py [ip field]"
             sys.exit(1)

         ipfield = sys.argv[1]

         infile = sys.stdin
         outfile = sys.stdout

         r = csv.DictReader(infile)
         header = r.fieldnames

         w = csv.DictWriter(outfile, fieldnames=r.fieldnames)
         w.writeheader()

         for result in r:
             if result[ipfield]:
                 result[src_ip] = result[ipfield]
                 result[abuseemail] = abuseEMail(ipfield)
                 result[company] = company(ipfield)
                 w.writerow(result)

     main()
0 Karma

asucrews
Path Finder

After playing with script i think my issue is with for result in r loop, but i not really sure.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...