Using Web Tools Add-On curl for API requiring field value substitution in URI

[docwindsor]

I need to hit an API with a dynamic path, not one that requires parameters or a body. We need to get information back from a custom, authenticated API that works like whois. I need to pass a header with an authentication token, but the URI needs to end in an IP, like a whois call.

Is there a way to pass each IP found in a search to a service like whois? For example, given the following three IPs how would I use the add-on to make these calls?

https://www.whois.com/whois/1.1.1.1
https://www.whois.com/whois/1.1.1.2
https://www.whois.com/whois/1.1.1.3

Thank you!

[jkat54]

Try this:

|makeresults count=1 | eval names="yahoo.com, msn.com, bing.com" | makemv names delim="," | mvexpand names | fields names | curl method=get uri="https://www.whois.com/whois/" datafield=names

If that works, do this

... some search ... | fields ips| curl method=get uri="https://www.whois.com/whois/" datafield=ips

Or this:

... some search ... 
| fields ipv4 
| map search="
  |curl method=method uri="https://www.whois.com/whois/$ipv4$" user=username pass=password debug=true 
  | table *
  "

If you want to take that map search and push it to summary index, or append it to a lookup for later use... you can do that OR you can continue to enrich and play with the data in the pipeline with standard splunk commands.

[docwindsor]

Really appreciate your help with this!

I actually tried both of those techniques and more, but unfortunately nothing works so far. Our internal REST API requires authentication so I'm using the "headerfield" option to pass a custom header containing an auth token. The method is "GET".

• If I specify the "headerfield" and explicitly list the IP as part of the text of the URI then I get the expected response from the API for the IP in question. So it works for "hard coded" IPs. This shows that it's working and I have the header correct, but isn't useful for searches.
• I also tried concatenating the IP field value to the URI (like you'd do with an eval: uri="http://www.ourcompany.com/internalapi/".ip) but that doesn't work either, and I get a 502 error. From the "curl_message" I can see a message from the API that no IP was passed.
• The "datafield" method above doesn't work because the API I'm hitting won't accept a URL like "http://www.ourcompany.com/internalapi/?1.1.1.1". When I try that technique I get a 404 error. My guess is that "whois" wasn't as good an example as I'd intended since it appears more flexible than our API.
• The "map" method above doesn't seem to pass the "headerfield", which I need for our authentication (we're passing a token in the header field). Using the map technique I get a 401 error. The "curl_response_url" appears correct (and it's hitting the API otherwise it would get a 404), but I guess the header is not being passed? The "curl_status" is 401, but the "curl_message" field contains "{… "error":"received Error Code: 405","success":false}".

(Apologies for lack of specifics but I'm trying to hit a company proprietary, internal API.)

Any other suggestions or thoughts are appreciated!