Enable REST API response logging in the Splunk Add-on Builder

[samyool36]

I am currently using the Splunk Add-On Builder app to connect to an external REST API in an attempt to ingest data into Splunk. I have configured the token as a REST request header in the Data Input Definition section and also defined a URL. When I attempt to test this connection I receive the following message in the output window:

2019-03-18 11:11:24,600 - test_name - [ERROR] - [test] The response status=500 for request which url=https://myresturl.com/input and method=GET.

I have tested the connection using Postman and a simple curl command and both of these return the required data. Is it possible to turn on response logging for the Splunk Add-On Builder app to determine why the server is returning a 500 error? If so, which files need to be updated for this to happen?

[jeffrey_berry]

I have recently been investigating using the "Splunk Add-on Builder" to ingest data from a REST API, and I had with a similar issue diagnosing the http requests sent from the created add-on app. The Burp Suite Community Edition app (Burp Proxy manual tool) helped troubleshoot the https request from the created add-on app and the https response from the REST API. The created add-on was configured to use the Burp Suite app as a proxy. The https requests from the created add-on are logged in the Burp Suite app, and the responses from the REST API are logged also. Also, the Burp Suite allows the user to intercept the https requests from the created add-on app, and the user can manually modify before forwarding to the REST API.

The Burp Suite Community Edition app is free to download at the https://portswigger.net/burp url.

[johnkimber]

"Internal Server Error" is that the error can only be resolved by fixes to the Web server software. It is not a client-side problem meaning that the problem is not with your browser, your computer, or your internet connection. This error can only be resolved by fixes to the Web server software . It is up to the administrators of the Web server site to locate and analyse the logs which should give further information about the error. However, if you are a web visitor and want to rule out whether the problem is on your end:

• Clear your browser cookies and cache
• Reload or Refresh the Webpage

From Server end:

• Server permission
• Server timeout
• Script timeout
• Errors in .htaccess files
• Check the Error Logs

[lakshman239]

As 500 error code is generic, you may want to go through https://www.lifewire.com/500-internal-server-error-explained-2622938 to rule out any of them.

In the Add-on Setup parameters, you can enable 'Logging settings' so, you can use logging within the add-on logs to troubleshoot further (look at https://docs.splunk.com/Documentation/AddonBuilder/2.2.0/UserGuide/Createasetuppage)

You can then use python helper functions to log the errors
https://docs.splunk.com/Documentation/AddonBuilder/2.2.0/UserGuide/PythonHelperFunctions

helper.log_info('The error code={}".format(error_code))

[chli_splunk]

Do you enable any proxy in AoB? Any specific headers? Or any default license files in your OS?
You can setup log level in global settings, when building the UI.

[samyool36]

Thanks for the response. I don't have any proxy enabled and the only header being sent is the API token. I also am not aware of any default license files.

Would you be able to advise where to set this log level in the global settings? Would this allow me to see the full response from the server?