Why am I getting different results from AppInspect API vs CLI?

[splunkian]

I have an app that has modular inputs built with Node.js. When I inspect the app using the AppInspect CLI (1.6.1), it passes no problem. When I try against the AppInspect API, it fails because I have removed the node_modules folder that contains the splunk-sdk, which is used in the modular inputs.

Because the splunk-sdk doesnt exist, the API complains in the response from the AppInspect API:
The xml returned by the scheme is not valid... Cannot find module 'splunk-sdk'

I could leave in the node_modules folder, but AppInspect will fail it because it contains a bunch of hidden files and it adds extra bloat to the app.

Which one should I trust? The CLI or the API? If the API is correct, then how do I resolve this issue so it passes?

[sni_splunk]

Hi splunkian,

The AppInspect API is slightly different with AppInspect CLI and it has more checks, including some runtime checks requiring installing the app into a running Splunk instance and verify the given app's conformance. They are not included in AppInspect CLI for the sake of ease of distribution.

At the same time, AppInspect API service may use a more recent version of AppInspect library (used by the AppInspect CLI as well), and this may cause some difference.

You can trust the result of AppInspect API service since it is more complete and more recent.

In your case, the difference is because of the runtime checks included in AppInspect API. If you remove the node_modules folder, could you confirm if it (after removing the node_modules folder) still works for generating a valid modular input scheme (http://dev.splunk.com/view/python-sdk/SP-CAAAER3) after installing it into Splunk Enterprise?

[splunkian]

Hi Sni – Thanks for the information regarding the differences between the API and CLI versions of AppInspect. To answer your question, if I remove the node_modules folder the modular input will fail to work as it depends on the Node version of the Splunk SDK, which resides in the node_modules folder. Currently, the install process would be to run npm install in the bin/ folder to download the node.js dependencies defined in the package.json file. Let me know if there is a better way to handle this.

[sni_splunk]

@splunkian this means your app is a partial app requiring further update to work, and this makes the verification fail to work. Since you already ask your app users to manually install some dependencies to work, can you try comment out some parts of your app, like the modular input you have, and add an additional step for your users, saying something like "once you install the dependencies, uncomment these lines in inputs.conf to enable it"?

[splunkian]

Thanks @sni, I can try to do something like that. Is that considered to be a 'good practice' in such a scenario?

[sni_splunk]

I don't see many apps requiring install additional dependencies previously. If you create a modular input, and users install your app, they can access the modular input from Splunk web (https://docs.splunk.com/File:Mod_inputs_splunk_web.png a screen shot in the doc, https://docs.splunk.com/Documentation/Splunk/7.2.5/AdvancedDev/ModInputsBasicExample).

As far as I can tell, many users will not read your manual to run npm install before installing your app (check out http://www.readthefuckingmanual.com), they will simply install it and see the modular input fails to work. In that case, one approach is commenting them out so that they don't see it and will have to start reading the manual.