Accessing results via API: User 'appID' could not act as : userID

[tavor999]

For this question consider we have two environments 'test' and 'prod'. We have the exact same scheduled search setup to target the similar data streams for each environment. The action is to use the HTTP POST. Upon receiving this POST our servers make a request to the API to fetch the full results. In the 'test' Splunk instance this is working fine. In the 'prod' instance we get the following error.

<?xml version="1.0" encoding="UTF-8"?>
<response>
  <messages>
    <msg type="FATAL">User '<appID>' could not act as: <userID></msg>
  </messages>
</response>

I have built the scheduled search in each UI using my but set the permissions wide open. The is our application ID which is used for our API interactions. The appID has no problems getting a session but when trying to get the results we see the above.

URL example for getting results:

https://<host>:<port>/servicesNS/<appID>/search/search/jobs/<sid>/results?offset=0&count=0

In both Splunk instances the search is in the "search" app with permissions set to read write for all users.

[tavor999]

We still have a bit of work to identify low level details but have solved this problem. It was solved by moving the scheduled search to a pool of search heads in another data center which are behind our API load balancer vip. Essentially my API request was going to a pool of servers in data center A and the scheduled search was running on a pool of servers in data center B. It worked without issue when we moved the scheduled search to data center A.

Thanks to jkat54 to throwing out ideas.

[jkat54]

The search is being dispatched as userID, and appID doesnt have permission to ready userID's searches.

Please check dispatchAs in savedsearches.conf:

dispatchAs = [user|owner]
* When the saved search is dispatched via the "saved/searches/{name}/dispatch"
  endpoint, this setting controls, what user that search is dispatched as.
* This setting is only meaningful for shared saved searches.
* When dispatched as user it will be executed as if the requesting user owned
  the search.
* When dispatched as owner it will be executed as if the owner of the search
  dispatched it no matter what user requested it.
* If the 'force_saved_search_dispatch_as_user' attribute, in the limits.conf
  file, is set to true then the dispatchAs attribute is reset to 'user' while
  the saved search is dispatching.
* Defaults to owner.

[jkat54]

Is <appid> in your example a placeholder for an actual appid you have? As in you're not sharing the app id with us?

It looks like the user doesnt exist in production and its telling you "yourAppId" cant act as "bob" when youre accessing
https://:/servicesNS/yourAppId/search/search/jobs/bob/results?offset=0&count=0

Is it possible that your appId is being used for both appID and SID in the link?

https://<host>:<port>/servicesNS/<appID>/search/search/jobs/<sid>/results?offset=0&count=0

However there isnt an SID in production that matches AppId?

[tavor999]

Correct. is simply my replacement for my real app id. When I query for the results I have a unique SID which I am using in the place of .

<appID> = my application user id with access to API
 <sid> = unique search id as provided by the HTTP POST from the 'action' associated with the scheduled search. 

thanks.

[jkat54]

and <appid> = <userid> in this message?

 <msg type="FATAL">User '<appID>' could not act as: <userID></msg>

[tavor999]

they are placeholders for the real names which I would prefer not to post in a forum 🙂

[jkat54]

Does  <appid> = <userid> in this message?

  <msg type="FATAL">User '<appID>' could not act as: <userID></msg>

[tavor999]

No. appID is our applicaiton ID with API access. userID is my user id which was used to create and schedule the search via the UI. Again in the test splunk instance it is working with identify permissions on the search but in prod it is not and I am receiving the above error.

[jkat54]

I assume it doesnt, which means the knowledge object (in this case a saved search), needs to be shared with appID. OR maybe the saved search is owned by userID, and set to execute as "user" vs "owner", and so when appID posts to the API, the search executes as userID and appID doesnt have permission on the SID but userID does.

Make sense?