Microsoft Azure Active Directory Reporting Add-on: HTTPError: 401 Client Error: Unauthorized for url: https://graph.microsoft.com

[nathanpyen]

Hello everyone,

We installed MS Azure AD Reporting Add-on version 1.1.0, and getting the following error messages in ta_ms_aad_MS_AAD_audit.log:
(signins.log has its own errors too, but would like to tackle this first)

2019-05-30 12:01:03,779 INFO pid=17581 tid=MainThread file=connectionpool.py:_new_conn:758 | Starting new HTTPS connection (1): 127.0.0.1
2019-05-30 12:01:06,523 INFO pid=17581 tid=MainThread file=connectionpool.py:_new_conn:758 | Starting new HTTPS connection (1): 127.0.0.1
2019-05-30 12:01:08,871 INFO pid=17581 tid=MainThread file=setup_util.py:log_info:114 | Log level is not set, use default INFO
2019-05-30 12:01:08,871 INFO pid=17581 tid=MainThread file=splunk_rest_client.py:_request_handler:100 | Use HTTP connection pooling
2019-05-30 12:01:08,873 INFO pid=17581 tid=MainThread file=connectionpool.py:_new_conn:758 | Starting new HTTPS connection (1): 127.0.0.1
2019-05-30 12:01:09,181 ERROR pid=17581 tid=MainThread file=base_modinput.py:log_error:307 | Get error when collecting events.
Traceback (most recent call last):
  File "/opt/splunk/etc/apps/TA-MS-AAD/bin/ta_ms_aad/modinput_wrapper/base_modinput.py", line 127, in stream_events
    self.collect_events(ew)
  File "/opt/splunk/etc/apps/TA-MS-AAD/bin/MS_AAD_audit.py", line 76, in collect_events
    input_module.collect_events(self, ew)
  File "/opt/splunk/etc/apps/TA-MS-AAD/bin/input_module_MS_AAD_audit.py", line 57, in collect_events
    audit_events = azutils.get_items(helper, access_token, url)
  File "/opt/splunk/etc/apps/TA-MS-AAD/bin/azure/utils.py", line 20, in get_items
    raise e
HTTPError: 401 Client Error: Unauthorized for url: https://graph.microsoft.com/beta/auditLogs/directoryAudits?$orderby=activityDateTime&$filter=activityDateTime+ge+2019-05-30T04:00:00Z

I read the questions related to "HTTPError: 401 Client Error" message on other post, and it could be related to API permission issue. However, I believe we have all the API permissions that we would need. In the Azure, the app has the following API permissions:

Azure Active Directory (7)
- Directory.AccessAsUser.All Delegated Access the directory as the signed-in user
- Directory.Read.All Application Read directory data
- Directory.ReadWrite.All Delegated Read and write directory data
- Member.Read.Hidden Application Read all hidden memberships
- User.Read Delegated Sign in and read user profile
- User.Read.All Delegated Read all users' full profiles
- User.ReadBasic.All Delegated Read all users' basic profiles
Azure Service Management (1)
- user_impersonation Delegated Access Azure Service Management as organization users (preview)
Microsoft Graph (5)
- AuditLog.Read.All Delegated Read audit log data
- Directory.Read.All Delegated Read directory data
- User.Read Delegated Sign in and read user profile
- User.Read.All Delegated Read all users' full profiles
- User.ReadBasic.All Delegated Read all users' basic profiles

Could anyone point us what are we doing wrong and are we missing?

Thank you.

[evania]

Hi @nathanpyen ,

Did you have a chance to check out any answers? If any work, please resolve this post by approving it! If your problem is still not solved, keep us updated so that someone else can help you.

Thanks for posting!

[jconger]

The application registration needs to be in the Security Reader role for the subscription also.

Here's a spreadsheet that's a work-in-progress detailing the add-ons and the permissions needed -> https://docs.google.com/spreadsheets/d/1YJAqNmcXZU-7O9CxVKupOkR6q2S8TXriMeLAUMYmMs4/edit?usp=sharing

[nathanpyen]

jconger,

The application is called, Splunk-Integration, and it has Security Reader, Monitoring Reader, and Reader roles for the subscription.

[nathanpyen]

Okay, after granting following 3 API Application permissions,
- AuditLog.Read.All
- Directory.Read.All
- User.Read.All
We are no longer seeing the original error message, HTTPError: 401 Client Error.

However, add-on is not returning any events. Do you know what are we missing?

2019-05-30 21:09:53,122 INFO pid=6852 tid=MainThread file=connectionpool.py:_new_conn:758 | Starting new HTTPS connection (1): 127.0.0.1
2019-05-30 21:09:54,399 INFO pid=6852 tid=MainThread file=connectionpool.py:_new_conn:758 | Starting new HTTPS connection (1): 127.0.0.1
2019-05-30 21:09:56,595 INFO pid=6852 tid=MainThread file=splunk_rest_client.py:_request_handler:100 | Use HTTP connection pooling
2019-05-30 21:09:56,596 DEBUG pid=6852 tid=MainThread file=binding.py:get:664 | GET request to https://127.0.0.1:8089/servicesNS/nobody/TA-MS-AAD/storage/collections/config/TA_MS_AAD_checkpointer (body: {})
2019-05-30 21:09:56,597 INFO pid=6852 tid=MainThread file=connectionpool.py:_new_conn:758 | Starting new HTTPS connection (1): 127.0.0.1
2019-05-30 21:09:56,602 DEBUG pid=6852 tid=MainThread file=connectionpool.py:_make_request:387 | "GET /servicesNS/nobody/TA-MS-AAD/storage/collections/config/TA_MS_AAD_checkpointer HTTP/1.1" 200 5307
2019-05-30 21:09:56,603 DEBUG pid=6852 tid=MainThread file=binding.py:new_f:71 | Operation took 0:00:00.007515
2019-05-30 21:09:56,604 DEBUG pid=6852 tid=MainThread file=binding.py:get:664 | GET request to https://127.0.0.1:8089/servicesNS/nobody/TA-MS-AAD/storage/collections/config/ (body: {'search': 'TA_MS_AAD_checkpointer', 'offset': 0, 'count': -1})
2019-05-30 21:09:56,608 DEBUG pid=6852 tid=MainThread file=connectionpool.py:_make_request:387 | "GET /servicesNS/nobody/TA-MS-AAD/storage/collections/config/?search=TA_MS_AAD_checkpointer&offset=0&count=-1 HTTP/1.1" 200 4505
2019-05-30 21:09:56,609 DEBUG pid=6852 tid=MainThread file=binding.py:new_f:71 | Operation took 0:00:00.005022
2019-05-30 21:09:56,611 DEBUG pid=6852 tid=MainThread file=binding.py:get:664 | GET request to https://127.0.0.1:8089/servicesNS/nobody/TA-MS-AAD/storage/collections/data/TA_MS_AAD_checkpointer/F... (body: {})
2019-05-30 21:09:56,614 DEBUG pid=6852 tid=MainThread file=connectionpool.py:_make_request:387 | "GET /servicesNS/nobody/TA-MS-AAD/storage/collections/data/TA_MS_AAD_checkpointer/FCC_Azure_AD_Audits_last_date HTTP/1.1" 200 102
2019-05-30 21:09:56,615 DEBUG pid=6852 tid=MainThread file=binding.py:new_f:71 | Operation took 0:00:00.003437
2019-05-30 21:09:56,621 DEBUG pid=6852 tid=MainThread file=connectionpool.py:_new_conn:809 | Starting new HTTPS connection (1): login.microsoftonline.com
2019-05-30 21:09:56,839 DEBUG pid=6852 tid=MainThread file=connectionpool.py:_make_request:400 | https://login.microsoftonline.com:443 "POST /bb320f37-44f0-4d6d-bd7e-1e5b79f0e15d/oauth2/v2.0/token HTTP/1.1" 200 1582
2019-05-30 21:09:56,843 DEBUG pid=6852 tid=MainThread file=connectionpool.py:_new_conn:809 | Starting new HTTPS connection (1): graph.microsoft.com
2019-05-30 21:09:56,942 DEBUG pid=6852 tid=MainThread file=connectionpool.py:_make_request:400 | https://graph.microsoft.com:443 "GET /beta/auditLogs/directoryAudits?$orderby=activityDateTime&$filter=activityDateTime+ge+2019-05-30T04:00:00Z HTTP/1.1" 200 None
2019-05-30 21:09:56,944 DEBUG pid=6852 tid=MainThread file=base_modinput.py:log_debug:286 | **Total directory audit events returned: 0*
2019-05-30 21:09:56,944 DEBUG pid=6852 tid=MainThread file=binding.py:post:736 | POST request to https://127.0.0.1:8089/servicesNS/nobody/TA-MS-AAD/storage/collections/data/TA_MS_AAD_checkpointer/b... (body: {'body': '[{"_key": "FCC_Azure_AD_Audits_last_date", "state": "\"2019-05-30T04:00:00Z\""}]'})
2019-05-30 21:09:56,953 DEBUG pid=6852 tid=MainThread file=connectionpool.py:_make_request:387 | "POST /servicesNS/nobody/TA-MS-AAD/storage/collections/data/TA_MS_AAD_checkpointer/batch_save HTTP/1.1" 200 35
2019-05-30 21:09:56,953 DEBUG pid=6852 tid=MainThread file=binding.py:new_f:71 | Operation took 0:00:00.008755*