There is actually one more possible issue - when using a remote Indexer. You need to enable replication of KVStore lookups by added to <app>/local/collections.conf:   [<automatic_lookup_name>_kvstore] replicate = true   ... View more

I ran into this problem and while the dependency for cpu.sh is to have sysstat installed, I also found that df.sh wouldn't parse correctly on Ubuntu until the gawk package was installed (mawk package was already present). Gord T. ... View more

This is a very old posting, but ran into this today with only 1 of my inputs from O365 (total of 6) after changing the Secret token (and even rebooting). I found that deleting and re-creating the input resolved it. I'm guessing that after the Secret was changed, something in the input didn't get properly updated. Thanks, Gord T. ... View more

Sorry... Never closed the loop on this. Yes, adding the AppID to Global Reader role (in addition to the API mentioned above) resolved the issue. Thanks, Gord T. ... View more

Had to do this today because VPN logs for eStreamer returns the IPv4 client_ip field in hex encoded IPv6 format (e.g. 104.33.245.146 is listed as 0000:0000:0000:0000:0000:ffff:6821:f592) index=estreamer | eval A=if(like(client_ip,"0000:0000:0000:0000:0000:ffff:%"), substr(ipv6,31,9) , client_ip) | eval src_ip=if(len(A)==9, tonumber(substr(A,1,2),16). "." .tonumber(substr(A,3,2),16). "." .tonumber(substr(A,6,2),16). "." .tonumber(substr(A,8,2),16), A) Not sure what the use case OP originally was looking for, but hope this helps someone. First eval compares set temp variable A to the hex-encoded IPv4 if matches the format, otherwise stores the IPv6 Second eval parses & converts each IPv4 octet if A is 9 characters long, othewise returns the IPv6 Hope this helps someone else.     ... View more

I know this is an old thread, but since a google search led me to it, others will probably read it too. This solution is defined in this other article: https://community.splunk.com/t5/All-Apps-and-Add-ons/Cisco-eStreamer-eNcore-delay-in-logs-getting-to-Splunk/m-p/464251 TL;DR (or in case the URL above breaks), by default eStreamer only picks up events if there are 100 or more to collect. On low-volume systems, this could be an issue, so there is a batchSize parameter to adjust. The end of my estreamer.conf now has this added line at the end: "workerProcesses": 4, "batchSize": 5 } ... View more

Just don't forget to put a comma at the end of the previous line (figured out my error quick when running ./splencore.sh test). So, then end of your .conf file should look like this: "workerProcesses": 4, "batchSize": 5 } ... View more

@splunklabs Any feedback on this? Has anyone managed to get this working? I've played around with various settings, like providing Organization.Read.All, etc, with no luck. BTW, the confusion around error code is because _auth.log returns 403, but the other log returns 401 (see below). From ta_ms_o365_reporting_ms_o365_message_trace_oauth.log 2022-08-18 19:10:06,228 INFO pid=1745907 tid=MainThread file=setup_util.py:log_info:142 | Log level is not set, use default INFO 2022-08-18 19:10:06,229 INFO pid=1745907 tid=MainThread file=splunk_rest_client.py:_request_handler:99 | Use HTTP connection pooling 2022-08-18 19:10:06,241 INFO pid=1745907 tid=MainThread file=setup_util.py:log_info:142 | Proxy is not enabled! 2022-08-18 19:10:06,443 INFO pid=1745907 tid=MainThread file=setup_util.py:log_info:142 | Proxy is not enabled! 2022-08-18 19:10:07,546 ERROR pid=1745907 tid=MainThread file=base_modinput.py:log_error:316 | _Splunk_ HTTP Request error: 403 Client Error: for url: https://reports.office365.com/ecp/reportingwebservice/reporting.svc/MessageTrace?$filter=StartDate%20eq%20datetime'2022-08-13T19:10:06.241464Z'%20and%20EndDate%20eq%20datetime'2022-08-13T20:10:06.241464Z' 2022-08-18 19:10:07,547 ERROR pid=1745907 tid=MainThread file=base_modinput.py:log_error:316 | Get error when collecting events. Traceback (most recent call last): File "/opt/splunk/etc/apps/TA-MS_O365_Reporting/lib/splunktaucclib/modinput_wrapper/base_modinput.py", line 140, in stream_events self.collect_events(ew) File "/opt/splunk/etc/apps/TA-MS_O365_Reporting/bin/ms_o365_message_trace_oauth.py", line 355, in collect_events get_events_continuous(helper, ew) File "/opt/splunk/etc/apps/TA-MS_O365_Reporting/bin/ms_o365_message_trace_oauth.py", line 96, in get_events_continuous message_response = get_messages(helper, microsoft_trace_url) File "/opt/splunk/etc/apps/TA-MS_O365_Reporting/bin/ms_o365_message_trace_oauth.py", line 74, in get_messages raise e File "/opt/splunk/etc/apps/TA-MS_O365_Reporting/bin/ms_o365_message_trace_oauth.py", line 66, in get_messages r.raise_for_status() File "/opt/splunk/lib/python3.7/site-packages/requests/models.py", line 940, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 403 Client Error: for url: https://reports.office365.com/ecp/reportingwebservice/reporting.svc/MessageTrace?$filter=StartDate%20eq%20datetime'2022-08-13T19:10:06.241464Z'%20and%20EndDate%20eq%20datetime'2022-08-13T20:10:06.241464Z'   From ta_ms_o365_reporting_ms_o365_message_trace.log 2022-08-18 19:06:22,623 INFO pid=1743945 tid=MainThread file=setup_util.py:log_info:142 | Log level is not set, use default INFO 2022-08-18 19:06:22,623 INFO pid=1743945 tid=MainThread file=splunk_rest_client.py:_request_handler:99 | Use HTTP connection pooling 2022-08-18 19:06:22,692 INFO pid=1743945 tid=MainThread file=setup_util.py:log_info:142 | Proxy is not enabled! 2022-08-18 19:06:27,816 ERROR pid=1743945 tid=MainThread file=base_modinput.py:log_error:316 | _Splunk_ HTTP Request error: 401 Client Error: for url: https://reports.office365.com/ecp/reportingwebservice/reporting.svc/MessageTrace?$filter=StartDate%20eq%20datetime'2022-08-03T01:59:37.734559Z'%20and%20EndDate%20eq%20datetime'2022-08-03T02:59:37.734559Z' 2022-08-18 19:06:27,817 ERROR pid=1743945 tid=MainThread file=base_modinput.py:log_error:316 | Get error when collecting events. Traceback (most recent call last): File "/opt/splunk/etc/apps/TA-MS_O365_Reporting/lib/splunktaucclib/modinput_wrapper/base_modinput.py", line 140, in stream_events self.collect_events(ew) File "/opt/splunk/etc/apps/TA-MS_O365_Reporting/bin/ms_o365_message_trace.py", line 357, in collect_events get_events_continuous(helper, ew) File "/opt/splunk/etc/apps/TA-MS_O365_Reporting/bin/ms_o365_message_trace.py", line 99, in get_events_continuous message_response = get_messages(helper, microsoft_trace_url, global_microsoft_office_365_username, global_microsoft_office_365_password) File "/opt/splunk/etc/apps/TA-MS_O365_Reporting/bin/ms_o365_message_trace.py", line 74, in get_messages raise e File "/opt/splunk/etc/apps/TA-MS_O365_Reporting/bin/ms_o365_message_trace.py", line 68, in get_messages r.raise_for_status() File "/opt/splunk/lib/python3.7/site-packages/requests/models.py", line 940, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 401 Client Error: for url: https://reports.office365.com/ecp/reportingwebservice/reporting.svc/MessageTrace?$filter=StartDate%20eq%20datetime'2022-08-03T01:59:37.734559Z'%20and%20EndDate%20eq%20datetime'2022-08-03T02:59:37.734559Z'   ... View more

There's a new version of the TA just released that now supports OAuth, but the permissions required are unclear: https://community.splunk.com/t5/All-Apps-and-Add-ons/OAuth-permissions-for-Splunk-Add-on-for-Microsoft-Office-365/m-p/608348#M77317   ... View more

There's an updated TA posted now to support OAuth, but the permission requirements are unclear: https://community.splunk.com/t5/All-Apps-and-Add-ons/OAuth-permissions-for-Splunk-Add-on-for-Microsoft-Office-365/m-p/608348#M77317 ... View more

A little more investigation on this, and there appears to be inconsistent information in the Meraki documentation on this. The top row of the table in this document states in the "device flow" information is available via API, but this document list URLs & Flows as Syslog messages, and documents Event Log separately (and I believe ONLY eventlog details are sent to Meraki Cloud). Can anyone confirm whether Flows and/or URL events are eventually planned? For now, it looks like syslog is my best choice. ... View more

What event should I be looking for that tells me exactly which lookup caused the reference cylce? I can't seem to find the relevant event. Thanks, Gord T. ... View more

I haven't seen this behaviour. Did you change *boht* lines that contain $filter ?? ... View more

That add-on should point to this URL for instructions on how to configure Cloudflare logging: https://developers.cloudflare.com/logs/about If you download and unzip the add-on, you'll find this URL in the readme.txt, but it should really be added to the Overview page on Splunkbase. Gord T. ... View more

I had the exact same error with the ServiceNow add-on (which uses a script to inject incident tickets into SNow). The permission I had to grant was "list_storage_passwords". I believe, because the script is being run as the user who created the alert, the user must be able to read the credentials from storage, and pass these to the python script. ... View more

I resolved this in part due to Raúl Marín's excellent writeup and youtube video (https://raulmarin.me/2020/04/26/netflow-traffic-ingestion-with-splunk-stream-and-an-independent-stream-forwarder/ & https://www.youtube.com/watch?v=Usjy5NF0rwE, respectively). He was dealing with a similar issue of errors when choosing the 2nd option (Collect data from other machines). It turns out *both* options will give an error if the host's name isn't defined in /etc/system/local/inputs.conf. So, by adding this stanza, and restarting Splunk, that problem went away:   [default] host = splunk-hostname   The wizard then started to prompt me to run the set_permissions.sh  command. After adjusting  permissions on the script using the command below, then running the script and restarting Splunk again, everything went smoothly sudo chmod +x ./set_permissions.sh Thanks, Gord T. ... View more

@mikefg Thanks for this. Don't know why this isn't in the installation portion of the documentation instead of finding it in the Troubleshooting section. Splunk Gods should fix that (or use a relative path in props.conf instead so no manual changes needed). ... View more

I believe the key permission that you're missing is Directory.Read.All (Application).   I consolidated my O365, AzureAD, and GraphAPI permissions into a single appID with the following permissions which work fine: Microsoft Graph:     AuditLog.Read.All (Application)     Diretory.Read.All (Application)     SecurityEvents.Read.All (Application)     User.Read (Delegated) <--- this is set by default     User.Read.All (Application) Office 365 Management APIs:     ActivityFeed.Read (Application)     ActivityFeed.ReadDlp (Application)     ServiceHealth.Read (Application) Hope this helps! ... View more

That looks fine. Agree with @isoutamo - time to look at UF splunkd.log & metrics.log files. As an aside, you might want to make sure there aren't any Firewall, HIPS, or other  security tools that might also be blocking outbound connections (and "ps -ef" to make sure splunk is actually running and not just crashing). ... View more

You should have 2 stanzas in outputs.conf: [tcpout] ...bunch of stuff from default.conf... defaultGroup = MyIndexer ...bunch of stuff from default.conf... [tcpout:MyIndexer] server = indexer.domain.com:9997   ... View more