Splunk Dev

Error messages using Google Cloud Platform Add-on

c2bi
New Member

I have configured the add-on for Google Cloud Platform and verified that pub/sub messages are being written to the pub/sub topic from GCP and that messages are successfully being pulled from the Splunk subscription. No GCP logs are appearing in Splunk. The error seems to be internal to Splunk. I have captured error messages from 2 different relevant logs below.

The following log messages are repeated in the /opt/splunk/var/log/splunk/splunk_ta_google_pubsub_util.log

2018-01-04 18:59:53,807 ERROR pid=470 tid=Thread-2 file=event_writer.py:write_events:268 | Failed to post events to HEC_URI=https://127.0.0.1:8088/services/collector, error_code=400, reason={"text":"Invalid data format","code":6,"invalid-event-number":0}
2018-01-04 18:59:54,591 ERROR pid=470 tid=Thread-2 file=event_writer.py:write_events:268 | Failed to post events to HEC_URI=https://127.0.0.1:8088/services/collector, error_code=400, reason={"text":"Invalid data format","code":6,"invalid-event-number":0}

The following log messages are repeated in the /opt/splunk/var/log/splunk/splunk_ta_google_pubsub_main.log

2018-01-04 19:59:23,387 ERROR pid=470 tid=Thread-2 file=google_pubsub_data_loader.py:index_data:70 | Failed to collect data for project=[MY_PROJECT]t, subscription=[MY_SUBSCRIPTION], error=Traceback (most recent call last):
File "/opt/splunk/etc/apps/Splunk_TA_google-cloudplatform/bin/pubsub_mod/google_pubsub_data_loader.py", line 64, in index_data
self._do_safe_index()
File "/opt/splunk/etc/apps/Splunk_TA_google-cloudplatform/bin/pubsub_mod/google_pubsub_data_loader.py", line 86, in _do_safe_index
for msgs in sub.pull_messages():
File "/opt/splunk/etc/apps/Splunk_TA_google-cloudplatform/bin/google_wrapper/pubsub_wrapper.py", line 85, in pull_messages
for message in messages:
TypeError: 'NoneType' object is not iterable

Tags (2)
0 Karma

amarrazzo
Engager

This sounds similar to an issue experienced when attempting to use the add-on with version 7.x of Splunk. I don't see the version of Splunk you are using listed in your question..but if you are using a version newer then 6.5 (the last supported version listed on splunk base for this particular add-on), you might try the below fix.

If google_global_settings.conf does not already exist in the local directory, create the file, if not, add the below stanza to it, then restart your instance. The conf file path would be:

$SPLUNK_HOME/etc/apps/Splunk_TA_googlecloudplatform/local/google_global_settings.conf

And the stanza:

[global_settings]
 use_hec = 0
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...