In the process of creating a new app, I found I was getting cross-app module imports if another app has Python modules with the same name. For example: app1/bin/test_script.py app1/bin/test_module.py app2/bin/test_script.py app2/bin/test_module.py If test_script.py does an "import test_module", and I execute "app2/bin/test_script.py", it actually imports/executes "app1/bin/test_module.py" I am writing a new app to import mail headers from audited Gmail accounts, and I used some of the same modules for the credential auth process from the Gsuite for Splunk app. I found when I tried to auth credentials in the Gsuite for Splunk app, it imported and executed the "ga_authorize.py" from my app directory, not the GSuiteForSplunk/bin directory. I added some logging into my copy of the "ga_authorize.py" script to print the sys.path, PYTHONPATH and the path of the script being executed to prove this: 2019-05-27 13:36:07,277 log_level=INFO pid=57869 tid=MainThread file="ga_authorize.py" function="handle_GET" line_number="51" version="GSuiteForSplunk.v1.2.3.b220" operation=build_url 2019-05-27 13:36:07,277 log_level=INFO pid=57869 tid=MainThread file="ga_authorize.py" function="<module>" line_number="32" version="GSuiteForSplunk.v1.2.3.b220" sys.path=['/opt/splunk/lib/python2.7/site-packages/splunk/appserver/mrsparkle', '/opt/splunk/bin', '/opt/splunk/lib/python2.7/site-packages', '/opt/splunk/lib/python27.zip', '/opt/splunk/lib/python2.7', '/opt/splunk/lib/python2.7/plat-linux2', '/opt/splunk/lib/python2.7/lib-tk', '/opt/splunk/lib/python2.7/lib-old', '/opt/splunk/lib/python2.7/lib-dynload', '/opt/splunk/bin', '/opt/splunk/etc/apps/FitbitAddonforSplunk/bin', '/opt/splunk/etc/apps/GSuiteForSplunk/bin', '/opt/splunk/etc/apps/Splunk_SA_CIM/bin', '/opt/splunk/etc/apps/Splunk_Security_Essentials/bin', '/opt/splunk/etc/apps/Splunk_TA_aws/bin', '/opt/splunk/etc/apps/Splunk_TA_nix/bin', '/opt/splunk/etc/apps/TA-GMail-audit/bin', '/opt/splunk/etc/apps/TA-GMail-audit_PROD/bin', '/opt/splunk/etc/apps/TA-cyberchef/bin', '/opt/splunk/etc/apps/TA-fitbit/bin', '/opt/splunk/etc/apps/TA-gmail-audit/bin', '/opt/splunk/etc/apps/alert_logevent/bin', '/opt/splunk/etc/apps/alert_webhook/bin', '/opt/splunk/etc/apps/base64/bin', '/opt/splunk/etc/apps/cis-controls-app-for-splunk/bin', '/opt/splunk/etc/apps/introspection_generator_addon/bin', '/opt/splunk/etc/apps/kapsch_roadside/bin', '/opt/splunk/etc/apps/lookup_editor/bin', '/opt/splunk/etc/apps/rest-storage-passwords-manager/bin', '/opt/splunk/etc/apps/search/bin', '/opt/splunk/etc/apps/splunk_app_addon-builder/bin', '/opt/splunk/etc/apps/splunk_archiver/bin', '/opt/splunk/etc/apps/splunk_instrumentation/bin', '/opt/splunk/etc/apps/splunk_monitoring_console/bin', '/opt/splunk/etc/system/bin', '/opt/splunk/etc/apps/TA-gmail-audit/bin/lib'] 2019-05-27 13:36:07,277 log_level=INFO pid=57869 tid=MainThread file="ga_authorize.py" function="<module>" line_number="31" version="GSuiteForSplunk.v1.2.3.b220" PYTHONPATH=/opt/splunk/lib/python2.7/site-packages 2019-05-27 13:36:07,274 INFO SCRIPTPATH=/opt/splunk/etc/apps/TA-gmail-audit/bin/ga_authorize.py You can see that the 'app/bin' directory of all apps is included in the sys.path and the name of the script being executed is 'TA-gmail-audit/bin/ga_authorize.py' not 'GSuiteforSplunk/bin/ga_authorize.py' Is this expected behaviour? Is there any way to force Splunk/Python to only import/execute modules under the current app directory, and not pick up the same named modules in other app directories? Keen to hear if anyone else has run into this issue, and if so, what was done to fix/workaround the issue. Thanks. ... View more

SSO with Splunk and PingFederate works well in terms of authenticating, but I can only get it working if "defaultRoleIfMissing" is configured in authentication.conf. If I remove that setting, Splunk does not allow the login and displays: "No valid Splunk role is found in the local mapping or in the assertion." I have confirmed that the Splunk role to SAML group mapping is definitely configured correctly, trying through both the config files and the web UI, but neither seems to work. I turned on debugging and got the below XML from the idP response. What is strange is that the 'realName' and 'mail' attributes are being pulled correctly from the response, but not the 'role' section. <saml:AttributeStatement> <saml:Attribute Name="realName" NameFormat="urn:oasis:names:tc:SAML:2.0:attrname-format:unspecified"> <saml:AttributeValue xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="xs:string">SplunkUser</saml:AttributeValue> </saml:Attribute> <saml:Attribute Name="role" NameFormat="urn:oasis:names:tc:SAML:2.0:attrname-format:unspecified"> <saml:AttributeValue xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="xs:string">AllUsers</saml:AttributeValue> <saml:AttributeValue xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="xs:string">User</saml:AttributeValue> <saml:AttributeValue xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="xs:string">Employee</saml:AttributeValue> <saml:AttributeValue xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="xs:string">admin2</saml:AttributeValue> <saml:AttributeValue xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="xs:string">admin</saml:AttributeValue> </saml:Attribute> <saml:Attribute Name="mail" NameFormat="urn:oasis:names:tc:SAML:2.0:attrname-format:unspecified"> <saml:AttributeValue xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="xs:string">nick@example.com</saml:AttributeValue> </saml:Attribute> </saml:AttributeStatement> Has anyone else experienced this issue? Any pointers? ... View more

Hi all, I have tried modifying the scheduled alert email actions to use raw and table format for the emailed alert, but both seem to strip out all line breaks from the original _raw field, meaning it is far more difficult to read long, multiline events with deliberate line breaking for legibility. Is there any way to force the emailed alerts to keep the original line breaking? Or any way to make the 'table' command keep the original line breaks? Cheers, Nick v K ... View more

Background: Active and Standby server with key directories replicated periodically (every 5 mins) via rsync, including shell scripts and logs. The active server syncs changes to standby server. All scheduled scripts check for the existence of a 'live server flag file' e.g. /opt/LIVESERVER.txt, on the local filesystem and will not execute if the file does not exist. This way, crontab can be enabled on both systems but scripts will only execute on the live side, where the LIVERSERVER flag file exists. I have a Splunk Universal Forwarder installed on each node. Any scripted inputs can simply have the flag file logic added to them, however I am struggling to work out a way that I can conditionally monitor a log file, i.e. only monitor if some file exists on the local filesystem. At the moment, we manually stop the Splunk Forwarder on the standby side and only ever run the forwarder on the active node. Ideally, I want both forwarders running all the time, so that I can monitor other files/services on both sides, but I don't want to have the rsynced log files read at the live side, the re-read at the standby side. I only ever want the forwarder to monitor log files on the active node. I know that a "monitor_if_exists" option doesn't exist, but something like below would be exactly what I am after: [monitor:///apps/log/test.log] index = test sourcetype = TestLog monitor_if_exists = /opt/LIVESERVER.txt Anyone know of any way to achieve this? ... View more

Hi, I am trying to get an Advanced XML Dashboard with Google Maps setup, and I have got almost everything working, except that I cannot figure out a way to have two Google Maps on the same row in the dashboard. Each map seems to occupy an entire row. I specify "panel_row1_col1" on the first one, the "panel_row1_col2" on the next one, but they always appear on separate rows. I also tried putting a normal Splunk chart next to a Google Map, but that does not work either. Please let me know if there is any way around this. I cannot find one. Thanks and regards, Nick v K ... View more

One of the Splunk users today said to me "Wouldn't it be great if you could get a widget for your desktop that displayed Splunk information?" I can certainly see that this would be handy, where users can subscribe to particular gauges and charts that they want to be able to readily see without even logging onto Splunk. Is this possible, and has anyone already done anything like this with Splunk? Cheers, Nick v K ... View more