The current symptom is more relevant to default value of returned event of Splunk rest api, in this case, "services/data/models" without  specifying "count" parameter. The relevant rest api is referenced in the following python script and by default, the rest api retrieves 30 items. $SPLUNK_HOME/etc/apps/splunk_app_addon-builder/bin/tabuilder_utility/ko_util/cim_util.py  def _get_cim_by_rest(service):     url = "/services/data/models" Please check how many data models are with global permission in the affected Search Head where AoB(Addon Builder is installed) by clicking "Settings" -> "Data models" and count the number of Datamodels with "Global" under "Sharing" field(the very end). If "Vulnerabilities" or "Web" data models are displayed at the end(beyond 30th), that might be the cause of this symptom.  Changing the permission a couple of data models(which are not being used) from global to app, so that "Vulnerabilities" or "Web" data model can be returned within the first 30 items from  "services/data/models" rest api without "count" parameter. ... View more

According to https://docs.splunk.com/Documentation/DBX/3.1.4/DeployDBX/Commands, first parameter OUT SYS_REFCURSOR for Oracle database procedure. If you use an Oracle database, you must store the result in the first parameter and set it as OUT SYS_REFCURSOR. See the Oracle Stored procedure example below for details. If analyticsutil.closebatch does not contain OUT SYS_REFCURSOR as the first parameter, please try to include it. Simple example can be found for an Oracle procedure call in the following URL that I created some time ago. https://answers.splunk.com/answers/657219/procedure-in-splunk-dbconnect-31.html ... View more

dbxquery command has been rewritten in Java in Splunk DB Connect 3.x and has been included in $SPLUNK_HOME/etc/apps/splunk_app_db_connect/jars/command.jar file. You can find the relevant information in commands.conf DBX 2.4.x : $SPLUNK_HOME/etc/apps/splunk_app_db_connect/default/command.conf [dbxquery] filename = dbxquery.py supports_getinfo = true supports_rawargs = true passauth = true run_in_preview = false local = true generating = true streaming = false DBX 3.x.x : $SPLUNK_HOME/etc/apps/splunk_app_db_connect/default/command.conf [dbxquery] filename = java.path chunked = true #command.arg.1 = -Xdebug #command.arg.2 = -Xrunjdwp:transport=dt_socket,address=192.168.33.1:9999,suspend=y command.arg.1 = -Dlogback.configurationFile=../config/command_logback.xml command.arg.2 = -DDBX_COMMAND_LOG_LEVEL=INFO command.arg.3 = -cp command.arg.4 = ../jars/command.jar command.arg.5 = com.splunk.dbx.command.DbxQueryCommand # Keystore password #command.arg.6 = changeme ... View more

On creating a DB connection, there is a section(please see below) where identity previously create should be put. ORA-01017 error indicates that the credential on Oracle DB side does not match. You can revisit the identity you created and double check whether username and password is correct. ... View more

As HTTP 400 Error in DB Connect app typically can come from wrong events data itself, you can check HEC events whether they show any missing metadata or garbage data by enabling TRACE before they are uploaded to HEC. 1] TRACE can be enabled without stopping Splunk using DB Connect -> Configuration -> Settings -> Logging 2] Once the affected input is run and events are not shown on Splunk side after TRACE is enabled, you can check splunk_app_db_connect_server.log if there is any missing metadata like source/sourcetype/index/host at the end of events or any garbage or null included in the event. Example 2019-03-12 20:09:00.025 -0400 [QuartzScheduler_Worker-16] DEBUG c.s.d.s.dbinput.task.processors.EventMarshaller - action=finish_format_hec_events record=Record: {header=[RisingInputRecordHeader{risingColumnValue='25059'} number=8, source="dbx3", creationDate="2019-03-12 16:08:07.0"], payload=[{"time":"1552421287.000","event":"2019-03-12 16:08:07.000, ID=\"25059\", SALES_MANAGER=\"Nick Everd\", PRODUCT=\"0\", SALES_QTY=\"52\", SALES_AMOUNT=\"52000\", SALES_DATE=\"2019-03-12 20:08:07.0\"",**"source":"dbx3","sourcetype":"dbx3","index":"dbx3","host":"kheolin01"**}]} ... View more

The symptom is supposed to be fixed in the latest release(DB Connect app 3.14) as below: https://docs.splunk.com/Documentation/DBX/3.1.4/ReleaseNotes/Releasenotes DBX-4623 Compatibility issue with splunk_app_db_connect 3.1.3 and TA-Proofpoint-TAP 1.3.117 keeps DBX from loading. ... View more

The symptom is supposed to be fixed in the latest release(DB Connect app 3.14) as below: https://docs.splunk.com/Documentation/DBX/3.1.4/ReleaseNotes/Releasenotes DBX-4623 Compatibility issue with splunk_app_db_connect 3.1.3 and TA-Proofpoint-TAP 1.3.117 keeps DBX from loading. ... View more

The alert is triggered when the percentage of a small bucket(by definition, less than 10% of maxDataSize for the index) is created more than the current thresholds(30) for the last 24 hours. Please check the relevant configuration file as below: $SPLUNK_HOME/etc/system/default/health.conf [feature:buckets] display_name = Buckets .. indicator:buckets_created_last_60m:yellow = 40 indicator:percent_small_buckets_created_last_24h:description = This indicator tracks the percentage of small buckets created over the last 24 hours. A small bucket is defined as less than 10 % of the ‘maxDataSize’ setting in indexes.conf. indicator:percent_small_buckets_created_last_24h:red = 50 indicator:percent_small_buckets_created_last_24h:yellow = 30 If you'd like to disable or suppress the message, please check the following Splunk doc. http://docs.splunk.com/Documentation/Splunk/7.2.1/DMC/Configurefeaturemonitoring As hot buckets roll to warm whenever Splunk is restarted, warm buckets can be created with a smaller size than the specified max size(‘maxDataSize’) for the index. If this is the case in your environment, Splunk does not have control over this behaviour and having smaller buckets will not affect any performance issue. If smaller buckets come from other reasons, you may need to investigate the reason. ... View more

If you'd like to create a custom field at search time in Splunk, props.conf is the one where you need to put proper regex. Please start looking at http://docs.splunk.com/Documentation/Splunk/7.2.0/Knowledge/Createandmaintainsearch-timefieldextractionsthroughconfigurationfiles and check whether the field you have configured does appear on "INTERESTING FIELDS" pane on the left side in Search & Reporting app. ... View more

At the time of this writing, mentioned in http://docs.splunk.com/Documentation/DBX/3.1.3/DeployDBX/Prerequisites#Java_Runtime_Environment_.28JRE.29, Splunk DB Connect only supports the Oracle JRE. taskserver may not be invoked with the following similar message. JRE Status: Unsupported JRE detected Using Oracle Corporation JRE version 1.8, OpenJDK 64-Bit Server VM Need Oracle Corporation JRE version 1.8 But supporting Open JDK is on the way starting from DB Connect 3.1.4 which is the next release. This schedule can change without any notice. If this is not supported even with DB Connect 3.1.4, please submit a Splunk support case and request the progress of DBX-2531. ... View more

There is a few steps to check whether taskserver functions properly. 1] Check whether java program is running using "ps -ef | grep java" command Example output splunk 8371 8123 0 01:11 ? 00:05:20 /opt/splunk/jdk1.8.0_181/bin/java -Ddw.server.applicationConnectors[0].port=9998 -jar /opt/splunk/HF2/etc/apps/splunk_app_db_connect/linux_x86_64/bin/../../jars/server.jar 2] If java process with server.jar is not up and running, check whether java path is configured properly in $SPLUNK_HOME/splunk_app_db_connect/your_platform/bin/customized.java.path and then correct java path in your system if it points to the wrong path. Example of customized.java.path in /opt/splunk/HF2/etc/apps/splunk_app_db_connect/linux_x86_64/bin /opt/splunk/jdk1.8.0_181/bin/java 3] If Java process is running with server.jar, check the connectivity to task server using CURL command after login to the Splunk server where DB Connect app is installed Example curl -k -u admin:password https://localhost:<mgmt_port>/servicesNS/nobody/splunk_app_db_connect/db_connect/dbxproxy/taskserver curl -k http://localhost:9998/api/taskserver curl --noproxy localhost -k http://localhost:9998/api/taskserver Example output {"status":"**RUNNING**","schedulerStatus":0,"message":"task server is running","info":{"ProcessName":"8371@kheolin05.sv.splunk.com","Uptime":"57931358","javaVMName":"Java HotSpot(TM) Server VM","javaVersion":"1.8.0_181","TotalMemory":"357302272","javaVendor":"Oracle Corporation","javaSpecificationVersion":"1.8","MaxMemory":"954466304","FreeMemory":"92405888"},"serverId":"16df13b1-549c-4d42-9a79-1e0ae5d163fd"}[/opt/splunk/HF2/etc/apps/splunk_app_db_connect/linux_x86_64/bin:HF2]$ ** If curl command only works with "--noproxy localhost" option, check whether http_proxy is configured in your system environment variable and then configure no_proxy environment variable for localhost and restart Splunk instance Example no_proxy=localhost,127.0.0.1 ** Here tcp port# 9998 is default port for taskserver, port for taskserver can be found in step#1 above in port=9998 ... View more

The reason why all the available sourcetypes are not lised is because currently, the following SPL is used to bring available sourcetypes internally. | metadata type=sourcetypes Splunk has identified this issue internally and the fix is on the way using the following SPL, instead. | metadata type=sourcetypes index=* You can run the SPL above to check the difference from your DB Connect search screen. That being said, DB Connect will not block the input creation and users still can enter available sourcetype although the drop down does not list the sourcetype. Please check the following Splunk documentation as reference. http://docs.splunk.com/Documentation/DBX/3.1.3/DeployDBX/Createandmanagedatabaseinputs#Parameter_settings Source type: Enter a sourcetype field value for Splunk Enterprise to assign to queried data as it is indexed. Click the field and enter a value, or choose an existing value from the menu that appears. ... View more

Windows Authentication Domain setting is only Microsoft SQL database, not for Oracle database as described in the following Splunk Document. http://docs.splunk.com/Documentation/DBX/3.1.3/DeployDBX/Createandmanageidentities Use Windows Authentication Domain: This setting is for identities that connect to Microsoft SQL Server databases. Enable this setting if you need to specify a Windows Authentication Domain. ... View more

Can you try the following syntax? | dbxquery connection="Monitor_MSSQL" procedure="{call Staging.[dbo].[mon_GetCountRecentOrdersFailures_App](?)}" params="1" I could not test the above on MS SQL server but I have tested stored procedures on Oracle database in which the first parameter is reserved for output for dbxquery command. 1] Create a stored procedure using the following SQL CREATE OR REPLACE PROCEDURE dbx( p_ref_cursor OUT SYS_REFCURSOR, id IN VARCHAR, dep_id IN VARCHAR) AS BEGIN OPEN p_ref_cursor FOR SELECT * FROM employees WHERE employee_id = id and department_id = dep_id; END dbx; 2] Run dbxquery command. | dbxquery connection=oracle procedure="{call dbx(?,?,?) }" params="173,81" This worked for me. ... View more

"Splunkd daemon is not responding: The read operation timed out" error indicates that generating PDF program timed out. It's worth increasing splunkdConnectionTimeout attribute in web.conf. splunkdConnectionTimeout = <integer> * Number of seconds to wait before timing out when communicating with splunkd * Must be at least 30 * Values smaller than 30 will be ignored, resulting in the use of the default value * Defaults to 30 ... View more

The current issue has been fixed in Windows TA 4.8.4 onwards, so please download the latest version Windows TA and test or try using the following Regex. blacklist1 = EventCode="4662" Message="Object Type:(?!\s*groupPolicyContainer)" ... View more