All Apps and Add-ons

DB Connect and SQLite database locking

responsys_cm
Builder

I have about 30 searches that run once per day and pull data from a SQLite database and write them to Splunk lookup tables. Most of the searches only take a few seconds to run. Each search is scheduled a minute apart. Yet it seems that I'm getting errors when most of the scripts run indicating the database is locked because it is busy:

2016-09-20 00:00:34.936 dbx8905:ERROR:DatabaseQueryCommand - SQL exception occurred: [SQLITE_BUSY] The database file is locked (database is locked) java.sql.SQLException: [SQLITE_BUSY] The database file is locked (database is locked) at org.sqlite.DB.newSQLException(DB.java:383) at org.sqlite.DB.newSQLException(DB.java:387) at org.sqlite.DB.execute(DB.java:339) at org.sqlite.Stmt.exec(Stmt.java:65) at org.sqlite.Stmt.executeQuery(Stmt.java:122) at com.splunk.dbx.sql.Database.execute(Database.java:236) at com.splunk.dbx.command.DatabaseQueryCommand.executeCommand(DatabaseQueryCommand.java:83) at com.splunk.runtime.SplunkCommand.invoke(SplunkCommand.java:237) at com.splunk.bridge.session.BridgeSession.call(BridgeSession.java:92) at com.splunk.bridge.session.BridgeSession.call(BridgeSession.java:30) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source)

Is there any way to adjust the DBX timeout or force it to only use a single thread when accessing a SQLite database?

0 Karma

jcoates_splunk
Splunk Employee
Splunk Employee

Once upon a time I worked for a company that reengineered its policy engine to sit on SQLite... and that happened. I think the query_timeout value in inputs.conf might do it for you? http://docs.splunk.com/Documentation/DBX/2.3.0/DeployDBX/inputsspec

Another idea might be to change your searches to dbxquery statements that | collect into indexes, though doing that 30 times sounds tedious.

0 Karma
Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Wondering How to Build Resiliency in the Cloud?

IT leaders are choosing Splunk Cloud as an ideal cloud transformation platform to drive business resilience,  ...

Updated Data Management and AWS GDI Inventory in Splunk Observability

We’re making some changes to Data Management and Infrastructure Inventory for AWS. The Data Management page, ...