Splunk Search

Splunk DB Connect Tail Input command not working

scc00
Contributor

Hi,

I'm trying to use the Tail command for Splunk DB Connect to pull the newest logs into a Splunk Index. Unfortunately, it is not pulling the data, even when I generate new logs. I have tried the DUMP option which works just fine but duplicates the entire database each dump so I don't want to use it. I have configured the Tail option as noted within the documentation and have used the same configurations as I used with the DUMP option but it still does not work. I am not using any specific SQL query either. What am I doing wrong?

Tags (2)
0 Karma

pmagee
Explorer

One thing I'm discovering is that the "rising column" and "log date" column names must be specified in ALL CAPS or they won't be recognized. There may be other issues with your particular query as well. I got some insight from checking the $SPLUNK_HOME/var/log/splunk/dbx.log file.

scc00
Contributor

That's probably correct but after a database restart, it started working. I can't say that I can say what the real solution was though. Which is annoying.

0 Karma

pmagee
Explorer

Based on your errors it looks like your connection to the database is failing. I would suspect that there is a space shortage in the database, probably in regards to the audit trail. The audit trail can't expand, so the login fails, which is why the column doesn't show up in the result set.

0 Karma

scc00
Contributor

Well I tried the CAPS option with no results. Though I did check the dbx.log file and found two SQL exceptions. I'm checking with my DBAs but I doubt the database is full. Any other thoughts?

dbx5836:WARN:PooledConnectionFactory - Pooled Connection validation failed: com.splunk.dbx.sql.Database$DatabaseValidationException: Error validating database: java.sql.SQLException: I/O Error: No space left on device
dbx5836:ERROR:TailDatabaseMonitor - Error while executing database monitor: java.sql.SQLException: The column "ERR_ADD_DATE" is not present in the result set

0 Karma
Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

REGISTER NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If ...

Observability | Use Synthetic Monitoring for Website Metadata Verification

If you are on Splunk Observability Cloud, you may already have Synthetic Monitoringin your observability ...

More Ways To Control Your Costs With Archived Metrics | Register for Tech Talk

Tuesday, May 14, 2024  |  11AM PT / 2PM ET Register to Attend Join us for this Tech Talk and learn how to ...