Monitoring Splunk

DB Connect - OutOfMemoryError for db input

flle
Path Finder

Hi,

I am experiencing an OutOfMemory Error for a specific Oracle DB input. It is one of three dbmon-tail inputs configured within DB connect. Two of them are fine but the third one will crash after a couple of hours. Related messages:

2013-08-16 23:44:07.793 dbx2345:INFO:TailDatabaseMonitor - Executing database monitor=[dbmon-tail://mydb/mytable]
2013-08-16 23:44:07.809 dbx2345:INFO:TailDatabaseMonitor - Applying latest tail.rising.column value=9773010
...
2013-08-16 23:44:19.500 dbx2345:INFO:ExecutionContext - Execution finished in duration=11707 ms
2013-08-16 23:44:19.505 monsch1:ERROR:Scheduler - Caught ExecutionException: java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError
...

So it seems that the query itself is finished successfully again but when the scheduler tries to evaluate the result it crashes (expected would be something like: 2013-08-16 23:44:18.936 monsch1:INFO:Scheduler - Execution of input=[dbmon-tail://mydb/mytable] finished in duration=11707 ms with resultCount=0 success=true continueMonitoring=true). Afterwards this query/input is not executed again until the forwarder is restarted. Important to note is that the resultCount of the query is usually very small or even 0. The table however is pretty big.

I did not find any helpful information on the docu or forum, nor was I able to find a cause for this behaviour.

Has anyone experienced something similar and found a reason for it?

Can I specifically assign an amount of memory to an input or the db connect app / java bridge?

Thanks for any hints.

Florian

0 Karma

vstolya
Explorer

I set DBConnect JVM Commandline options to: -Xmx512m ...

Then from "Java Bridge Server Status" see that almost all memory was in use:
Uptime: 0 days 04:03:05
Memory: TOTAL=288.44 MB MAX=455.13 MB FREE=11.47 MB (96% used)

But, it appears that for my DBConnect it was enough.
Please try to increase memory as mentioned above.

0 Karma

lukejadamec
Super Champion

In case you are wondering, the etc//apps//dbx//default//java.conf file is where the java memory is set. The default is -Xmx256m, which stands for 256MB.

0 Karma

flle
Path Finder

Oracle. Added that info to question.

0 Karma

ziegfried
Influencer

What kind of database are you querying?

0 Karma

vstolya
Explorer

Had similar issue with DBConnect.
What I see from logs above:
2013-08-16 23:44:07.809 dbx2345:INFO:TailDatabaseMonitor - Applying latest tail.rising.column value=9773010
that rising column value is the number, but I think you expected date value.
If so, you need to put proper rising column expression.

0 Karma

flle
Path Finder

No, rising collumn is a number field. As I said, the input works without problems for a few hours but then crashes with OutOfMemory error. You had a similar error? How did you solve it?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...