- Splunk Answers
- :
- Splunk Administration
- :
- Deployment Architecture
- :
- ConfObjectManagerDB & HIGH CPU
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I have noticed that every 20-30 minutes we see a hugh CPU spike on two of our indexers. And looking at the splunkd.log file, the CPU spike co-incides with a large number of errors as follows:
02-27-2012 17:33:14.197 +0000 ERROR ConfObjectManagerDB - Ignoring invalid database setting: [inputs/monitor%3A%2F%2F%2Fvar%2Flog] modtime = 1321526706.498255000
02-27-2012 17:33:14.197 +0000 ERROR ConfObjectManagerDB - Ignoring invalid database setting: [inputs/monitor%3A%2F%2F~%2FLibrary%2FLogs] modtime = 1321526706.628675000
02-27-2012 17:33:14.197 +0000 ERROR ConfObjectManagerDB - Ignoring invalid database setting: [inputs/script%3A%2F%2F.%2Fbin%2Fcpu.sh] modtime = 1321526706.896315000
02-27-2012 17:33:14.197 +0000 ERROR ConfObjectManagerDB - Ignoring invalid database setting: [inputs/script%3A%2F%2F.%2Fbin%2Fdf.sh] modtime = 1321526706.990800000
02-27-2012 17:33:14.197 +0000 ERROR ConfObjectManagerDB - Ignoring invalid database setting: [inputs/script%3A%2F%2F.%2Fbin%2Fhardware.sh] modtime = 1321526707.087146000
02-27-2012 17:33:14.197 +0000 ERROR ConfObjectManagerDB - Ignoring invalid database setting: [inputs/script%3A%2F%2F.%2Fbin%2Finterfaces.sh] modtime = 1321526707.179597000
perhaps 500 or so lines...
I am assuming this is related to the deployment manager or bundle replication, but a quick google hasn't given me much to go on...
Does anyone have any suggestions where to start looking?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
After a bit of a wait, Splunk suggested that this could be a bundle issue between the 4.3 searchhead, and 4.2.x indexers.
Whilst this was not listed as a possible problem on the compatibility matrix, (here: http://docs.splunk.com/Documentation/Splunk/4.3.1/Deploy/Whatisdistributedsearch#Cross-version_compa...) since upgrading everything to 4.3.1 the problem has been resolved.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
After a bit of a wait, Splunk suggested that this could be a bundle issue between the 4.3 searchhead, and 4.2.x indexers.
Whilst this was not listed as a possible problem on the compatibility matrix, (here: http://docs.splunk.com/Documentation/Splunk/4.3.1/Deploy/Whatisdistributedsearch#Cross-version_compa...) since upgrading everything to 4.3.1 the problem has been resolved.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
We have a similar setup, running on 4.2.1, and 4.2.0 on our two indexer servers (Linux 64bit on search heads and indexers). I see that both servers threw this message but I cannot see any CPU problems on any of the servers. We use HW based servers. We around 80 forwarders connected to our system and it lost connections to many of them at the same time.
- Unable to distribute to peer named indexer02 at uri https://IP:8089 because replication was unsuccessful. replicationStatus Failed
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Do you have any update on this error? We are experiencing the same problems here.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
No. I also have a ticket open with spunk support since the 27th, and have not heard anything back yet.
Can you post an overview of your environment, incase there are any similarities i can update the ticket with.
We have 1 search head, with a pool of two indexers.
We see the errors on BOTH indexers at the same time.
Indexer1 is a deployment server, which deploys to about 12 universal forwarders, and we have a few manually configured forwarders too.
Our entire deployment is hosted in AWS.
i'll update this ticket when i get an answer from support, but in the meantime.... any ideas????
Splunk Custom Visualizations App End of Life
Introducing Splunk Enterprise 9.2
Adoption of RUM and APM at Splunk
