Monitoring Splunk

Audit.log restart_splunkd- What produces these messages, and how can I tell if splunkd restarted?

hartsoftware
Engager

I'm seeing many action=restart_splunkd messages from my "_audit" index. I can tell from my processor status that splunkd is not restarting, yet I'm receiving these messages in my _audit index. Can someone help me understand what produces these messages? Also, how can I tell when splunkd actually did restart?

Thanks.

Labels (1)
Tags (2)

som_shekhar
New Member

Hi ,I see this noise in Splunk 8.0.1 also.

0 Karma

andrewtrobec
Motivator

Splunk 8.0.5 too.

0 Karma

araitz
Splunk Employee
Splunk Employee

This is some unfortunate noise from the audit handler. In the future, we hope to improve the audit logging. Genti's answer is correct regarding detecting actual shut downs.

ckurtz
Path Finder

Occurring in 5.0.4, too. Always nice to see the official answer from Genti! (He was here last week helping us)

0 Karma

the_wolverine
Champion

It is still occurring in version 5.0.3.

0 Karma

Genti
Splunk Employee
Splunk Employee

Yeap, 2 more bugs submitted regarding the above

0 Karma

Genti
Splunk Employee
Splunk Employee

Actually, if you notice audit.log will have this message logged every minute, and sometimes more then once per minute. (ie. it sends the action twice - or at least logs it twice)
For real splunkd restart check your splunkd.log (located at /spluhome/var/log/splunk/) for messages like:

10-21-2010 14:40:17.044 INFO  loader - Splunkd starting (build 82143).

and

10-21-2010 14:40:13.029 INFO  ShutdownHandler - Shutdown complete in 2125.5 milliseconds

wandrilleD
Engager

It looks like it's still occuring in newer versions, we are currently in 6.4 and still the same problem.

My question is, with your solution above, it's not possible to track which user did launch the restart?

0 Karma

samsplunks
Explorer

Fast forward to 2019, Splunk 7, the bug is still happening.

One dashboard queries and evals action="restart_splunkd" which causes an Audit:[timestamp=XXX, user=XXX, action=restart_splunkd, info=granted][n/a] log to appear in the _audit index with an audittrail sourcetype (everytime the dashboad is reloaded).

0 Karma

JosephHobbs
Path Finder

Almost 2023 in Splunk 9.x and it's still an issue...

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...