All Apps and Add-ons

Data not ingesting into splunk from RabbitMQ queue

getmesomedata
Explorer

I'm having some issues trying to get my data from my RabbitMQ instance into splunk.

I've completed the following steps:
- Enabled the STOMP protocol in my installation of RabbitMQ
- Installed the STOMP app in my spunk instance and setup a data input to listen to my queue (127.0.0.1\topic\testQueue)
- Published some messages onto the queue which results in no data in splunk.

I've checked the list of connections within RabbitMQ and there is a connection from splunk so I know that part has worked. I've checked the splunk internal errors and I can't see anything relating to the STOMP app.

Can you suggest any other logs for me to check or is there anything obvious I've missed out?

0 Karma
1 Solution

allenta
Explorer

And the 'mysterious' buffering issue is now fixed! Please, upgrade to v0.3 and check if your problem persists.

Thank you!

View solution in original post

0 Karma

allenta
Explorer

And the 'mysterious' buffering issue is now fixed! Please, upgrade to v0.3 and check if your problem persists.

Thank you!

0 Karma

allenta
Explorer

Great 🙂

The issue was trivial. A forgot flush call in the stream which connects the modular input and Splunk. A beginner's mistake.

0 Karma

getmesomedata
Explorer

Success, v0.3 works a charm! Thanks

Out of curiosity what was the issue?

0 Karma

allenta
Explorer

Hi getmesomedata!

The steps you've followed are perfectly correct. It would be helpful if you can make a quick test in order to check if the issue you're experiencing is related with a strange behaviour we are still researching.

We've detected some kind of event buffering somewhere in between RabbitMQ and Splunk. Due to that 'mysterious' buffering, if you test the STOMP modular input with only a few messages, they arrive at Splunk, but they are never rendered in the UI until the buffer is completely filled. So, please, repeat your test with 100 or more messages (you can use the producer.py script in https://github.com/allenta/splunk-stomp/tree/master/extras/clients if you want). Let us know if that way you are able to see the enqueued messages in the Splunk Search UI.

Thank you for the report!

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...