Hi,
I have something like the following, where I have a message producer and consumer.
I am using ActiveMQ for messaging.
Sometimes I notice that consumer didn't get messages and I'm logging this way:
Producer code: log.info("Status=Produced, TransactionId=123");
Consumer.code: log.info("Status=Consumed, TransactionId=123");
I also have a Dead Letter queue consumer, which logs something like:
DLQConsumer: log.info("Status=Discarded, TransactionId=123");
The whole Producer/Consumer flow is Async.
I need Splunk to alert me when it sees a transaction, that is not processed by Consumer.
How do I write a Splunk search to alert me for these?
In a nutshell what I would like to get reported is that:
All messages produced should be consumed, if not, then I need to get alerted with TransactionId.
Also I don't want to deal with a situation where a message was just produced and not yet consumed, still Splunk reporting it to me.
Maybe I can set the time range as current time - 15 minutes to current time - 1 minute to avoid a situation where a message was just produced and not yet consumed.
... View more