All Apps and Add-ons

AMQP Messaging Modular Input Documentation: What are Index Message Properties and Index Message Envelope?

mathiask
Communicator

I have some Questions that are not specified in the Addon Documentation

Message Processing

Index Message Envelope: What does it do?
Index Message Properties: What does it do?

0 Karma
1 Solution

Damien_Dallimor
Ultra Champion

When an AMQP message is received , there are several parts to it : message envelope , message properties , message body.

The options in the stanza setup allow you to choose what parts of the message you wish to index in Splunk using the default AMQP response handler. (you can also create your own custom response handlers if you wish).

Message Properties

Message Envelope

View solution in original post

Damien_Dallimor
Ultra Champion

When an AMQP message is received , there are several parts to it : message envelope , message properties , message body.

The options in the stanza setup allow you to choose what parts of the message you wish to index in Splunk using the default AMQP response handler. (you can also create your own custom response handlers if you wish).

Message Properties

Message Envelope

mathiask
Communicator

As far as i can see these parameters do not tell Splunk to index these message parts. To my knowledge only the default fields get indexed during index time (except defined otherwise).

But when options are selected some additional information is written to splunk, but this is not indexed.
Is this maybe my fault because i selected sourcetype="_json" (which obviously also doesn't work since the AMQP meta information is given als key=value pairs).

If and only if my observation is correct (the data is indeed not indexed but simply made available) I would suggest rather terms like "written to splunk" or "made available to splunk".

0 Karma

Damien_Dallimor
Ultra Champion

I think you are confusing yourself over definitions of what indexed means vs perhaps what an index time extraction is.

All the data from the mod input gets indexed (written to a bucket on disk) by Splunk.
No additional fields beyond the core meta data are index time extracted by default , they are search time extracted.

0 Karma

mathiask
Communicator

Okay I think I now see the definition problem...

The underlying definition problem is what the scope of the term 'indexed' covers
- The event/data as a whole can be called indexed if it is listed by any parameter in any index, i.e. splunk context: the event is indexed and therefore searchable
- The field/part/subset of the event/data by which the event/data is indexed, i.e. splunk context: index time extracted fields, by which the event gets indexed, http://docs.splunk.com/Splexicon:Indexedfield

So to be more precise.
- Yes the event (including message envelope/properties) is indexed by the default indexed fields and therefore can be retrieved 'faster' using the index, i.e. splunk context: the event and the message envelope/properties are searchable
- No the event is not indexed by additional data found in the message envelope/properties and therefore cannot be retrieved 'faster' by using parameters found in the message envelope/properties, i.e. splunk context: no index time field extraction for the message envelope/properties

0 Karma

Damien_Dallimor
Ultra Champion

Indexing raw events and indexed field extractions are different things by defintion.If you choose to perform your own indexed field extractions, well this is up to the user, but search time field extractions are more optimal.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...