Dashboards & Visualizations

How do I extract an XML payload from a Java CXF LoggingInInterceptor log entry?

sjnorman
Explorer

Our application is using CXF interceptors to log XML SOAP requests and responses. The format of the log entries is:

2014-06-24 07:35:03,597 INFO  com.foo.bar.Test WebContainer : 5 - Inbound Message
 ---------------------------
ID: 7232  
Response-Code: 200  
Encoding: UTF-8  
Content-Type: text/xml  
Headers: {$WSCS=[RC4-MD5], $WSIS=[true], ...  
Payload: <soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"><soap:Body><MyXmlMessage>....</MyXmlMessage></soap:Body></soap:Envelope>
 --------------------------------------

Is there any way to have Splunk (whether through configuration, a search query, etc.) extract the XML payload part of the log entry? We'd like to be able to run queries against the XML to look for specific element values.

In case it makes a difference, the SOAP payload log entries are intermixed with other application-specific log entries.

Tags (3)
0 Karma
1 Solution

martin_mueller
SplunkTrust
SplunkTrust

Once you've extracted the XML string into the Payload field you could do this in a search:

... | spath input=Payload

That'll look at the content of Payload and extract all fields it can find. If you're looking for a specific value only you can add an xpath-style selector as well, see http://docs.splunk.com/Documentation/Splunk/6.1.1/SearchReference/spath for reference.

View solution in original post

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

Once you've extracted the XML string into the Payload field you could do this in a search:

... | spath input=Payload

That'll look at the content of Payload and extract all fields it can find. If you're looking for a specific value only you can add an xpath-style selector as well, see http://docs.splunk.com/Documentation/Splunk/6.1.1/SearchReference/spath for reference.

0 Karma

sjnorman
Explorer

Works, thanks!

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

That'll create a field called soap:Envelope.soap:Body.MyXmlMessage with the value ...., just as you'd expect.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

Splunk will eat that. Look at this:

| stats count | eval _raw = "2014-06-24 07:35:03,597 INFO  com.foo.bar.Test WebContainer : 5 - Inbound Message
 ---------------------------
ID: 7232  
Response-Code: 200  
Encoding: UTF-8  
Content-Type: text/xml  
Headers: {$WSCS=[RC4-MD5], $WSIS=[true], ...  
Payload: <soap:Envelope xmlns:soap=\"http://schemas.xmlsoap.org/soap/envelope/\"><soap:Body><MyXmlMessage>....</MyXmlMessage></soap:Body></soap:Envelope>
 --------------------------------------"
| rex "(?s)Payload: (?<payload>.*)\s+-{30,}" | spath input=payload
0 Karma

sjnorman
Explorer

The format for a single log entry is as you see above (starts with date and ends with a dashed line) and contains line breaks, even between XML elements.

In addition, there are XML namespaces and prefixes in some of the XML elements such as soap:Body -- when I tried substituting the SOAP body with Splunk returns an error indicating that there is a missing terminator. I assume the ':' needs to be escaped somehow?

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

That shouldn't be a problem. Add a field extraction with this expression:

Payload: (?<Payload>[^\n\r]+)

That's assuming your XML has no line breaks. To test you can use inline rex like this:

... | rex "Payload: (?<Payload>[^\n\r]+)" | spath input=Payload
0 Karma

sjnorman
Explorer

Thanks...just need to find out how to get the Payload field extracted first.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...