Splunk Search

Extracting fields in payload xml multiline log

changux
Builder

Hi all.

I need help setting an input and extracting multiline fields with one entry like this:

####<May 2, 2015 23:37:26 PM PCT> <Warning> <TGFG Logging> <host> <source> <[ACTIVE] ExecuteThread: '26' for queue: 'jboss.kernel.Default (self-tuning)'> <<user>> <> <y7433edf553abf7c3:-12a453ef2:148c05609dd:-5000-000000000000342> <54564444> <JB-000000> < [FILE34, null, null, REQUEST] <!--- Input BD Profiles -->: <soapenv:Body xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
  <ns:RunService xmlns:ns="http://mysite.com/s3" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
    <ns:data>
      <con:java-content ref="jcid:-12ac44f2:35t54ed33:-4ea8" xmlns:con="http://www.jobss.com/wli/sb/context"/>
    </ns:data>
  </ns:RunService>
</soapenv:Body>>

Useful info:

  • All the events begin with the same:

  • I need to extract the field "Status", in the example is the single word between the < > after the timestamp (<Warning>).

  • Also, i need to extract the <host>

  • I need to extract the <source>

  • Everything left must be called "Payload".

Anyone can help me?

Thanks!

0 Karma
1 Solution

woodcock
Esteemed Legend

Like I said, fix your linebreaking and timestampinginprops.conf` first or you'll never get anything off the ground but once you do, this will work:

| rex "^####<[^>]+>\s*<(?<Status>[^>]+)>\s*<[^>]+>\s*<(?<Host>[^>]+)>\s*<(?<Source>[^>]+)>\s*(?<Payload>.*)$"

View solution in original post

woodcock
Esteemed Legend

Like I said, fix your linebreaking and timestampinginprops.conf` first or you'll never get anything off the ground but once you do, this will work:

| rex "^####<[^>]+>\s*<(?<Status>[^>]+)>\s*<[^>]+>\s*<(?<Host>[^>]+)>\s*<(?<Source>[^>]+)>\s*(?<Payload>.*)$"

changux
Builder

You rock! thanks!

0 Karma

changux
Builder

Works very nice. How i can extract only the "payload"?

Thank you so much!

0 Karma

woodcock
Esteemed Legend
| rex "^####<[^>]+>\s*<[^>]+>\s*<[^>]+>\s*<[^>]+>\s*<[^>]+>\s*(?<Payload>.*)$"

Don't forget to click "Accept".

0 Karma

aljohnson_splun
Splunk Employee
Splunk Employee

If they're already editing props.conf, why would you suggest they use rex instead of adding an EXTRACT ?

0 Karma

lcrielaa
Communicator

| rex field=_raw "####<[^>]+>\s<(?[^>]+)>\s<[^>]+>\s<(?[^>]+)>\s<(?[^>]+)>\s(?.*)"

That'll do the extracting you want but like the other said, you'll need to fix your linebreaking and timestamping via your props.conf.

0 Karma

woodcock
Esteemed Legend

All that you are asking is possible and not too difficult but you are WAY ahead of yourself. You need to get timestamping and linebreaking working first. Are you working on that?

changux
Builder

Yes, but doesn't work properly.

0 Karma

woodcock
Esteemed Legend

Do you have anything at all coming into Splunk yet? In other words, do you have inputs.conf working and timestamp recognition working?

0 Karma

changux
Builder

Not for now. My test extracts fields with the symbols ( <> ), not only the content 😞

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...