Getting Data In

How to add and parse XML data in Splunk?

prijila
Engager

Hi,
I have an XML file as my source file. It has the following structure:

<photos>
     <photo id="123" title="Birthday" format="jpg">
          <owner id="1111">Jason</owner>
          <comments>
              <comment ownerid="112">Good pic!</comment>
              <comment ownerif="223">Happy birthday</comment>
          <comments>
      </photo>
</photos>

How do i parse this and load this data into splunk?

Thank you in advance.

1 Solution

vasanthmss
Motivator

Hi,

sample xml,

<photos>
<photo id="1231" title="Birthday" format="jpg">
<owner id="1112">Jason</owner>
<comments>
<comment ownerid="113">Good pic!</comment>
<comment ownerid="224">Happy birthday</comment>
<comments>
</photo>
</photos>

props.conf

[yoursourcetype]
DATETIME_CONFIG = CURRENT
KV_MODE = xml
LINE_BREAKER = (<photos>)
MUST_BREAK_AFTER = \</photos\>
NO_BINARY_CHECK = 1
SHOULD_LINEMERGE = false
TRUNCATE = 0
pulldown_type = 1
FIELDALIAS-rootfields = photo.owner as Owner photo{@id} as PhotoID photo{@title} as PhotoTitle photo.owner{@id} as PhotoOwnerID photo{@format} as PhotoFormat photo.comments.comment{@ownerid} as CommentOwnerID photo.comments.comment as Comment

Search

index=<yourindex> sourcetype=<yoursourcetype> | table  PhotoID,PhotoTitle,PhotoFormat,CommentOwnerID,Comment,Owner
V

View solution in original post

vasanthmss
Motivator

Hi,

sample xml,

<photos>
<photo id="1231" title="Birthday" format="jpg">
<owner id="1112">Jason</owner>
<comments>
<comment ownerid="113">Good pic!</comment>
<comment ownerid="224">Happy birthday</comment>
<comments>
</photo>
</photos>

props.conf

[yoursourcetype]
DATETIME_CONFIG = CURRENT
KV_MODE = xml
LINE_BREAKER = (<photos>)
MUST_BREAK_AFTER = \</photos\>
NO_BINARY_CHECK = 1
SHOULD_LINEMERGE = false
TRUNCATE = 0
pulldown_type = 1
FIELDALIAS-rootfields = photo.owner as Owner photo{@id} as PhotoID photo{@title} as PhotoTitle photo.owner{@id} as PhotoOwnerID photo{@format} as PhotoFormat photo.comments.comment{@ownerid} as CommentOwnerID photo.comments.comment as Comment

Search

index=<yourindex> sourcetype=<yoursourcetype> | table  PhotoID,PhotoTitle,PhotoFormat,CommentOwnerID,Comment,Owner
V

donaldwayne1975
Path Finder

Very helpful. working with an XML that the designer decided to embed more fields in the value between the tags like this:
$field1=stuff,$field2=morestuff,$field3=evenmorestuff
Have not found a way to get these $fields to extract successfully. Suggestions?

0 Karma

akocak
Contributor

@donaldwayne1975
You can add EVAL into your props for this.

0 Karma

ifremer2015
New Member

Which props.con should we modify ? ../etc/system or ..etc/apps

Thank you !

0 Karma

krishnarajb2304
Explorer

In-case if the sourcetype belongs to one application you can update this in application (/etc/apps//local/props.conf.

The same sourcetype can be re-used in more than one application you can add it in system props.conf
/etc/system/local/props.conf.

0 Karma

aljohnson_splun
Splunk Employee
Splunk Employee
0 Karma

Raghav2384
Motivator

xmlkv and path should help you. Please refer this answer http://answers.splunk.com/answers/54957/how-to-extract-data-from-xml.html
Hope this helps.
Thanks,
Raghav

0 Karma

RMcCurdyDOTcom
Explorer

nmap XML to SPLUNK HEC !!! 

 

https://github.com/freeload101/SCRIPTS/blob/b3f83288a9f289d86f6cdd04898478d0427097ce/Bash/NMAP_FRUIT.sh#L80

 

 

0 Karma

RMcCurdyDOTcom
Explorer

got nasty gram for posting links

search online for freeload101 github in scripts nmap_fruit.sh 

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...