Getting Data In

Is it possible to ingest XML?

nick405060
Motivator

It is 2019 and there is still not a comprehensive Splunk Answer or Documentation on how to ingest XML.

Can someone explain to me how to configure props to ingest

<?xml version="1.0" encoding="utf-8"?>
  <ArrayOfUser xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <User>
      <Id>removed</Id>
      <Uuid>removed</Uuid>
      ... many more attributes at this same level ...
    <User>
    <User>
      <Id>removed</Id>
      <Uuid>removed</Uuid>
      ... many more attributes at this same level ...
    <User>
0 Karma

woodcock
Esteemed Legend

Just bring it in and set KV_MODE = xml in props.conf for your sourcetype on your Search Head(s).

0 Karma

smoir_splunk
Splunk Employee
Splunk Employee

This is not going to be props for your specific case (assuming you want each one of those users to be a separate event) but a similar example for props and transforms that I have to extract tracks from an itunes library XML file.
They're also visible here:
https://github.com/smoreface/music_app_for_splunk/blob/master/default/transforms.conf
https://github.com/smoreface/music_app_for_splunk/blob/master/default/props.conf

transforms stanza:

[itunes_xml]
CLEAN_KEYS = true
FORMAT = $1::$2
REGEX = <key>([^<]+)</key><[^>]+>([^<]+)</

props stanza:
[itunes_xml]
FIELDALIAS-iTunes_xml_Normie = Album AS album Artist AS artist Date_Added AS date_added Name AS track_name Play_Count AS play_count Play_Date AS last_played Play_Date_UTC AS last_played_utc Rating AS rating Release_Date AS release_date Size AS file_size Total_Time AS track_length Track_Number AS track_number
SEDCMD-xml&to& = s/&#38;/&/g

Example XML being parsed:

<dict>
            <key>Track ID</key><integer>10815</integer>
            <key>Size</key><integer>4338490</integer>
            <key>Total Time</key><integer>216816</integer>
            <key>Track Number</key><integer>1</integer>
            <key>Track Count</key><integer>10</integer>
            <key>Year</key><integer>2004</integer>
            <key>Date Modified</key><date>2007-01-20T22:07:34Z</date>
            <key>Date Added</key><date>2008-07-27T03:52:43Z</date>
            <key>Bit Rate</key><integer>160</integer>
            <key>Sample Rate</key><integer>44100</integer>
            <key>Play Count</key><integer>1</integer>
            <key>Play Date</key><integer>3319660819</integer>
            <key>Play Date UTC</key><date>2009-03-12T07:00:19Z</date>
            <key>Skip Count</key><integer>1</integer>
            <key>Skip Date</key><date>2010-06-14T22:40:10Z</date>
            <key>Persistent ID</key><string>36990211F06BD125</string>
            <key>Track Type</key><string>File</string>
            <key>File Folder Count</key><integer>5</integer>
            <key>Library Folder Count</key><integer>1</integer>
            <key>Name</key><string>Cry</string>
            <key>Artist</key><string>Sirens</string>
            <key>Album</key><string>Tied To The Mast</string>
            <key>Genre</key><string>Pop</string>
            <key>Kind</key><string>MPEG audio file</string>
            <key>Location</key><string>file:///Users/user/Music/iTunes/iTunes%20Music/Music/Sirens/Tied%20To%20The%20Mast/01%20Cry.mp3</string>
        </dict>
        <key>10817</key>
        <dict>
            <key>Track ID</key><integer>10817</integer>
            <key>Size</key><integer>4082943</integer>
            <key>Total Time</key><integer>254093</integer>
            <key>Track Number</key><integer>1</integer>
            <key>Track Count</key><integer>2</integer>
            <key>Date Modified</key><date>2008-01-15T02:13:52Z</date>
            <key>Date Added</key><date>2008-07-27T03:52:43Z</date>
            <key>Bit Rate</key><integer>128</integer>
            <key>Sample Rate</key><integer>44100</integer>
            <key>Play Count</key><integer>19</integer>
            <key>Play Date</key><integer>3441386101</integer>
            <key>Play Date UTC</key><date>2013-01-19T04:35:01Z</date>
            <key>Skip Count</key><integer>1</integer>
            <key>Skip Date</key><date>2009-02-10T22:07:13Z</date>
            <key>Rating</key><integer>40</integer>
            <key>Album Rating</key><integer>20</integer>
            <key>Album Rating Computed</key><true/>
            <key>Persistent ID</key><string>36990211F06BD130</string>
            <key>Track Type</key><string>File</string>
            <key>File Folder Count</key><integer>5</integer>
            <key>Library Folder Count</key><integer>1</integer>
            <key>Name</key><string>Gone</string>
            <key>Artist</key><string>Straight No Chaser</string>
            <key>Kind</key><string>AAC audio file</string>
            <key>Location</key><string>file:///Users/user/Music/iTunes/iTunes%20Music/Music/Straight%20No%20Chaser/Unknown%20Album/01%20Gone.m4a</string>
        </dict>

Hope this helps!

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...