Getting Data In

Can someone help me parse this XML into Splunk on the backend? Also should I add this to the props.config in my Deployment Server or my Search Head Deployer?

rshoun
Explorer

Here is what I have tried and it is not working:

Edit the local/inputs.conf file and add this:

[monitor:///directory/*.xml]
sourcetype = panorama:api:templates
index =  qt_palo
crcSalt = Belton Palo-Templates
alwaysOpenFile = 1
disabled = false

[monitor:///directory/*.xml]
sourcetype = panorama:api:templates
index =  qt_palo
crcSalt = Tulsa Palo-Templates
alwaysOpenFile = 1
disabled = false

[monitor:///directory/*.xml]
sourcetype = panorama:api:devicegroups
index =  qt_palo
crcSalt = Belton Palo-Templates
alwaysOpenFile = 1
disabled = false

[monitor:///directory/*.xml]
sourcetype = panorama:api:devicegroups
index =  qt_palo
crcSalt =  Belton Palo-Device Groups
alwaysOpenFile = 1
disabled = false

Create a local/props.conf file and add this:

[panorama:api:templates]
EXTRACT-Status = (?i)<response status>(?P<Status>[^<]+)
EXTRACT-Name = (?i)<entry name>(?P<Name>[^<]+)
EXTRACT-TemplateStack = (?i)<template-stack>(?P<TemplateStack>[^<]+)
EXTRACT-EntryName = (?i)<entry name>(?P<EntryName>[^<]+)
EXTRACT-Serial = (?i)<serial>(?P<Serial>[^<]+)
EXTRACT-Connected = (?i)<connected>(?P<Connected>[^<]+)
EXTRACT-UnsupportedVersion = (?i)<unsupported-version>(?P<UnsupportedVersion>[^<]+)
EXTRACT-LastCommitAllStateTPL = (?i)<last-commit-all-state-tpl>(?P<LastCommitAllStateTPL>[^<]+)
EXTRACT-LastValidateAllStateTPL= (?i)<last-validate-all-state-tpl>(?P<LastValidateAllStateTPL>[^<]+)


EXTRACT-Connected = (?i)<XMLtagInRawEvent1>(?P<YourXMLExtractionName1>[^<]+)
EXTRACT-Connected = (?i)<XMLtagInRawEvent1>(?P<YourXMLExtractionName1>[^<]+)
EXTRACT-Connected = (?i)<XMLtagInRawEvent1>(?P<YourXMLExtractionName1>[^<]+)
EXTRACT-Connected = (?i)<XMLtagInRawEvent1>(?P<YourXMLExtractionName1>[^<]+)
EXTRACT-Connected = (?i)<XMLtagInRawEvent1>(?P<YourXMLExtractionName1>[^<]+)
EXTRACT-Connected = (?i)<XMLtagInRawEvent1>(?P<YourXMLExtractionName1>[^<]+)
EXTRACT-Connected = (?i)<XMLtagInRawEvent1>(?P<YourXMLExtractionName1>[^<]+)
1 Solution

skoelpin
SplunkTrust
SplunkTrust

Are you having issues parsing the XML file? If so you need to include KV_MODE =1 in your props.conf on the indexer(s)

View solution in original post

0 Karma

skoelpin
SplunkTrust
SplunkTrust

One thing to note here. Your deployment server will make changes to your forwarders while the deployer will make changes to your search head members in the cluster. Neither of these are responsible for parsing the data, you will need to make changes to your indexer(s) (aka search peers).. If your in an index cluster then you will make changes on the Master Node (aka the cluster master). These changes then will take affect on each search peer

To do this, you need to edit props.conf in $SPLUNK_HOME/etc/master-apps/_cluster/local/props.conf on master and push cluster-bundle with command 'splunk apply cluster-bundle'. Peers will restart and props.conf, in $SPLUNK_HOME/etc/slave-apps/_cluster/local/props.conf, will be layered when splunkd start.

0 Karma

rshoun
Explorer

I just thought of something, will I need to add all of my XML (all the code above: ie: "[monitor:", etc. - all my XML extracted fields) to the props.conf Master Node Indexer as well? Or I just need to add the KV_MODE =1 to my Master indexer correct? I should not need to add everything else here or am I wrong? (Please let me know if my question does not make sense to you.)

0 Karma

skoelpin
SplunkTrust
SplunkTrust

It should auto-extract most of the fields without having to explicitly extract them, but yes you just need to apply KV_MODE = 1 to your master node and it will apply those config settings to all of your search peers. If the fields aren't auto-extracted then you can apply these extractions to $SPLUNK_HOME/etc/master-apps/_cluster/local/props.conf and push it out to all your search peers. If you pushed this out to only 1 indexer then only 1 indexer will have these fields available which will give bad data output

0 Karma

rshoun
Explorer

Oh okay! That makes more sense! Geeze, I am really glad you are here! Thank you so much!

rshoun
Explorer

Wow, you have been an outstanding resource! I will do this! Thank you sooo much for adding this extra content!

0 Karma

skoelpin
SplunkTrust
SplunkTrust

Are you having issues parsing the XML file? If so you need to include KV_MODE =1 in your props.conf on the indexer(s)

0 Karma

rshoun
Explorer

I was wondering if that was the problem! Let me try the KV_MODE =1 and then I will see if that corrects the issue. Thanks!

0 Karma

skoelpin
SplunkTrust
SplunkTrust

This will defiantly fix your issue. Make sure to restart splunkd after making changes to props.conf

0 Karma

rshoun
Explorer

Dang it. I knew a restart was going to be necessary. Should I restart all places where my props.conf and inputs.config are located as well or just the place where the index KV_MODE =1 will be applied?

0 Karma

skoelpin
SplunkTrust
SplunkTrust

Yeah change management is tough to get around.. You should restart splunkd on every indexer you made changes to (You shouldn't have to make changes to inputs.conf on the indexer)

How many search peers are in your index cluster? I would strongly recommend testing this in a dev environment and verify it's working before making changes in production. You could download Splunk on your local machine, apply your props.conf changes and restart splunkd to verify the correct changes applied.

0 Karma

rshoun
Explorer

We have 7 search peers in our index cluster, if you count the index cluster master. The dev environment idea is actually a really good idea. I will see if I can get my team on board with this. I absolutely agree with the dev environment idea but not everyone will always understand the logical need for this. In the meantime, your idea of doing of local Splunk download is actually a really good idea! Thank you for your help!

Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...