Dashboards & Visualizations

is possible to index XML ?

dhavamanis
Builder

is possible to index the XML pattern data into splunk and do Splunk search?

In our case, we need to index the XML and co-relate the other logs using Splunk. Can you please suggest the best approach.

Sample Data:



firstName
firstName
STRING
true
2008-07-03 02:41:19.0


lastName
Last Name
STRING
false
2008-10-14 02:35:24.0


middleName
Middle Name
STRING
true
2007-11-30 01:12:55.0

Tags (2)
0 Karma
1 Solution

somesoni2
SplunkTrust
SplunkTrust

Yes indeed. Have look at the other post on similar lines.

http://answers.splunk.com/answers/70619/parsing-xml-log-files

http://answers.splunk.com/answers/2141/xml-log-source-type

http://answers.splunk.com/answers/28619/indexing-xml-log-file-input

Update:
Try this (corrected regex and added MAX_DAYS_AGO to accommodate your older date values, increase more if you have timestamp older than then 4000 days( close to 12 years)

props.conf

[fastone]
BREAK_ONLY_BEFORE = (\<personattribute\sid|\</listpersonattribute)
NO_BINARY_CHECK = 1
REPORT-xmlext = xml-extr
SHOULD_LINEMERGE = true
TIME_FORMAT = %Y-%m-%d %H:%M:%S.%3N
TIME_PREFIX = \<createddatetime\>
pulldown_type = 1
REPORT-xmlext = xml-extr
MAX_DAYS_AGO = 4000

transforms.conf

[xml-extr] 
REGEX = \<(\w+)\>([^\>]*)\</
FORMAT = $1::$2
MV_ADD = true
REPEAT_MATCH = true

Regarding the date

View solution in original post

lguinn2
Legend

@somesoni2 - if you put answers in the Answers box, then good things can happen: first, you can get credit for your answers. Second, and much more important: other users will see this as an ANSWERED question and so they will look at if they need help. So you will be helping many more people. UNANSWERED questions (like this one) are ignored by people who are looking for answers.
Please put answers in the Answers box!

somesoni2
SplunkTrust
SplunkTrust

Yes indeed. Have look at the other post on similar lines.

http://answers.splunk.com/answers/70619/parsing-xml-log-files

http://answers.splunk.com/answers/2141/xml-log-source-type

http://answers.splunk.com/answers/28619/indexing-xml-log-file-input

Update:
Try this (corrected regex and added MAX_DAYS_AGO to accommodate your older date values, increase more if you have timestamp older than then 4000 days( close to 12 years)

props.conf

[fastone]
BREAK_ONLY_BEFORE = (\<personattribute\sid|\</listpersonattribute)
NO_BINARY_CHECK = 1
REPORT-xmlext = xml-extr
SHOULD_LINEMERGE = true
TIME_FORMAT = %Y-%m-%d %H:%M:%S.%3N
TIME_PREFIX = \<createddatetime\>
pulldown_type = 1
REPORT-xmlext = xml-extr
MAX_DAYS_AGO = 4000

transforms.conf

[xml-extr] 
REGEX = \<(\w+)\>([^\>]*)\</
FORMAT = $1::$2
MV_ADD = true
REPEAT_MATCH = true

Regarding the date

somesoni2
SplunkTrust
SplunkTrust

Try the updated answer.

0 Karma

dhavamanis
Builder

Thank you, i am getting this error for line breaking and also event date not extracting from the xml attribute "createdDateTime",

Line breaking regex has no capturing groups: >\s*(?=<personAttribute>)

We want to extract as a fields from each element in the xml. Can you please review the below,

transforms.conf
[xml]
LINE_BREAKER = >\s*(?=<personAttribute>)
TIME_PREFIX = <createdDateTime>
TIME_FORMAT = %Y-%m-%d %H:%M:%S.%3N
SHOULD_LINEMERGE = true
KV_MODE=xml
REPORT-xmlext = xml-extr

props.conf
[xml-extr]
REGEX = <(w+)>([^<]*)
FORMAT = $1::$2
MV_ADD = true
REPEAT_MATCH = true

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...