Dashboards & Visualizations

is possible to index XML ?

dhavamanis
Builder

is possible to index the XML pattern data into splunk and do Splunk search?

In our case, we need to index the XML and co-relate the other logs using Splunk. Can you please suggest the best approach.

Sample Data:



firstName
firstName
STRING
true
2008-07-03 02:41:19.0


lastName
Last Name
STRING
false
2008-10-14 02:35:24.0


middleName
Middle Name
STRING
true
2007-11-30 01:12:55.0

Tags (2)
0 Karma
1 Solution

somesoni2
Revered Legend

Yes indeed. Have look at the other post on similar lines.

http://answers.splunk.com/answers/70619/parsing-xml-log-files

http://answers.splunk.com/answers/2141/xml-log-source-type

http://answers.splunk.com/answers/28619/indexing-xml-log-file-input

Update:
Try this (corrected regex and added MAX_DAYS_AGO to accommodate your older date values, increase more if you have timestamp older than then 4000 days( close to 12 years)

props.conf

[fastone]
BREAK_ONLY_BEFORE = (\<personattribute\sid|\</listpersonattribute)
NO_BINARY_CHECK = 1
REPORT-xmlext = xml-extr
SHOULD_LINEMERGE = true
TIME_FORMAT = %Y-%m-%d %H:%M:%S.%3N
TIME_PREFIX = \<createddatetime\>
pulldown_type = 1
REPORT-xmlext = xml-extr
MAX_DAYS_AGO = 4000

transforms.conf

[xml-extr] 
REGEX = \<(\w+)\>([^\>]*)\</
FORMAT = $1::$2
MV_ADD = true
REPEAT_MATCH = true

Regarding the date

View solution in original post

lguinn2
Legend

@somesoni2 - if you put answers in the Answers box, then good things can happen: first, you can get credit for your answers. Second, and much more important: other users will see this as an ANSWERED question and so they will look at if they need help. So you will be helping many more people. UNANSWERED questions (like this one) are ignored by people who are looking for answers.
Please put answers in the Answers box!

somesoni2
Revered Legend

Yes indeed. Have look at the other post on similar lines.

http://answers.splunk.com/answers/70619/parsing-xml-log-files

http://answers.splunk.com/answers/2141/xml-log-source-type

http://answers.splunk.com/answers/28619/indexing-xml-log-file-input

Update:
Try this (corrected regex and added MAX_DAYS_AGO to accommodate your older date values, increase more if you have timestamp older than then 4000 days( close to 12 years)

props.conf

[fastone]
BREAK_ONLY_BEFORE = (\<personattribute\sid|\</listpersonattribute)
NO_BINARY_CHECK = 1
REPORT-xmlext = xml-extr
SHOULD_LINEMERGE = true
TIME_FORMAT = %Y-%m-%d %H:%M:%S.%3N
TIME_PREFIX = \<createddatetime\>
pulldown_type = 1
REPORT-xmlext = xml-extr
MAX_DAYS_AGO = 4000

transforms.conf

[xml-extr] 
REGEX = \<(\w+)\>([^\>]*)\</
FORMAT = $1::$2
MV_ADD = true
REPEAT_MATCH = true

Regarding the date

somesoni2
Revered Legend

Try the updated answer.

0 Karma

dhavamanis
Builder

Thank you, i am getting this error for line breaking and also event date not extracting from the xml attribute "createdDateTime",

Line breaking regex has no capturing groups: >\s*(?=<personAttribute>)

We want to extract as a fields from each element in the xml. Can you please review the below,

transforms.conf
[xml]
LINE_BREAKER = >\s*(?=<personAttribute>)
TIME_PREFIX = <createdDateTime>
TIME_FORMAT = %Y-%m-%d %H:%M:%S.%3N
SHOULD_LINEMERGE = true
KV_MODE=xml
REPORT-xmlext = xml-extr

props.conf
[xml-extr]
REGEX = <(w+)>([^<]*)
FORMAT = $1::$2
MV_ADD = true
REPEAT_MATCH = true

0 Karma
Get Updates on the Splunk Community!

Join Us for Splunk University and Get Your Bootcamp Game On!

If you know, you know! Splunk University is the vibe this summer so register today for bootcamps galore ...

.conf24 | Learning Tracks for Security, Observability, Platform, and Developers!

.conf24 is taking place at The Venetian in Las Vegas from June 11 - 14. Continue reading to learn about the ...

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...