Getting Data In

How to index XML files in a directory location?

nir_sheep
New Member

Hi,

I am new to Splunk. i was able to get data indexed for regular lg files., but we have some Ora audit XML files that we want to index to search data for truobleshooting some issues. to try this, we have put some of the XML files on the splunk server in a location /mnt/avtest

name of files = idprd1_ora_17472_1.xml, idprd1_ora_17482_2.xml ...

sample data :

11.2
1684097112014-11-25T18:09:48.792252Zuseruserhost17482unknown110000000000000000000Authenticated by: DATABASE; Client address: (ADDRESS=(PROTOCOL=tcp)(HOST=IP_address)(PORT=port))511111

1684097322014-11-25T18:09:48.795447ZUSERuserhost17482unknown1OwnerPLD_GROUP_PERSON103011111---------S------111111
 #1(6):147037
select pldgrouppe0_.id as id40_, pldgrouppe0_.CREATION_TIMESTAMP as CREATION2_40_, pldgrouppe0_.created_by as created3_40_, pldgrouppe0_.pld_group_id as pld6_40_, pldgrouppe0_.pld_person_id as pld7_40_, pldgrouppe0_.LAST_UPDATE_TIMESTAMP as LAST4_40_, pldgrouppe0_.LAST_UPDATED_BY as LAST5_40_ from pld_group_person pldgrouppe0_ where pldgrouppe0_.pld_group_id=:1

Can you please guide me how to add the XML data and parse it? I have tried doing it using the splunk web also followed examples to edit props.conf and inputs.conf to index data, but its still not indexing data.

Thanks
Navd

Tags (3)
0 Karma

cpetterborg
SplunkTrust
SplunkTrust

I have XML files doing something like this in the props.conf file:

[<your_source_type>]
KV_MODE = xml
MAX_TIMESTAMP_LOOKAHEAD=150
SHOULD_LINEMERGE=true
BREAK_ONLY_BEFORE=^\<msg
NO_BINARY_CHECK=true

That should catch the fact that it is XML. You may have additional things you are needing to add to the configuration, and you may have to modify some of the parameters (like the value for MAX_TIMESTAMP_LOOKAHEAD and BREAK_ONLY_BEFORE,, but otherwise this should get you started.

You should follow the normal standards for specifying the inputs.conf file to have the same sourcetype that you specify in the props.conf file on your indexer(s).

nir_sheep
New Member

Thanks much.
It started parsing the data now., the actual issue looked with no data getting indexed at all after we downloaded splunk DB app, that was not configured. I deleted it and restarted splunk. its indexing data now, but its doing it for current time stamp. Needs to extract the time stamps :

props.conf
[aud_xml]
KV_MODE = xml
MAX_TIMESTAMP_LOOKAHEAD=165
SHOULD_LINEMERGE=true
BREAK_ONLY_BEFORE=<\s*\/AuditRecord\s*>
NO_BINARY_CHECK=true

inputs.conf

[monitor:///mnt/avtest]
disabled = 0
followTail = 0
host = host
sourcetype = aud_xml
whitelist = idprd1_ora_[\d]*_[\d].xml$
crcSalt =

sample file format :

11.2
1697608112014-11-25T22:07:27.695390ZUSERuserhost17498unknown110000000000000000000Authenticated by: DATABASE; Client address: (ADDRESS=(PROTOCOL=tcp)(HOST=ip)(PORT=port))5id

Thanks
Navd

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...