How to index a text file in the xml format?

[madhanbaskar]

/getClientProfileV1Request></SOAP-ENV:Body></SOAP-ENV:Envelope></soap-env:Body>-- HTTP Header values -<tp:headers xsi:type="http:HttpRequestHeaders" xmlns:http="http://www.bea.com/wli/sb/transports/http" xmlns:tp="http://www.bea.com/wli/sb/transports" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance&quot;&gt;&lt;tp:user-header name="Authorization" value="Basic cHJvZmlsZXVzZXI6cHJvZmlsM3VzM3I="/><tp:user-header name="EData" value="19=1.2,15=RS-Ro,16=Dn,13=getCtRequest,11=REQ,2=getes,10=3xbhkxv0p9,7=Assigned,6=PCK069,5=getAssignedRoles,4=com,9=roscha3c,8=UNKNOWN,1.2=APP10865:3xbhkxw1kw:3xbhkxxefh,1.1=APP09470::3xbhkxv28o"/><tp:user-header name="Syste" value="RO"/><http:Accept>application/xml</http:Accept><http:Content-Type>text/xml</http:Content-Type></tp:headers>

Above is the log in text format inside a text file. I need to index the same in my indexer in XML format itself. what would be the easy way to read the text file and convert it to an XML file so that I can view the logs in xml format in splunk GUI?

[niketn]

Can you please try the following:

eval xml_data=replace(replace(replace(_raw,"&lt;","<"),"&gt;",">"),"&quot;","\"")
| table xml_data

[madhanbaskar]

@niketnilay ,

Thanks that actually helped 🙂

[niketn]

@madhanbaskar, I have converted the comment to answer. Please accept to mark this as answered.

[ddrillic]

A similar case I had recently at How can we index an entire XML document as one event?

[niketn]

@madhanbaskar, you can either define KV_MODE=xml in props.conf for your input for Splunk to perform field discovery for XML data.

Other option would be to use spath to traverse the xml data once it is indexed.

Refer to documentation: https://docs.splunk.com/Documentation/Splunk/latest/Knowledge/Automatickey-valuefieldextractionsatse...

https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Spath

[madhanbaskar]

@niketnilay, Thanks

Will the above really help?

My problem is, in the client server the logs are stored in text format(.txt) but I need to forward that log to splunk server and index those logs in XML format(.xml).

What would be the easy way to do that?

I'm not sure of how the Spath or KV_mode help in this transformation case?

[niketn]

On a test machine or your local machine, you can take a sample log containing XML data and upload to Splunk (follow steps from the documentation: http://docs.splunk.com/Documentation/Splunk/latest/SearchTutorial/GetthetutorialdataintoSplunk)
Once you are in Sourcetype screen, from Advanced settings you can add KV_MODE=xml

One of the other things you can try out is upload the data as txt and just runspath command to extract fields as you need. https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Spath. PS: if your file has content other XML you can use rex command to extract xml data first, before applying spath.

Does the text file contain only XML or does it contain XML between some text data as delimiters? Would you be able to post the content of a sample file after masking/anonymizing sensitive data?

[madhanbaskar]

@niketnilay,

The difference is these 3 tags < , " , &gt; in the text file needs to be changed to < , " ,> respectively

If so then the file will be automatically converted to a xml file.... better?

[niketn]

Sorry I just noticed that you might not be interested in XML spath rather value containing within XML nodes as Key Value Pairs. Splunk would have been able to directly extract Key Value pairs. However, in this case I see value=key, rather than key=value. Splunk's default fields can not start with numbers, hence can not be all numbers also. Besides you have multiple fields with name. Only first one will get extracted by default.

So, in your case I would suggest to go with Regular Expression based Field Extraction:http://docs.splunk.com/Documentation/Splunk/latest/Knowledge/ExtractfieldsinteractivelywithIFX

In the above sample data can you please let me know what are the fields you are interested in?

[madhanbaskar]

@niketnilay,

si=& quot ;http://www.w3.org/2001/XMLSchema-instance & quot;& gt ;& lt ;tp:headers

In the above sample i need to change the & quot ; & gt; & lt ;

to " , > , < respectively.

That should help me 😞

[niketn]

Can you re-post XML or edit your question with Code button (101010) here on Splunk Answers so that special characters are not escaped?

I think your original post does not have < " and > characters, rather have <, ", >. Please confirm if this is correct.

[madhanbaskar]

@niketnilay , Exactly!!

You are right. The original post doesnt have < > " rather it has lt,quot,gt ....

I need to know how to change that in splunk so that in the place of lt,quot and gt , < " > will get replaced??