Solved: How can I index JSON that's embedded in HMTL?

mgagliardi · ‎02-10-2017

I'm querying an endpoint that is returning JSON embedded in HTML. Like so:

<HTML><BODY> <pre id="info">...</pre> &lt;br/&gt; <script>document.getElementById("info").innerHTML = JSON.stringify({"perfInfo":{"processes":79,"threads":1662,"PhysicalAvailable":11717,"PhysicalTotal":16383,"CommitTotal":5856,"CommitLimit":22525,"CommitPeak":7155,"KernelNonpaged":100,"KernelPaged":209,"KernelTotal":310},"sysInfo":{"powerProfile":"7","hostname":"HOSTNAME","Version":"7.5.0.206","UDI":"(01)00853866003209(11)170203(10)7.5.0.206","BootstrapperProfile":"Enterprise - Management","hwProductName":"VMware Virtual Platform","hwBIOSVersion":"6.00","osVersion":"Windows Server 2012 (R2) (6.3 Server build 9600)"},"gpuInfo":{"numGPUs":0},"Sessions":{"available":1,"capacity":1,"assigned":[],"passive":0,"allowingNewConnections":true},"slots":{"totalSlots":1,"availableSlots":1,"passiveSlots":0,"userRamReservedMB":15559,"userRamRemainingMB":13639}}
, null, 4); </script>&lt;br/&gt;</BODY></HTML>

I want just the JSON but I'm having a hell of a time figuring out how to strip away or ignore the HTML. Could anyone offer some pointers? TIA!

s2_splunk · ‎02-10-2017

Configure this in props.conf for your sourcetype:

[your_sourcetype]
SEDCMD-removeHTMLBeginning = s/<HTML>[\s\S]*\(\{/{/g
SEDCMD-removeHTMLEnd = s/\}\}[\s\S]*$/}}/g

That worked for me with your sample response.

View solution in original post

s2_splunk · ‎02-10-2017

Configure this in props.conf for your sourcetype:

[your_sourcetype]
SEDCMD-removeHTMLBeginning = s/<HTML>[\s\S]*\(\{/{/g
SEDCMD-removeHTMLEnd = s/\}\}[\s\S]*$/}}/g

That worked for me with your sample response.

mgagliardi · ‎02-10-2017

FWIW your answer works, but I also found that setting an HTTP header of Accept=application/json got me where I wanted to go (via the REST API data source).

s2_splunk · ‎02-10-2017

Even better! 🙂

How can I index JSON that's embedded in HMTL?

.conf24 | Registration Open!

ICYMI - Check out the latest releases of Splunk Edge Processor

Introducing the 2024 SplunkTrust!