Splunk Search

Help! I cannot configure the proper search XML for a search feeding a table

cbeyonder12
Engager

This is my current idea of how a table with latency data should fit together, I am trying to mimic the "Real-Time measured indexing rate and latency" table at the top of the Indexing tab of Splunk On Splunk (SOS). Please let me know your suggestions... asterisks have been removed for security..

<dashboard>
<label>Index Performace</label>
<row>
<table>
<searchString>index=_internal host="*.*.net" | eval latency=round((_indextime - _time),2) | eval seconds_elapsed=(time() - now()) | eval secs=if(seconds_elapsed<0,"1",seconds_elapsed) | eval esize=((len(_raw)/1024)) | eventstats max(secs) AS seconds | eventstats count AS ecount, sum(esize) as sum_esize $type$ | stats last(ecount) AS "event count", last(eval(ecount/seconds)) as eps, last(eval(sum_esize/seconds)) as KBps, min(latency) as "minimum latency (seconds)", avg(latency) as avglat, max(latency) AS "maximum latency (seconds)" first(_time) AS oldestTime last(_time) AS newestTime $type$ | eval avglat=round(avglat,2)| eval eps=round(eps,2) | eval KBps=round(KBps,2) | convert timeformat="%m/%d/%Y %H:%M:%S" ctime(newestTime) | convert timeformat="%m/%d/%Y %H:%M:%S" ctime(oldestTime) | rename newestTime AS "Time stamp of newest event" oldestTime AS "Time stamp of oldest event" avglat AS "average latency (seconds)" eps AS "events per second" KBps AS "indexing rate (KBps)"</searchString>

<title>Indexing Perf new</title>
<earliestTime>-24h@h</earliestTime>
<latestTime>now</latestTime>
</table>
</row>
</dashboard>

hexx
Splunk Employee
Splunk Employee

There are two problems in your <searchString> parameter:

  • The "lesser than" sign in the | eval secs=if(seconds_elapsed<0,"1",seconds_elapsed) expression is interpreted as the beginning of an XML tag, which causes the view to break. It needs to be replaced by its HTML-safe encoding of "&lt;".
  • You need to replace the $type$ token references by a valid split-by clause. Typically, this would be a metadata field such as source/sourcetype/host/index, but you can use any field that makes sense to you.

In the example below, I have corrected both problems - $type$ has been replaced by the by sourcetype clause:


index=_internal host="*"
| eval latency=round((_indextime - _time),2)
| eval seconds_elapsed=(time() - now())
| eval secs=if(seconds_elapsed&lt;0,"1",seconds_elapsed)
| eval esize=((len(_raw)/1024))
| eventstats max(secs) AS seconds
| eventstats count AS ecount, sum(esize) as sum_esize by sourcetype
| stats last(ecount) AS "event count", last(eval(ecount/seconds)) as eps, last(eval(sum_esize/seconds)) as KBps, min(latency) as "minimum latency (seconds)", avg(latency) as avglat, max(latency) AS "maximum latency (seconds)" first(_time) AS oldestTime last(_time) AS newestTime by sourcetype
| eval avglat=round(avglat,2)
| eval eps=round(eps,2)
| eval KBps=round(KBps,2)
| convert timeformat="%m/%d/%Y %H:%M:%S" ctime(newestTime)
| convert timeformat="%m/%d/%Y %H:%M:%S" ctime(oldestTime)
| rename newestTime AS "Time stamp of newest event" oldestTime AS "Time stamp of oldest event" avglat AS "average latency (seconds)" eps AS "events per second" KBps AS "indexing rate (KBps)"

Get Updates on the Splunk Community!

More Ways To Control Your Costs With Archived Metrics | Register for Tech Talk

Tuesday, May 14, 2024  |  11AM PT / 2PM ET Register to Attend Join us for this Tech Talk and learn how to ...

.conf24 | Personalize your .conf experience with Learning Paths!

Personalize your .conf24 Experience Learning paths allow you to level up your skill sets and dive deeper ...

Threat Hunting Unlocked: How to Uplevel Your Threat Hunting With the PEAK Framework ...

WATCH NOWAs AI starts tackling low level alerts, it's more critical than ever to uplevel your threat hunting ...