I have an extremely simple search that inexplicably does not work. It returns blank values for latest(ewma):
index=myIndex msg=instrumentationResults |
eval ewma=('timerStats.ewma'-1) |
timechart limit=0 span=15m latest(ewma)
This also returns blank values for latest(ewma):
index=myIndex msg=instrumentationResults |
eval ewmaNumMinusOne=('timerStats.ewma'-1) |
convert num(ewmaNumMinusOne) as ewma |
timechart limit=0 span=15m latest(ewma)
This also returns blank values for latest(ewma):
index=myIndex msg=instrumentationResults |
convert num('timerStats.ewma') as ewmaNum |
eval ewmaNumMinusOne=(ewmaNum-1) |
convert num(ewmaNumMinusOne) as ewma |
timechart limit=0 span=15m latest(ewma)
However, this returns lots of numbers for latest(ewma):
index=myIndex msg=instrumentationResults |
eval ewma=('timerStats.ewma') |
timechart limit=0 span=15m latest(ewma)
Can anyone explain how to make eval work as expected here?
Give this a try
index=myIndex msg=instrumentationResults |
eval ewma=tonumber('timerStats.ewma')-1 |
timechart limit=0 span=15m latest(ewma)
That also yields blank values for latest(ewma).
This is does seems like conversion issue. Could you try this and see if you see value for ewma column.
index=myIndex msg=instrumentationResults | head 1 | table "timerStats.ewma" | eval ewma=tonumber('timerStats.ewma')
and
index=myIndex msg=instrumentationResults | head 1 | table "timerStats.ewma" | eval ewma=tonumber(substr('timerStats.ewma',1,10))
The bizarre results of both queries:
1106445.3503729277
1106445.3503729277
Note that EACH query returns BOTH of those values in the timerStats.ewma column!
Here is the event that is getting borked:
{"name":"myApp","hostname":"myHost","pid":95299,"level":30,"timerName":"sendToClient_Kafka","timerStats":{"ewma":1106445.3503729277,"min":585249,"max":50912895079,"lastPublish":[39357007,36446006]},"msg":"instrumentationResults","time":"2015-09-23T03:59:59.990Z","v":0}
So emwa is blank for both the query?
EACH query returns the following result in the timerStats.ewma column:
1106445.3503729277
1106445.3503729277
That's right. Somehow two identical values are being output from tonumber('timerStats.ewma').
Even simpler, the following also outputs a single column with two identical values:
index=myIndex msg=instrumentationResults
| head 1 | table "timerStats.ewma"
Ohk.. so the problem is that you've a multivalued field in the equation and that is causing issues with all eval. It will work fine if directly used in Stats/aggregation commands.
Since your logs don't have two values for timeStats field, it seems like you've your props.conf/sourcetype definition causing it to be extracted twice. Could you please provide your props.conf for your sourcetype??
The relevant stanza of props.conf:
[bunyan]
TZ = GMT
TIMESTAMP_FIELDS = time
TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%3NZ
pulldown_type = true
INDEXED_EXTRACTIONS = json
KV_MODE = none
category = Structured
TRUNCATE = 1000000
SHOULD_LINEMERGE = False
description = JavaScript Object Notation format. With bunyan timestamp (time) field extraction
I would suggest this to add following to your props.conf on Search Head, may need to reload/refresh/restart
AUTO_KV_JSON = false
This should completely stop doing JSON field extraction as search time as you already have fields extracted during index-time (INDEX_EXTRACTIONS = json). Once you apply this and reloaded the configuration, try your query again.