My data files are in Avro, and I have a props.conf that looks like
[source::/logs/...]
sourcetype = api
[api]
KV_MODE = json
TIME_PREFIX = "timestamp"
TIME_FORMAT = %10s%3N
For a given time range,
How can I resolve this?
I was given a workaround.
EXTRACT-_time = strptime('timestamp', "%s%3N")
In Hunk 6.2 the recommended way of solving this would be to tell Hunk to always return the timestamp field
[my-virtual-index]
....
# required fields (6.2 or later)
vix.input.[N].required.fields = <comma delimited, optionally wildcarded, list of fields to always output for this input>
In Hunk 6.1.x the recommended way of solving this issue is to disable the column projection optimization that leads to this problem:
[my-provider]
...
# disable column projection
vix.splunk.search.column.filter = false
The reason for the difference in behavior is due to Hunk's optimization based on required fields - which in "Fast mode" are whatever the search requires, while "Smart mode", which for an event search (e.g. search index=avro) is the same as "Verbose mode", all fields are required. The avro record reader honors the required fields and since Hunk needs/expects "_time" while the data contains "timestamp" the time related field is omitted and thus causing the problem.
I was given a workaround.
EXTRACT-_time = strptime('timestamp', "%s%3N")
Stick to Smart Mode 😛 there rarely is a reason to not use Smart Mode.
What search are you running?