I'm going to explain this the best way possible. Our server running the indexer is located in EST. We are querying a database located in a different timezone and timestamping against epoch time, but the indexer seems to get the timestamp wrong and add 4 hours to it. Is there something we can do to make it use GMT time? What do you guys do to prevent timezone issues? Looking for best practices.
The best best-practice is this:
1: Always run host OS in GMT.
2: Always timestamp source data in GMT.
3: Always use NTP.
In a perfect world, everybody would do that and things would be fine. Short of that:
1: Always explicitly timestamp (do not use DATETIME_CONFIG=CURRENT
nor DATETIME_CONFIG=NONE
).
2: Always timestamp source data with per-event TZ.
3: When that is not possible, always set a TZ=
value in props.conf
for everything (this can be verified by checking date_zone
; it should NEVER be local
but it is OK if it is null
for some TA
inputs).
4: Always keep an eye on your calculated lag ( _indextime
- _time
) so you see when things go sideways.
5: Always trackdown the reason any events have timestamp=none
.
It sounds like you are having a type-3 problem.
Thank you! This was great and will hopefully solve future issues!