Hi - I had splunk import a fairly simple two column file - column 1 was a date/time column2 is some info... the problem seems to be that some of the values in column 1 are in EST and some are in UTC.
I don't think splunk is interpreting these correctly - is there a way I can verify this?
Data sample
02/07/18 03:55:00 PM EST String=2
02/07/18 03:55:04 PM EST String=3
02/07/18 03:55:08 PM EST String=0
02/09/18 11:10:01 PM UTC String=1
02/09/18 11:10:04 PM UTC String=0
02/09/18 11:10:07 PM UTC String=0
So, your sample data already has mixed timezones? Or is that sample data representative of how Splunk parsed it? If the latter, can you include the actual CSV prior to having Splunk handle it?
Sample data has mixed time zones
So when you view the data in Splunk, does the extracted time agree with what your CSV has?
If column 1 has time zone information in it ("EST", "UTC", "-0500", "Z", or similar) then your props.conf settings can be tweaked to interpret times correctly. If not, do you have any control over how the file is written?
Thanks I do have time zone UTC/EST Time zone designators...Guidance on the best way to modify the props file?
Oh also I thought I read somewhere that Splunk should automatically be able to pick that up from the file? Meaning that the props file does not need to be changed?
Automatic doesn't always work like it should, as you've discovered. If you can post some sample events (with private into masked) we can help with the right props settings.
Thanks - its really a very basic file 🙂 - added sample to the post
Try these props.conf settings.
TIME_PREFIX = ^
TIME_FORMAT = %m/%d/%y %H:%M:%S %p %Z
MAX_TIMESTAMP_LOOKAHEAD = 25
LINE_BREAKER = ([\r\n]+)
SHOULD_LINEMERGE = false
Thanks , do I need to re-index the data too?
"reindeer"? Is that auto-correct for re-index? If so, yes, you need to re-index the data for the new props to be applied.
OK I tried but can't seem to get the formatting to work
_time comes out as
2/6/18 9:27:42.000 AM
where as the time in the file is
2/06/18 04:27:42 PM EST
Did you restart Splunk after changing props.conf?