Hi all! Sorry, if this question was already asked by someone, but i'm stuck with a time configuration.
So, i just installed Splunk and configured it to listen on UDP port in my network. All hosts send data to it and everything is great, but Splunk shows the wrong time in search results.
This is how i see it:
https://www.dropbox.com/s/e4rf3kxete9qgpv/splunk_f.PNG
Also Splunk shows me the wrong time on all another hosts. Every time I type another ip - Splunk muss time.
This my date on server:
root@monsrv:~# date
Птн Авг 15 09:55:11 IRKT 2014
What do I need to configure to see the right time in search results?
Sorry for my bad English. Hope you understand me. 🙂
You should read this http://docs.splunk.com/Documentation/Splunk/6.1.3/Admin/Aboutconfigurationfiles and the following couple of pages.
This is direct from splunk documentation:
By default, Splunk Enterprise applies time zones using these rules, in this order:
Splunk Enterprise uses any time zone specified in raw event data (for example, PST, -0800).
Splunk Enterprise uses the value of a TZ attribute set in props.conf, if the event matches the host, source, or source type specified by the stanza.
If an event that arrives at an indexer originated at a forwarder, and both the forwarder and the receiving indexer run Splunk Enterprise 6.0 or later, then Splunk Enterprise uses the time zone that the forwarder provides.
Otherwise, Splunk Enterprise uses the time zone of the server that indexes the event.
Note: If you change the time zone setting in the system Splunk Enterprise runs on, you must restart Splunk Enterprise for it to pick up the change.
So in your case:
Point 1 is not applicable as your events do not contain time zone information.
Point 2 is also not applicable. Since you have not modified any props.conf settings. Also, you are not aware which props.conf contains the settings.
Then in that case either point 3 or point 4 is applicable. Since you have mentioned in your comment that both the host and the system(receiver) both are in UTC+9 timezone. That timezone is considered for indexing events.
What you need to do is this:
Step 1: Create props.conf file under /opt/splunk/etc/system/local/
directory of your indexer. The full path will look like this /opt/splunk/etc/system/local/props.conf
on indexer node.
Note: You can also create this props.conf file under /opt/splunk/etc/apps/<your_app>/local/
directory. Here your_app is the dedicated app that you have created for your indexer node.
Step 2: Add a stanza with your sourcetype
[Your_Sourcetype]
Note: you can have stanza with source, host and sourcetype. I have chosen sourcetype here.
Step 3: Under that stanza specify the timezone
[Your_Sourcetype]
TZ = UTC
For more information on setting timezones read http://docs.splunk.com/Documentation/Splunk/6.1.3/Data/Applytimezoneoffsetstotimestamps
I suggest you to read all these:
http://docs.splunk.com/Documentation/Splunk/6.1.3/Data/HowSplunkextractstimestamps
http://docs.splunk.com/Documentation/Splunk/6.1.3/Data/Configuretimestamprecognition
http://docs.splunk.com/Documentation/Splunk/6.1.3/Data/Configurepositionaltimestampextraction
http://docs.splunk.com/Documentation/Splunk/6.1.3/Data/Applytimezoneoffsetstotimestamps
HI Strive,
Thanks for your response, I am facing the same problem but with a weird twist that is this problem is not for all the records, I am having few records (about 100) that are having the this time stamp issue and rest are absolutely fine. Any idea what could possibly be the reason for that?
Thanks,
Vinod.
So where is required props.conf?
root@monsrv:~# find /opt/splunk/ -name props.conf
/opt/splunk/etc/apps/search/default/props.conf
/opt/splunk/etc/apps/legacy/default/props.conf
/opt/splunk/etc/apps/SplunkLightForwarder/default/props.conf
/opt/splunk/etc/apps/learned/local/props.conf
/opt/splunk/etc/apps/sample_app/default/props.conf
/opt/splunk/etc/system/default/props.conf
Yes UTC+9. Timezone on system is right, also on the hosts who sending logs for splunk. On sceenshot on right side is actual date. Splunk shows incorrect date. (mark as circle 🙂 ) And i have no idea where i need to config it.
Looking at the timestamps in your screenshot it seems this is a timezone issue. What time zone is the source and your user in? Your server seems to be in UTC+9?
Also, who's prepending the timestamp and host to the syslog event? Is your Splunk doing that, or is that already prepended before it gets to Splunk? If that's prepended before it gets to Splunk, what timezone is that system in?
Haven't you set your configurations in props.conf file. Your custom configurations should be under /etc/system/local. If you have written a separate app for heavy forwarder or indexer then the props.conf file should be under that app's local directory.
The file in etc/system/default is useless to us because it only contains default values.
Thanks for the reply!
/opt/splunk/etc/system/default/props.conf:
http://pastebin.com/pDzwZA6G
Post your props.conf configurations.
What values you have set for TIME_FORMAT, TIME_PREFIX