How do I set timezone properly in props.conf?

[vpuri6004]

Our data source is generating syslog data using UTC. Time in the syslog header is formatted as Oct 22 15:51:14. We made the following changes to $SPLUNK_HOME/etc/system/default/props.conf:

[host::<hostname>]
TZ = UTC

The <hostname> specified above is the host generating the syslog message. The CentOS server on which Splunk is installed is initialized to be in the EDT timezone.

We also modified our Splunk application's props.conf as follows:

[source:tcp:<port>]
TZ = UTC

However, when we search the data from Splunk, we don't see the data converted to the local time (EDT/Eastern). Splunk is able to parse the date/time field though.

Are there other configuration changes needed to handle timezone changes?

[woodcock · Accepted solution]

The only file that you should need to use is this:

$SPLUNK_HOME/etc/apps/<YourApp>/default/props.conf

Assuming that all hosts are UTC, you can just add this:

[default]
TZ=UTC

You should NOT modify anything about how Splunk talks to itself. Splunk always normalizes every event to UTC for indexing (creating the _time field) so all it needs to know is the TZ that is used for each event's timestamp. The Indexers know how to check host OS to determine UTC of system clock.

The reason that you "do not see data converted to EDT" is because you are misunderstanding how Splunk works. You need to go to Your Account Name -> Edit Account -> Time zone and select the TZ that you will be using to talk to Splunk. Next, on the Events tab, find the Raw/List/Table link on your Search Head that is just above your search results and just under the graph and all the way to the left and make sure it is set to List. This will add a column to your search results called Time which will show your preferred normalized timestamp (in your case, EDT). The timestamp inside the raw event will never change and will always be exactly the way it was when the thing that generated it sent it to splunk.

Also, only newly indexed events are effected by changes to this setting; older events will stay "wrong".

[santosh_sshanbh]

With respect to above answer, in my case the user has to see data for more than 20 servers which are in different time zone. So it is practcally impossible to chnage the TZ from settings menu each time user navigates to different server. I want the dashboard to automatically show the data in source server time zone. How can this be done automatically?

[woodcock]

This is indeed a flaw in Splunk's overall design. Please open an Enhancement Request Ticket requesting that dashboard panels with a timepicker have the option for a control to select any TZ to apply.

[somesoni2]

First of all, you shouldn't change anything in the folder $SPLUNK_HOME/etc/system/default. You should make all the changes to $SPLUNK_HOME/etc/system/local. Secondly, which server (Forwarder/Indexer) did you make this props.conf changes and did you restart the splunk server after making the change?