Getting Data In

Logging in UTC, not displaying in correct timezone

twhisnant
New Member

We're receiving logs from Bluecoat Proxies via syslog. The logs are written locally where a UF picks them up, sends them to balanced HFs and onto multiple indexers. For fields extractions we are using the Bluecoat app from Splunk.

Sample event:
"Jul 31 14:05:59 proxy 2012-07-31: 18:05:59 294 192.168.10.1 userID - - - OBSERVED "CategoryA" - 200 TCP_NC_MISS POST application/octet-stream http sample.domain.com 80 /Somerandom.bin - bin "User-Agent string/9.2.1" 192.168.50.50 406 802 -"

To get the correct timestamp we modified props.conf in the Bluecoat app and updated it via our Deployment Server.

props.conf under the Bluecoat app ($SPLUNK_HOME/etc/deployment-apps/Bluecoat/default/props.conf)
TIME_PREFIX = \w{3}\s\d{2}\s\d{2}:\d{2}:\d{2}\s\w+\s
MAX_TIMESTAMP_LOOKAHEAD = 20
TIME_FORMAT = %y-%m-%d: %H:%M:%S
TZ = UTC

After the indexers received the update app, they started using the correct timestamp (the second in the event). The problem lies with displaying the events.

Our timezone is US/Eastern while the timestamp in the logs is UTC. I confirmed that my user account in Splunk is configured for US/Eastern (GMT -0500).

I've tried changing the TZ under the apps props.conf to "TZ = US/Eastern" with no luck. The search head timezone is EDT.

How do I display these logs in my timezone?

any help is appreciated.

Tags (2)
0 Karma

woodcock
Esteemed Legend

In the Events tab just above the thin line that marks where the raw events are shown, just to the right of the fields area, but still the farthest thing to the left on that line, you will see a thing that says either list, raw or table. If you change this to list, then it will add a column called Time to the left of each event in your search results that shows you the event's _time value formatted for the TZ setting in your user profile. This is as close as you can get to what you are asking: there is no setting that changes the values of the raw data's data, but you can add a field that contains a differently formatted version of any time.

m4him7
Path Finder

The Splunk for Blue Coat ProxySG App 3.0.7 converts the old sourcetype bcoat_log to bcoat_proxysg. The TA-BlueCoat addon for the heavy forwarder sets the timezone to UTC with TZ=UTC under the [bcoat_log] property. The [bcoat_proxysg] property does not set the timezone.

The default is for Blue Coat to send UTC. That will cause all of your events to be indexed at UTC but Splunk believes the timestamp to be local time. You can see the issue if you monitor the events in realtime. You will notice that the Blue Coat timestamp is UTC and the _time is set to the same time which is UTC although Splunk believes this to be local time. For EST timezone that caused events to be indexed 4 hours in the future. Any searches or alerts would not see the events until the 4 hours passed.

The solution is to set the TZ to UTC within the [bcoat_proxysg] property. I did not want to modify the TA-BlueCoat addon so I modified the props.conf on the heavy forwarder indexing the Blue Coat syslogs under splunk/etc/system/local with:

[bcoat_proxysg]
TZ = UTC

Now Splunk understand the Blue Coat event times to be UTC and Splunk properly converts to local time. Looking at the _raw event the _time is local while the _raw event shows the UTC. This is now what is expected.

yannK
Splunk Employee
Splunk Employee

If your timestamp is like : Jul 31 14:05:59, it doesn't include a timezone.
the timezone applied will be the one defined on the HEAVY forwarder parsing the events.
(usually this is the indexer but not in your case)

Make sure that the sourcetype definition (props.conf) is present on the heavy forwarder, otherwise, it will default to the HF system timezone.

edit :
if the issue is a display on the indexer, double check the users timezone preferences.
and compare with
mysearch | eval utctime=_time | table _time utctime date_zone

0 Karma

twhisnant
New Member

The timestamp needed is "2012-07-31: 18:05:59", also no timezone.

When you say "the timezone applied will be the one defined on the HEAVY forwarder parsing the events" do you mean the timezone shown by the date command? Ex:
"$ date
Wed Aug 1 16:36:01 EDT 2012"

If so, the HF is in EDT (US/Eastern).

The heavy forwarder receives the props.conf in the bluecoat app. Does it need to defined elsewhere?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...