Getting Data In

Splunk for BlueCoat

pillowhead
Explorer

Hi, I am using version 4.1 of Splunk and have installed Splunk for BlueCoat. The logs from BlueCoat are using UTC time and I want them to show up as localtime in Splunk. When I change the time format in BlueCoat to use localtime in the log format (W3C ELFF), my Splunk for BlueCoat reports page displays incorrectly. The IP's don't show up correctly, they show up as a 3 digit number and the URL's are missing the domain portion of the URL.

Any suggestions?

Tags (1)
0 Karma
1 Solution

Dan
Splunk Employee
Splunk Employee

The fields are h0rked because the W3C ELFF format introduces spaces into the timestamp, and the space character is being used as the delimiter for field extraction. You would need to modify the list of expected fields in $SPLUNK_HOME/etc/apps/SplunkforBluecoat/default/transforms.conf, here:

[delimExtractions]
DELIMS=" "
FIELDS="date","time","time_taken","dvc_ip","user","user_group","x_exception_id","filter_result","category","http_referrer","holder","http_response","action","http_method","http_content_type","uri_scheme","dest_host","dest_port","uri_path","uri_query","uri_extension","http_user_agent","src_ip","sc_bytes","cs_bytes","x_virus_id"

In addition, if you're changing the format of the timestamp you'll probably also have to change the following line in $SPLUNK_HOME/etc/apps/SplunkforBluecoat/default/props.conf:

TIME_FORMAT=%Y-%m-%d %T

For both changes, the caveat applies of not updating default configs, or any upgrade will revert the changes. The practice is to create the config in local/ and only copy over the settings that are being changed.

Lastly, I would expect that all of this should be moot, since Splunk normalizes any timestamp in the events and stores it internally as UTC anyway. When a user searches for data, the time is then converted to the localtime of the browser. Perhaps I'm not understanding the original issue that prompted you to change to W3C?

View solution in original post

choustonweather
New Member

Error. In Splunk, I see logs from bluecoat as UTC logs. Everything else I have in Splunk show up as localtime. I have to manually search into the future to see my bluecoat logs which are UTC all the way from bluecoat to the search app in Splunk. I don't see anything in bluecoat for splunk.

0 Karma

Dan
Splunk Employee
Splunk Employee

The fields are h0rked because the W3C ELFF format introduces spaces into the timestamp, and the space character is being used as the delimiter for field extraction. You would need to modify the list of expected fields in $SPLUNK_HOME/etc/apps/SplunkforBluecoat/default/transforms.conf, here:

[delimExtractions]
DELIMS=" "
FIELDS="date","time","time_taken","dvc_ip","user","user_group","x_exception_id","filter_result","category","http_referrer","holder","http_response","action","http_method","http_content_type","uri_scheme","dest_host","dest_port","uri_path","uri_query","uri_extension","http_user_agent","src_ip","sc_bytes","cs_bytes","x_virus_id"

In addition, if you're changing the format of the timestamp you'll probably also have to change the following line in $SPLUNK_HOME/etc/apps/SplunkforBluecoat/default/props.conf:

TIME_FORMAT=%Y-%m-%d %T

For both changes, the caveat applies of not updating default configs, or any upgrade will revert the changes. The practice is to create the config in local/ and only copy over the settings that are being changed.

Lastly, I would expect that all of this should be moot, since Splunk normalizes any timestamp in the events and stores it internally as UTC anyway. When a user searches for data, the time is then converted to the localtime of the browser. Perhaps I'm not understanding the original issue that prompted you to change to W3C?

pillowhead
Explorer

I thought it was an issue because when I viewed the traffic in realtime, it was in UTC time, so I wanted to see it in localtime when I was viewing it in realtime. I didn't realize that the time got normalized. I will just leave it the way it is.

Thank you!

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...