Getting Data In

Multiple Worldwide Searchers, Multiple Timezones, Multiple Indexers and Search Heads

dskillman
Splunk Employee
Splunk Employee

I've looked at a few Q&As but nothing really seems to answer my question but this one seems to explains the problem most simply:

http://answers.splunk.com/questions/8116/users-searching-in-different-timezones

But the use case is a bit larger than this. Let's say I have people who need to run searches that live on 3 continents but they all need global visibility into the data that lives in indexers in 3 different global data centers. Their job is to make sure that all locations are fufulling time based (relative to the local time zone of the data center) SLAs and would like to have a single view that shows all of these events.

A 'simple' search example would be show me all transactions that finished later than 9:15AM wherever they may occurred.

The 'best' answer is to log into 3 different search heads in each location and get the results, but what if you have 10 DCs and/or timezones? The 'hack' is to mess with timezone settings at the indexer lever and use one search head, which has it own issues of breaking time series correlation.

It's painfully close and seems like Splunk should have a search command, function or subset of distributed search that could ignore the offsets imposed during a default distributed search and take the actual hour of the event. Any ideas?

0 Karma

bbingham
Builder

I'm not sure if this is correct on my assumption, but, Splunk can search based on epoch time, and since epoch is a reference to GMT, in theory it would be the exact same time no matter what the server is that indexed the data. Have you tried just using the epoch time for your "earliest" and "latest" dashboard search?

example:

 index="main" "*Blah*" earliest=1292544000

would search for Blah at midnight GMT on dec. 17. Not sure it will accomplish what you're looking for, but might work.

0 Karma
Get Updates on the Splunk Community!

Introducing Splunk Enterprise 9.2

WATCH HERE! Watch this Tech Talk to learn about the latest features and enhancements shipped in the new Splunk ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...