Deployment Architecture

Virtual Index with Hunk

boulderdevdude
New Member

Platform: CDH5.0.0 + Splunk 6.1 and Hunk.
We have configured virtual indexes. We are successfully issuing queries against Splunk and getting data back from MR jobs.

When MR jobs run, should they produce splunk index files, so next time Splunk makes a query against this data it doesn't have to run an MR job again?

I see result files with names like 1401212205.1082/0/part-m-00043 (1 per MR task, I think). Each file contains what looks like a hash and a path to a different file. Example
81a76d523faab948b16fcd2eb6dd56a0 /user/splunk/scratch/dispatch/1401212205.1082/0/splunk-m-0000043

But the splunk-m-0000043 file (or any like them) do not exist at that directory location. Are they supposed to exist?

Tags (2)
0 Karma

Ledion_Bitincka
Splunk Employee
Splunk Employee

Hunk does not create any persistent artifacts or index after running a search, ie it does "index" the data after first search. However, for searches that you intend to run frequently in Hunk 6.1 you can enable report acceleration to cache the intermediate results so they are reused next time

Here's another related question - not sure if it's the same question from two co-workers 🙂

0 Karma
Get Updates on the Splunk Community!

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...