All Apps and Add-ons

Can Splunk process data that is "updated" over time?

mpdude
Explorer

Dear fellow Splunkers,

I have a use case where I believe Splunk could provide great insight, alerts and dashboards, but I do not know if the way data has to be acquired makes it the right tool for the job.

The data in question is timesheet reporting, with the additional challenge that timesheets might be updated (data entry errors fixed) later on.

For example, I could run a script every day that would import records consisting of:

  • ID/Name of the user
  • Current timestamp = the time that data was read from the underlying operational system
  • Timesheet period: Date, begin and end time
  • Project being worked on
  • Maybe additional categories

So, it might happen that I import some of these tuples, but then – say the next day – re-run the import and one of the following happens:

  • A particular period is no longer present, maybe because it has been deleted (time recorded by mistake)
  • A particular period has changed in duration (e. g. forgot to stop timer)
  • New periods are added (forgot to start timer)

Would it be feasible to work with this data in Splunk at all? I guess the problem is that Splunk is not a (relational) database but an append-only index, right? I mean, how could I easily add to all relevant searches that for a particular day, only those events (imported records) are to be considered that have been imported at the time where data for that day has last been updated?

Does that problem description make sense?

Tags (1)
0 Karma

woodcock
Esteemed Legend

This kind of thing is often done using DB Connect:
https://splunkbase.splunk.com/app/2686/
Be aware that v3 is a complete rewrite of v3 and there are many feature changes. If v3 isn't working for you, try v2.

The easiest way to use it is to use dbxquery to access the data in the DB when you need it, but you can also pull it in and index it with Splunk if you like.

0 Karma
Get Updates on the Splunk Community!

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...