All Apps and Add-ons

Can Splunk process data that is "updated" over time?

mpdude
Explorer

Dear fellow Splunkers,

I have a use case where I believe Splunk could provide great insight, alerts and dashboards, but I do not know if the way data has to be acquired makes it the right tool for the job.

The data in question is timesheet reporting, with the additional challenge that timesheets might be updated (data entry errors fixed) later on.

For example, I could run a script every day that would import records consisting of:

  • ID/Name of the user
  • Current timestamp = the time that data was read from the underlying operational system
  • Timesheet period: Date, begin and end time
  • Project being worked on
  • Maybe additional categories

So, it might happen that I import some of these tuples, but then – say the next day – re-run the import and one of the following happens:

  • A particular period is no longer present, maybe because it has been deleted (time recorded by mistake)
  • A particular period has changed in duration (e. g. forgot to stop timer)
  • New periods are added (forgot to start timer)

Would it be feasible to work with this data in Splunk at all? I guess the problem is that Splunk is not a (relational) database but an append-only index, right? I mean, how could I easily add to all relevant searches that for a particular day, only those events (imported records) are to be considered that have been imported at the time where data for that day has last been updated?

Does that problem description make sense?

Tags (1)
0 Karma

woodcock
Esteemed Legend

This kind of thing is often done using DB Connect:
https://splunkbase.splunk.com/app/2686/
Be aware that v3 is a complete rewrite of v3 and there are many feature changes. If v3 isn't working for you, try v2.

The easiest way to use it is to use dbxquery to access the data in the DB when you need it, but you can also pull it in and index it with Splunk if you like.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...