All Apps and Add-ons

Splunk DB Connect: How do I avoid indexing duplicate data from an Oracle database table if some lines can be created with a delay up to 7 days?

jbechchar
New Member

Hi everybody,

Here is my problem:

I have a table in an Oracle database.
My Goal is to index each invoice once, and it's possible for some lines to be created with a 7 day delay.
Example : yesterday, a new line was added in my table with and older creation_invoice date.

So to be sure to get the line it, means that each day I have to get the data from the last 7 days. (I do not have a system creation date of each line) and use my creation_invoice date from timestamp.

The problem is when I get the data from the last 7 days, it will index some data which is already indexed.
I would like to index only the new line to avoid duplicate events.

Do you have any ideas ? I heard about kv stor,e but no idea if it can be help me.

Thanks in advance for yours answers.

Regards,
Jordan

0 Karma

araitz
Splunk Employee
Splunk Employee

Try using _index_time, which is the time that data was indexed, rather than _time:

 index=whatever_indextime>[|stats count | eval search=(now()-300)]
0 Karma

jbechchar
New Member

Thanks for your answer.

But the script help to get the data I want but the data are already indexed.

I really would like to have a unique event for each invoice in my index. And not to have 2 events for an invoice.

I do not know if it's possible with my problem.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...