Getting Data In

Modular Input Checkpoint

sanjay_shrestha
Contributor

I am writing a modular input and the script pulls list of the records in each interval when it runs.

e.g.

Name       Address
Joe         Ankeny
Bob        Clive

I do get duplicate events as I have not implemented Checkpoint yet. Since the script would bring all rows every time, do I need to save this every single row in checkpoint file and run through verification if row exists in the file or not?

Thanks,
Sanjay

0 Karma
1 Solution

MuS
SplunkTrust
SplunkTrust

Hi sanjay.shrestha,

if your script pulls in all rows on each run, your approach sounds good.

It would be easier if you could use some timestamp to get the data in; in this case you can safe the last time the script ran in the checkpoint and use this last time stamp in the next script run.
I have some modular inputs doing exactly this.

Hope this helps ...

cheers, MuS

View solution in original post

0 Karma

MuS
SplunkTrust
SplunkTrust

Hi sanjay.shrestha,

if your script pulls in all rows on each run, your approach sounds good.

It would be easier if you could use some timestamp to get the data in; in this case you can safe the last time the script ran in the checkpoint and use this last time stamp in the next script run.
I have some modular inputs doing exactly this.

Hope this helps ...

cheers, MuS

0 Karma

sanjay_shrestha
Contributor

Thanks Michael. I will implement your advise and let you know.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...