Splunk Search

Extract Data which ever is newly added or updated

pramodkumar
Path Finder

Hi Team,

Am facing one issue, my requriement is to continously monitor the file and want to pick only the latest records which are newly added and should ignore older events, i just want to take only latest events which option should i enable to satisfy my requirement.

Tags (2)
0 Karma
1 Solution

lguinn2
Legend

There is no option to take only the latest events. However, there are many ways to do it, depending on your specific needs.

For example, if you want to see only the latest event for each ip address, use the dedup command:

yoursearchhere | dedup src_ip

This example assumes that the ip address is in a field called src_ip.

If your definition of "the latest events" means "all events in the last minute", then you can use Custom Time in the time selection drop down to set that for your search. Or you could use a realtime search if you need to see the events continuously updated.

View solution in original post

lguinn2
Legend

There is no option to take only the latest events. However, there are many ways to do it, depending on your specific needs.

For example, if you want to see only the latest event for each ip address, use the dedup command:

yoursearchhere | dedup src_ip

This example assumes that the ip address is in a field called src_ip.

If your definition of "the latest events" means "all events in the last minute", then you can use Custom Time in the time selection drop down to set that for your search. Or you could use a realtime search if you need to see the events continuously updated.

Drainy
Champion

Um, surely you just search for earliest=-5h@h ?

pramodkumar
Path Finder

Hi lguinn,

Yours answer looks good but my scenario is monitor for every 5 hours, in first hour i have fetched 4000 events and analyzed the data, at the second 5th hour i got 5000 events. Now i had to only analyze the latest 5000 events not total 9000 events.. plz help me

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...