Monitoring Splunk

Get all events/fields from latest data of an Index

rajeshjlnt
Path Finder

We have an index 'abc' to which data gets fed in non-uniform intervals. I would like to get all events of this index that were indexed recently. Could i get some guidance on how to achieve this?

Ex: Data indexed on 1st of March, 5th of March and 10th of March. I want to get all events indexed on 10th of March.

Tags (1)
0 Karma
1 Solution

gcusello
SplunkTrust
SplunkTrust

Hi @rajeshjlnt
I think that you're speaking of events that are indexed with a timestamp really different with the indexing date.

To do this at first you have to find a time period where you are sure that contains all the events indexed in the monitoring period; in other words, if you index today events of last year, and you select as time frame the last month, you'll not have all the events in you results.
Then you have to use the _indextime field that are present in all the events (in epochtime).

You have to run something like this (e.g. in the last 30 days):

index=your_index earliest=-30d latest=now
| eval indextime=strftime(_indextime, "%Y-%m-%d %H:%M:%S"), diff=_time-_indextime
| table _time indextime diff

Ciao,
Giuseppe

View solution in original post

woodcock
Esteemed Legend

You can use _index_earliest=-1h _index_latest=now in your foundational search; for March 10, use this:

index="abc" earliest=0 latetst=@d+100d _index_earliest=1583816400 _index_latest=1583902800
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @rajeshjlnt
I think that you're speaking of events that are indexed with a timestamp really different with the indexing date.

To do this at first you have to find a time period where you are sure that contains all the events indexed in the monitoring period; in other words, if you index today events of last year, and you select as time frame the last month, you'll not have all the events in you results.
Then you have to use the _indextime field that are present in all the events (in epochtime).

You have to run something like this (e.g. in the last 30 days):

index=your_index earliest=-30d latest=now
| eval indextime=strftime(_indextime, "%Y-%m-%d %H:%M:%S"), diff=_time-_indextime
| table _time indextime diff

Ciao,
Giuseppe

rajeshjlnt
Path Finder

@gcusello , in my case _time and _indextime. I understand how timestamps work in splunk. Let me explain my requirement in more detail.
I upload a CSV file with n entries every day with a fixed timestamp. now i want to search and get events from latest uploaded file.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @rajeshjlnt,
if the csv file has a different name for each file (in other words a date in its name), you can use this to filter events, something like this:
e.g. if the csv is named your_csv_2020-03-11.csv and you have a different one every day with a different name, you can run something like this:
index=your_index [ index=your_index | head 1 | fields source ]
| ...

Ciao.
Giuseppe

0 Karma

rajeshjlnt
Path Finder

This works for me. Great thanks @gcusello

I made a small modification to your suggestion by adding 'search', without which i got an error

index=your_index [ search index=your_index | head 1 | fields source ]

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @rajeshjlnt,
sorry a little missed!
If this answer solves your problem, please accept and/or upvote it for the other users of the Community.

Ciao and next time!
Giuseppe

0 Karma

manjunathmeti
SplunkTrust
SplunkTrust

You can filter data on index time also. Use index command with _index_earliest and _index_latest.

index=_internal  _index_earliest=-5m@m _index_latest=@m
0 Karma

rajeshjlnt
Path Finder

@manjunathmeti , this is similar to time range picker. May be i am missing something, how can this help in getting the latest indexed set of events?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...