Getting Data In

Is it possible to get more granular on Windows system events as far as retention is concerned on the indexer?

careybrucem
Explorer

Or am I at the mercy of the settings for index rotation settings?

In other words, we have the following requirements to save the following events for the following amount of time:

Successful logon/logoff - 90 days

Unsuccessful logons - 1 year

Creation/deletion of user accounts - 2 years

Anyway to do this or do I have to save everything for 2 years?

Thanks,

Bruce

0 Karma
1 Solution

jeffland
SplunkTrust
SplunkTrust

As far as I know, you can't directly influence the time your data is stored on a per-event basis.

One solution for you is to index your events on different indexes and set different retention policies per index. This would require a modest effort to adjust your searches (and of course to assign indexes to events), but it should allow you to apply accurate settings on a granularity you require. Keep in mind that if you set your data to expire (or archive) after 90 days, this needs to apply to every event in your bucket in order to delete/archive the bucket - see here for a quick read.

Another idea (though I'm not sure how feasible this is) could be to regularly export the data you need to keep for longer and look it up from those external places while keeping your index for only 90 days. If you really only need the five items you mentioned above, those would easily fit into a csv (or whatever you want to export and look up from). Then again, if you really only need those five items, you shouldn't have any trouble keeping all of them them on your indexes (if storage capacity is the reason you'd like to remove certain elements sooner).

Lastly, and this is also just an idea, you could try to set your archiving policy to run by script - this could possibly enable you to archive only selected events, although I have to admit I haven't done it or seen it done, so I don't know if it's possible.

View solution in original post

jeffland
SplunkTrust
SplunkTrust

As far as I know, you can't directly influence the time your data is stored on a per-event basis.

One solution for you is to index your events on different indexes and set different retention policies per index. This would require a modest effort to adjust your searches (and of course to assign indexes to events), but it should allow you to apply accurate settings on a granularity you require. Keep in mind that if you set your data to expire (or archive) after 90 days, this needs to apply to every event in your bucket in order to delete/archive the bucket - see here for a quick read.

Another idea (though I'm not sure how feasible this is) could be to regularly export the data you need to keep for longer and look it up from those external places while keeping your index for only 90 days. If you really only need the five items you mentioned above, those would easily fit into a csv (or whatever you want to export and look up from). Then again, if you really only need those five items, you shouldn't have any trouble keeping all of them them on your indexes (if storage capacity is the reason you'd like to remove certain elements sooner).

Lastly, and this is also just an idea, you could try to set your archiving policy to run by script - this could possibly enable you to archive only selected events, although I have to admit I haven't done it or seen it done, so I don't know if it's possible.

careybrucem
Explorer

Thanks for your input Jeffland,
Bruce

0 Karma
Get Updates on the Splunk Community!

Join Us for Splunk University and Get Your Bootcamp Game On!

If you know, you know! Splunk University is the vibe this summer so register today for bootcamps galore ...

.conf24 | Learning Tracks for Security, Observability, Platform, and Developers!

.conf24 is taking place at The Venetian in Las Vegas from June 11 - 14. Continue reading to learn about the ...

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...