Getting Data In

Is it possible to get more granular on Windows system events as far as retention is concerned on the indexer?

careybrucem
Explorer

Or am I at the mercy of the settings for index rotation settings?

In other words, we have the following requirements to save the following events for the following amount of time:

Successful logon/logoff - 90 days

Unsuccessful logons - 1 year

Creation/deletion of user accounts - 2 years

Anyway to do this or do I have to save everything for 2 years?

Thanks,

Bruce

0 Karma
1 Solution

jeffland
SplunkTrust
SplunkTrust

As far as I know, you can't directly influence the time your data is stored on a per-event basis.

One solution for you is to index your events on different indexes and set different retention policies per index. This would require a modest effort to adjust your searches (and of course to assign indexes to events), but it should allow you to apply accurate settings on a granularity you require. Keep in mind that if you set your data to expire (or archive) after 90 days, this needs to apply to every event in your bucket in order to delete/archive the bucket - see here for a quick read.

Another idea (though I'm not sure how feasible this is) could be to regularly export the data you need to keep for longer and look it up from those external places while keeping your index for only 90 days. If you really only need the five items you mentioned above, those would easily fit into a csv (or whatever you want to export and look up from). Then again, if you really only need those five items, you shouldn't have any trouble keeping all of them them on your indexes (if storage capacity is the reason you'd like to remove certain elements sooner).

Lastly, and this is also just an idea, you could try to set your archiving policy to run by script - this could possibly enable you to archive only selected events, although I have to admit I haven't done it or seen it done, so I don't know if it's possible.

View solution in original post

jeffland
SplunkTrust
SplunkTrust

As far as I know, you can't directly influence the time your data is stored on a per-event basis.

One solution for you is to index your events on different indexes and set different retention policies per index. This would require a modest effort to adjust your searches (and of course to assign indexes to events), but it should allow you to apply accurate settings on a granularity you require. Keep in mind that if you set your data to expire (or archive) after 90 days, this needs to apply to every event in your bucket in order to delete/archive the bucket - see here for a quick read.

Another idea (though I'm not sure how feasible this is) could be to regularly export the data you need to keep for longer and look it up from those external places while keeping your index for only 90 days. If you really only need the five items you mentioned above, those would easily fit into a csv (or whatever you want to export and look up from). Then again, if you really only need those five items, you shouldn't have any trouble keeping all of them them on your indexes (if storage capacity is the reason you'd like to remove certain elements sooner).

Lastly, and this is also just an idea, you could try to set your archiving policy to run by script - this could possibly enable you to archive only selected events, although I have to admit I haven't done it or seen it done, so I don't know if it's possible.

careybrucem
Explorer

Thanks for your input Jeffland,
Bruce

0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...