Getting Data In

User wants to retain data for 12 months

pratapa
Explorer

Following query diplays user logon events for the last 10 days.

index=main sourcetype=WinEventLog (EventCode=4624 OR EventCode=4634) user=pratapa.ln earliest=-12mon
| eval day=strftime(_time,"%d/%m/%Y") | stats earliest(_time) AS earliest latest(_time) AS latest by user host day
| eval earliest=strftime(earliest,"%d/%m/%Y %H.%M.%S"), latest=strftime(latest,"%d/%m/%Y %H.%M.%S")

But user wants the data to be retained for 12 months.
To achieve this, we have created a new index with name "retention" with the following parameters.

[retention]
coldPath = $SPLUNK_DB/retention/colddb
homePath = $SPLUNK_DB/retention/db
thawedPath = $SPLUNK_DB/retention/thaweddb
maxDataSize = 150
maxHotSpanSecs = 86400
maxTotalDataSizeMB = 54000
frozenTimePeriodInSecs = 31104000

What are the next steps that we need to follow?

0 Karma

dindu
Contributor

Hi,

As the data is very much structured my recommendation would be to go with summary indexes.
You could use the same query you created and navigate that to summary index -retention.
Modify the query to push data to summary index and schedule it as a report and run it periodically.

index=main sourcetype=WinEventLog (EventCode=4624 OR EventCode=4634) user=pratapa.ln earliest=-12mon
| eval day=strftime(_time,"%d/%m/%Y") | stats earliest(_time) AS earliest latest(_time) AS latest by user host day
| eval earliest=strftime(earliest,"%d/%m/%Y %H.%M.%S"), latest=strftime(latest,"%d/%m/%Y %H.%M.%S")
|collect index=retention

OR

You could also modify the configuration to inject the data to newly created index and use it like a regular index

0 Karma

pratapa
Explorer

Can I use the below query.

Can I use the query as below.

index=retention sourcetype=WinEventLog (EventCode=4624 OR EventCode=4634) user=pratapa.ln earliest=-12mon
| eval day=strftime(_time,"%d/%m/%Y") | stats earliest(_time) AS earliest latest(_time) AS latest by user host day
| eval earliest=strftime(earliest,"%d/%m/%Y %H.%M.%S"), latest=strftime(latest,"%d/%m/%Y %H.%M.%S")

0 Karma

pratapa
Explorer

Can I use the query as below.

index=retention sourcetype=WinEventLog (EventCode=4624 OR EventCode=4634) user=pratapa.ln earliest=-12mon
| eval day=strftime(_time,"%d/%m/%Y") | stats earliest(_time) AS earliest latest(_time) AS latest by user host day
| eval earliest=strftime(earliest,"%d/%m/%Y %H.%M.%S"), latest=strftime(latest,"%d/%m/%Y %H.%M.%S")
|collect index=retention

0 Karma

dindu
Contributor

Hi,
If you are using the summary index and the data is stored as table.You could directly fetch from the summary index later.

    index=retention 
   |table  earliest,latest ,user ,host,day

Read more about summary index - https://docs.splunk.com/Documentation/Splunk/8.0.2/Knowledge/Usesummaryindexing
https://docs.splunk.com/Documentation/Splunk/8.0.2/Knowledge/Configuresummaryindexes

OR
if you want the raw data to be persisted to the index "retention".
Deploy the above index.conf in indexer and direct the Splunk forwarder to inject data to the "retention " index.

0 Karma
Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...