Getting Data In

Splunk and AIDE -- How do I ignore the first line of an AIDE log file?

matthewssa
Path Finder

Right now AIDE runs a check every 5 minutes and comes back with the same results each time of files Added, Removed, or Changed. The issue is the timestamp changes and the same results are being indexed over and over even though there has been no change. I would like to prevent indexing the same log file, but Splunk sees the log as a different file because the timestamp is changing on the first line. Is there a way to prevent Splunk from indexing the AIDE logs and only index them when there is a change in the rest of the AIDE log below the timestamp?

Example AIDE log.

Start timestamp: 2016-06-11 01:53:00
Summary:
Total number of files: 1116
Added files: 0
Removed files: 1
Changed files: 3


Removed files:


removed: /var/log/aide/aideCIM.log


Changed files:


changed: /var/log/aide
changed: /var/log/aide/aide.log
changed: /var/log/aide/aide_files.log


0 Karma
1 Solution

nickhills
Ultra Champion

Are you using the AIDE-TA?
https://splunkbase.splunk.com/app/2864/#/details

From a compliance point of view (if thats something you are working towards) its probably 'correct' to re-read the alert each time it is updated. AIDE is modifying the log file, so whilst you could drop the timestamp, its still a modified log.

Your compliance officer may take the view that the issue should be properly investigated (and resolved) to quiesce the alerts, rather than 'hiding' them - but thats a conversation to have with them.

With all of that said, your on a 'hiding to nothing' by trying to FSIM the FISM log 🙂
Its common to add an exception so that you don't monitor the log files that the monitor is writing, which triggers the monitor, which updates the file, and ..... and ....

If my comment helps, please give it a thumbs up!

View solution in original post

matthewssa
Path Finder

Thanks for the response.
I did actually try to use the AIDE-TA, but the issue was having to run Splunk as root. That of course was not an option.

I took into account your consideration about bringing in the log over and over which makes since. You would want to keep seeing the alerts until you fix them. What I ended up doing was creating a TA that would parse all the events and only bring in those that matched the words changed: removed: added: This actually worked well because now we don't have to deal with all the rest of the junk lines.

0 Karma

nickhills
Ultra Champion

Are you using the AIDE-TA?
https://splunkbase.splunk.com/app/2864/#/details

From a compliance point of view (if thats something you are working towards) its probably 'correct' to re-read the alert each time it is updated. AIDE is modifying the log file, so whilst you could drop the timestamp, its still a modified log.

Your compliance officer may take the view that the issue should be properly investigated (and resolved) to quiesce the alerts, rather than 'hiding' them - but thats a conversation to have with them.

With all of that said, your on a 'hiding to nothing' by trying to FSIM the FISM log 🙂
Its common to add an exception so that you don't monitor the log files that the monitor is writing, which triggers the monitor, which updates the file, and ..... and ....

If my comment helps, please give it a thumbs up!
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...