Deployment Architecture

Unix Log File Tampering

brianokelly
Explorer

I would like to understand if there is a way to monitor if a unix log file has been tampered with (lines deleted or modified). I dont believe this can be done with fschange monitor as it will record all updates to the log as changes.

Tags (3)

dwaddle
SplunkTrust
SplunkTrust

This is a very complicated subject to comprehensively answer. The only way to be entirely sure is to be able to (essentially) system-call trace every single process running on the system at all times.

Remember that users don't change files - processes running on behalf of those users change files. In a traditional Unix system, the root user is assumed to have absolute authority to have any of his processes change a file. Without keeping detailed trace data of what read() and write() system calls were made by each and every process, how do you know which ones were "authentic" and which ones were not?

This is one of the gaps that SELinux and other similar security layers attempt to close. SELinux makes it possible for additional controls to be placed around certain system operations -- even for the root user. A properly configured selinux policy can make it much harder for someone to tamper with logfiles by restricting what processes can write to those logfiles, regardless of what user is running the process.

That said, though, even at its current state, SELinux is not able to fully withstand the attack of a complicit root user. See http://marc.info/?l=selinux&m=105458123321234&w=2 for comments from an NSA representative regarding how much of root's capabilities have to be taken away in order to make it substantially harder for the root user to subvert the system.

mw's answer of the operating system auditing facilities is a good one - the audit subsystem could give you "commercially reasonable best effort" assurance that you knew about each time a log file was changed.

But at some point you have to just stop and trust your sysadmin to not subvert the systems you've hired her/him to manage for you.

mw
Splunk Employee
Splunk Employee

I believe the answer to this is going to be: turn on OS level auditing. That will tell you as much as is possible to know. You could probably splunk user's .bash_history, etc files as well, but of course, they can modify their HISTFILE if they want to.

jbsplunk
Splunk Employee
Splunk Employee

fschange tells you what the change that occured was, including deletes and modifications. If you use the 'fullevent=true' option, you can even compare before and after to see how the file looked, and how it lo oks now.

brianokelly
Explorer

Unfortunately as I stated in my post, fschange wont work. Consider what happens when a user logs in to the system. Entries are placed into /var/log/secure, fschange would record these changes. If the root user echo's the same lines into the file, fschange has no way of distinguishing between the two events. Remember, we want to know if a USER has modified a log, not that the log was written to as normal.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...