On our Linux servers, we see that audit policies are re-applied to the audit service whenever the service is restarted. I'm trying to find a way to only generate events if a restart of the server has not occurred within a given period of time (likely 10-15) minutes. How would I configure my search to search for audit change events that are not within 15 minutes of a server restart?
Search for finding a change in audit settings:
index=os* key="MAC-policy" type=CONFIG_CHANGE
Search for finding server restarts:
index=os* type=SYSTEM_SHUTDOWN
Let's try a different approach
index=os ("SYSTEM_SHUTDOWN" OR "CONFIG_CHANGE") | rex "(?<action>SYSTEM_SHUTDOWN|CONFIG_CHANGE)" | autogress action as next_action | where action=next_action
Let's try a different approach
index=os ("SYSTEM_SHUTDOWN" OR "CONFIG_CHANGE") | rex "(?<action>SYSTEM_SHUTDOWN|CONFIG_CHANGE)" | autogress action as next_action | where action=next_action
That does the trick.
So I understand it, are you skipping the first CONFIG_CHANGE event that occurs after a SYSTEM_SHUTDOWN event?
That's Correct
That works for me. Thanks.
See if this works. What this should do, is group all events that start with Shutdown and have a CONFIG_CHANGE within 15min timespan. All other events will not be grouped. closed_txn=f, returns ungrouped events. You may have to experiment with keepevicted
vs keeporphans
index=os | transaction maxspan=16m startswith="SYSTEM_SHUTDOWN" endswith="CONFIG_CHANGE" keepevicted=t | where closed_txn=0
http://docs.splunk.com/Documentation/Splunk/6.4.2/SearchReference/Transaction
Wouldn't marking all transactions as open mean that the "CONFIG_CHANGE" wasn't taking place? 100% of the time it will take place after a reboot so I need to be able to see this event if the "SYSTEM_SHUTDOWN" doesn't occur.
This doesn't mark all transaction as open. It will mark transactions as open if it DOES NOT meet any of these criteris
1) Find SYSTEM_SHUTDOWN
2) Find CONFIG_CHANGE within 15mins of SYSTEM_SHUTDOWN
Once it groups the transactions, (SYSTEM_SHUTDOWN & CONFIG_CHANGE within 15min span) remaining events will be marked as "open" transactions. The where clause will keep only the open transactions.
To make this cleaner, you could add ("SYSTEM_SHUTDOWN" OR "CONFIG_CHANGE") to your base search to minimize the events returned.
Were you able to try the query? What results did you get?
The results showed transactions but the ones I saw didn't start with "SYSTEM_SHUTDOWN" so I flipped the keepevicted value to false and the events that were generated are not showing now but I need to do an effective test by rebooting my test machine then adding an audit rule.
Update: Looks like that did not produce an event when I added an audit rule. Here is what happened:
First I kept the "keepevicted=t". This produced events but the events produced are within the span of time. If the idea was to produce events outside the span of time, this did not work.
Next I changed it to "keepevicted=f". This produced no events so I changed to span down to 1s to force a result and still no result so this did not work.
Next I changed it from "keepevicted=f" to "keeporphans=t". This produced no results so it did not work.
Next I changed it to "keeporphans=f". This produced no results so it did not work.
My understanding is you DO NOT want CONFIG_CHANGE" right after a "SYSTEM_SHUTDOWN". If that's true, you should set keepevicted=t
. You want all events that are NOT grouped in a transaction. To verify the findings, do a table _time _raw closed_txn
and compare the events where closed_txn=0
and closed_txn=1
. The one's with 1 should have "SYSTEM_SHUTDOWN" AND/OR "CONFIG_CHANGE" where as the the other should only have "CONFIG_CHANGE"
I want to see the "CONFIG_CHANGE" event ONLY if the "SYSTEM_SHUTDOWN" event hasn't occurred in the last 15m. That would be the only time the "CONFIG_CHANGE" event would be relevant to our security team.