Hi,
My problem is duplicated windows security logs. 2 or more log same as each other.
03/18/2019 10:53:50 AM
LogName=Security
SourceName=Microsoft Windows security auditing.
EventCode=4799
EventType=0
Type=Information
ComputerName=TestClient.kvp
TaskCategory=Security Group Management
OpCode=Info
RecordNumber=21040
Keywords=Audit Success
Message=A security-enabled local group membership was enumerated.
Subject:
Security ID: NT AUTHORITY\SYSTEM
Account Name: TestClient$
Account Domain: KVP
Logon ID: 0x3E7
Group:
Security ID: BUILTIN\Administrators
Group Name: Administrators
Group Domain: Builtin
Process Information:
Process ID: 0x46c
Process Name: C:\Windows\System32\VSSVC.exe
03/18/2019 10:53:50 AM
LogName=Security
SourceName=Microsoft Windows security auditing.
EventCode=4799
EventType=0
Type=Information
ComputerName=TestClient.kvp
TaskCategory=Security Group Management
OpCode=Info
RecordNumber=21040
Keywords=Audit Success
Message=A security-enabled local group membership was enumerated.
Subject:
Security ID: NT AUTHORITY\SYSTEM
Account Name: TestClient$
Account Domain: KVP
Logon ID: 0x3E7
Group:
Security ID: BUILTIN\Administrators
Group Name: Administrators
Group Domain: Builtin
Process Information:
Process ID: 0x46c
Process Name: C:\Windows\System32\VSSVC.exe
useAck is possibly the cause then. (With reference to the comments above)
useAck ensures that you never 'loose' a message, but it can result in data duplication.
https://docs.splunk.com/Documentation/Forwarder/7.2.4/Forwarder/Protectagainstthelossofin-flightdata...
Is this forwarded with useAck = true set on the forwarders outputs.conf?
no this option is
useACK = false
should I change?
İf I change this option , Do I have a log loss?
if option is true , wait a 7mb logs, ı think is very long ?
No.
Have you checked on the source windows server to see if the actual event is duplicated in event viewer?
Can I find out what caused them by looking at the logs ?
Yes I check the win event viewer. and logs it just one,
try this:
[your search which finds duplicate events]|eval it=strftime(_indextime, "%Y-%m-%d %H:%M:%S.%N3"|table _time it host splunk_server Message
Look at the two rows for your duplicated events - is the index time the same, are they from the same splunk_server?
Sorry my wrong anser,
ı check today and useAck = true
and
this query result is
"_time",it,host,RecordNumber,"splunk_server"
"2019-03-19T07:56:29.000+0300","2019-03-19 07:56:30.0000000003","TestClient",3778048,a4idx07p SameMessage
"2019-03-19T07:56:29.000+0300","2019-03-19 07:56:55.0000000003","TestClient",3778048,a4idx06p SameMessage
This suggests that useAck is the problem - You can see both events are generated at exactly the same time, but they are indexed 25 seconds apart, by different indexers.
What likely happened is that the first message was received, but the indexer either did not ack the event in time (or it got lost on the network) so the forwarder resent it, resulting in the duplication.
This is by design - useAck means no messages should ever get 'lost' but it can result in duplication - this is the tradeoff.