How to stop getting duplicate events from WindowsUpdateLog?

[hettervik]

Hi. I have a problem that I'm getting duplicate events from WindowsUpdate.log. I'm pretty sure it's related to the following message I'm getting in the internal logs Logfile truncated while open, original pathname="C:/winupdate/blabla", will begin reading from start. Though, if the logfile really was truncated (all the data in the file deleted), then I wouldn't get duplicate events. It seems to me that Splunk only "thinks" that the log file was truncated, but in reality it's not.

Can anyone explain to me how this "truncating" of the WindowsUpdateLog works, and why it can cause duplicate events in Splunk? Also, if someone could offer a possible solution, that would be nice as well!

[hettervik · Accepted solution]

Seems like this problem has been addressed in newer versions of the supported Splunk Windows TA.

https://docs.splunk.com/Documentation/AddOns/released/Windows/Configuration#Configure_Windows_Update...

[DavidHourani]

hi @hettervi,

This is an informational message, isn't it ? Doesn't seem like an error to me. It only means that the log file got rotated, if you're getting duplicates it could be because the file is generated with the same data again.

Is there any timestamp in the file or anything that makes the events unique ? You could try to play around with initCrcLength and crcSaltto avoid having the file indexed over and over

[hettervik]

Yes. It's not an error message per se. It doesn't necessarily mean that the windowsupdatelog got rotated, and that's the problem I think. For some reason, instead of rotating the logs in a normal fashion, Windows truncate the log file and starts writing it over again, if I understand correctly, but I'm not sure. Though it doesn't make any sense that Windows would generate the same file over again with the same data.

I have timestamps, and they are not unique. That is, I'm positive that the events I'm getting are in fact duplicate events from the same machine. Same timestamp, same everything, except indextime of course.

[DavidHourani]

so from what you're saying it could be that since Windows is modifying something in the header of the file the rest of the file is getting indexes all over again.

Could you please share your inputs configuration ?

[hettervik]

I don't have the input configuration available right now unfortunately, but it's straight from the official Windows TA from Splunk. Is uses a normal monitoring stanza for tailing the log. I think the path is C:\Windows\Logs\WindowsUpdate.

[hettervik]

For the record I've also tried using the setting ignoreOlderThan=1h, but it has no effect. I will try using another inputs.conf setting followTail=1. It's kind of a hacky solution, but if it stops duplicate entries I'm happy.