I have not yet started ingesting IIS logs from my systems. The systems have roughly 2 years of logs stored on them, that I need to ingest for IA compliance reasons. I'm going to utilize ignoreOlderThan = 0 setting to ingest all of the logs that are stored on the system.
My situation is complicated by the fact that all systems in our environment are clones, so some of the logs that are present on the systems are valid, and some of the logs that are present are remnants from the system that it was cloned from. I became aware of this when I was testing and a co-worker pointed out that Splunk had logs for my test system going back 4 months, but the system was just cloned 2 weeks ago.
For example, the following 2 logs came from the same system, which is currently named ZA70027.
2015-03-10 13:24:08 W3SVC2 ZA70043 10.1.101.116 GET /XenApp/loading.htm - 443 - 204.223.38.5 Mozilla/5.0+(compatible;+MSIE+10.0;+Windows+NT+6.1;+WOW64;+Trident/6.0) <Address Removed> 200 0 0 1957 345 1872
2015-06-15 18:32:22 W3SVC2 ZA70027 10.1.101.124 GET /XenApp/media/LoginPaneFooterMidBorderGlow.png - 443 <Account Removed> 192.156.37.34 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.1;+WOW64;+Trident/7.0;+SLCC2;+.NET+CLR+2.0.50727;+.NET+CLR+3.5.30729;+.NET+CLR+3.0.30729;+Media+Center+PC+6.0;+InfoPath.3;+.NET4.0C;+.NET4.0E) <Address Removed> 200 0 0 549 600 46
I am already running all my logs through heavy forwarder. I would like to utilize a Regex search string in my transforms to drop unwanted logs.
Is there a way to verify that the current system's hostname exist in the IIS log before sending them to the indexer?
