The whole design of the app is to duplicate as little data a possible.
- We only index data based on the firstSeen or first_found date for each vulnerability on each host.
- We only create a new event when the state of the vulnerability changes.
- After the initial run of the input we only pull differential data.
Given the basic design above we rely on splunk deduping functionality on event creation to see events are identical and not create duplicate events if for some reason duplicate data happens during a differential pull, which is very rare it could even happen.
