how to identify how many times that same timestamp is occured .....

[dilstn]

I have logs in which some events occured twice in same timestamp.......so i need to identify and eliminate those repeated timestamp alone.... I found that with dedup it provide once occurence of timestamp... but i need to eliminated repeated timestamps alone .....

For example .,,,.. these are the fields obtained by using regex ...Fields--> Date & CountryCode

Date CountryCode
Mar 28, 2013 5:42:30 AM AUS
Mar 28, 2013 5:42:30 AM AUS
Mar 28, 2013 5:50:21 AM PAK
Mar 28, 2013 5:57:14 AM USA
Mar 28, 2013 5:59:45 AM SGP
. .
. .
. .

So here i need to eliminate repeated timestamp... here the first two field values are repeated....
So it must be eliminated......

The result should be like this......

Date CountryCode

Mar 28, 2013 5:50:21 AM PAK
Mar 28, 2013 5:57:14 AM USA
Mar 28, 2013 5:59:45 AM SGP
.

So the repeated timestamp should be avoided .... Can u guide me .....plz...............

[BobM]

Splunk has a dedup command to remove duplicates. You just mention the fields to compare and it will keep only the first it detects.

... | dedup Date, CountryCode

Bob

[kristian_kolb]

So you want to remove all of the results for the duplicated events, not just the extras, right?

Assuming there are fields called Date and CountryCode;

...| stats c(CountryCode) as count first(CountryCode) as CountryCode by Date | where count < 2 | fields - count

/K

[martin_mueller · Accepted solution]

So... you want to drop any row where the timestamp occurs more than once? Try this:

... | eventstats count by _time | where count=1

[dilstn]

you are really rocking man !!! . It worked finally ...thank U

[martin_mueller]

If that's what he wants he can do an eventstats count by _time CountryCode to eliminate that risk.

[kristian_kolb]

there's a risk that two events with different CountryCodes may happen in the same second..