Splunk Search

How do I keep a sum total of events before a dedup?

nfieglein
Path Finder

Not sure if I am 100% clear in the question, but here is what I am looking to do. I have a stream of incoming messages which have an identifier which can be duplicated (updated statuses). I have a flag which indicates whether the message is a new message and a flag which indicates whether the message is a close of the event. I want to run a dedup to have the current status of the message, but I want to count all of the new messages and compare them to all of the open and closed messages. I cannot seem to find a mechanism for counting a set of events and then counting a subset of those events created through the dedup command. I cannot use transactions as the number of events can range into the millions and the number of messages per event can range in the 20-30s.

Here are the two searches that I want to combine:

index=dccmtdit
| `GetDatesAndTimes`
| stats sum(IsNewDispatch) as DispatchesOpened by QuarterCreated

And:

search index=dccmtdit
| `GetDatesAndTimes`
| dedup  DpsNum sortby -IsClosed -CurrentStatusDateEpoch
| stats sum(IsOpen) as OpenDispatches, sum(IsClosed) as ClosedDispatches, by QuarterCreated

I want to take those results and work further like this:

| search OpenDispatches!=0 OR DispatchesOpened!=0
| eval PercentOpen=exact(OpenDispatches/DispatchesOpened*100)
| eval PercentClosed=exact(ClosedDispatches/DispatchesOpened*100)
Tags (3)
0 Karma
1 Solution

somesoni2
SplunkTrust
SplunkTrust

GIve this a try

 index=dccmtdit
| `GetDatesAndTimes`
| eventstats sum(IsNewDispatch) as DispatchesOpened by QuarterCreated
 | dedup DpsNum sortby -IsClosed -CurrentStatusDateEpoch
| stats sum(IsOpen) as OpenDispatches, sum(IsClosed) as ClosedDispatches first(DispatchesOpened) as DispatchesOpened by QuarterCreated
| search OpenDispatches!=0 OR DispatchesOpened!=0
| eval PercentOpen=exact(OpenDispatches/DispatchesOpened*100)
| eval PercentClosed=exact(ClosedDispatches/DispatchesOpened*100)

View solution in original post

somesoni2
SplunkTrust
SplunkTrust

GIve this a try

 index=dccmtdit
| `GetDatesAndTimes`
| eventstats sum(IsNewDispatch) as DispatchesOpened by QuarterCreated
 | dedup DpsNum sortby -IsClosed -CurrentStatusDateEpoch
| stats sum(IsOpen) as OpenDispatches, sum(IsClosed) as ClosedDispatches first(DispatchesOpened) as DispatchesOpened by QuarterCreated
| search OpenDispatches!=0 OR DispatchesOpened!=0
| eval PercentOpen=exact(OpenDispatches/DispatchesOpened*100)
| eval PercentClosed=exact(ClosedDispatches/DispatchesOpened*100)
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...