Splunk Search

How to create a timechart in Splunk that shows how many accounts are in processing at different intervals?

benjillaz
Explorer

Hello Splunkers

Hope you are doing good, appreciate beforehand all the time you take helping us out here.

So I'm in the dilemma of simulating a "PipeLine" in Splunk. This is to know at certain time intervals how many, let's say accounts, are being processed. That means, if account 1 started processing at 3:00 pm, then at the interval of 3:05, if account 1, is still not finished, I will have a pipeline of 1 and so on. Then, let's say account 1 finished processing, so it will be subtracted from the pipeline. I need a search for this is to be able to see how many accounts are in processing at certain time intervals.

For this matter, I have something like this:

index="app_log" sourcetype=accounts_calcs calc_status="accountcalc(1)" OR calc_status="accountcalc(2)" | timechart count by calc_status span=10m | ??

So, I know if an account started calculation if the status is equal to accountcalc(1) and an account finished calculation if the status is accountcalc(2). I need to say at certain time intervals how many accounts are in the process of calculation, so when I find a start, I will sum it and when I have a finish, I will perform a deduction. I need to take into consideration that I have different accounts of course, so even though I have 300 accounts that started calculation and then 200 finished, I need to check that the actual account that started is the same that finished in that time interval. If not, it will still be on the pipeline until that particular account finishes.

Truth is, I don't have too much idea how I can do it, so I'm asking for help

Regards

1 Solution

martin_mueller
SplunkTrust
SplunkTrust

Assuming you have a field account_id you could do something like this:

  index="app_log" sourcetype=accounts_calcs calc_status="accountcalc(1)" OR calc_status="accountcalc(2)"
| stats min(_time) as _time range(_time) as duration values(calc_status) as calc_status by account_id
| search calc_status="accountcalc(1)"
| addinfo
| eval duration = if(searchmatch("calc_status=\"accountcalc(2)\""), duration, info_max_time-_time
| concurrency duration=duration
| timechart max(concurrency)
  • grab data
  • collect start, duration, status values for each account
  • only keep accounts that have a start status value
  • add end of time range to results
  • keep duration if account has an end status value, else set duration to "end of time range minus start time" to denote "still processing"
  • compute concurrently processing account count
  • chart maximum concurrency

Assuming you don't have a field account_id, you could try to hack yourself a pipeline count like this:

  index="app_log" sourcetype=accounts_calcs calc_status="accountcalc(1)" OR calc_status="accountcalc(2)"
| delta = if(calc_status="accountcalc(1)", 1, -1)
| streamstats sum(delta) as concurrency
| timechart max(concurrency)

This is counting each transaction end as a -1, each start as a +1, and keeps a running total. Note, you may need to reverse the +/-, or the order of events, or both in case you get upside-down or flipped charts.

View solution in original post

martin_mueller
SplunkTrust
SplunkTrust

Assuming you have a field account_id you could do something like this:

  index="app_log" sourcetype=accounts_calcs calc_status="accountcalc(1)" OR calc_status="accountcalc(2)"
| stats min(_time) as _time range(_time) as duration values(calc_status) as calc_status by account_id
| search calc_status="accountcalc(1)"
| addinfo
| eval duration = if(searchmatch("calc_status=\"accountcalc(2)\""), duration, info_max_time-_time
| concurrency duration=duration
| timechart max(concurrency)
  • grab data
  • collect start, duration, status values for each account
  • only keep accounts that have a start status value
  • add end of time range to results
  • keep duration if account has an end status value, else set duration to "end of time range minus start time" to denote "still processing"
  • compute concurrently processing account count
  • chart maximum concurrency

Assuming you don't have a field account_id, you could try to hack yourself a pipeline count like this:

  index="app_log" sourcetype=accounts_calcs calc_status="accountcalc(1)" OR calc_status="accountcalc(2)"
| delta = if(calc_status="accountcalc(1)", 1, -1)
| streamstats sum(delta) as concurrency
| timechart max(concurrency)

This is counting each transaction end as a -1, each start as a +1, and keeps a running total. Note, you may need to reverse the +/-, or the order of events, or both in case you get upside-down or flipped charts.

benjillaz
Explorer

Wow that is pretty advanced i would not have been able to come with something like that,i really appreciate the help.
Will use them and communicate how it looked

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...