Why does using bucket span before transaction command give wrong duration in results?

[RohiniJindam]

sourcetype = abc | bucket span=1h _time | transaction user_ip destination_domain maxspan=20s maxpause=2s | stats count  by duration destination_domain  

This search renders duration for all fields as "0"

sourcetype = abc | transaction user_ip destination_domain maxspan=20s maxpause=2s | stats count  by duration destination_domain  

Whereas the above search gives the correct duration between the events. Any thoughts where I am going wrong?

Thanks

[sideview · Accepted solution]

the bucket command basically rounds down all _time values to the nearest hour. The transaction command then calculates duration using the _time values of the events. Since in your transactions the _time values will all be rounded to the same hour, they will all appear to occur at the exact same time. Hence the durations will all be zero.

Plus, given your search, there appears to be no legitimate reason to bucket _time... You're not using the _time values anywhere except implicitly in transaction, and the only thing it's doing in transaction is bad.

If your stats clause at the end was trying to group your transactions by duration, destination_domain, and the hour in which they occurred, then this search would make sense fwiw:

sourcetype = abc | transaction user_ip destination_domain maxspan=20s maxpause=2s | bucket span=1h _time | stats count by duration destination_domain _time

[RohiniJindam]

Thank you for your help.

[sideview]

You got it.

[RohiniJindam]

True that. I do need | stats count by duration destination_domain _time in the base search that's generating the summary rows.
So if i understand the following search correctly,

sourcetype = abc | transaction user_ip destination_domain maxspan=20s maxpause=2s | bucket span=1h _time | stats count by duration destination_domain _time

transaction command will render the _time values as those of the first event in the transaction. bucket will round off these values to the nearest hour, right?

[sideview]

OK. If that | stats count by duration destination_domain is in the search generating the summary rows, then it'll be throwing the _time values away though. If the stats is in a search against the summary events and you've just simplified the question down, that's kind of a different issue.

Either way though it sounds like you want a | stats count by duration destination_domain _time in the base search that's generating the summary rows.

[RohiniJindam]

Thankyou for your response, nick.

I understand now. The reason I am using bucket _time is that I am summary indexing my data. So I have searches aggregating and working on data from the past hour. These results are further used in the daily and weekly searches. So i need the _time values.

[linu1988]

What i checked, Bucket command is making it an interval of 1 hour to latest events to older ones. When you apply transaction it doesn't find the respective end to complete the transaction.