I have a table that contains several columns. The table looks something like this:
timestamp,region,product_number,status,count
time1,americas,12345,done,5
time2,americas,23456,fail,4
time3,emea,34567,done,3
time4,emea,56789,fail,1
time5,emea,56789,done,2
I would like to reformat it to remove the column values that are duplicated:
timestamp,region,product_number,status,count
time1,americas,12345,done,5
time2, ,23456,fail,4
time3,emea,34567,done,3
time4, ,56789,fail,1
time5, , ,done,2
I don't think there's a generic solution to remove "duplicates" in any field (at least I can't think of one), but if it's OK to specify fields you could achieve this with streamstats
.
... | streamstats current=f last(region) as _lastregion, last(product_number) as _last_product_number | eval region=if(region==_lastregion,"",region) | eval product_number=if(product_number==_last_product_number,"",product_number) | ... (and so on)
I don't think there's a generic solution to remove "duplicates" in any field (at least I can't think of one), but if it's OK to specify fields you could achieve this with streamstats
.
... | streamstats current=f last(region) as _lastregion, last(product_number) as _last_product_number | eval region=if(region==_lastregion,"",region) | eval product_number=if(product_number==_last_product_number,"",product_number) | ... (and so on)
Um, it works. Turns out I had bad data. Thanks, Ayn!
have you considered using selfjoin on region?
It doesn't produce the table I'm looking for unfortunately. What I get from your suggestion is:
timestamp,region,product_number,status,count
time1, ,12345,done,5
time2, ,23456,fail,4
time3,emea,34567,done,3
time4,emea,56789,fail,1
time5,emea, ,done,2