Working out % change of field value based on events logged 1 day apart

[373782073]

Hi,
Apologies for the unclear title. I could not think of a logical description for the problem statement.

I have created a table from a search with two columns being filename and size(bytes)

filename size(bytes)
abc-1986-01-08-16:00:43-level1.tar 1000
abc-1986-01-09-16:00:43-level1.tar 1200

The filename field results have dates embedded (shown as examples 1986-01-08 and 1986-01-09 above). I'd like to create a separate column that will show the percentage difference in size for the same file name (but with different dates).

Example:

filename size(bytes) Increase-from-previous-date(%)
abc-1986-01-08-16:00:43-level1.tar 1000 0
abc-1986-01-09-16:00:43-level1.tar 1200 20

Note: The table results only contains files with 2 dates n and n-1 ( n = today -1 day)

Thanks in advance

[to4kawa · Accepted solution]

| makeresults
| eval _raw="filename size
abc-1986-01-08-16:00:43-level1.tar 1000
def-1986-01-08-16:00:43-level1.tar 700
abc-1986-01-09-16:00:43-level1.tar 1200
def-1986-01-09-16:00:43-level1.tar 800"
| multikv forceheader=1
| table filename size
| rename COMMENT as "this is sample you provided"
| rex field=filename "(?<time>\d{4}-\d{2}-\d{2}-\d{2}:\d{2}:\d{2})"
| eval files=mvindex(split(filename,"-"),0)
| eval time=strptime(time,"%F-%T")
| eval _time=time
| bin span=1d _time
| streamstats current=f values(size) as prev_size by files
| eval increase= round((size-prev_size)/prev_size * 100)
| sort files time
| fillnull increase
| table filename size increase
| rename size as "size(bytes)" , increase as "Increase-from-previous-date(%)"

hi, @373782073
how about this?

[373782073]

That's awesome, it works perfectly, now I just need to tweak it turn run on my search and extracted fields!

[soumyasaha25]

you can try something like this:

index=_internal earliest=-3d latest=-2d
| stats count as previous_count 
| appendcols 
    [ search index=_internal earliest=-2d latest=-1d
    | stats count as current_count] 
| eval Increase= (current_count-previous_count)/previous_count
| eval inc_perc=Increase*100 
| table previous_count current_count inc_perc

Make sure you modify the search with your logic to include the filename and size(bytes) as in your existing logic.

[373782073]

Thanks that is roughly what I want to achieve but I have the problem that my search finds 30 files dated with -3 days dates and the exact same 30 files dated -2 days from the present date, this being due to daily backups across 30 hosts being kept for -3days and -2days.... The response above provides a single row compounded result for all files.
To explain this better I am seeing this from my search:

filename size(bytes)
abc-1986-01-08-16:00:43-level1.tar 1000
def-1986-01-08-16:00:43-level1.tar 700
abc-1986-01-09-16:00:43-level1.tar 1200
def-1986-01-09-16:00:43-level1.tar 800
...x 30 instances of separate files dates with both -3days and 30 files with -2days

the command provided does a count and % increase computation of all disregarding that different files and their sizes are being used in the percentage increase computation. I am after a per file % increase difference in size for the same file name from the previous date.
Would you know how to separate individual files size increases per file names across all 30 files and list them per row?
Eg:
filename size(bytes) Increase-from-previous-date(%)
abc-1986-01-08-16:00:43-level1.tar 1000 0
abc-1986-01-09-16:00:43-level1.tar 1200 20
def-1986-01-08-16:00:43-level1.tar 700 0
def-1986-01-09-16:00:43-level1.tar 800 14

thanks