Answers for "How to perform spectrum analysis?"
https://answers.splunk.com/answers/147907/how-to-perform-spectrum-analysis.html
The latest answers for the question "How to perform spectrum analysis?"Answer by yuanliu
https://answers.splunk.com/answering/149675/view.html
Following @martin_mueller's R-rated suggestion and help from R-rated app author @rfujara_splunk;-) as well as a frantic search for cheap interpolation, the following is a recipe to analyse event count.
| timechart count
| appendpipe [
| stats count
| addinfo
| eval temp=info_min_time."##".info_max_time
| fields temp count
| makemv temp delim="##"
| mvexpand temp
| rename temp as _time
] | timechart max(count) as COUNT
| fillnull
| eventstats count as TOTAL
| r "output=transform(input,FFT=Mod(fft(COUNT)),Freq=((1:TOTAL)-1)/(TOTAL*X_span))"
Application notes
1. You need to install the **R app**. See @martin_meuller's answer above.
2. For event counts, gaps should be interpreted as 0. The largest part of the above search is to do just that, thanks to @somesoni2's [answer to my question][1].
3. The `eventstats` to obtain `TOTAL` is superficial and a waste of computation. There should be a better way to do this within R.
4. The above only outputs modulus of the transformation because counts are all real numbers. You can output the complex numbers by ridding `Mod()` from the above. (Interestingly, although Splunk lacks complex number arithmetics, its stats functions accepts complex numbers. Maybe it takes the real part and discards imaginary part as NaN.)
5. `Freq` is a dummy sequence for interpretation, expressed in hertz. You can chart over `Freq`, for example.
6. Maximum frequency you can analyse is 0.5/`span`. `span` in both `timechart` calls must be equal.
2. Beware of an undesirable side effect of `timechart` used to fill gaps: It forces an extra interval.
A few F(FT)-words
1. As discrete Fourier transform goes, you only look at half of the output sequence (positive frequencies) when inputs are all real.
2. When analyzing (all-positive) event counts, output at frequency 0 is meaningless, as this component contains the strong DC bias.
4. `fft()` uses a square sampling window. Spectrum leakage could diffuse your analysis especially when dealing with black-and-white data such as event counts.
R-rated notes
1. Object `input` from Splunk is in "data frame” class. You need to “transform" it into arrays that most R functions deal with. The `transform()` function in the above has nothing to do with Fourier *transformation*. The latter is performed in `fft()` function.
2. In addition to fields you pass to R, `input` also passes certain Splunk internal fields as X-rated objects. In the above, X\_span is `span` in the last stats function (`timechart`); you also have access to X\_time which corresponds to \_time in Splunk. (This is perhaps not limited to R app.)
The above doesn’t address how to separate data series into R arrays then output transformed objects. That will be my end goal. But it’s a good start.
[1]: http://answers.splunk.com/answers/149425/how-to-produce-empty-time-bucketsThu, 07 Aug 2014 00:18:32 GMTyuanliuAnswer by martin_mueller
https://answers.splunk.com/answering/147929/view.html
I believe R is capable of FFT, take a look at http://apps.splunk.com/app/1735/ for using R within Splunk.Tue, 29 Jul 2014 07:14:47 GMTmartin_mueller