Questions in topic: "forecast"
https://answers.splunk.com/answers/topics/single/164311.html
The latest questions for the topic "forecast"Forecast Time series
https://answers.splunk.com/answers/676403/forecast-time-series.html
Hi Ninjas,
I have a query that looks like this:
sourcetype="x" index=y source="z" host="S"
| bin _time span=10m
| stats dc(CN) as Actual by _time | lookup CN_Forecast_S.csv _time OUTPUT lowerBound pred upperBound
| eval isOutlierLow=if(Actual < lowerBound , abs(Actual-lowerBound)/lowerBound, 0)
| eval isOutlierHigh=if(Actual > upperBound, abs(Actual-upperBound)/upperBound, 0)
| eval isOutlier=if(Actual < lowerBound OR Actual > upperBound, abs(Actual)/abs(upperBound-lowerBound), 0)
| fields _time, Actual, lowerBound, pred, upperBound, isOutlier, isOutlierLow, isOutlierHigh
The **CN_Forecast_S.csv** is a lookup file generated by a savedsearch that predict +2days of data.
The problem is my query display data until "now" only and I would like to show data for the rest +xdays that I already have predicted in the same graph. I tried to specify `lastest=+2d@d` , but that didn't work
That's the result of my query
![alt text][1]
Thank you in advance.
[1]: /storage/temp/253575-predict.pngsplunk-enterprisetimeseriesforecastWed, 25 Jul 2018 14:27:30 GMTBenImenCan I "teach" the machine learning tool kit projects in our company converge through time, based on project name and the field?
https://answers.splunk.com/answers/654562/can-i-teach-the-machine-learning-tool-kit-projects.html
Hi
I want to "teach" the machine learning tool kit how our projects in our company converges through time based on project name and the field 'SLOC'.
How can I tell the algorithm to see the different projects in our company and forecast the current project?
I can build a table that contains 3 fields: project name, date, sloc
Example:
project 1, 01.01.18, 22
project 1, 01.02.18, 40
project 1, 01.03.18, 2
project 2, 01.01.18, 400
project 2, 01.02.18, 88
project 2, 01.03.18, 16
Is it possible to "feed" the algorithm to know how our projects are going, so it will be able to forecast the next projects based on the previous project's data?
Thankssplunk-enterprisetimechartalgorithmsforecastSun, 29 Apr 2018 12:29:08 GMTmatansocherHow do I make a predict function more aggressive?
https://answers.splunk.com/answers/628858/how-do-i-make-a-predict-function-more-aggressive.html
How do I make a predict function more aggressive?
Below is an example of my predict example, search and graph:
`... | predict Total as predict future_timespan=12 holdback=0 | fields - upper* lower*`
![pic of graph with predict function used][1]
It is something I probably need to understand more of, and I am possibly entering the topic of polynomial or exponential types of growth. And maybe this is the case and the answer lies outside of `predict`.
Appreciate any advice/pointers to further reading/explanations on this.
----------
Some useful questions i have been reading up on this:
[how-to-create-a-search-to-predict-license-violation][2]
[prediction-function-algorithms-questions][3]
[predict-95-confidence-interval][4] - good at explaining some basics
[Predict Documentation][5]
Note: I could use the upperX values, which would be more arressive(give me higher future values) but again I don't think this will be aggressive enough. maybe I need to look at the [forecast option][6]?
tks
[1]: /storage/temp/229817-predict-make-more-aggressive-question.png
[2]: https://answers.splunk.com/answers/187080/how-to-create-a-search-to-predict-license-violatio.html?utm_source=typeahead&utm_medium=newquestion&utm_campaign=no_votes_sort_relev
[3]: https://answers.splunk.com/answers/95610/prediction-function-algorithms-questions.html
[4]: https://answers.splunk.com/answers/514892/predict-95-confidence-interval.html
[5]: https://docs.splunk.com/Documentation/Splunk/7.0.2/SearchReference/Predict
[6]: https://docs.splunk.com/Documentation/MLApp/3.1.0/User/ForecastTimeSeriespredictpredictionpredictiveforecastforecastingThu, 22 Mar 2018 22:00:17 GMTHattrickNZView time result of search as epoch
https://answers.splunk.com/answers/624938/view-time-result-of-search-as-epoch.html
I've got this search
index=vpn sourcetype=vpn_device login
| bin _time span=1h
| stats count AS logins by _time
The output looks like this
2018-02-03 09:00 65
2018-02-03 10:00 123
2018-02-03 11:00 92
What I want to see is this instead
1512813600 65
1512817200 123
1512820800 92
I want to take the output of the search and send it into a forecast time series in the Splunk Machine Learning section. I know I can use this
index=vpn sourcetype=vpn_device login
| bin _time AS "Time" span=1h
| stats count AS logins by "Time"
to get output 2, but the model requires _time, count to work.
Forecast Time Series
Predict likely future values given past values of a metric (numerical time series).
Choose an example dataset or enter a search (should contain "_time" field with unix timestamp values)
So any thoughts on how I can do this?splunk-enterpriseepochforecastFri, 09 Mar 2018 21:16:46 GMTjwhughes58How to forecast multiple time series from one search ?
https://answers.splunk.com/answers/556134/how-to-forecast-multiple-time-series-from-one-sear.html
Hello!
I'm really new to Splunk's Machine Learning Toolkit, so any help would be greatly appreciated. Thank you.
I'm trying to forecast time series for multiple apps in my query. My current query is:
<code> index=... report=1min_rollup app="..." earliest="06/07/2017:10:00:00" latest="06/07/2017:11:00:00" | stats sum(COUNT) as sum_count by _time,app | stats avg(sum_count) as avgCount by _time, app | bin _time span=5m | eval time=_time%3600 stats values(avgCount ) by _time, State | outputlookup eg.csv <code>
This gives me the lookup table eg.csv which looks like:
_time |app| avgCount
...
Now, I want to forecast the avgCount of all the apps on seperate time series. How can I generate multiple forecasted time series (one forecasted time series per app) from the search that I do have right now???
Thank you! Your help is greatly appreciated!
</code></code>splunk-enterprisemultiplesearchtime-seriesforecastTue, 18 Jul 2017 20:52:35 GMTEvaRexLooking for the Forecasting Code/Macro from Mike Fisher's Splunk Conf 2016 Presentation
https://answers.splunk.com/answers/476681/looking-for-the-forecasting-codemacro-from-mike-fi.html
Ok, I found this great post from Conf 2016 by Mike Fisher about using Splunk for forecasting:
https://conf.splunk.com/files/2016/slides/building-a-crystal-ball-forecasting-future-values-for-multi-cyclic-time-series-metrics-in-splunk.pdf
I've been looking for the macros in a Splunk App or post, however, I haven't found them and the version in the PDF isn't copying correctly for me. So I've retyped the macro and a static version, however, I'm having trouble getting the search to produce results.
I've posted my copy of the macro below and also a hard coded version of the search.
Macro version:
eval w=case
(
( _time > relative_time ( now(), "$reltime$@d-5w-30m" ) AND _time <= relative_time ( now(), "$reltime$@d-5w+$days$d+30m" ) ), 5,
( _time > relative_time ( now(), "$reltime$@d-4w-30m" ) AND _time <= relative_time ( now(), "$reltime$@d-4w+$days$d+30m" ) ), 4,
( _time > relative_time ( now(), "$reltime$@d-3w-30m" ) AND _time <= relative_time ( now(), "$reltime$@d-3w+$days$d+30m" ) ), 3,
( _time > relative_time ( now(), "$reltime$@d-2w-30m" ) AND _time <= relative_time ( now(), "$reltime$@d-2w+$days$d+30m" ) ), 2,
( _time > relative_time ( now(), "$reltime$@d-1w-30m" ) AND _time <= relative_time ( now(), "$reltime$@d-1w+$days$d+30m" ) ), 1
)
| eval shift=case( isnotnull(w),"+"+w+"w-30m+"+w+"w-20m+"+w+"w-10m+"+w+"w-0m+"+w+"w+10m+"+w+"w+20m+"+w+"w+30m" )
| where isnotnull(shift)
| makemv shift
| mvexpand shift
| eval time=relative_time(_time, shift)
| eventstats avg($val$) as pred by time
| eval upper=if($val$ > pred, $val$, pred)
| eval lower=if($val$ < pred, $val$, pred)
| stats avg($val$) as pred, stdev(upper) as ustdev, stdev(lower) as lstdev by time
| eval low=pred-lstdev*(sqrt(1/(1-$confidence$/100)))
| eval low=if(low<0, 0, low)
| eval high=pred+ustdev*(sqrt(1/(1-$confidence$/100)))
| eval _time=time
| timechart span=10m min(pred) as pred, min(low) as low, min(high) as high
| where _time > relative_time( now(), "$reltime$@d" ) AND _time <= relative_time( now(), "$reltime$+$days$d@d" )
Hard Coded version:
index=summary_trend source=orders_10min earliest=-5w
| timechart span=10m sum(OrderCount) as actual
| eval w=case
(
( _time > relative_time ( now(), "+1d@d-5w-30m" ) AND _time <= relative_time ( now(), "+1d@d-5w+3d+30m" ) ), 5,
( _time > relative_time ( now(), "+1d@d-4w-30m" ) AND _time <= relative_time ( now(), "+1d@d-4w+3d+30m" ) ), 4,
( _time > relative_time ( now(), "+1d@d-3w-30m" ) AND _time <= relative_time ( now(), "+1d@d-3w+3d+30m" ) ), 3,
( _time > relative_time ( now(), "+1d@d-2w-30m" ) AND _time <= relative_time ( now(), "+1d@d-2w+3d+30m" ) ), 2,
( _time > relative_time ( now(), "+1d@d-1w-30m" ) AND _time <= relative_time ( now(), "+1d@d-1w+3d+30m" ) ), 1
)
| eval shift=case( isnotnull(w),"+"+w+"w-30m+"+w+"w-20m+"+w+"w-10m+"+w+"w-0m+"+w+"w+10m+"+w+"w+20m+"+w+"w+30m" )
| where isnotnull(shift)
| makemv shift
| mvexpand shift
| eval time=relative_time(_time, shift)
| eventstats avg(actual) as pred by time
| eval upper=if(actual > pred, actual, pred)
| eval lower=if(actual < pred, actual, pred)
| stats avg(actual) as pred, stdev(upper) as ustdev, stdev(lower) as lstdev by time
| eval low=pred-lstdev*(sqrt(1/(1-90.0/100)))
| eval low=if(low<0, 0, low)
| eval high=pred+ustdev*(sqrt(1/(1-90.0/100)))
| eval _time=time
| timechart span=10m min(pred) as pred, min(low) as low, min(high) as high
| where _time > relative_time( now(), "+1d@d" ) AND _time <= relative_time( now(), "+1d+3d@d" )
Thanks for your help,
RobmacroforecastforecastingThu, 24 Nov 2016 23:53:36 GMTrob_jordanExponential Smoothing Implementation in Splunk
https://answers.splunk.com/answers/70620/exponential-smoothing-implementation-in-splunk.html
Hi,
I am planning to implement exponential smoothing in Splunk based on below formula where
s1 is the forecasted value. At time t=0, it is equal to first event. For time=t, it is calculated based on below formula. I can hard code value for "alpha".
s1=x0
s{t}=[alpha * x{t-1}] + [(1-alpha)s{t-1}], t>1
For time=t, it is referring to previously calculated forecast value (s{t-1}) and previous event value (x{t-1}) so not sure how this can be achieved using Splunk.
Say the log data is like below and "total" is the field which needs to be used(x{t}) to calcuate forecasted value(s{t}). I know there will be a field named "total" created which contains all the values but is there a way I can refer to say first value in field "total" like total[0] (as in arrays) which will be equal to 4, total[1] which will be equal to 6?
1/2/13 2:30:00.000 PM total=4
1/2/13 2:31:00.000 PM total=6
1/2/13 2:32:00.000 PM total=8
1/2/13 2:33:00.000 PM total=10
Any help is greatly appreciated.predictfunctionpredictionforecastWed, 02 Jan 2013 22:59:50 GMTsamsplunkd