Splunk Search

Is the "Predict command" causing the search results return, taking too much time?

Naren26
Path Finder

I have to forecast data for next 15 days, based on the last 30 days data. I have used the following query:

sourcetype=mylogs (message=1234*)
| timechart count as msgs span=1m 
| timechart avg(msgs) as msgs_daily_avg span=1d 
| predict msgs_daily_avg algorithm=LLP period=30 future_timespan=15

The search is getting me the results. But the performance is taken for beating. The time frame to collect data is "Last 30 days".

It is almost taking 20-25 minutes to fetch the entire results along with the predicted values. Is it because of the predict keyword which I used or the time frame which I set?

Can I update the results in a file by running the above query in a report and use the data in the file to populate them in dashboard panel?

How can this be done to avoid the performance issues?

0 Karma

FrankVl
Ultra Champion

No experience with the predict command, so I'll leave commenting on that part of your question to others, but to determine which part of your search is causing the slow performance: have you tried running only parts of the search in incremental steps? So only the base search first, then add the first timechart, then the second, then as a last test add the predict statement.

Also: since it seems you are just counting events, any chance you can start with a tstats? Or is the filter for certain message field content essential?

You're mentioning a 30 day time frame, but how many events are being searched?

0 Karma

skoelpin
SplunkTrust
SplunkTrust

The predict command is VERY memory intensive, so most likely he is trying to do more than what his hardware can do in a reasonable amount of time. There's better approaches then using the predict command

0 Karma

skoelpin
SplunkTrust
SplunkTrust

By default, Splunk is set to predict 2 days into the future which is configurable through props.conf and changing MAX_DAYS_HENCE

Using this to predict 2 days into the future will provide inaccurate results. Why not just use relative_time to look back and push the prediction forward? This will give you much better accuracy and give results in seconds since your searching a much smaller time frame

http://docs.splunk.com/Documentation/Splunk/7.0.2/Data/Configuretimestamprecognition

0 Karma
Get Updates on the Splunk Community!

Introducing Splunk Enterprise 9.2

WATCH HERE! Watch this Tech Talk to learn about the latest features and enhancements shipped in the new Splunk ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...