All Apps and Add-ons

Splunk Machine Learning Toolkit: Can I apply a machine learning model to external data via REST?

walterd
Engager

Hi all,

I work in a Security Operations Center for an organization, which is targeted by lots of Spam and Phishing mails.

Our users have the opportunity to report those Phishing mails to us, where we continue analysing them.
The issue is, that we get plenty of Spam mails reported too.

Now we would like to filter out only the Phishing ones.
Therefore, I have obtained a dataset and trained a model in Splunk.

Regarding the worklow, I am not sure on how to integrate Splunk in the chain.
In our current situation, the raw metadata of the extracted features cannot be streamed in realtime to Splunk but has to be uploaded manually one a week.

I am looking for a solution something like this:
1. Extract the features from newly reported mails
2. Call a ML algorithm with the previously obtained features as paramaters in Splunk via REST
3. Get the result of the classification in an instant

Can this be done within Splunk without having the data to be first forwarded and indexed?

Cheers,
Dan

1 Solution

astein_splunk
Splunk Employee
Splunk Employee

Hi

"In our current situation, the raw metadata of the extracted features cannot be streamed in realtime to Splunk but has to be uploaded manually one a week."

Why is this? Most of the customers I work with can extract their features from the streaming data in Splunk, which are used in the training of the model (|fit) . Can you elaborate?

If you go through the SDK/API documentation for Splunk, you can submit a search to splunk and get the results back from an external system. An example search can be making an empty result set (makeresults) , filling the correct fields with features (|eval) and then applying the ML model to get the result back (|apply).

View solution in original post

astein_splunk
Splunk Employee
Splunk Employee

Hi

"In our current situation, the raw metadata of the extracted features cannot be streamed in realtime to Splunk but has to be uploaded manually one a week."

Why is this? Most of the customers I work with can extract their features from the streaming data in Splunk, which are used in the training of the model (|fit) . Can you elaborate?

If you go through the SDK/API documentation for Splunk, you can submit a search to splunk and get the results back from an external system. An example search can be making an empty result set (makeresults) , filling the correct fields with features (|eval) and then applying the ML model to get the result back (|apply).

walterd
Engager

I only saw now your reply.

Thank you very much! This was exactly the solution I was looking for.

Regarding your question, we have some sort of a process bottleneck in place, which will need time to get removed.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...