All Apps and Add-ons

How do I train my sentiment analysis model with my own data?

nawneel
Communicator

How To train model with training_data . what is command and how do i start using sentiment for analysing my twitter data.please give some insight on this.

1 Solution

carasso
Splunk Employee
Splunk Employee

$ python sentiment/bin/train.py

Usage: train

For example, "python sentiment/bin/train.py mydata mymodel"

(I have a small question in my head if you have to be in the bin directory when you run it)

See the sentiment/training_data directory for sample data and how it's laid out. See the README.

View solution in original post

Inteliment_Splu
New Member

hadoop@master:/opt/splunk/etc/apps/sentiment/bin$ $SPLUNK_HOME/bin/python train.py train Demo/ Demo_Demo
Training Directory: ../training_data/Demo/train

loaded 0 reviews from ../training_data/Demo/train
Error: float division by zero

Can anyone help me out in solving this error

0 Karma

_jgpm_
Communicator

this is an answer a long time coming, but you have to point to the right "train".
python has to be declared in the location SPLUNKHOME/bin/python
train has to be declared in the location SPLUNKHOME/etc/apps/sentiment/bin/train.py
demo has to be declared in the location SPLUNKHOME/etc/apps/sentiment/training_data/demo
model name has to be declared as lowercase "demo"

Then it works. It should look something like this:
Training Directory: SPLUNKHOME/etc/apps/sentiment/training_data/demo/train
names = ['SPLUNKHOME/etc/apps/sentiment/training_data/demo/train/-1.txt', SPLUNKHOME/etc/apps/sentiment/training_data/demo/train/1.txt']

+++++4k
++++len data = 9501
corpdir = 'SPLUNKHOME/etc/apps/sentiment/training_data/demo/train'

loaded 9501 reviews from SPLUNKHOME/etc/apps/sentiment/training_data/demo/train
Original Token Count: 5876
After removing rare: 5493
After removing weakly correlated: 156
...

0 Karma

_jgpm_
Communicator

correction, your pwd has to be 'SPLUNKHOME/etc/apps/sentiment/bin' for it to work without errors.

0 Karma

marcoscala
Builder

Hi! just trained my own set!!!

  1. Move into the "$SPLUNK_HOME/etc/apps/sentiment/bin" directory
  2. type "$SPLUNK_HOME/bin/python train.py train "root_set_dir" "Model_name"

where
"root_set_dir" is the name ofthe first segment of the dir you created under "sentiment/training_data" containing the train subdir with the sample data.

if you have:
/opt/splunk/etc/apps/sentiment/training_data/my_model/train
you have to run the following command:

"$SPLUNK_HOME/bin/python train.py train my_model/ my_model

Marco Scala

carasso
Splunk Employee
Splunk Employee

$ python sentiment/bin/train.py

Usage: train

For example, "python sentiment/bin/train.py mydata mymodel"

(I have a small question in my head if you have to be in the bin directory when you run it)

See the sentiment/training_data directory for sample data and how it's laid out. See the README.

marcoscala
Builder

David,
you forgot the "train" command after train.py:

"$SPLUNK_HOME/bin/python train.py train "root_set_dir" "Model_name"

it really needs "train" again!

Marco

carasso
Splunk Employee
Splunk Employee

$ python sentiment/bin/train.py
Usage: train
For example, "python sentiment/bin/train.py mydata mymodel"
(I have a small question in my head if you have to be in the bin directory when you run it)

See the sentiment/training_data directory for sample data and how it's laid out. See the README.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...