My REST API works fine, apart from the fact that new processes of rest.py get launched whithout any clear reason why this happens. This is causing Splunk to hammer twitter.com API and in no time I hit the rate limit (max 180 hits per 15 minutes) and I get blocked for a while.
The processes:
splunk 12085 1 0 13:46 00:00:00 python /opt/splunk/etc/apps/rest_ta/bin/rest.py |
splunk 12095 1 0 13:46 00:00:00 python /opt/splunk/etc/apps/rest_ta/bin/rest.py |
splunk 12100 1 0 13:46 00:00:00 python /opt/splunk/etc/apps/rest_ta/bin/rest.py |
splunk 12107 1 1 13:46 00:00:00 python /opt/splunk/etc/apps/rest_ta/bin/rest.py |
splunk 12113 1 1 13:46 00:00:00 python /opt/splunk/etc/apps/rest_ta/bin/rest.py |
splunk 12118 1 1 13:46 00:00:00 python /opt/splunk/etc/apps/rest_ta/bin/rest.py |
splunk 12126 1 2 13:46 00:00:00 python /opt/splunk/etc/apps/rest_ta/bin/rest.py |
splunk 12137 1 3 13:46 00:00:00 python /opt/splunk/etc/apps/rest_ta/bin/rest.py |
...as you can see there are only a few seconds between the two ExecProcessor instances and goes on and on until I disable the REST data input.
I would like to see only one active process that only talks to Twitter every polling_interval.
Any help is welcome.
Ah yes... Splunk version=6.1.1, REST API version: 1.3.4.
Best Regards,
Peter
Damien, I have installed another clean instance of 6.1.1 on yet another fresh Debian. Same issue.
I see this on my Debian 16.04 and 18.04 installs with both Splunk 6.6.5 and Splunk 7.2.4.
Fresh installs, no other data inputs and I get piled up rest.py processes.
From what I can tell is that it appears that Splunk is scheduling rest.py to run via its scheduler, but the rest.py script assumes that it is a run-once instance, since it loops after starting.
This seems to be a workaround but it is modifying the logic of the baboonbones code.
https://answers.splunk.com/answers/389392/issue-with-restpy-not-closing-and-running-out-of-m.html
Hi Damien, I have copied everything (api, REST stanza's)onto another Splunk server (running 6.0.3) and there it seems to run stable. Not sure what is wrong with that 6.1.1 instance but I guess we can leave it for now, though I don't dare to upgrade to 6.1.1 on that other server. Thanks for your time so far! What I will do at a later stage, is install a new server from scratch (format/OS Debian "wheezy"/Splunk 6.1.1/API 1.3.4/Stanzas) and comment on the thread again. Note that I have not changed the code and even reinstalled the REST API multiple times. To be continued..
I can't replicate this , and I have a very diverse set of inputs.conf rest stanzas setup , and I have never heard of this from any other users before. So this is firmly in the "scratching my head" category. Have you changed any of the python code perchance ?
Damien,
Is this perhaps a Debian issue?
I am having this problem as well. I have been researching forums and boards for the answer and I haven't found anyone who has resolved it yet.
One common thread is that it seems that this seen on debian installs.
Damien, still happens after reinstalling the app and configuration completely (rm -Rf style). Any ideas?
Damien, I have done the exercise again and I must say that it does happen even with only 1 REST stanza configured and enabled.
By the way, the host is a fresh Debian with all updates installed.
Thanks for your time Damien. This problem occurs even with only 1 single REST stanza configured and enabled. So yes I still see them, but for the record, I will remove all and start from scratch. Will get back to you soon.
See if this shows the multiple rest.py stanzas:
/$SPLUNK_HOME$/bin/splunk btool inputs list|grep rest.py
jkat54, this command does not show any output, as the script (rest.py) is not shown in the inputs.conf file (it is called somewhere else by the Splunk REST API).
Thanks jkat54, will give that a try, will be back soon.
Stop Splunk. Ensure there are no rest.py processes running. Ensure that you have only 1 single REST stanza configured and enabled. Start Splunk. Do you still see multiple rest.py processes running ?