Hi All,
So I am trying to make a modular input which ingests the journal in newer LInux distributions which no longer log to a file.
I have this test Python script which works from the command line.
import select
import systemd
from systemd import journal
j = systemd.journal.Reader()
#j.log_level(journal.LOG_DEBUG)
# j.add_match(_SYSTEMD_UNIT="systemd-udevd.service")
j.seek_tail()
# need to get previous event as seek_tail went right to the very end of the journal
j.get_previous()
p = select.poll()
p.register(j, j.get_events())
while p.poll():
if j.process() != journal.APPEND:
continue
for entry in j:
event = str(entry['__REALTIME_TIMESTAMP'])
for key, value in entry.iteritems():
event = event + ',' + str(key) + '="' + str(value).replace('"','\\"') + '"'
print event
Now the issue I have is when I try to add it to a modular input script. Splunk's Python obviously doesn't have the library for systemd and I get this error
[root@phoenix bin]# /opt/splunk/bin/splunk cmd /opt/splunk/bin/python /opt/splunk/etc/apps/journal_input/bin/journal_input.py
Traceback (most recent call last):
File "/opt/splunk/etc/apps/journal_input/bin/journal_input.py", line 20, in <module>
from systemd import journal
ImportError: No module named systemd
Everything I have read says to leave Splunk's python alone and not to try to install custom modules.
Does anyone have any suggestions how I can get this into a modular input or to get systemd accessible to Splunk's Python binary without having support freak out?
Hi, have a look on my comment on this topic: http://answers.splunk.com/answers/109009/splunk6-django-app-importing-sqlite.html
I basically download the source of same version of Python used by Splunk and compiled modules outsite... after that I moved the compiled modules to the Splunk/lib/python dir.
Hi, have a look on my comment on this topic: http://answers.splunk.com/answers/109009/splunk6-django-app-importing-sqlite.html
I basically download the source of same version of Python used by Splunk and compiled modules outsite... after that I moved the compiled modules to the Splunk/lib/python dir.
Thanks this has definitely stopped the importing of the library issue.
cp -r /usr/lib64/python2.7/site-packages/systemd /opt/splunk/lib/python2.7/site-packages/
chown -R splunk:users /opt/splunk/lib/python2.7/site-packages/
I did want to try to avoid this however as I know Splunk officially does not recommend this.
I note in your thread Splunk were looking into resolving the need for this in 2003. It seems they moved onto more important issues since.
There will be always the need to additional packages if you developed custom search/lookup scripts. I would prefer Splunk to ship the Python source/dev packages + virtualenv so would be easier to just run pip install PKG_NAME
🙂
Agreed it would make things a hell of a lot easier. I am guessing they don't want to support tweaking the Python libraries and in a way I can see why.
Sadly it really limits the power of what we can achieve with modular and scripted inputs though.