Hi Splunk community,
I am facing some issue in using the Splunk modular input.
The modular input is built around events and event writers that are constructing an xml event. But, when i search the events in Splunk, i would like to see the included JSON event; see attached event.
Below you see a code snippet of the code that generates the event.
for refence to splunk modular input
https://github.com/splunk/splunk-sdk-python/tree/master/splunklib
Is there anybody who knows how this is suppose to work ?
You can try passing all the details while initializing the object for Event class.
Ex. event = Event(data="{}", stanza="", time="%.3f" % time.time())
And than use ew.write_event(event)
Note: You'll require to import time module. (import time)
It looks like you're using simple mode for modular input rather than xml:
http://docs.splunk.com/Documentation/Splunk/7.2.0/AdvancedDev/ModInputsStream
This is setup in your introspection in the tag:
http://docs.splunk.com/Documentation/Splunk/7.2.0/AdvancedDev/ModInputsScripts
Hi, And thanks for you reply. This was one of the parameters that i have been testing out, but to no luck.
I tried to make my test case more simple.
import sys
import os
import six
import json
import time
import datetime
#import splunklib.client as client
# import splunklib
from splunklib.modularinput import Script, Scheme, Argument, EventWriter, Event
FUNCTION_NAME = "testModularInput"
class testModularInput(Script):
""" Class for importing jira data
"""
def get_scheme(self):
scheme = Scheme(self.__class__.__name__)
scheme.description = self.__class__.__name__
scheme.use_external_validation = False
scheme.use_single_instance = True
scheme.streaming_mode = Scheme.streaming_mode_xml
username_arg = Argument(
name="username",
title="User name",
data_type=Argument.data_type_string,
required_on_create=True,
required_on_edit=True
)
scheme.add_argument(username_arg)
return scheme
def validate_input(self, validation_definition):
return None
def stream_events(self, inputs, ew):
for input_name, input_item in inputs.inputs.items():
event = Event()
event.stanza = input_name
event.time = datetime.datetime.now().isoformat()+"+0200"
event.data = "test input of date"
event.index = "main"
ew.write_event(event)
return None
if __name__ == "__main__":
instance = testModularInput()
if len(sys.argv) > 1:
if sys.argv[1] == "--scheme":
instance.get_scheme()
elif sys.argv[1] == "--validate-arguments":
instance.validate_input()
else:
instance.usage()
pass
else:
instance.run(sys.argv)
sys.exit(0)
result in Splunk ( i have added dash to ensure that the xml is not hidden.
<stream>
<event stanza="testModularInput://testDebug" unbroken="1">
<time>2018-10-23T08:30:03.869000+0200</time>
<index>main</index>
<data>test input of date</data>
<done />
</event>
</stream>
Just for info, I newer got this to work. I found a work around of using print(evemt.data) instead.
Hi,
A little bit more details, regarding conf files:
inputs.conf
[myModularInput://firsttest]
sourcetype = myModularInput:json
props.conf
[myModularInput:json]
TRUNCATE = 0
INDEXED_EXTRACTIONS = json
KV_MODE=none
no transform
this current setup fails as the data can not be parsed as json ( it is a xml document).
To clarify my question, how do i get rid of the XML part of the event. It is fine that the values in the xml event is used to modify the time, index, sourcetype, source .... but in Splunk when i do a search i want the data available as json.
script output:
2018-10-22T14:39:24.915000+0200
{'time': '2018-10-22T14:39:24.915000+0200', 'test': 'Issue'}
index event:
2018-10-22T14:39:24.915000+0200
{'time': '2018-10-22T14:39:24.915000+0200', 'test': 'Issue'}
expected event:
{'time': '2018-10-22T14:39:24.915000+0200', 'test': 'Issue'}
Hi, It looks like the text viewer is interpreting the xml data.
To clarify my question, how do i get rid of the XML part of the event. It is fine that the values in the xml event is used to modify the time, index, sourcetype, source .... but in Splunk when i do a search i want the data available as json.
script output:
stream
event stanza="myModularInput://issuedebug" unbroken="1"
time>2018-10-22T14:55:12.821000+0200OrderedDict([('time', '2018-10-22T14:55:12.821000+0200'), ('test', 'Issue')])2018-10-22T14:55:12.821000+0200OrderedDict([('time', '2018-10-22T14:55:12.821000+0200'), ('test', 'Issue')])