Splunk Search

Custom Search Command - Can I emit multiple records for each input record?

kmarx
Explorer

I'm have a custom command that parses an input field in each given record and emits 0 to N records as its output. I'm doing this to avoid a bunch of mvzip/mvexp and logic in the calling SPL. It does seem to work fine, but I'd like some reasurance that this is really supported in the SDK.

The SDK doc for StreamingCommand of the PythonSDK (http://docs.splunk.com/Documentation/PythonSDK) says (bold highlighting mine),

Streaming commands typically filter, augment, or update, search result records. Splunk will send them in batches of up to 50,000 records....

This (and the rest of the article and others like it that I've found) don't really seem to specify how many records can be returned. The above seems to suggest that it's really supposed to be 1:1 and not 1:n.

Here's an edited down version of my code. (I'm also a Python newbie, so apologies for any ugliness there.)

import sys
from mytokeninfo import Info
sys.path.append("splunk_sdk-1.6.5-py2.7.egg")
from splunklib.searchcommands import dispatch, StreamingCommand, Configuration

@Configuration(local=True)    # Per doc on "stateful" streaming commands
class ExStatefulCommand(StreamingCommand):
    def stream(self, records):
    for record in records:
         tokens = self.parseRecordForTokens(record)
         for token in tokens:
             info = self.processToken(token)
             record['newField1']  = info.field1  # Application specifics simplified here for clarity (hopefully)
             record['newField2'] = info.field2   # ...etc
             yield record

So, for each record, I'm augmenting it one or more times and also yield'ing it each time. If so, I'd love to see the doc for it. If not, can I get an explanation as to why and also suggestions for how best to deal with this in a clean and proper manner?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...