Splunk Search

Can a custom search script invoke an eval/where expression?

Lowell
Super Champion

I would like to be able to create a custom search command that uses an eval-expression as an argument. (Similar to how you can say ... | transaction startswith=(eval-expression) ...)

Does anyone know if it's possible to invoke an eval-expression from within the python custom search command? Obviously this can be done for built-in commands like transaction, but that doesn't seem to be written in python.

If this can't be done, any ideas as a functional workaround?

Lowell
Super Champion

Here is one hacked up way to do this. BTW, I'm going to let this unanswered for now, in the hopes that someone come up with a better (less-hackish) answer. Also, the ability to directly issue an eval/where expression from within a custom search command could be extended to many more situations than this limited hack.

After some playing around, I came up with an approach that works by using a combination of getinfo and a dynamic preop command. This approach leverages the fact that the getinfo mechanism calls the script twice, and passes in the arguments to the script both times. The first __GETINFO__ call dynamically builds a pre-streaming operation based on the argument passed to the script. This causes the pre-operation command to execute an eval command before my custom search command is run; and so the temporary field is accessible from my search command when it's called in __EXECUTE__ mode.


Below is a demo version of my script. If the "filter" eval expression evaluates to true, then function f1 is used to handle the event, and if it evaluates to false then function f2 is used instead. In this simple example, f1() adds two fields, and f2() multiples two fields. If no "filter" is given, then f2() is always called.

The script is called like this:

... | mycmd filter="date_hour>12" | table a b sum product

mycmd.py:

import splunk.Intersplunk

(isgetinfo, sys.argv) = splunk.Intersplunk.isGetInfo(sys.argv)
args, kwargs = splunk.Intersplunk.getKeywordsAndOptions()

# Process args
eval_filter = kvargs.get("filter", None)

if isgetinfo:
    # streaming, generating, retevs, reqsop, preop
    if eval_filter:
        splunk.Intersplunk.outputInfo(False, False, False, True, "addinfo | eval _filter=if(%s,1,0)" % eval_filter)
    else:
        splunk.Intersplunk.outputInfo(False, False, False, True, "addinfo")

def f1(r):
    # Add field 'a' and 'b'
    r["sum"] = int(r["a"]) + int(r["b"])
    return r

def f2(r):
    # Multiply field 'a' and 'b' 
    r["product"] = int(r["a"]) * int(r["b"])
    return r

if __name__ == '__main__':
    (results, dummyresults, settings) = splunk.Intersplunk.getOrganizedResults()
    if len(results) >= 1:
        search_id = results[0]["info_sid"]
    else:
        search_id = None
    logger.info("SID:  %r", search_id)
    output = [] 
    try:
        for result in results:
            if "_filter" in result:
                if result["_filter") == "1":
                    result = f1(result)
                else:
                    result = f2(result)
                del result["_filter"]
            else:
                result = f2(result)
            output.append(result)
        splunk.Intersplunk.outputResults(output)
    except Exception, e:
        splunk.Intersplunk.generateErrorResults("Unhandled exception:  %s" % (e,))

MuS
SplunkTrust
SplunkTrust

Hi Lowell, is this script working in 4.2.x? I'm trying to run it but it returns 'error 1' although I fixed your typo and change a|b field to existing ones that match my search

0 Karma

southeringtonp
Motivator

You'd need to build the evaluation logic into your Python script.

Parameters like this are going to be passed into your Python script as command-line arguments -- you can get them from sys.argv and do whatever you want from there.

0 Karma

Lowell
Super Champion

Sorry, but I think your missing the point. I know I can do whatever I want within python, but the point is that I don't want to go build by own eval-engine. I simply want to leverage the one Splunk already built. (The eval command already has over 40 functions and that list is growing each release. I certainly don't want to go build my own mini expression language and all those functions. And then try to keep that in sync with every new splunk release.)

0 Karma

ziegfried
Influencer

if it is not supported, it would be a great enhancement!

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...