Throughout the custom search command process, splunkd and the Python script exchange metadata through a series of getinfo and execute commands. splunkd sends the getinfo command to request information, including the command type and required fields, from the Python script. splunkd sends a separate execute command for each chunk of search results in the pipeline So what you're seeing is the getinfo step. As far as I know there's no way to disable it or easily shorten it. This also means that any latency that your script has will be doubled and given to the next person. Great stuff. It looks to me like Splunk doesn't care about custom search commands and the feature is pretty much unmaintained.  This is understandable of course, since it's only a small, young company, run by amateurs with only few resources to spare. ... View more

When running   | inputlookup testlookup   (which is an external lookup) I get the error message: The lookup table 'testlookup' requires a .csv or KV store lookup definition ... so I assume this isn't an intended use case. Quite a bummer because (as per some of my earlier posts) custom search commands kind of suck. ... View more

Tabling works, but that's not enough if you need to carry along hidden values for Drilldowns. Tokens "kind of" work with the <fields> tag, but they don't seem to update when the token changes. Classic Splunk W. ... View more

Thank you, but ... what do I do with that? Say I want to implement a command in Java. The page has a helpful side note on the java path, but the table says to implement a command in Python I have to use ... The Splunk SDK for Python? And the guide just stops at how to point Splunk to my application, which is like 5% of the way there. Do you expect people to know the protocol already? Because again, I don't see ANY documentation on it and I don't find the Python source to be all that readable either, though arguably that *is* the best documentation I've seen.  A hint on how the DB Connect folks did it would be helpful. Though then again they also use a ton of shims until they get to launch their java app, but at least the protocol appears to be implemented there. ... View more

What a crazy oversight, right? This isn't just some authenticated port that you can't do anything with or something someone would ever actually normally connect to from the outside. No, this thing opens up a very direct tunnel into the java server to the whole network using the Splunk custom command protocol. What?!?! Splunk has been negatively impressing me regarding everything surrounding custom search commands (the DB Connect commands are implemented as such) and DB Connect is an especially hard-to-predict-hard-to-debug example. While I can't answer your question directly, I can tell you what didn't work for me. I was hoping that setting the vmopts option -Daddress=127.0.0.1 or -Dserver.address=127.0.0.1 would help, since that's what works for some Spring Boot applications (which I'm not sure DB Connect is, but it might be). Perhaps firewalling would work or network namespaces/containerization. ... View more

This doesn't seem to work for clearing the scripts cache ($SPLUNK_API/servicesNS/-/-/admin/script/_reload). It's not equivalent to $SPLUNK_WEB/en-US/debug/refresh?entity=admin/script. ... View more

Holy crap guys, I found some hints in the Splunk Dev For All app!!! This has a fairly small (Python 2) utility library called "cexec" implementing the chunked search protocol! Here's a part of its docstring: This library abstracts away some of the low-level details of writing "chunked" custom search commands for Splunk (e.g. byte-level protocol parsing). However, it still requires a fair bit of background on how the chunked protocol works at a semantic level. For a detailed description of the protocol, read: https://confluence.splunk.com/display/PROD/Chunked+External+Command+Protocol+v1.0 At a high-level, the Splunk search pipeline operates on "chunks" of search results. Thus, when a "chunked" custom search command is in a search pipeline, Splunk will send chunks to the external command (on stdin) and expect chunks in reply (on stdout). This library implements a BaseChunkHandler class that handles most of the details of receiving and sending chunks. Developers are expected to extend this class with their own handler() method to actually do useful work on search results. Sadly the linked Confluence page is offline, not saved by the Internet Archive and a web search for "Chunked External Command Protocol v1.0" yields exactly 0 results (how often does that happen?). Luckily the library only has 368 lines, is well commented and quite readable! So that's probably some of the best documentation we have. Still baffling that Splunk Inc. seems to want to keep information about the chunked search protocol a secret. That together with the atrocious performance (200ms MINIMUM) makes using them for utility functionality inviable. And do you know what that means? We'll use the custom command exactly once and do all processing outside of Splunk. If the bad experience of custom search commands is meant to hamper migration away, congratulations, you played yourselves. ... View more

What a joke to have to rely on this. Simply insane. Thank you.  ... View more

I'm looking for this as well, for colocation reasons. But so far I haven't found a way. For people playing along at home: web.conf has a few options for dashboards (like dashboard_html_allow_embeddable_content), but seemingly nothing to allow embedding scripts. ... View more

Is there really no better way than this? ... View more

Now here's an interesting tidbit: Where a bare-bones Python script using Splunklib takes 170ms to dispatch.evaluate, a dbxquery from DB Connect takes 80ms for a whole query! And that's also with a chunked python3 search command. So what's different? It doesn't use splunklib! Or at least not the Python version. The Python script acting as the bridge simply forwards the search protocol messages to a TCP server which likely uses the Java implementation. I've said before that splunklib could use some attention - case in point. ... View more

Noah Dietrich, the most official source of information on custom search commands (!) doesn't exactly know why this happens either (13:00). ... View more

Despite our use cases being slightly different I think we want the same thing: Some form of persistence rather than the file being called over and over. I ask the question a bit more generally here: https://community.splunk.com/t5/Building-for-the-Splunk-Platform/Custom-Search-Command-V2-Chunked-Persist-script/m-p/626063#M10884 ... View more

I wouldn't consider this problem solved. It remains annoyingly hard to run custom search commands from the command line because the protocol is entirely undocumented and no tooling is available. ... View more

I also have this issue with a chunked custom search command. I simply call the command in a search (Fast/Verbose doesn't make a difference) and the command is called twice. Preview is off. The job inspector shows 2 invocations, but doesn't explain them. My logs also show 2 invocations, with some differences: The metadata field is a bit shorter on the first invocation Only in the second invocation is the search_results_info field filled (with a lot of general information). Interestingly both times the action is getinfo, preview is False and streaming_command_will_restart is True. That getinfo is the first call seems to be a thing of the V2-Protocol from my reading of the Splunklib code. Edit: At some point I get this message:   ERROR ChunkedExternProcessor [27114 ChunkedExternProcessorStderrLogger] - stderr: Exception ignored in: <_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>   It  should be noted that even though the file is loaded twice and the prepare method is called twice, the generate method is not. Too bad that for my application loading the file is exactly the problem. ... View more

Ha, yep, I had to guess this myself. The custom search commands (or splunklib in general) docs could use some work. ... View more

Thank you! This may come in handy if I don't manage to find another option to keep the search command's process alive and end up having to manage one myself. ... View more

[expose:all] methods = GET,POST pattern = ** skipCSRFProtection = 1 I'm trying to disable CSRF protection for other reasons and can confirm that this approach doesn't work for me. ... View more

This solves all my "why the hell isn't it adding this field no matter what I do" related problems. My previous workaround was doing a makeresults and doing an addcols to it, which is of course terrible. Why does Splunk behave like this by default? ... View more