Splunk Search

What is the procedure to build your own Splunk (search related) function?

Glenn
Builder

I have heard that this is possible - please correct me if I am wrong.

Firstly, the reason I want to do this. We index a large volume of financial logs, which are in the Financial Information eXchange (FIX) format. These are not really in an easily human readable format as they contain a bunch of numeric codes for fields and values, so I am trying to get Splunk to translate these logs so when my users search for them, they can understand them without having to reference their FIX documentation.

As I imagine this is quite a common problem, I'm going to ask another question about whether anyone has solved this problem already. In the meantime, knowing how to create my own search functions would be useful for me and others anyway, and I couldn't find instructions in the documentation.

Search/replace can easily be done by piping the search to "rex" sed mode. My FIX guys have selected a "top 100" translations they want, which means that the rex command (while it does actually work) is quite an inelegant way to do it, since it is about 40 lines long.

I'd like to create my own custom function "fixtranslate" using python, so I could encapsulate this search/replace inside the function. I would use Splunk web to run the search, pipe it to fixtranslate (passing raw search results to the script) which would do the search replace, and pass the modified results back to Splunk web to display.

How do I do this?

0 Karma
1 Solution

gkanapathy
Splunk Employee
Splunk Employee

Please see my answer to your other question, as it might not be necessary to write a script as this could possibly be handled by existing Splunk functions.

But this doc: http://docs.splunk.com/Documentation/Splunk/5.0/Search/Aboutcustomsearchcommands describes how to write custom external search commands. There are also several out-of-the-box and examples scripts included with Splunk.

You should also look into how to write custom lookup scripts, which are similar but potentially more efficient and integrate into your search in a slightly different way.

View solution in original post

gkanapathy
Splunk Employee
Splunk Employee

Please see my answer to your other question, as it might not be necessary to write a script as this could possibly be handled by existing Splunk functions.

But this doc: http://docs.splunk.com/Documentation/Splunk/5.0/Search/Aboutcustomsearchcommands describes how to write custom external search commands. There are also several out-of-the-box and examples scripts included with Splunk.

You should also look into how to write custom lookup scripts, which are similar but potentially more efficient and integrate into your search in a slightly different way.

Glenn
Builder

Solution to original problem with FIX logs, using a custom search command, can be found here: http://answers.splunk.com/questions/887/has-anyone-got-a-method-for-decoding-fix-financial-format-lo...

0 Karma

Glenn
Builder

Looks like I would still need a separate custom lookup for each field, and since there are so many potential fields, this will be complex to configure and difficult to maintain. Custom search command still looks the most promising.

0 Karma

Glenn
Builder

Dang, thats it. I didn't see the documentation as was looking in the wrong place (Developer manual rather than Search manual). Thanks I'll take a look. I will look into a custom lookup script as well.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...