How to use my plugin on Splunk cluster?

[lremember]

Dear splunkers:

I want to adapt my plugin to the Splunk cluster.  I have already set up the search head and peer cluster (one main node, two search head nodes, two peer nodes).

But I don't know how Splunk clusters work. How can I test to prove that my plugin is suitable for Splunk clusters. Currently, the index database created from peer nodes can search for relevant data in the search header node. Can you help me?

By the way: 

The data I input from the peer node takes a long time to be found in the search header node. Why is this?

[PickleRick]

1. What is this "plugin" you're speaking of?

2. What is your architecture and general config? Please use the standard splunk vocabulary - it helps greatly if we don't have to guess what the "main node" is for example.

3. "the index database created from peer nodes can search for relevant data in the search header node" - honestly, I have no idea what's going on here. Normally it's the other way around - you use search head to search for data from indexes stored on indexers (search peers).

[lremember]

We built the Splunk cluster using a 16 core 32GB virtual machine，I don't know if this configuration is low for the cluster.

In addition, what I am referring to as a plugin is an application based on Splink add-on, which is released to the Splink application after our development and testing are completed. Currently, cluster testing is underway. My application runs normally on a single node Splink platform, but I am not sure what modifications I need to make if it is a clustered version of the Splink platform. Many of the configurations in my application are saved in configuration files, rather than the Splunk platform's kv database.

[richgalloway]

It's hard to answer your questions about getting your app to work on a SHC without knowing what the app DOES.  It's good that it uses config files, but that's true of 99% of apps.

Tell us what actions your app/plug-in performs and we'll tell you how to get it to work on a cluster.

[lremember]

My plugin function is very simple. We have our own data comparison system, which uses the powerful log collection and processing capabilities of the Splunk platform to compare the data extracted from the logs with our database by calling our data comparison interface. Then, the comparison results are saved to a local file for display and reading by other interfaces developed based on Splunk SDK. Because I am not very familiar with the working principle of Splunk's clustering, I have added the option of saving the result file to the kV database. I don't know if this operation is foolish

[isoutamo]

Here is couple of links where you can see the Splunk's officially supported and preferred architecture and what are correct terms to use:

• https://www.splunk.com/en_us/pdfs/tech-brief/splunk-validated-architectures.pdf probably page 13 is what your are trying?
• https://docs.splunk.com/Documentation/Splunk/9.0.5/Indexer/Basicclusterarchitecture
• https://docs.splunk.com/Documentation/Splunk/9.0.5/DistSearch/SHCarchitecture

[lremember]

Thank you for your help. I have read the link documents you provided and have received great help from them. But there are still some unclear aspects.

The application we have developed is based on Splunk and we are preparing to publish it to Splunk applications, either on a single node Splunk platform or search peers with a search head.

But now we want the application to work based on the search head cluster. We have already built the search head cluster (three search heads), but the process of building a search head cluster is not very clear and familiar. What I want to know is how my application is distributed to the search head cluster and works properly.

I would greatly appreciate it if you could provide me with some additional suggestions on building a search head cluster

BestRregars

[richgalloway]

It would help tremendously if you told us what your app does and why you think it won't work on a SHC.

The best way to find out if it works is to try it.  Of course, you'll need a cluster which requires at least three search heads rather than the two you have now.

I don't understand what you mean by "peer nodes can search for relevant data in the search header node" because that's not the way Splunk works.  Data is stored on peers and is searched for there.  Normally, search heads forward all of their data to peers for storage and searching.

[lremember]

I am eager to know how to configure SHC and push the app now. It may be that I am too foolish and did not succeed in following the tutorial provided in the document. It would be better if you could describe the steps and methods for me

[lremember]

Thanks for you reply!

Sorry, I am not very familiar with the working principle of the Splunk search header cluster. So when I was building a search header cluster, it wasn't very smooth. What I want to know is how I can make my application work based on search header clusters. At present, it works normally based on a single search header and a cluster of peer nodes.

Some of the relevant configurations in my application are stored in files. How should I modify them to achieve the goal of working in a cluster environment？

[lremember]

@richgalloway  can you help me? Thank you!