control the indexing of the forwarders

[efaundez]

Good afternoon

     Currently we work in a cluster environment and we need to have a better control of the data that is indexed, I mean to control the indexing of new sources.

    To exemplify we have the following scenario

    elyos server
[input: ... / data.gz]
index = dps
sourcetype = damage
 
     As in the previous example, the indexing of index = dps is enabled, sourcetype = damage, but as you know, another stanza can be created

[input: ... / data2.gz]
index = heal
sourcetype = support

      How can we prevent this from happening? can be managed from the cluster the new sources, having a control of the new data ?, because if for some reason they decided to create new stanzas loading more data at index = dps .... there would be no control.

     I have reviewed documentation but changes are applied to the UF, but it is required that the control is in the cluster and not in each UF that is receiving new information.

[esix_splunk]

There is no mechanism internal to Splunk that would prevent a 3rd party from editing the inputs in a forwarder and adding additional data sources.

There are a few ways you can remediate this kind of scenario though..

1) Restrict access to indexers for only authorized forwarders. But if you have thousands of forwards, this is not easy, and additionally if someone has access to the forwarder, they can still add inputs/datasources

2) Update permissions on the forwarders themselves so that only authorized users can add inputs / data sources. This is not done within Splunk, but at the OS level. This also means that you someone on the other end should have access / permissions to Splunk and that you trust them not to add more sources.

3) Use Splunk to monitor your data sources. E.g., set an alert when there is an increase in data sources over the previous day. And then you can remediate by contacting the owner / deleting the data etc.

Those are just a few ideas i have based on experience. There are other ways this could be done, im sure there will be more feedback on this.

[efaundez]

would have to configure the option of forwarder management? to have control of the new sources towards the indexers?

[tiagofbmm]

I think there may be some confusion here between Cluster and the Deployment Server.
Whether or not you have an Indexer Cluster, the Splunk best practice here is to manage all you inputs in the Forwarder layer is solely through the Deployment Server.

Not only you as Administrator have full control of what you have, but also no-one but the DS makes any changes to what is ingested in each one of the Universal Forwarders.

Please follow this doc to fully understand the desired architecture:

https://docs.splunk.com/Documentation/Splunk/7.0.2/Updating/Deploymentserverarchitecture

[efaundez]

thanks for your reply.
  
  But now it works so long ago, and 550GB is indexed daily I do not think I can make a new configuration or suggest changes, the idea would be that it is currently working, apply some policies or see how to control the new sources from splunk and not in each UF

[esix_splunk]

There is no way to control onboarding of data sources from the indexer / cluster side. If the forwarder has the indexer splunktcp port, ssl cert, or autodiscovery password, they are trusted and data is accepted.

[ddrillic]

The big question is whether the deployment server controls all the deployment apps or each UF is being administered locally or maybe you have an hybrid solution like we do, and I believe it's pretty prevalent like this out there...

[efaundez]

Good afternoon, UFs are currently managed by third parties, but they are authorized in the first instance to index the new sources. As they know, they can index new sources by reference to the first index created, or index data at a known index.

If the user wants to index new data he will do it and we will not have total control. The idea would be from the master server to restrict the new sources not informed.

[ddrillic]

-- If the user wants to index new data he will do it and we will not have total control. The idea would be from the master server to restrict the new sources not informed.

Not sure if it's possible from the Splunk side...