Getting Data In

Index 10 different companies data into splunk

rupeshn
Explorer

Hi All,

I've got into weird situation where in i need to get data of 10 different companies into Splunk with two indexers in single site clustering with one SH.

I'm under planning stage.

I'm wondering how can we install apps for each company individually on same indexer since each company will have their own indexes.

For example: If i install windows add-on, It would give results of any one company. Then , How can i get data of other 9 companies as well.

Please suggest.

Tags (1)
0 Karma
1 Solution

nickhills
Ultra Champion

Generally apps apply their knowledge objects to sourcetypes.
This means that if you install the Windows specific TA, it will apply its field extractions, dashboards and reports to all data that matches that sourcetype, regardless of the index it is in.

This means that if you have 10 (or more) separate indexes (with permissions set correctly) a user from any org can use the same App and TAs to access data ONLY from their index.

If some orgs have apps that others do not use, you can set the permissions on an app to show/hide it from that group of users if you need to.

You need to take extra care to ensure your dashboards correctly search multiple indexes, but this is not too complicated.

You will need to be VERY careful about your index security and how you manage your roles. A slip up here could reveal other orgs data, which would be a huge privacy concern and depending on your territory, significant fines.

There is no real technical limitation to what you are proposing - just make sure your processes are robust.

If my comment helps, please give it a thumbs up!

View solution in original post

0 Karma

nickhills
Ultra Champion

Generally apps apply their knowledge objects to sourcetypes.
This means that if you install the Windows specific TA, it will apply its field extractions, dashboards and reports to all data that matches that sourcetype, regardless of the index it is in.

This means that if you have 10 (or more) separate indexes (with permissions set correctly) a user from any org can use the same App and TAs to access data ONLY from their index.

If some orgs have apps that others do not use, you can set the permissions on an app to show/hide it from that group of users if you need to.

You need to take extra care to ensure your dashboards correctly search multiple indexes, but this is not too complicated.

You will need to be VERY careful about your index security and how you manage your roles. A slip up here could reveal other orgs data, which would be a huge privacy concern and depending on your territory, significant fines.

There is no real technical limitation to what you are proposing - just make sure your processes are robust.

If my comment helps, please give it a thumbs up!
0 Karma

rupeshn
Explorer

Thanks @nickhillscpl

0 Karma

morethanyell
Builder

Assuming that the 10 companies are in the same network the "single site" you're referring to, there shouldn't be any problem.

Install Universalforwarders or Heavyforwarders on the 10 companies you mentioned. And using your Deployment Server, you can install apps you can manage--all remotely. These are apps such as TAs that collect logs from different sources and/or configuration files (e.g. outputs.conf) that tells Splunk to send logs to your intermediate Heavyforwarder (or direct to the indexer cluster).

Each company "may" have their own indexes but if you configure them to send logs to your indexer, that should solve the issue.

However, if the 10 other comapanies you're referring to are in totally different network of their own, then, you might want to consider other solutions like using HEC on a heavyforwarder that has rules to allow incoming HEC posts.

0 Karma

rupeshn
Explorer

@morethanyell ,

Once we install add-ons on universal forwarders and send the logs to indexer. We would need to install the same add-on on indexer as well. Then which logs would the add-on on indexer consider? I believe there would be conflict as there are logs of 10 companies coming to indexer. What logs would add-on indexer consider?

Company 1 logs or company 2 logs or.................company10 logs

For example one company has index=wineventlog_company1 for all windows related logs where i installed "Windows add-on" and another company has index=wineventlog_company2 for all windows related logs where i installed same add-on "Windows add-on". When windows logs from both these companies goes to indexer where we installed the same add-on. What logs would this add-on consider and on what basis?

0 Karma

morethanyell
Builder

The differentiating information is host, not name of index. Also, you don't install wineventlog on an indexer because and indexer should be a dedicated server (cluster of servers) which purpose is solely for indexing.

0 Karma

rupeshn
Explorer

@morethanyell

I believe you've misunderstood my question.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...