Installation

Splunk Hadoop Dat Roll query w.r.t Permission.

Harishma
Communicator

Our Splunk and Hadoop Clusters are in 2 different Domains ABC/splunk and XYZ/hadoop.

Splunk Doc states the below:

A subdirectory under jobtracker.staging.root.dir (usually /user/) with the name of the user account under which Splunk Analytics for Hadoop is running on the search head. For example, if Splunk Analytics for Hadoop is started by user "BigDataUser" and jobtracker.staging.root.dir=/user/ you need a directory /user/HadoopAnalytics that is accessible by user "BigDataUser".

Does this mean I need to have the same service account ABC/splunk created under /user/ in Hadoop?
OR
does this mean the directory name should be the same as the name that splunk is running as? is it should be /user/splunk i.e
only name should be same?

Tags (2)
0 Karma
1 Solution

rdagan_splunk
Splunk Employee
Splunk Employee

The user that installed splunk has to have a write permission in HDFS.

For example, I used user root to install Splunk.

In the Splunk Provider I setup an HDFS working directory to /user/root/splunkmr ( vix.splunk.home.hdfs = /user/root/splunkmr )

And in HDFS I made sure all of directories under /user/root are owned by root.
For example:
[root@localhost local]# /opt/hadoop-2.7.4/bin/hadoop fs -ls /user/root
Found 3 items
drwxrwxrwx - root root 0 2017-10-16 14:03 /user/root/archive
drwxrwxrwx - root root 0 2017-10-16 13:51 /user/root/data
drwx--x--x - root root 0 2017-10-16 14:01 /user/root/splunkmr

View solution in original post

rdagan_splunk
Splunk Employee
Splunk Employee

The user that installed splunk has to have a write permission in HDFS.

For example, I used user root to install Splunk.

In the Splunk Provider I setup an HDFS working directory to /user/root/splunkmr ( vix.splunk.home.hdfs = /user/root/splunkmr )

And in HDFS I made sure all of directories under /user/root are owned by root.
For example:
[root@localhost local]# /opt/hadoop-2.7.4/bin/hadoop fs -ls /user/root
Found 3 items
drwxrwxrwx - root root 0 2017-10-16 14:03 /user/root/archive
drwxrwxrwx - root root 0 2017-10-16 13:51 /user/root/data
drwx--x--x - root root 0 2017-10-16 14:01 /user/root/splunkmr

Harishma
Communicator

Hi @rdagan ,

I read the below in this doc :

https://docs.splunk.com/Documentation/Splunk/7.0.0/HadoopAnalytics/Importantinformationaboutinstalla...

" Many Splunk Anaytics for Hadoop features require communication between various aspects of third-party databases, Splunk Analytics for Hadoop, and Splunk Enterprise. To make it easier to configure these features as you need them, we recommend that you install or configure everything with the same user names and credentials: "

However my splunk account name in splunk cluster is splunkac and in Hadoop cluster also ive created a new account called splunkac and it exists under under /user .

Howevre both are two diff account with different credentials and the clusters also exist in defferent realms/domain. Does that matter?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...