How to troubleshoot why block data is not coming in when using the Splunk Add-on for EMC VNX?

[pgadhari]

We have installed Splunk Add on for EMC VNX in the splunk instance in our Windows server and configured the inputs for both file and block data. The file data is being indexed, but there is no block data coming in. The following error is seen in data_loader.log:

2015-07-07 14:01:48,828 ERROR 12928 - Traceback (most recent call last):
  File "C:\Program Files\Splunk\etc\apps\Splunk_TA_emc-vnx\bin\thread_pool.py", line 116, in _run
    job()
  File "C:\Program Files\Splunk\etc\apps\Splunk_TA_emc-vnx\bin\job_factory.py", line 27, in __call__
    results = self._func()
  File "C:\Program Files\Splunk\etc\apps\Splunk_TA_emc-vnx\bin\vnx_block_loader.py", line 92, in collect_array_states
    self._get_agents()
  File "C:\Program Files\Splunk\etc\apps\Splunk_TA_emc-vnx\bin\vnx_block_loader.py", line 348, in _get_agents
    output = timed_popen(self.cli_common + cli, 15)
  File "C:\Program Files\Splunk\etc\apps\Splunk_TA_emc-vnx\bin\timed_popen.py", line 55, in timed_popen
    return _do_timed_popen(args, timeout)
  File "C:\Program Files\Splunk\etc\apps\Splunk_TA_emc-vnx\bin\timed_popen.py", line 41, in _do_timed_popen
    sub = Popen(args, stdout=PIPE, stderr=PIPE)
  File "C:\Program Files\Splunk\Python-2.7\Lib\subprocess.py", line 710, in __init__
    errread, errwrite)
  File "C:\Program Files\Splunk\Python-2.7\Lib\subprocess.py", line 958, in _execute_child
    startupinfo)
WindowsError: [Error 2] The system cannot find the file specified

The ta_vnx.log file shows that the jobs have been started, but simultaneously, we get the above error in data_loader.log.
For file data, we have specified the control station IP in inputs, and for block data, we have specified the SP IPs.
How can we troubleshoot this issue? Please help! Thanks !!

[johnansett]

Just had this issue occur when I created a new .bash_profile. Check that Navisphere can run against the arrays, then make sure you add the naviseccli bin directory which is usually /opt/Navisphere/bin to your system PATH.
For example, add the following line to ~/.bash_profile:

PATH=$PATH:/opt/Navisphere/bin
export PATH

Then execute this file to make the setting effective by running source ~/.bash_profile

Ref: http://docs.splunk.com/Documentation/AddOns/released/EMCVNX/InstallVNXCLI

[jcoates_splunk]

" WindowsError: [Error 2] The system cannot find the file specified"

It can't find a file that it needs -- not entirely clear which one, but I'd walk through the setup instructions again and make sure that nothing got missed.

[pgadhari]

Yes, I have gone through the setup instructions and configurations several times. I have also tried configuring naviseccli on Linux host, but still getting the same error. Currently, I am using Windows 2008 R2 as my data collector node. I have also opened the support ticket with Splunk, but haven't got any response on it yet.

I also tried running vnx_data_loader.py script using Splunk cmd python.... command, it runs. But when it runs through the app, gives above error. I thought that the Splunk_home variable is not set properly, but it is also configured properly. Can you help us resolve this issue ?

Thanks
Pankaj

[jcoates_splunk]

please open a support ticket so that we can isolate the problem.

[pgadhari]

Yes, the support ticket is opened with Splunk. But not getting any reply on it yet. The response from Splunk support is very slow. Suppor ticket number is 252136. Appreciate if you can help me to resolve this.

Thanks
Pankaj

[z5pate]

@pgadhari did you ever get a response from Splunk support? Just installed the add-on and I'm getting the same thing with ver 1.2.0 and Splunk Enterprise ver 6.6.2. We are running this on a RHEL 6.5 box

[pgadhari]

@z5pate.. nope I did not get any response on the issue from support. That's an old post, hence I did not follow up after that.

[peterfilardo]

Make sure you have a copy of plink.exe in your PATH. That fixed it for me.