Virtual index causing metadata command to error out for other sourcetypes

[hortonew]

1. Without a virtual index enabled, running | metadata type=sourcetypes index=* will return correctly.
2. Adding a virtual index that uses a hadoop provider, this command now fails due to the fact that it can't find sourcetype details. Searching the virtual index however returns correct sourcetype details.

What is necessary for the metadata command to return successfully? Is there a file I need next to the data to dictate the sourcetype info? Can I remove this index from the metadata results without having to manually specify all indexes I want in the command?

Error:
01-15-2020 20:57:40.884 ERROR metadata - No 'sourcetype' key found in results. Cannot merge metadata.
01-15-2020 20:57:40.884 INFO PreviewExecutor - Finished preview generation in 0.002741056 seconds.
01-15-2020 20:57:40.901 INFO ReducePhaseExecutor - Ending phase_1
01-15-2020 20:57:40.901 INFO UserManager - Unwound user context: x@y.com -> NULL
01-15-2020 20:57:40.901 ERROR SearchOrchestrator - Phase_1 failed due to : Error in 'metadata': No 'sourcetype' key found in results. Cannot merge metadata.
01-15-2020 20:57:40.901 INFO ReducePhaseExecutor - ReducePhaseExecutor=1 action=CANCEL
01-15-2020 20:57:40.901 INFO DispatchExecutor - User applied action=CANCEL while status=0
01-15-2020 20:57:40.901 ERROR SearchStatusEnforcer - sid:md_1579121855.178190 Error in 'metadata': No 'sourcetype' key found in results. Cannot merge metadata.

Version info:
Splunk 7.3.3
Hadoop cli 2.8.4
AWS EMR emr-5.28.0

[jhornsby_splunk · Accepted solution]

Hi @hortonew,

Unfortunately this is a bug in the version of Splunk that you are using. It is fixed in 8.0.

Cheers,

- Jo.

[hortonew]

Hey thanks for the response. Any chance you can post which release notes items directly corrects this? I need to read up on what's causing it. Thanks!

[jhornsby_splunk]

Hi @hortonew,

The specific issue is internally tracked as ERP-2150, which has the summary | metadata type=sourcetypes index=* fails when you have virtual indexes. Unfortunately I've realized that there's an issue that means it isn't currently in the release notes.

Cheers,

- Jo.

[hortonew]

Thanks Jo. Just to confirm, this is currently unresolved even in the latest release of splunk? If so, is there any fix planned that will be applied to the 7.3.x chain in say, 7.3.5? Thanks again.

Or did you mean it was patched, but it never made it in to release notes?

[hortonew]

Nevermind - it seems 8.0 does in fact resolve the issue. I just tested.

[hortonew]

Probably unsupported, but you can take an 8.0 install, copy /opt/splunk/bin/jars/SplunkMR-hy2.jar and copy into your 7.3.3 install to fix this issue.