Windows Infrastructure App: Why am I getting "Invalid key in stanza [powershell://...]" errors after installing TA-DomainController-2012R2?

[Wallen]

We upgraded to Server2012R2 Domain controllers and so I changed the addon TA for Windows Infrastructure from TA-DomainController-NT6 to TA-DomainController-2012R2, and I get the following Stanza errors when starting the splunk forwarder:

(Replaced Backslashes with Forwardslashes for pasting here)

Checking prerequisites...
        Checking mgmt port [8089]: open
        Checking conf files for problems...
                Invalid key in stanza [powershell://Replication-Stats] in D:/Apps/SplunkUniversalForwarder/etc/apps/TA-DomainController-2012R2/default/inputs.conf, line 75: script  (value:  & "$SplunkHome/etc/apps/TA-DomainController-2012R2/bin/Invoke-MonitoredScript.ps1" -Command "./replication-stats.ps1")
                Invalid key in stanza [powershell://Replication-Stats] in D:/Apps/SplunkUniversalForwarder/etc/apps/TA-DomainController-2012R2/default/inputs.conf, line 76: schedule  (value:  30 */5 * ? * *)
                Invalid key in stanza [powershell://AD-Health] in D:/Apps/SplunkUniversalForwarder/etc/apps/TA-DomainController-2012R2/default/inputs.conf, line 86: script  (value:  & "$SplunkHome/etc/apps/TA-DomainController-2012R2/bin/Inv
oke-MonitoredScript.ps1" -Command "./ad-health.ps1")
                Invalid key in stanza [powershell://AD-Health] in D:/Apps/SplunkUniversalForwarder/etc/apps/TA-DomainController-2012R2/default/inputs.conf, line 87: schedule  (value:  0 */5 * ? * *)
                Invalid key in stanza [powershell://Siteinfo] in D:/Apps/SplunkUniversalForwarder/etc/apps/TA-DomainController-2012R2/default/inputs.conf, line 97: script  (value:  & "$SplunkHome/etc/apps/TA-DomainController-2012R2/bin/Invo
ke-MonitoredScript.ps1" -Command "./siteinfo.ps1")
                Invalid key in stanza [powershell://Siteinfo] in D:/Apps/SplunkUniversalForwarder/etc/apps/TA-DomainController-2012R2/default/inputs.conf, line 98: schedule  (value:  0 15 * ? * *)
                Your indexes and inputs configurations are not internally consistent. For more information, run 'splunk btool check --debug'
        Done

So, pretty much... the powershell scripts won't run anymore.

[halr9000]

I fixed the formatting for you. When you have a bit of code or output, select the text and click the "101010" button. That will preserve linebreaks and make things easier to read.

[jeff · Accepted solution]

Just hazarding a guess: did you also deploy the Splunk App for Microsoft Powershell? The Exchange App Documentation says it's a requirement for TA-DomainController-2012R2.

[Wallen]

Yes, The Powershell App was a requirement prior to updating from TA-DomainController-NT6 to TA-DomainController-2012R2. The only thing I did was upgrade the addon to the Infrastructure App, since we upgraded our domain controllers. The stanzas to launch Powershell scripts are completely different between the two. One Stanza starts with "Script", the Other Starts with "Powershell".

TA-DomainController-NT6:
[script://.\bin\runpowershell.cmd dns-zoneinfo.ps1]
source=Powershell
sourcetype=MSAD:NT6:DNS-Zone-Information
index=msad
interval=3600
disabled=false

TA-DomainController-2012R2
[powershell://Replication-Stats]
script = & "$SplunkHome\etc\apps\TA-DomainController-2012R2\bin\Invoke-MonitoredScript.ps1" -Command ".\replication-stats.ps1"
schedule = 30 */5 * ? * *
index = msad
source = Powershell
sourcetype=MSAD:NT6:Replication
disabled=false

[halr9000]

I think this is the right root cause--something to do with the PowerShell modinput. That error message is saying that splunkd doesn't know what to do with the powershell lines. I suspect that the modular input is not installed correctly. On the same machine which is running the TA-DomainController-2012R2 app (your DC, presumably), be sure that the PowerShell modinput is also in the etc/apps folder. Double-check permissions?

You could also try running (at an elevated prompt):

splunk.exe btool check --debug

[Wallen]

You were right. I didn't have the Splunk App for Microsoft Powershell installed. I could have sworn that I did, though. Thanks!