Why are events not being indexed into the indexes specified in the inputs.conf on my Windows universal forwarder?

[normangoh]

Hi guys,

I have one Universal Forwarder that has a deployed app from the deployment server. Inside the inputs.conf of the app contains the following.

[WinEventLog:Application] disabled=0
index=windowseventlog

[WinEventLog:Security] disabled=0
index=windowseventlog

[WinEventLog:System] disabled = 0
index=windowseventlog

[monitor://c:\*.log] disabled=0
index=web source=sample.log
sourcetype=IIS

From what I understand, events will be indexed to the stated indexes in the inputs.conf, but when I look into the indexes from my Indexer, events are directed into "wineventlog". I do not remember I have specified to push events there and I cannot find out why is it pushing events to "wineventlog" index. I did the same thing on another Universal Forwarder which is behaving correctly and are pushing the events to the stated indexes in the inputs.conf.

Does anyone have the same issue? Or did I do anything wrong? This Splunk setup is for learning purposes by the way.

[darrenfuller]

Use btool to inspect your inputs on the forwarder:

/opt/splunk/bin/splunk btool indexes list

see if the output is what you expect.

If the problem persists, go to the indexer and do the same with props, transforms and indexes.conf, see if there are hints in there why the system is not working as you expect....

/opt/splunk/bin/splunk props list
/opt/splunk/bin/splunk transforms list
/opt/splunk/bin/splunk indexes list

[normangoh]

The forwarder that is having issue is installed on a windows. When I enter the command "btool indexes list" on the folder /splunkuniversalforwarder is returns command not recognized. Am I doing it wrong?

[darrenfuller]

ok, on windows.... 1) make sure you are in an administrator command prompt (title of window should be Administrator: command prompt, if not close and reopen with right click run as administrator) and 2) change to the \splunk\bin directory and type splunk btool indexes list

[normangoh]

I managed to find out the reason for this issue. It was because Splunk Universal Forwarder installed windows TA by default hence it is overriding the rule on my custom deployed app.

But there is something I don't understand. Because my custom app is named "winevt" and the folder ranking is lower than "Splunk TA Windows" hence I tried creating another app named "awinevt" hoping that it will be fetching rules from the higher ranking "awinevt" in folder view. But in the end it is still fetching from "Splunk TA Windows". Anyone know if there is a configuration to allow an app to have the highest priority? How does the overriding system works?

Thanks in advance!

[darrenfuller]

The precedence is working correctly. The uppercase S in Splunk_TA_Windows comes before the lowercase a in awinevt, if you rename it to Awinevt, everything should work out correctly.

[normangoh]

Thank you for clearing my doubts. So uppercase and lowercase takes a more important role than the alphabetical order of the folders.

Thanks!