Not getting data from sample files on search head /forwarder to index cluster

[greatdane]

I am new to trying to set up a dev environment with 1 deployment server, 1 search head/forwarder 1 master cluster, 2 indexers within the cluster. I have taken data samples and placed them in directories on the search head ex. opt/splunk//hops/asalogs. I have configured the inputs.conf file to monitor for these to ingest and be sent towards the indexers in my outputs.conf file. My props.conf file has been created to do the parsing of the logs. Bear in mind the hops index mentioned in the stanzas is located on the cluster master

Inputs.conf

[monitor:///opt/splunk/HOPS/asalogs/*.log]
sourcetype=apps:hops:websphere
index=hops
disabled=false

[monitor:///opt/splunk/HOPS/jse/*.xml]
sourcetype=apps:hops:jse
index=hops
disabled=false

Outputs.conf

[indexAndForward]
index = false

[tcpout]
defaultGroup=indexcluster
forwardedindex.filter.disable = true
indexAndForward = false

[tcpout:indexcluster]
server=1.1.1.1:9997,1.1.1.2:9997
disabled = false

[tcpout-server://1.1.1.1:9997]

Props.conf

[apps:hops:jse]
CHARSET=UTF-8
MAX_TIMESTAMP_LOOKAHEAD=32
SHOULD_LINEMERGE=true
disabled=false
TIME_PREFIX=\

[apps:hops:websphere]
SHOULD_LINEMERGE=true
NO_BINARY_CHECK=true
CHARSET=UTF-8
MAX_TIMESTAMP_LOOKAHEAD=32
disabled=false
TIME_PREFIX=[

By all indication from all the documentation that I have read is that it should be working, I am not getting a lot of support locally so I am having to go out to other splunk denizens for assistance.

Thanks!

[pradeepkumarg]

You mentioned the index stanza is located on the master. I assume you created in an app under master-apps and did a bundle push. You can login to the indexer and go to settings -> indexes to see if the index is actually created on the indexer.

1. Do you see partial data missing or all data?
2. Do you see internal data from the forwarder host? index=_internal host=your_forwarder_host

[greatdane]

I tried and I have 0 amount of data searching all time running the query you provided.

[pradeepkumarg]

so, that looks like the forwarder is unable to communicate to the indexers. Verify your outputs.conf and check for any errors in splunkd.log $SPLUNK_HOME/var/log/splunk/splunkd.log