Getting Data In

Forwarder data not being indexed, but is shown in metrics as processed

Kindred
Path Finder

Hey,

I'm trying to debug a really strange issue with a forwarder on one of our VMs. Basically we cloned a working VM (that does get indexed), but the new VM data doesn't get indexed, yet metrics shows it's processing data from it:

08-20-2013 16:02:15.229 +0000 INFO Metrics - group=tcpin_connections, 10.55.57.68:46601:9997, connectionType=cooked, sourcePort=46601, sourceHost=10.55.57.68, sourceIp=10.55.57.68, destPort=9997, _tcp_Bps=14.58, _tcp_KBps=0.01, _tcp_avg_thruput=0.03, kb=0.42, _tcp_Kprocessed=72.13, _tcp_eps=0.03, build=163460, version=5.0.3, os=Linux, arch=x86_64, hostname=xx-xx-mgmt, guid=A9B4ADDB-3751-41ED-9109-64F6060552D5, fwdType=uf, ssl=false, lastIndexer=50.xx.xx.xx:9997, ack=false

I've gone through warnings and errors in S.o.S and nothing shows anything related to connection issues.

Are there any other places to look for information? The config is the same as the working machine (same inputs etc) so I don't believe it's a config issue.

0 Karma
1 Solution

yannK
Splunk Employee
Splunk Employee

Maybe not related but : If you clone a forwarder, it's required to give him a unique identity.
Look if the metrics logs of the indexer is you do not have multiple forwarders with the exact same guid=A9B4ADDB-3751-41ED-9109-64F6060552D5

Then go on the forwarder and delete the $SPLUNK_HOME/etc/instance.cfg and restart to recreate a new one.

For detail about cloning an instance
http://docs.splunk.com/Documentation/Splunk/5.0.4/Deploy/Makeadfpartofasystemimage

View solution in original post

0 Karma

yannK
Splunk Employee
Splunk Employee

Maybe not related but : If you clone a forwarder, it's required to give him a unique identity.
Look if the metrics logs of the indexer is you do not have multiple forwarders with the exact same guid=A9B4ADDB-3751-41ED-9109-64F6060552D5

Then go on the forwarder and delete the $SPLUNK_HOME/etc/instance.cfg and restart to recreate a new one.

For detail about cloning an instance
http://docs.splunk.com/Documentation/Splunk/5.0.4/Deploy/Makeadfpartofasystemimage

0 Karma

Kindred
Path Finder

Thanks, I found the file and it made a new one on restart, and I can see the data now so looks like it was that.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...