Deployment Architecture
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Ask a Question

Migration Deployment server error

jprakash_netgea
Engager
‎04-30-2019 03:26 PM

I did a migration from old splunk to new splunk. after configuring serverclasses i found some devices sending logs but all of sudden i am not seeing any devices phoned in. Here is the logs for one Device. the last log says the action:Uninstall. what does that mean ? Now when i try to restart splunk from client machine is giving me "ERROR: pid 2968 terminated with signal 11 (core dumped)
SSL certificate generation failed." Error.

04-30-2019 09:32:18.609 -0700 INFO ClientSessionsManager:Listener_AppEvents - Received count=3 AppEvents from DC ip=10.100.0.135 name=4A4C403F-B870-4D58-8259-189DB74E4D44
04-30-2019 09:42:19.562 -0700 INFO ClientSessionsManager:Listener_AppEvents - Received count=3 AppEvents from DC ip=10.100.0.135 name=4A4C403F-B870-4D58-8259-189DB74E4D44
04-30-2019 09:52:20.515 -0700 INFO ClientSessionsManager:Listener_AppEvents - Received count=3 AppEvents from DC ip=10.100.0.135 name=4A4C403F-B870-4D58-8259-189DB74E4D44
04-30-2019 10:02:21.493 -0700 INFO ClientSessionsManager:Listener_AppEvents - Received count=3 AppEvents from DC ip=10.100.0.135 name=4A4C403F-B870-4D58-8259-189DB74E4D44
04-30-2019 10:11:25.399 -0700 INFO PubSubSvr - Subscribed: channel=deploymentServer/phoneHome/default/reply/sjebsdb04.netgear.com/4A4C403F-B870-4D58-8259-189DB74E4D44 connectionId=connection_10.100.0.135_8089_sjebsdb04.netgear.com_sjebsdb04.netgear.com_4A4C403F-B870-4D58-8259-189DB74E4D44 listener=0x7fd2d46dda00
04-30-2019 10:11:25.423 -0700 INFO PubSubSvr - Subscribed: channel=tenantService/handshake/reply/sjebsdb04.netgear.com/4A4C403F-B870-4D58-8259-189DB74E4D44 connectionId=connection_10.100.0.135_8089_sjebsdb04.netgear.com_sjebsdb04.netgear.com_4A4C403F-B870-4D58-8259-189DB74E4D44 listener=0x7fd2d46dda00
04-30-2019 10:12:25.265 -0700 INFO ClientSessionsManager - Adding client: ip=10.100.0.135 uts=linux-x86_64 id=4A4C403F-B870-4D58-8259-189DB74E4D44 name=4A4C403F-B870-4D58-8259-189DB74E4D44
04-30-2019 10:12:25.265 -0700 INFO ClientSessionsManager - ip=10.100.0.135 name=4A4C403F-B870-4D58-8259-189DB74E4D44 New record for sc=netgear_all_nix app=Splunk_TA_nix: action=Phonehome result=Ok checksum=0
04-30-2019 10:12:25.266 -0700 INFO ClientSessionsManager - ip=10.100.0.135 name=4A4C403F-B870-4D58-8259-189DB74E4D44 New record for sc=netgear_all_deploymentclient app=netgear_all_deploymentclient: action=Phonehome result=Ok checksum=0
04-30-2019 10:12:25.266 -0700 INFO ClientSessionsManager - ip=10.100.0.135 name=4A4C403F-B870-4D58-8259-189DB74E4D44 New record for sc=netgear_all_deploymentclient app=netgear_all_forwarder_outputs: action=Phonehome result=Ok checksum=0
04-30-2019 10:12:25.266 -0700 INFO ClientSessionsManager - ip=10.100.0.135 name=4A4C403F-B870-4D58-8259-189DB74E4D44 New record for sc=netgear_all_forwarder_outputs app=netgear_all_forwarder_outputs: action=Phonehome result=Ok checksum=0
04-30-2019 10:12:25.266 -0700 INFO ClientSessionsManager - ip=10.100.0.135 name=4A4C403F-B870-4D58-8259-189DB74E4D44 New record for sc=netgear_all_forwarders app=netgear_all_forwarder_outputs: action=Phonehome result=Ok checksum=0
04-30-2019 10:12:25.289 -0700 INFO ClientSessionsManager:Listener_AppEvents - Received count=3 AppEvents from DC ip=10.100.0.135 name=4A4C403F-B870-4D58-8259-189DB74E4D44
04-30-2019 10:12:25.289 -0700 INFO ClientSessionsManager - ip=10.100.0.135 name=4A4C403F-B870-4D58-8259-189DB74E4D44 Updating record for sc=netgear_all_deploymentclient app=netgear_all_deploymentclient: action=Install result=Ok checksum=13412163369467275487
04-30-2019 10:12:25.289 -0700 INFO ClientSessionsManager - ip=10.100.0.135 name=4A4C403F-B870-4D58-8259-189DB74E4D44 Updating record for sc=netgear_all_deploymentclient app=netgear_all_forwarder_outputs: action=Install result=Ok checksum=15491986698083571770
04-30-2019 10:12:25.289 -0700 INFO ClientSessionsManager - ip=10.100.0.135 name=4A4C403F-B870-4D58-8259-189DB74E4D44 Updating record for sc=netgear_all_nix app=Splunk_TA_nix: action=Install result=Ok checksum=13350837051104395546
04-30-2019 10:15:25.616 -0700 INFO PubSubSvr - Subscribed: channel=deploymentServer/phoneHome/default/reply/sjebsdb04.netgear.com/4A4C403F-B870-4D58-8259-189DB74E4D44 connectionId=connection_10.100.0.135_8089_sjebsdb04.netgear.com_sjebsdb04.netgear.com_4A4C403F-B870-4D58-8259-189DB74E4D44 listener=0x7fa537a57a00
04-30-2019 10:15:25.639 -0700 INFO PubSubSvr - Subscribed: channel=tenantService/handshake/reply/sjebsdb04.netgear.com/4A4C403F-B870-4D58-8259-189DB74E4D44 connectionId=connection_10.100.0.135_8089_sjebsdb04.netgear.com_sjebsdb04.netgear.com_4A4C403F-B870-4D58-8259-189DB74E4D44 listener=0x7fa537a57a00
04-30-2019 10:16:25.482 -0700 INFO ClientSessionsManager - Adding client: ip=10.100.0.135 uts=linux-x86_64 id=4A4C403F-B870-4D58-8259-189DB74E4D44 name=4A4C403F-B870-4D58-8259-189DB74E4D44
04-30-2019 10:16:25.483 -0700 INFO ClientSessionsManager - ip=10.100.0.135 name=4A4C403F-B870-4D58-8259-189DB74E4D44 New record for sc=netgear_all_nix app=Splunk_TA_nix: action=Phonehome result=Ok checksum=0
04-30-2019 10:16:25.483 -0700 INFO ClientSessionsManager - ip=10.100.0.135 name=4A4C403F-B870-4D58-8259-189DB74E4D44 New record for sc=netgear_all_deploymentclient app=netgear_all_deploymentclient: action=Phonehome result=Ok checksum=0
04-30-2019 10:16:25.483 -0700 INFO ClientSessionsManager - ip=10.100.0.135 name=4A4C403F-B870-4D58-8259-189DB74E4D44 New record for sc=netgear_all_deploymentclient app=netgear_all_forwarder_outputs: action=Phonehome result=Ok checksum=0
04-30-2019 10:16:25.483 -0700 INFO ClientSessionsManager - ip=10.100.0.135 name=4A4C403F-B870-4D58-8259-189DB74E4D44 New record for sc=netgear_all_forwarder_outputs app=netgear_all_forwarder_outputs: action=Phonehome result=Ok checksum=0
04-30-2019 10:16:25.483 -0700 INFO ClientSessionsManager - ip=10.100.0.135 name=4A4C403F-B870-4D58-8259-189DB74E4D44 New record for sc=netgear_all_forwarders app=netgear_all_forwarder_outputs: action=Phonehome result=Ok checksum=0
04-30-2019 10:21:26.000 -0700 WARN PubSubSvr - sender=connection_10.100.0.135_8089_sjebsdb04.netgear.com_sjebsdb04.netgear.com_4A4C403F-B870-4D58-8259-189DB74E4D44 channel=deploymentServer/phoneHome/default Message not dispatched (connection invalid)
04-30-2019 10:22:26.230 -0700 INFO PubSubSvr - Subscribed: channel=deploymentServer/phoneHome/default/reply/sjebsdb04.netgear.com/4A4C403F-B870-4D58-8259-189DB74E4D44 connectionId=connection_10.100.0.135_8089_sjebsdb04.netgear.com_sjebsdb04.netgear.com_4A4C403F-B870-4D58-8259-189DB74E4D44 listener=0x7faf1f6b9400
04-30-2019 10:22:26.253 -0700 INFO PubSubSvr - Subscribed: channel=tenantService/handshake/reply/sjebsdb04.netgear.com/4A4C403F-B870-4D58-8259-189DB74E4D44 connectionId=connection_10.100.0.135_8089_sjebsdb04.netgear.com_sjebsdb04.netgear.com_4A4C403F-B870-4D58-8259-189DB74E4D44 listener=0x7faf1f6b9400
04-30-2019 10:23:26.096 -0700 INFO ClientSessionsManager - Adding client: ip=10.100.0.135 uts=linux-x86_64 id=4A4C403F-B870-4D58-8259-189DB74E4D44 name=4A4C403F-B870-4D58-8259-189DB74E4D44
04-30-2019 10:23:26.096 -0700 INFO ClientSessionsManager - ip=10.100.0.135 name=4A4C403F-B870-4D58-8259-189DB74E4D44 New record for sc=netgear_all_nix app=Splunk_TA_nix: action=Phonehome result=Ok checksum=0
04-30-2019 10:23:26.096 -0700 INFO ClientSessionsManager - ip=10.100.0.135 name=4A4C403F-B870-4D58-8259-189DB74E4D44 New record for sc=netgear_all_deploymentclient app=netgear_all_deploymentclient: action=Phonehome result=Ok checksum=0
04-30-2019 10:23:26.096 -0700 INFO ClientSessionsManager - ip=10.100.0.135 name=4A4C403F-B870-4D58-8259-189DB74E4D44 New record for sc=netgear_all_forwarder_outputs app=netgear_all_forwarder_outputs: action=Phonehome result=Ok checksum=0
04-30-2019 10:23:26.096 -0700 INFO ClientSessionsManager - ip=10.100.0.135 name=4A4C403F-B870-4D58-8259-189DB74E4D44 New record for sc=netgear_all_forwarders app=netgear_all_forwarder_outputs: action=Phonehome result=Ok checksum=0
04-30-2019 10:23:26.096 -0700 INFO ClientSessionsManager - ip=10.100.0.135 name=4A4C403F-B870-4D58-8259-189DB74E4D44 New record for sc=netgear_all_deploymentclient app=netgear_all_forwarder_outputs: action=Phonehome result=Ok checksum=0
04-30-2019 10:23:26.120 -0700 INFO ClientSessionsManager:Listener_AppEvents - Received count=3 AppEvents from DC ip=10.100.0.135 name=4A4C403F-B870-4D58-8259-189DB74E4D44
04-30-2019 10:23:26.120 -0700 INFO ClientSessionsManager - ip=10.100.0.135 name=4A4C403F-B870-4D58-8259-189DB74E4D44 Updating record for sc=netgear_all_deploymentclient app=netgear_all_deploymentclient: action=Install result=Ok checksum=13412163369467275487
04-30-2019 10:23:26.120 -0700 INFO ClientSessionsManager - ip=10.100.0.135 name=4A4C403F-B870-4D58-8259-189DB74E4D44 Updating record for sc=netgear_all_deploymentclient app=netgear_all_forwarder_outputs: action=Install result=Ok checksum=15491986698083571770
04-30-2019 10:23:26.120 -0700 INFO ClientSessionsManager - ip=10.100.0.135 name=4A4C403F-B870-4D58-8259-189DB74E4D44 Updating record for sc=netgear_all_nix app=Splunk_TA_nix: action=Install result=Ok checksum=13350837051104395546
04-30-2019 10:29:26.698 -0700 WARN PubSubSvr - sender=connection_10.100.0.135_8089_sjebsdb04.netgear.com_sjebsdb04.netgear.com_4A4C403F-B870-4D58-8259-189DB74E4D44 channel=deploymentServer/phoneHome/default Message not dispatched (connection invalid)
04-30-2019 10:30:26.933 -0700 INFO PubSubSvr - Subscribed: channel=deploymentServer/phoneHome/default/reply/sjebsdb04.netgear.com/4A4C403F-B870-4D58-8259-189DB74E4D44 connectionId=connection_10.100.0.135_8089_sjebsdb04.netgear.com_sjebsdb04.netgear.com_4A4C403F-B870-4D58-8259-189DB74E4D44 listener=0x7f6454d4c400
04-30-2019 10:30:26.956 -0700 INFO PubSubSvr - Subscribed: channel=tenantService/handshake/reply/sjebsdb04.netgear.com/4A4C403F-B870-4D58-8259-189DB74E4D44 connectionId=connection_10.100.0.135_8089_sjebsdb04.netgear.com_sjebsdb04.netgear.com_4A4C403F-B870-4D58-8259-189DB74E4D44 listener=0x7f6454d4c400
04-30-2019 10:31:26.801 -0700 INFO ClientSessionsManager - Adding client: ip=10.100.0.135 uts=linux-x86_64 id=4A4C403F-B870-4D58-8259-189DB74E4D44 name=4A4C403F-B870-4D58-8259-189DB74E4D44
04-30-2019 10:31:26.801 -0700 INFO ClientSessionsManager - ip=10.100.0.135 name=4A4C403F-B870-4D58-8259-189DB74E4D44 New record for sc=netgear_all_nix app=Splunk_TA_nix: action=Phonehome result=Ok checksum=0
04-30-2019 10:31:26.801 -0700 INFO ClientSessionsManager - ip=10.100.0.135 name=4A4C403F-B870-4D58-8259-189DB74E4D44 New record for sc=netgear_all_deploymentclient app=netgear_all_deploymentclient: action=Phonehome result=Ok checksum=0
04-30-2019 10:31:26.801 -0700 INFO ClientSessionsManager - ip=10.100.0.135 name=4A4C403F-B870-4D58-8259-189DB74E4D44 New record for sc=netgear_all_deploymentclient app=netgear_all_forwarder_outputs: action=Phonehome result=Ok checksum=0
04-30-2019 10:31:26.801 -0700 INFO ClientSessionsManager - ip=10.100.0.135 name=4A4C403F-B870-4D58-8259-189DB74E4D44 New record for sc=netgear_all_forwarder_outputs app=netgear_all_forwarder_outputs: action=Phonehome result=Ok checksum=0
04-30-2019 10:31:26.801 -0700 INFO ClientSessionsManager - ip=10.100.0.135 name=4A4C403F-B870-4D58-8259-189DB74E4D44 New record for sc=netgear_all_forwarders app=netgear_all_forwarder_outputs: action=Phonehome result=Ok checksum=0
04-30-2019 10:40:23.257 -0700 INFO PubSubSvr - Subscribed: channel=deploymentServer/phoneHome/default/reply/sjebsdb04.netgear.com/4A4C403F-B870-4D58-8259-189DB74E4D44 connectionId=connection_10.100.0.135_8089_sjebsdb04.netgear.com_sjebsdb04.netgear.com_4A4C403F-B870-4D58-8259-189DB74E4D44 listener=0x7f0394ff0000
04-30-2019 10:40:23.281 -0700 INFO PubSubSvr - Subscribed: channel=tenantService/handshake/reply/sjebsdb04.netgear.com/4A4C403F-B870-4D58-8259-189DB74E4D44 connectionId=connection_10.100.0.135_8089_sjebsdb04.netgear.com_sjebsdb04.netgear.com_4A4C403F-B870-4D58-8259-189DB74E4D44 listener=0x7f0394ff0000
04-30-2019 10:40:26.873 -0700 INFO ClientSessionsManager - Adding client: ip=10.100.0.135 uts=linux-x86_64 id=4A4C403F-B870-4D58-8259-189DB74E4D44 name=4A4C403F-B870-4D58-8259-189DB74E4D44
04-30-2019 10:40:26.961 -0700 INFO ClientSessionsManager:Listener_AppEvents - Received count=3 AppEvents from DC ip=10.100.0.135 name=4A4C403F-B870-4D58-8259-189DB74E4D44
04-30-2019 10:40:26.961 -0700 INFO ClientSessionsManager - ip=10.100.0.135 name=4A4C403F-B870-4D58-8259-189DB74E4D44 New record for sc=netgear_all_deploymentclient app=netgear_all_deploymentclient: action=Uninstall result=Ok checksum=0
04-30-2019 10:40:26.961 -0700 INFO ClientSessionsManager - ip=10.100.0.135 name=4A4C403F-B870-4D58-8259-189DB74E4D44 New record for sc=netgear_all_deploymentclient app=netgear_all_forwarder_outputs: action=Uninstall result=Ok checksum=0
04-30-2019 10:40:26.961 -0700 INFO ClientSessionsManager - ip=10.100.0.135 name=4A4C403F-B870-4D58-8259-189DB74E4D44 New record for sc=netgear_all_nix app=Splunk_TA_nix: action=Uninstall result=Ok checksum=0

Tags (1)
0 Karma
Reply

jprakash_netgea
Engager
‎04-30-2019 04:13 PM

I tired restarting other server, the splunk service started on that machine. But no device phoned to new / old deployment server

0 Karma
Reply

lakshman239
Influencer
‎05-01-2019 02:01 AM

On the clients (forwarders), did you update the deploymentclient.conf to point to your new deployment server and restart it?

0 Karma
Reply

codebuilder
Influencer
‎05-01-2019 08:16 AM

Check the user:group permissions on your deploymentclient.conf file.

Also, if you defined your targetUri by hostname (not ip addr) then make certain that hostname resolves in DNS (or in /etc/hosts).

nslookup your.host.name.com
----
An upvote would be appreciated and Accept Solution if it helps!
0 Karma
Reply
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...