Guest

Preview Tool

Cisco Bug: CSCvv60255 - vManage cluster restore:After rebuilding cluster, statistics-db and message-server can't be started.

Last Modified

Sep 11, 2020

Products (1)

  • Cisco SD-WAN

Known Affected Releases

20.3

Description (partial)

Symptom:
We rebuild a new cluster for vManage restore test. (for more detail procedure, please see file 	
01_restore_procedure.txt )

After booting new vManager cluster , the nms service cannot be started properly.
_______________________________________________________
vManage-sw2-m1# request nms all status
NMS server proxy
        Enabled: true
        Status:  waiting
NMS application server
        Enabled: true
        Status:  waiting
NMS configuration database
        Enabled: true
        Status:  waiting
NMS coordination server
        Enabled: true
        Status:  waiting
NMS messaging server
        Enabled: true
        Status:  running PID:6717 for 3572s
NMS statistics database
        Enabled: true
        Status:  not running
NMS data collection agent
        Enabled: true
        Status:  not running
NMS cloud agent
        Enabled: true
        Status:  running PID:24252 for 26s
NMS container manager
        Enabled: true
        Status:  waiting
NMS SDAVC proxy
        Enabled: true
        Status:  running PID:929 for 5785s
vManage-sw2-m1#
_______________________________________________________





After running 「request nms all stop 」, we started every service by manually. but It didn't work..
The following issures have occurred.

Issure 1 :
statistics-db can't be started

vManage-sw2-m1#
vManage-sw2-m1# request nms statistics-db start
Successfully started NMS statistics database
vManage-sw2-m1#
vManage-sw2-m1#
vManage-sw2-m1# request nms statistics-db status
NMS statistics database
        Enabled: true
        Status:  not running
vManage-sw2-m1#



Issure 2 :
After runninng  「 request nms message-server start」 , the following message was displayed , and message server can't be started.

 vManage-sw2-m1# request nms messaging-server start
Starting NMS messaging server
Traceback (most recent call last):
  File "/usr/bin/vconfd_container_management.py", line 1025, in <module>
    main()
  File "/usr/bin/vconfd_container_management.py", line 1011, in main
    parser = arg_parsing()
  File "/usr/bin/vconfd_container_management.py", line 928, in arg_parsing
    choice_list = listdir(CURR_CONT_PKG)
OSError: [Errno 2] No such file or directory: '/opt/data/extra-packages/20.3.916/containers/'
vManage-sw2-m1#




Issure 3 :
The state of nms service not changed to running  after starting nms service manually.

for more detail about the issues, please see  the following files
--02_nms_service_start-failed_after_vManage-boot.txt 
--03_nms_service_start-failed_by manually_.txt

Conditions:
3 nodes vManages in Multi-Tenant Cluster . It works on the OpenStack .

The cluster was working in the 20.3.1 . (upgraded from 20.1.12 to 20.3.1)

For the restore test. we used following procedure to backup and rebuilding a new vManage cluster .
--01_restore_procedure.txt

After booting vManages , the state of nms service was not corect.
Bug details contain sensitive information and therefore require a Cisco.com account to be viewed.

Bug Details Include

  • Full Description (including symptoms, conditions and workarounds)
  • Status
  • Severity
  • Known Fixed Releases
  • Related Community Discussions
  • Number of Related Support Cases
Bug information is viewable for customers and partners who have a service contract. Registered users can view up to 200 bugs per month without a service contract.