Guest

Preview Tool

Cisco Bug: CSCvv27069 - the device connector should not attempt to upgrade if /var/sysmgr does not have enough space

Last Modified

Aug 13, 2020

Products (1)

  • Cisco Unified Computing System

Known Affected Releases

4.0(4i)A

Description (partial)

Symptom:
When the /var/sysmgr is full, DC is not able to successfully upgrade itself.

Linux(debug)# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/root             300M  195M  106M  65% /
none                  2.0M  4.0K  2.0M   1% /post
none                  300M  195M  106M  65% /var
none                  3.0G  1.5G  1.5G  51% /isan
none                  600M  3.0M  598M   1% /var/tmp
none                  2.0G  2.0G     0 100% /var/sysmgr<<<
none                  140M   72K  140M   1% /var/sysmgr/ftp
none                  128M     0  128M   0% /callhome
none                  512M  230M  283M  45% /dev/shm
none                  240M  1.9M  239M   1% /volatile
none                  8.0M     0  8.0M   0% /debug
/dev/mtdblock2         48M  1.4M   47M   3% /mnt/plog
/dev/sda5              84M  4.3M   75M   6% /mnt/cfg/0
/dev/sda6              84M  4.3M   75M   6% /mnt/cfg/1
none                  400M  4.8M  396M   2% /var/sysmgr/startup-cfg
/dev/sda4              84M   17M   64M  21% /mnt/pss
/dev/sda3              16G   10G  5.2G  66% /bootflash
/dev/sda7             3.8G  108M  3.5G   3% /opt
/dev/mtdblock3        3.9M  2.2M  1.6M  58% /opt/db/nvram
/dev/sda8             3.8G   50M  3.6G   2% /workspace
tmpfs                 1.0M  4.0K 1020K   1% /bootflash/sysdebug/tftpd_logs
none                   50M  6.4M   44M  13% /debug-plugin

stop the DC
Linux(debug)# /opt/connector/sw/bin/conn_process_state.sh

Linux(debug)# cd /opt/connector/sw/etc/                         
Linux(debug)# cp pmon.conn.stop.conf pmon.conn.conf
Linux(debug)#  ps -ef | grep connector
root      5136  2127  0 Jul23 ?        00:00:07 /isan/bin/pmon -i -f /isan/etc/pmon.conf -g /opt/connector/sw/etc/pmon.conn.conf
root      5462  5136  0 Jul23 ?        00:22:05 /opt/connector//sw/bin/ucsfi_dc --log_path=/var/sysmgr/sam_logs/device_connector.log --dbdir=/opt/connector//db/
root     14352 11343  0 16:59 pts/1    00:00:00 grep connector
Linux(debug)# pkill -HUP pmon
Linux(debug)# ps -ef | grep connector
root      5136  2127  0 Jul23 ?        00:00:07 /isan/bin/pmon -i -f /isan/etc/pmon.conf -g /opt/connector/sw/etc/pmon.conn.conf
root     14400 11343  0 17:00 pts/1    00:00:00 grep connector
Linux(debug)# /opt/connector/sw/bin/conn_process_state.sh


the device connector does not start gracefully 

Linux(debug)# /opt/connector/sw/bin/conn_process_state.sh

SERVICE NAME             STATE     RETRY(MAX)    EXITCODE    SIGNAL    CORE
------------             -----     ----------    --------    ------    ----
ucsfi               terminated           0(4)           0         0      no

2. test downgrade the DC, the downgrade was sucessfully but DC is not working in the UI, not able to connect to intersight

tried to restart the process

Linux(debug)# /opt/connector/sw/bin/conn_process_state.sh

SERVICE NAME             STATE     RETRY(MAX)    EXITCODE    SIGNAL    CORE
------------             -----     ----------    --------    ------    ----
ucsfi                   failed           4(4)           1         0      no 
Linux(debug)#   ps -ef | grep -i ucsfi
root      9743  9430  0 19:19 pts/3    00:00:00 grep -i ucsfi
Linux(debug)#  cd /opt/connector/sw/etc/
Linux(debug)# cp pmon.conn.start.conf pmon.conn.conf
Linux(debug)# pkill -HUP pmon
Linux(debug)#  ps -ef | grep connector ps -ef | grep connector                        
grep: connector: No such file or directory
grep: ps: No such file or directory
Linux(debug)#  ps -ef | grep connector                        
root      5136  2127  0 Jul23 ?        00:00:08 /isan/bin/pmon -i -f /isan/etc/pmon.conf -g /opt/connector/sw/etc/pmon.conn.conf
root     10214  9430  0 19:22 pts/3    00:00:00 grep connector
Linux(debug)#  /opt/connector/sw/bin/conn_process_state.sh

SERVICE NAME             STATE     RETRY(MAX)    EXITCODE    SIGNAL    CORE
------------             -----     ----------    --------    ------    ----
ucsfi                   failed           4(4)           1         0      no 
Linux(debug)# /opt/connector//sw/bin/ucsfi_dc --log_path=/var/sysmgr/sam_logs/device_connector.log --dbdir=/opt/connector//db/
2020-08-05 19:24:36.004872734 +0000 UTC m=+0.005749002 write error: write /var/sysmgr/sam_logs/device_connector.log: no space left on device

not able to bring the DC up.

Conditions:
100% /var/sysmgr<<<

possible related with bug CSCvm21299
Bug details contain sensitive information and therefore require a Cisco.com account to be viewed.

Bug Details Include

  • Full Description (including symptoms, conditions and workarounds)
  • Status
  • Severity
  • Known Fixed Releases
  • Related Community Discussions
  • Number of Related Support Cases
Bug information is viewable for customers and partners who have a service contract. Registered users can view up to 200 bugs per month without a service contract.