Guest

Preview Tool

Cisco Bug: CSCup02981 - SR 630146459, Nexus 7K Stability Issues after ISSU

Last Modified

Feb 03, 2017

Products (12)

  • Cisco Nexus 7000 Series Switches
  • Cisco Nexus 7000 10-Slot Switch
  • Cisco MDS 9124 Multilayer Fabric Switch
  • Cisco MDS 8G FC HP Blade Switch
  • Cisco MDS 9222i Multiservice Modular Switch
  • Cisco MDS 9509 Multilayer Director
  • Cisco MDS 9134 Multilayer Fabric Switch
  • Cisco MDS 9506 Multilayer Director
  • Cisco Nexus 7000 18-Slot Switch
  • Cisco MDS 9148 Multilayer Fabric Switch
View all products in Bug Search Tool Login Required

Known Affected Releases

5.2(1)

Description (partial)

Symptom:
The following command was issued to upgrade the Nexus Operating System:
MTY-DI701001#install all kickstart n7000-s1-kickstart.5.2.9.bin system n7000-s1-dk9.5.2.9.bin
After a series of steps completed, the ISSU process detected a problem an interrupted itself, as shown on the following output:
MOD_REMOVE: Module 6 removed (Serial number JAF1539BMNH) 2014 May  3 20:27:28 MTY-DI701001 %$ VDC-1 %$ %IDEHSD-STANDBY-2-MOUNT: slot0: online  2014 May  3 20:28:20 MTY-DI701001 %$ VDC-1 %$ %CMPPROXY-STANDBY-2-LOG_CMP_UP: Connectivity Management processor(on module 6) is now UP 
Module 6: Waiting for module online.
 -- SUCCESS
2014 May  3 20:29:41 MTY-DI701001 %$ VDC-1 %$ %IDEHSD-STANDBY-2-MOUNT: logflash: online  
Notifying services about the switchover.
[#                   ]   0% -- FAIL.
Return code 0x401E007B (request was aborted by service).
Please issue "show install all failure-reason" to find the cause of the failure.

Conditions:
Software
  BIOS:      version 3.22.0
  kickstart: version 5.2(1)
  system:    version 5.2(1)
  BIOS compile time:       02/20/10
  kickstart image file is: bootflash:///n7000-s1-kickstart.5.2.1.bin
  kickstart compile time:  12/25/2020 12:00:00 [07/29/2011 06:34:35]
  system image file is:    bootflash:///n7000-s1-dk9.5.2.1.bin
  system compile time:     6/7/2011 13:00:00 [07/29/2011 08:29:25]

The process was attempted a second time with the same results.
The ISSU process first tries to upgrade the standby supervisor, and after it is upgraded, it performs an automatic switchover in order to make it the active one.  Before the switchover occurs, a series of validations takes place.  The following situations cause the installation to fail to complete:
•	If the standby supervisor module bootflash: file system does not have sufficient space to accept the updated image.
•	If the specified system and kickstart images are not compatible.
•	If the network or device is configured while the upgrade is in progress.
•	If a Spanning Tree Protocol (STP) topology change occurs while the upgrade is in progress.
•	If the install all command is entered on the standby supervisor module.
•	If the install all command does not reference the default bootflash: in a dual supervisor module configuration.
•	If a module is removed while the upgrade is in progress.
•	If the device has any power disruption while the upgrade is in progress.
•	If the entire path for the remote server location is not specified accurately.
•	If images are incompatible after an upgrade. For example, an I/O module image may be incompatible with the system image, or a kickstart image may be incompatible with a system image. This is also identified by the show install all impact command in the compatibility check section of the output (under the Bootable column).
•	If a linecard is in failure state, the ISSU will abort.
In this particular case, after issuing the show install all failure-reason command, it was determined that the install failed to complete due to a STP topology change in the network.  As a troubleshooting effort, the spanning-tree failure was identified to be on 3 different VLAN's: 73, 305 and 2038.  This topology changes were being noticed every 15 to 30 seconds for each of the 3 vlan's.  Tracking of the affected interfaces led to 2 different interfaces: Port-channel 6 and port-channel 10 on the switch.
Port-channels are bundling the following interfaces: Eth1/9 with Eth2/9, and Eth8/9 with Eth8/10.
Bug details contain sensitive information and therefore require a Cisco.com account to be viewed.

Bug Details Include

  • Full Description (including symptoms, conditions and workarounds)
  • Status
  • Severity
  • Known Fixed Releases
  • Related Community Discussions
  • Number of Related Support Cases
Bug information is viewable for customers and partners who have a service contract. Registered users can view up to 200 bugs per month without a service contract.