Guest

Preview Tool

Cisco Bug: CSCvr95394 - "The process cmand has been helddown (rc 139)" Crash seen on cat9300 stack on bootup

Last Modified

Jul 21, 2020

Products (1)

  • Cisco Catalyst 9300 Series Switches

Known Affected Releases

17.1.1

Description (partial)

Symptom:
A Cat9k stack running an affected version may observe an unexpected reload duruing bootup with the following errors:

%PMAN-3-PROCHOLDDOWN: R0/0: The process cmand has been helddown (rc 139)
%PMAN-0-PROCFAILCRIT: R0/0: pvp: A critical process cmand has failed (rc 139)
%PMAN-3-RELOAD_RP: R0/0: pvp: Reloading: Switch will be reloaded

Chassis 1 reloading, reason - Non participant detected
%PMAN-5-EXITACTION: F0/0: pvp: Process manager is exiting: reload fp action requested
%PMAN-5-EXITACTvp: Process manager is exiting: Critical process cmand fault on rp_0_0 (rc=139)
%PMAN-3-PROCESS_NOTIFICATION: R0/0: pvp: System report /crashinfo/system-report_local_20191105-041316-Universal.tar.gz (size: 2887 KB) generated

Leaving a system report with cores inside, whose decodes may point to specific POE functions.

The decoding of the functions may be done by TAC. In there we may observe that the IOmd process was sending a power sense message to a POE client. From the syslogs, we may see that the cmand process gets helddown eventually:

Conditions:
So far it has been seen on C9300 stacks which may be running PoE.

Crash is seen during bootup, as if a switch was failing to join the stack.

Upon inspecting the failing switch we may see system reports (tar.gz) files with core files inside.
Bug details contain sensitive information and therefore require a Cisco.com account to be viewed.

Bug Details Include

  • Full Description (including symptoms, conditions and workarounds)
  • Status
  • Severity
  • Known Fixed Releases
  • Related Community Discussions
  • Number of Related Support Cases
Bug information is viewable for customers and partners who have a service contract. Registered users can view up to 200 bugs per month without a service contract.