Guest

Preview Tool

Cisco Bug: CSCuw48283 - High CPU Ficon due to flush sync loop

Last Modified

Mar 02, 2018

Products (1)

  • Cisco MDS 9000 Series Multilayer Switches

Known Affected Releases

6.2(5b)

Description (partial)

Symptom:
Ficon process will have high CPU and the CPU will be maxed out on the switch:

`show processes cpu`

CPU utilization for five seconds: 99%/5%; one minute: 99%; five minutes: 99%
PID    Runtime(ms)  Invoked   uSecs  5Sec    1Min    5Min    TTY  Process
-----  -----------  --------  -----  ------  ------  ------  ---  -----------
...snip
3281   2207862740  1893123134      1   2.86%  56.23% 70.66%   -    ficon

The the following error messages may also be seen:

`show ficon internal event errors`

1) Event:E_DEBUG, length:61, at 574194 usecs after Fri Aug 28 11:34:46 2015
    [102] ficonm_demux: (Warning) unexpected mts msg (opcode - 7)



2) Event:E_DEBUG, length:61, at 572139 usecs after Fri Aug 28 11:34:46 2015
    [102] ficonm_demux: (Warning) unexpected mts msg (opcode - 7)



3) Event:E_DEBUG, length:61, at 572049 usecs after Fri Aug 28 11:34:46 2015
    [102] ficonm_demux: (Warning) unexpected mts msg (opcode - 7)

There will also be stuck messages in the MTS buffer:

`show system internal mts buffer detail`

Node/Sap/queue  Age(ms)         SrcNode         SrcSAP  DstNode         DstSAP  OPC     MsgId   MsgSize RRToken Offset
sup/4115/recv   2942600923      0x701           492     0x701           4115    7842    0xac3f  119832  0x8000ac3d      0xac0004

Conditions:
On very rare circumstances, the ficon process may miss the 10 second timeout period while trying to flush the MTS sync queue; as a result, the ficon process rejects a late flush status message. MTS bounces the rejected message back to the ficon process, causing the infinite message bouncing and consumes CPU cycles unnecessarily.
Bug details contain sensitive information and therefore require a Cisco.com account to be viewed.

Bug Details Include

  • Full Description (including symptoms, conditions and workarounds)
  • Status
  • Severity
  • Known Fixed Releases
  • Related Community Discussions
  • Number of Related Support Cases
Bug information is viewable for customers and partners who have a service contract. Registered users can view up to 200 bugs per month without a service contract.