Guest

Preview Tool

Cisco Bug: CSCuj25768 - Improve EOP error handing to prevent catastophic failures

Last Modified

Jan 18, 2017

Products (1)

  • Cisco Carrier Routing System

Known Affected Releases

4.0.3.BASE

Description (partial)

Symptom:
A CRS router can get into a situation in which all RPs and may get reloaded simultaneously due to fabricq asics taking multiple "Missing End of packet (EOP)" asic errors due to one failing linecard.

RP/1/RP0/CPU0: fabricq_mgr[200]: %FABRIC-FABRICQ-6-ERR_ASIC_EOP : Minor error in Fabricq ASIC 0. SQI Missing squidcast End of Packet error 
RP/1/RP0/CPU0: fabricq_mgr[200]: %FABRIC-FABRICQ-3-RESET :Reseting Fabricq ASIC Device 0. Reason: SQI Missing squidcast

When the fabricq on the other devices (RPs and LCs) report the EOP error the fabricq asic on those devices is reset. If this happens 5 times within 5 minutes the device (active RP on each rack will reload).

The fabricq ASIC reports "Missing End of packet (EOP)" when the reassembled packet size does not match with the actual packet length specified in BHDR field of packet. This can happen due to badly constructed packet at the source node. In rare cases, an ingress linecard taking repeated qdram MBE errors, that can cause the EOP errors on other LC/RPs. This can result in multiple RPs getting reloaded at the same time and the entire router reloading.

The development team is internally evaluating all possibilities to improve the fault isolation, recovery and enhancements to solve such issues.

This enhancement bug is opened to track the enhancements that will prevent this problem from happening in the future.

Conditions:
Faulty ingress linecard sends corrupted packets over the fabric to other RPs and Linecards
Bug details contain sensitive information and therefore require a Cisco.com account to be viewed.

Bug Details Include

  • Full Description (including symptoms, conditions and workarounds)
  • Status
  • Severity
  • Known Fixed Releases
  • Related Community Discussions
  • Number of Related Support Cases
Bug information is viewable for customers and partners who have a service contract. Registered users can view up to 200 bugs per month without a service contract.