Guest

Preview Tool

Cisco Bug: CSCvi13269 - Nexus(T+):Support for Asymmetric PFC behavior

Last Modified

Oct 09, 2019

Products (1)

  • Cisco Nexus 3000 Series Switches

Known Affected Releases

SMU_BUILD_6

Description (partial)

Symptom:
he Mellanox CX3Pro NIC maintains two queues to handle incoming traffic: lossy and lossless. Arriving packets are queued on one of the two queues according to their 802.11q tags. When the lossless queue is full, the NIC sends pause frames to the ToR. If lossy queue is full, packets arriving to that queue are dropped.  
on the Microsoft network, we use DSCP tags to distinguish between lossy and lossless traffic, since VLAN tags cannot be used with PXE boot. However, the Mellanox NIC cannot use DSCP tags for queuing purposes . To ensure that lossless traffic is not dropped, we are forced to queue all packets into the lossless queue on the NIC. The NIC is programmed to generate  for priorities 3 and 4 when the lossless queue is full.   
This architecture works well when the lossless traffic is carried on a physically separate network (e.g. storage backend), or when overall traffic volume is low. However, when traffic volume is high, lossless traffic may be starved: lossy traffic fills up the NIC queue, but the generated pause frames only affect the lossless traffic.  
Asymmetric PFC is a solution to this problem.  The Mellanox NIC will be configured to send pauses frames on all priorities when the lossless queue is full, instead of just priorities 3 and 4. The NIC, however, will only honor pauses on priority 3 and 4. PFC_ASYM will be ON for server-facing interfaces on the ToR.

Conditions:
On Nexus 3K, customer has a need to monitor ingress and egress ACLs. Customer requested the ability to detect when each entry in access-list is matched. This is related to the detection of transient loops that can form and impact performance and stability of RDMA/pfc queues. This was observed and replicated in labs. Transient loops triggered routing updates and excessive traffic, which caused build up of TX/RX counters in the Broadcom chip leading to deadlock in the N3K switches.
Bug details contain sensitive information and therefore require a Cisco.com account to be viewed.

Bug Details Include

  • Full Description (including symptoms, conditions and workarounds)
  • Status
  • Severity
  • Known Fixed Releases
  • Related Community Discussions
  • Number of Related Support Cases
Bug information is viewable for customers and partners who have a service contract. Registered users can view up to 200 bugs per month without a service contract.