Guest

Preview Tool

Cisco Bug: CSCvi70543 - Service SAP Qosmgr - (Operation timed out) in if_bind sequence

Last Modified

Oct 16, 2018

Products (9)

  • Cisco Nexus 7000 Series Switches
  • Cisco Nexus 7000 10-Slot Switch
  • Cisco Nexus 7000 4-Slot Switch
  • Cisco Nexus 7700 6-Slot Switch
  • Cisco Nexus 7700 18-Slot Switch
  • Cisco Nexus 7000 18-Slot Switch
  • Cisco Nexus 7000 9-Slot Switch
  • Cisco Nexus 7700 2-Slot Switch
  • Cisco Nexus 7700 10-Slot Switch

Known Affected Releases

7.2(2)D1(2)

Description (partial)

Symptom:
In attempt to allocate interfaces from M-series I/O module into production VDC you may hit the next issue:

2018 Mar 27 14:02:56.278 RTR2712 %$ VDC-2 %$ %VMM-2-VMM_SERVICE_ERR: VDC2: Service SAP Qosmgr SAP for slot 4 returned error 0x41170014 (Operation timed out) in if_bind sequence

Conditions:
Trying to allocate interfaces from M-series I/O module into production VDC after 10 minutes timeout this operation may fail with:

2018 Mar 27 14:02:56.278 RTR %$ VDC-2 %$ %VMM-2-VMM_SERVICE_ERR: VDC2: Service SAP Qosmgr SAP for slot 4 returned error 0x41170014 (Operation timed out) in if_bind sequence


vdc_id: 2 vdc_name: RTR interfaces:
Port        Status
----        ----------
Eth3/1      OK
Eth3/2      OK
...
Eth3/22     OK
Eth3/23     OK
Eth3/24     OK
Eth4/1      ERROR:Operation timed out (0x41170014)
Eth4/2      ERROR:Operation timed out (0x41170014)
Eth4/3      ERROR:Operation timed out (0x41170014)
Eth4/4      ERROR:Operation timed out (0x41170014)
Eth4/5      ERROR:Operation timed out (0x41170014)
Eth4/6      ERROR:Operation timed out (0x41170014)
Eth4/7      ERROR:Operation timed out (0x41170014)
Eth4/8      ERROR:Operation timed out (0x41170014)
Eth4/9      ERROR:Operation timed out (0x41170014)
Eth4/10     ERROR:Operation timed out (0x41170014)
Eth4/11     ERROR:Operation timed out (0x41170014)
...


RTR2712# show system internal mts buffers details
**Fast Sap Buffers are not displayed below**
Node/Sap/queue  Age(ms)         SrcNode         SrcSAP  DstNode         DstSAP  OPC     MsgId   MsgSize RRToken Offset
sup/4765/nper   215691          0x201           357     0x201           4765    7679    0xb41e  224     0x8000b41d      0xffdc404
sup/284/pers    1630019         0x201           4487    0x201           284     86017   0x2c92  4608    0x2c92  0xfab2004
sup/284/pers    222             0x201           4751    0x201           284     86017   0xbee3  4608    0xbee3  0xffde004
sup-1/284/pers  1453369         0x205           14798   0x205           284     86017   0x4e47  4608    0x4e47  0xffd8004
sup-1/284/pers  612             0x205           14799   0x205           284     86017   0x13263 4608    0x13263 0xfab0004
sup-1/351/log   1069009         0x205           377     0x205           0       6484    0xe6e9  12110   0       0xfabc004      <===





The exact conditions are not known so far, however:
 - this issue is constantly and easily reproducible with M2 I/O modules with specific running-configuration.


2018 Mar 27 14:00:26.267 RTR %IPQOSMGR-4-QOSMGR_LC_ERROR_MSG: Linecard 1 returned an error: Operation timed out
2018 Mar 27 14:02:56.277 RTR %IPQOSMGR-4-QOSMGR_LC_ERROR_MSG: Linecard 1 returned an error: Operation timed out
2018 Mar 27 14:02:56.278 RTR %VMM-2-VMM_SERVICE_ERR: VDC2: Service SAP Qosmgr SAP for slot 4 returned error 0x41170014 (Operation timed out) in if_bind sequence


Looks like this issue is driven by "ACLMgr" component and thus it depends on certain ACL configuration, you might use in your environment. It might be ACLs for route-map, table-map, QoS, PACL, RACL...

Further investigation is needed...



RTR# show system internal mts buffers details
...
sup-1/351/log   6258260         0x105           377     0x205           0       6484    0xe6e9  12110   0       0xfaac004
sup-1/351/log   79718           0x105           14933   0x105           351     15462   0x3c37  1155    0x3c37  0xfab2804
sup-1/351/log   59997           0x105           28      0x105           351     15460   0x960c  118     0x960c  0xfa45c04
sup-1/351/log   55003           0x105           28      0x105           351     15460   0x970b  118     0x970b  0xfab8e04
sup-1/351/log   49718           0x105           18417   0x105           351     15462   0x97a9  2084    0x97a9  0xfa2e004


RTR2# show system internal mts node sup-1 sap 351 description
Aclmgr SAP
Bug details contain sensitive information and therefore require a Cisco.com account to be viewed.

Bug Details Include

  • Full Description (including symptoms, conditions and workarounds)
  • Status
  • Severity
  • Known Fixed Releases
  • Related Community Discussions
  • Number of Related Support Cases
Bug information is viewable for customers and partners who have a service contract. Registered users can view up to 200 bugs per month without a service contract.