Guest

Preview Tool

Cisco Bug: CSCun00875 - Active ESP crashed after ~5 hours of churning

Last Modified

Apr 17, 2019

Products (1)

  • Cisco ASR 1000 Series Aggregation Services Routers

Known Affected Releases

15.3(3)S

Description (partial)

Symptom:
A crash may occur after a large amount of ISG session flapping and tunnel flapping.  ESP (QFP) memory exhaustion may be seen prior to the crash:

041954: Jun 20 03:22:37.632 UTC: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel1, changed state to up
041955: Jun 20 03:43:54.252 UTC: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel1, changed state to up
041956: Jun 20 04:10:17.644 UTC: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel1, changed state to up
041957: Jun 20 04:16:19.992 UTC: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel1, changed state to up
041958: Jun 20 04:26:05.046 UTC: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel1, changed state to up
041959: Jun 20 04:26:05.079 UTC: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel2, changed state to up
041960: Jun 20 04:29:30.800 UTC: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel1, changed state to up
041961: Jun 20 04:32:40.967 UTC: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel1, changed state to up
041962: Jun 20 04:32:41.008 UTC: %CPPOSLIB-3-ERROR_NOTIFY: F0: fman_fp_image:  fman-fp encountered an error -Traceback= 1#21fcabb7bf645a0ac69f1ea8c36c64eb   errmsg:C565000+2230 cpp_common_os:95E2000+C470 cpp_common_os:95E2000+C270 cpp_common_os:95E2000+19BCC cpp_plutlu_common:9D7A000+2FE0 cpp_plutlu_common:9D7A000+30A8 cpp_cef_mpls_common:9C27000+1A1F8 cpp_cef_mpls_common:9C27000+1CA50 cpp_cef_mpls_common:9C27000+1D374 :10000000+5F4974 :10000000+28B984 aobjman:DE0C000+11A78 :10000000+41345C evlib:9FD0000+E4AC evlib:9FD000
041963: Jun 20 04:35:37.120 UTC: %CPPOSLIB-3-ERROR_NOTIFY: F0: fman_fp_image:  fman-fp encountered an error -Traceback= 1#21fcabb7bf645a0ac69f1ea8c36c64eb   errmsg:C565000+2230 cpp_common_os:95E2000+C470 cpp_common_os:95E2000+C270 cpp_common_os:95E2000+19BCC cpp_plutlu_common:9D7A000+2FE0 cpp_plutlu_common:9D7A000+30A8 cpp_cef_mpls_common:9C27000+1A1F8 cpp_cef_mpls_common:9C27000+1CA50 cpp_cef_mpls_common:9C27000+1D374 :10000000+5F4974 :10000000+28B984 aobjman:DE0C000+11A78 :10000000+41345C evlib:9FD0000+E4AC evlib:9FD0
041964: Jun 20 04:35:37.832 UTC: %IOSXE_OIR-6-OFFLINECARD: Card (fp) offline in slot F0
041965: Jun 20 04:37:08.683 UTC: %CPPHA-3-FAULT: F0: cpp_ha:  CPP:0.0 desc:CPP Client process failed: FMAN-FP det:HA class:CLIENT_SW sev:FATAL id:1 cppstate:RUNNING res:UNKNOWN flags:0x0 cdmflags:0x0
041966: Jun 20 04:37:08.697 UTC: %IOSXE-6-PLATFORM: F0: cpp_ha: Shutting down CPP MDM while client(s) still connected 
041967: Jun 20 04:37:08.843 UTC: %PMAN-3-PROCHOLDDOWN: F0: pman.sh:  The process cpp_ha_top_level_server has been helddown (rc 69)

Conditions:
QFP memory exhaustion and interface flapping
Bug details contain sensitive information and therefore require a Cisco.com account to be viewed.

Bug Details Include

  • Full Description (including symptoms, conditions and workarounds)
  • Status
  • Severity
  • Known Fixed Releases
  • Related Community Discussions
  • Number of Related Support Cases
Bug information is viewable for customers and partners who have a service contract. Registered users can view up to 200 bugs per month without a service contract.