Guest

Preview Tool

Cisco Bug: CSCus36208 - Nexus 5K/6K Experiences a Crash in the "PTPLC" Process Due to a Memory Leak

Last Modified

Nov 15, 2018

Products (9)

  • Cisco Nexus 6000 Series Switches
  • Cisco Nexus 5548P Switch
  • Cisco Nexus 5596UP Switch
  • Cisco Nexus 6004 Switch
  • Cisco Nexus 6001 Switch
  • Cisco Nexus 5672UP Switch
  • Cisco Nexus 56128P Switch
  • Cisco Nexus 5548UP Switch
  • Cisco Nexus 5596T Switch

Known Affected Releases

7.0(2)N1(1)

Description (partial)

Symptom:
A Nexus 5K or 6K switch may experience a crash in the "PTPLC" process after extended uptime due to a memory leak. Because this is a critical process, the switch may reload itself via a HAP reset. In addition, upon checking "show log nvram" and "show core", a core file will typically be written from the event. Eg:

  SWITCH# show version
  <snip>
    Reason: Reset triggered due to HA policy of Reset
    System version: 7.0(5)N1(1)
    Service: ptplc hap reset

  SWITCH# show log nvram
  <snip>
  %SYSMGR-2-SERVICE_CRASHED: Service "ptplc" (PID XXXX) hasn't caught signal 6 (core will be saved).
  %SYSMGR-2-HAP_FAILURE_SUP_RESET: System reset due to service "ptplc" in vdc 1 has had a hap failure

  SWITCH# show core
  Module  Instance  Process-name     PID       Date(Year-Month-Day Time)
  ------  --------  ---------------  --------  -------------------------
  1       1         ptplc            XXXX      2014-01-01 12:00:00

Conditions:
Checking "show processes log detail" will typically show that:

1) The process uptime is very long (multiple years or so, but this depends on the rate of the memory leak)
2) The "total" memory at the time of the crash is close to RLIMIT_AS (upper bound for this process's max memory)

Eg:

  `show processes log details | begin ptplc`
  ======================================================
  Service: ptplc
  Description: PTPLC Daemon
  Executable: /isan/bin/ptplc

  Started at Tue Mar  3 17:04:08 2015 (280021 us)
  Stopped at Sat Apr 28 15:10:10 2018 (221012 us)
  Uptime: 1151 days 21 hours 6 minutes 2 seconds  <=== High uptime, ~3 years

  <snip>

  RLIMIT_AS:      359785676 <=== Maximum memory allowed

  Virtual Memory:
  <snip>
      TOTAL     351344 KB <=== Memory usage at time of crash

On versions of NX-OS where "show system internal ptplc mem-stats detail" works, memory usage under this process will be seen to be increasing due to the "PTPLC_MEM_ptplc_ppf_info_t" API. Note: Bug CSCus52281 prevents this command from working in some NX-OS releases.

  SWITCH# show system internal ptplc mem-stats detail | grep PTPLC_MEM_ptplc_ppf_info_t 
    22 PTPLC_MEM_ptplc_ppf_info_t                1062   1062      33984      33984

If you believe you hit this bug but the above "show sys internal ptplc mem-stats" command does not work or if you just want further confirmation of the bug, please open a TAC case. Via internal tools, TAC can confirm the leak in the "PTPLC_MEM_ptplc_ppf_info_t" API based on mem-stats that are embedded inside the core file that's generated at the time of crash.
Bug details contain sensitive information and therefore require a Cisco.com account to be viewed.

Bug Details Include

  • Full Description (including symptoms, conditions and workarounds)
  • Status
  • Severity
  • Known Fixed Releases
  • Related Community Discussions
  • Number of Related Support Cases
Bug information is viewable for customers and partners who have a service contract. Registered users can view up to 200 bugs per month without a service contract.