Guest

Preview Tool

Cisco Bug: CSCur52935 - 530: continuos fib_mgr tracebacks from A9K-8x100GE-L-SE

Last Modified

Jul 21, 2018

Products (8)

  • Cisco ASR 9000 Series Aggregation Services Routers
  • Cisco ASR 9922 Router
  • Cisco IOS XR Software
  • Cisco ASR 9010 Router
  • Cisco ASR 9904 Router
  • Cisco ASR 9006 Router
  • Cisco ASR 9001 Router
  • Cisco ASR 9912 Router

Known Affected Releases

5.3.0.BASE

Description (partial)

After running script for some 3-4 days on adjacent node for route churning observed below logs on TMHK card.

LC/0/8/CPU0:Nov  3 09:52:13.634 : fib_mgr[181]: %ROUTING-FIB-3-PLATF_UPD_FAIL : FIB platform update failed: Obj=DATA_TYPE_NHINFO[ptr=62b782f8,refc=0x1,flags=0x800008] Action=CREATE Proto=ipv4. Cerr='prm_server' detected the 'resource not available' condition 'lack of stats counters'  : pkg/bin/fib_mgr : (PID=192622) :  -Traceback= afdcd3b 429c1e2 42a2040 42a55f3 42aadb1 42abe7d 42f7e22 42f815b 42f92bf 420944a 82465d7 8244724 4201676 4213b2c 828a050
LC/0/8/CPU0:Nov  3 09:52:30.670 : fib_mgr[181]: %ROUTING-FIB-3-PLATF_UPD_FAIL : FIB platform update failed: Obj=DATA_TYPE_NHINFO[ptr=62b79b78,refc=0x1,flags=0x800008] Action=CREATE Proto=ipv4. Cerr='prm_server' detected the 'resource not available' condition 'lack of stats counters'  : pkg/bin/fib_mgr : (PID=192622) :  -Traceback= afdcd3b 429c1e2 42a2040 42a55f3 42aadb1 42abe7d 42f7e22 42f815b 42f92bf 420944a 82465d7 8244724 4201676 4213b2c 828a050
LC/0/8/CPU0:Nov  3 09:52:44.369 : pifibm_server_lc[290]: %OS-PLATFORM_LPTS_PIFIB-5-ERR_PRM_STATS_ALLOC : Failed to allocate HW counters. 
LC/0/8/CPU0:Nov  3 09:52:45.682 : fib_mgr[181]: %ROUTING-FIB-3-PLATF_UPD_FAIL : FIB platform update failed: Obj=DATA_TYPE_NHINFO[ptr=62b79558,refc=0x1,flags=0x800008] Action=CREATE Proto=ipv4. Cerr='prm_server' detected the 'resource not available' condition 'lack of stats counters'  : pkg/bin/fib_mgr : (PID=192622) :  -Traceback= afdcd3b 429c1e2 42a2040 42a55f3 42aadb1 42abe7d 42f7e22 42f815b 42f92bf 420944a 82465d7 8244724 4201676 4213b2c 828a050

On similar terms route churn induce from TGN resulting below logs on VSM-500 card :-

LC/0/4/CPU0:Jun 17 21:25:57.687 : l2fib[259]: %OS-SHMWIN-2-ERROR_ENCOUNTERED : SHMWIN: Error encountered: System memory state is severe, please check the availability of the system memory  
LC/0/4/CPU0:Jun 17 21:25:59.368 : fib_mgr[185]: %OS-SHMWIN-2-ERROR_ENCOUNTERED : SHMWIN: Error encountered: System memory state is severe, please check the availability of the system memory  
LC/0/4/CPU0:Jun 17 21:26:25.121 : l2fib[259]: %OS-SHMWIN-2-ERROR_ENCOUNTERED : SHMWIN: Error encountered: System memory state is severe, please check the availability of the system memory  
LC/0/4/CPU0:Jun 17 21:26:29.365 : fib_mgr[185]: %OS-SHMWIN-2-ERROR_ENCOUNTERED : SHMWIN: Error encountered: System memory state is severe, please check the availability of the system memory  
LC/0/4/CPU0:Jun 17 21:26:53.257 : l2fib[259]: %OS-SHMWIN-2-ERROR_ENCOUNTERED : SHMWIN: Error encountered: System memory state is severe, please check the availability of the system memory  
LC/0/4/CPU0:Jun 17 21:26:59.363 : fib_mgr[185]: %OS-SHMWIN-2-ERROR_ENCOUNTERED : SHMWIN: Error encountered: System memory state is severe, please check the availability of the system memory

Symptom:After running script for some 3-4 days on adjacent node for route churning observed below logs on TMHK card.

LC/0/8/CPU0:Nov  3 09:52:13.634 : fib_mgr[181]: %ROUTING-FIB-3-PLATF_UPD_FAIL : FIB platform update failed: Obj=DATA_TYPE_NHINFO[ptr=62b782f8,refc=0x1,flags=0x800008] Action=CREATE Proto=ipv4. Cerr='prm_server' detected the 'resource not available' condition 'lack of stats counters'  : pkg/bin/fib_mgr : (PID=192622) :  -Traceback= afdcd3b 429c1e2 42a2040 42a55f3 42aadb1 42abe7d 42f7e22 42f815b 42f92bf 420944a 82465d7 8244724 4201676 4213b2c 828a050
LC/0/8/CPU0:Nov  3 09:52:30.670 : fib_mgr[181]: %ROUTING-FIB-3-PLATF_UPD_FAIL : FIB platform update failed: Obj=DATA_TYPE_NHINFO[ptr=62b79b78,refc=0x1,flags=0x800008] Action=CREATE Proto=ipv4. Cerr='prm_server' detected the 'resource not available' condition 'lack of stats counters'  : pkg/bin/fib_mgr : (PID=192622) :  -Traceback= afdcd3b 429c1e2 42a2040 42a55f3 42aadb1 42abe7d 42f7e22 42f815b 42f92bf 420944a 82465d7 8244724 4201676 4213b2c 828a050
LC/0/8/CPU0:Nov  3 09:52:44.369 : pifibm_server_lc[290]: %OS-PLATFORM_LPTS_PIFIB-5-ERR_PRM_STATS_ALLOC : Failed to allocate HW counters. 
LC/0/8/CPU0:Nov  3 09:52:45.682 : fib_mgr[181]: %ROUTING-FIB-3-PLATF_UPD_FAIL : FIB platform update failed: Obj=DATA_TYPE_NHINFO[ptr=62b79558,refc=0x1,flags=0x800008] Action=CREATE Proto=ipv4. Cerr='prm_server' detected the 'resource not available' condition 'lack of stats counters'  : pkg/bin/fib_mgr : (PID=192622) :  -Traceback= afdcd3b 429c1e2 42a2040 42a55f3 42aadb1 42abe7d 42f7e22 42f815b 42f92bf 420944a 82465d7 8244724 4201676 4213b2c 828a050

On similar terms route churn induce from TGN resulting below logs on VSM-500 card :-

LC/0/4/CPU0:Jun 17 21:25:57.687 : l2fib[259]: %OS-SHMWIN-2-ERROR_ENCOUNTERED : SHMWIN: Error encountered: System memory state is severe, please check the availability of the system memory  
LC/0/4/CPU0:Jun 17 21:25:59.368 : fib_mgr[185]: %OS-SHMWIN-2-ERROR_ENCOUNTERED : SHMWIN: Error encountered: System memory state is severe, please check the availability of the system memory  
LC/0/4/CPU0:Jun 17 21:26:25.121 : l2fib[259]: %OS-SHMWIN-2-ERROR_ENCOUNTERED : SHMWIN: Error encountered: System memory state is severe, please check the availability of the system memory  
LC/0/4/CPU0:Jun 17 21:26:29.365 : fib_mgr[185]: %OS-SHMWIN-2-ERROR_ENCOUNTERED : SHMWIN: Error encountered: System memory state is severe, please check the availability of the system memory  
LC/0/4/CPU0:Jun 17 21:26:53.257 : l2fib[259]: %OS-SHMWIN-2-ERROR_ENCOUNTERED : SHMWIN: Error encountered: System memory state is severe, please check the availability of the system memory  
LC/0/4/CPU0:Jun 17 21:26:59.363 : fib_mgr[185]: %OS-SHMWIN-2-ERROR_ENCOUNTERED : SHMWIN: Error encountered: System memory state is severe, please check the availability of the system memory

Conditions:Continuously route churning for few days helped in seeing Route update fail or low Shared memory logs on ASR9k-VSM-500 card.
Bug details contain sensitive information and therefore require a Cisco.com account to be viewed.

Bug Details Include

  • Full Description (including symptoms, conditions and workarounds)
  • Status
  • Severity
  • Known Fixed Releases
  • Related Community Discussions
  • Number of Related Support Cases
Bug information is viewable for customers and partners who have a service contract. Registered users can view up to 200 bugs per month without a service contract.