Guest

Preview Tool

Cisco Bug: CSCur18503 - CE was rebooted by unknown reason

Last Modified

Mar 08, 2016

Products (1)

  • Cisco Videoscape Distribution Suite Transparent Caching

Known Affected Releases

5.1.1

Description (partial)

Symptom:
CE was rebooted

Conditions:
<logs>

<WEBUI log>

Sep 19 22:25:30  mg-1  cluster has been degraded

Sep 19 22:45:30  mg-1  cluster has been enabled

 

<mg-1:/opt/pang/mgmt/avalon/var/log/snmpd.log>

2014-09-19 22:25:18.618  [peerapp:polling] ce-1:10161 request failed:
timeout

2014-09-19 22:25:30.604  [peerapp:polling] ce-1:10161 overtime

2014-09-19 22:25:30.628  cluster has been degraded

2014-09-19 22:25:36.655  [peerapp:polling] ce-1:10161 request failed:
timeout

2014-09-19 22:25:54.683  [peerapp:polling] ce-1:10161 request failed:
timeout

2014-09-19 22:26:00.610  [peerapp:polling] ce-1:10161 overtime

2014-09-19 22:26:12.706  [peerapp:polling] ce-1:10161 request failed:
timeout

2014-09-19 22:26:30.612  [peerapp:polling] ce-1:10161 overtime

2014-09-19 22:26:30.755  [peerapp:polling] ce-1:10161 request failed:
timeout

2014-09-19 22:26:48.789  [peerapp:polling] ce-1:10161 request failed:
timeout

2014-09-19 22:27:00.614  [peerapp:polling] ce-1:10161 overtime

2014-09-19 22:27:06.815  [peerapp:polling] ce-1:10161 request failed:
timeout

2014-09-19 22:27:24.839  [peerapp:polling] ce-1:10161 request failed:
timeout

2014-09-19 22:27:30.614  [peerapp:polling] ce-1:10161 overtime

2014-09-19 22:27:42.866  [peerapp:polling] ce-1:10161 request failed:
timeout

2014-09-19 22:28:00.622  [peerapp:polling] ce-1:10161 overtime

2014-09-19 22:28:00.899  [peerapp:polling] ce-1:10161 request failed:
timeout

2014-09-19 22:28:18.926  [peerapp:polling] ce-1:10161 request failed:
timeout

2014-09-19 22:28:30.622  [peerapp:polling] ce-1:10161 overtime

2014-09-19 22:28:36.946  [peerapp:polling] ce-1:10161 request failed:
timeout

2014-09-19 22:28:54.971  [peerapp:polling] ce-1:10161 request failed:
timeout

2014-09-19 22:29:00.626  [peerapp:polling] ce-1:10161 overtime

2014-09-19 22:29:12.995  [peerapp:polling] ce-1:10161 request failed:
timeout

2014-09-19 22:29:30.627  [peerapp:polling] ce-1:10161 overtime

2014-09-19 22:29:31. 18  [peerapp:polling] ce-1:10161 request failed:
timeout

2014-09-19 22:29:49. 42  [peerapp:polling] ce-1:10161 request failed:
timeout

2014-09-19 22:30:00.630  [peerapp:polling] ce-1:10161 overtime

2014-09-19 22:30:07. 66  [peerapp:polling] ce-1:10161 request failed:
timeout

2014-09-19 22:30:25. 86  [peerapp:polling] ce-1:10161 request failed:
timeout

2014-09-19 22:30:30.630  [peerapp:polling] ce-1:10161 overtime

2014-09-19 22:30:43.106  [peerapp:polling] ce-1:10161 request failed:
timeout

2014-09-19 22:31:00.634  [peerapp:polling] ce-1:10161 overtime

2014-09-19 22:31:01.130  [peerapp:polling] ce-1:10161 request failed:
timeout

2014-09-19 22:31:19.151  [peerapp:polling] ce-1:10161 request failed:
timeout

2014-09-19 22:31:30.635  [peerapp:polling] ce-1:10161 overtime

2014-09-19 22:31:37.175  [peerapp:polling] ce-1:10161 request failed:
timeout

2014-09-19 22:31:55.199  [peerapp:polling] ce-1:10161 request failed:
timeout

2014-09-19 22:32:00.639  [peerapp:polling] ce-1:10161 overtime

2014-09-19 22:32:13.230  [peerapp:polling] ce-1:10161 request failed:
timeout

2014-09-19 22:32:30.639  [peerapp:polling] ce-1:10161 overtime

2014-09-19 22:32:31.255  [peerapp:polling] ce-1:10161 request failed:
timeout

2014-09-19 22:32:49.278  [peerapp:polling] ce-1:10161 request failed:
timeout

2014-09-19 22:33:00.643  [peerapp:polling] ce-1:10161 overtime

2014-09-19 22:33:07.295  [peerapp:polling] ce-1:10161 request failed:
timeout

2014-09-19 22:33:25.311  [peerapp:polling] ce-1:10161 request failed:
timeout

2014-09-19 22:33:30.647  [peerapp:polling] ce-1:10161 overtime

2014-09-19 22:33:43.332  [peerapp:polling] ce-1:10161 request failed:
timeout

2014-09-19 22:34:00.652  [peerapp:polling] ce-1:10161 overtime

2014-09-19 22:34:01.352  [peerapp:polling] ce-1:10161 request failed:
timeout

2014-09-19 22:34:19.378  [peerapp:polling] ce-1:10161 request failed:
timeout

2014-09-19 22:34:30.655  [peerapp:polling] ce-1:10161 overtime

2014-09-19 22:34:37.396  [peerapp:polling] ce-1:10161 request failed:
timeout

2014-09-19 22:34:55.418  [peerapp:polling] ce-1:10161 request failed:
timeout

2014-09-19 22:35:00.659  [peerapp:polling] ce-1:10161 overtime

2014-09-19 22:45:30.749  cluster has been enabled

 

</var/log/peerapp/peerapp_system2014.09.19.log>

Sep 19 22:25:30 mg-1 snmpd[5365]: cluster has been degraded

Sep 19 22:25:43 ce-2 pang[3825]: grace timeout wait_to restore indication
is true 

Sep 19 22:25:43 ce-2 pang[3825]: time to say goodbye to blade 1 

Sep 19 22:25:43 ce-2 pang[3825]: Reallocating resources ...  

Sep 19 22:25:49 ce-2 pang[3825]: detected   major:  too many one direction
sessions 

Sep 19 22:26:52 ce-2 pang[3825]: detected   major:  too many one direction
sessions 

Sep 19 22:27:56 ce-2 pang[3825]: detected   major:  too many one direction
sessions 

Sep 19 22:27:56 ce-2 pang[3825]: volume             state      availability
owner      total      free       used       usage      t

otal_writes 

Sep 19 22:27:56 ce-2 pang[3825]: /mnt/vol1          mounted     active
ce-2          538        32         506        93.94      

0          

Sep 19 22:27:56 ce-2 pang[3825]: /mnt/vol2          mounted     active
ce-2          538        32         506        94.02      

0          

Sep 19 22:27:56 ce-2 pang[3825]: /mnt/vol3          mounted     active
ce-2          538        32         506        94.04      

0          

Sep 19 22:27:56 ce-2 pang[3825]: /mnt/vol4          mounted     active
ce-2          538        33         505        93.83      

0          

Sep 19 22:27:56 ce-2 pang[3825]: /mnt/vol5          mounted     active
ce-2          538        31         506        94.06      

0          

Sep 19 22:27:56 ce-2 pang[3825]: /mnt/vol6          mounted     active
ce-2          538        35         503        93.40      

0          

Sep 19 22:27:56 ce-2 pang[3825]: /mnt/vol7          mounted     active
ce-2          538        32         506        93.95      

0          

Sep 19 22:27:56 ce-2 pang[3825]: /mnt/vol8          mounted     active
ce-2          538        31         507        94.14      

0          

Sep 19 22:27:56 ce-2 pang[3825]: /mnt/vol9          mounted     active
ce-2          538        35         503        93.40      

0          

Sep 19 22:27:56 ce-2 pang[3825]: /mnt/vol10         mounted     active
ce-2          538        30         508        94.30      

0          

Sep 19 22:27:56 ce-2 pang[3825]: /mnt/vol11         mounted     active
ce-2          538        32         506        94.01      

0          

Sep 19 22:27:56 ce-2 pang[3825]: /mnt/vol21         mounted_cmdb
active    ce-2          538        35         502        93.

31      0          

Sep 19 22:29:01 ce-2 pang[3825]: detected   major:  too many one direction
sessions 

Sep 19 22:30:05 ce-2 pang[3825]: detected   major:  too many one direction
sessions 

Sep 19 22:31:10 ce-2 pang[3825]: detected   major:  too many one direction
sessions 

Sep 19 22:32:15 ce-2 pang[3825]: detected   major:  too many one direction
sessions 

Sep 19 22:32:38 ce-2 pang[3825]: Verifying blade 1 found mounted volumes or
xfs_repair - will reboot it 

Sep 19 22:32:38 ce-2 pang[3825]: Going to execute reboot to blade 1... 

Sep 19 22:33:22 ce-2 pang[3825]: detected   major:  too many one direction
sessions 

Sep 19 22:33:39 ce-2 pang[3825]: Volume /dev/sdn3 (id 13) was turned ON 

Sep 19 22:34:29 ce-2 pang[3825]: detected   major:  too many one direction
sessions 

Sep 19 22:34:46 ce-1 iscsid: iSCSI logger with pid=11468 started!

Sep 19 22:34:48 ce-1 iscsid: transport class version 2.0-870. iscsid
version 2.0-870

Sep 19 22:34:48 ce-1 iscsid: iSCSI daemon with pid=11475 started!

Sep 19 22:34:48 ce-1 iscsid: connection1:0 is operational now

Sep 19 22:34:48 ce-1 iscsid: connection2:0 is operational now

Sep 19 22:34:48 ce-1 iscsid: connection5:0 is operational now

Sep 19 22:34:48 ce-1 iscsid: connection6:0 is operational now

Sep 19 22:34:48 ce-1 iscsid: connection3:0 is operational now

Sep 19 22:34:48 ce-1 iscsid: connection4:0 is operational now

Sep 19 22:34:48 ce-1 iscsid: connection8:0 is operational now

Sep 19 22:34:48 ce-1 iscsid: connection7:0 is operational now

Sep 19 22:34:50 ce-2 pang[3825]: Volume /dev/sdm3 (id 12) was turned ON 

Sep 19 22:35:00 ce-1 snmpd[13603]: watchdog_threshold_reboots_time=54000 ,
reboot_time_diff=0 , g_start_kernel_watchdog 1

Sep 19 22:35:00 ce-1 snmpd[13603]: peerapp snmp agent has been restarted: 0
self-reboots till now , since 01-01-70 09:00:00 

Sep 19 22:35:05 ce-2 pang[3825]: volume             state      availability
owner      total      free       used       usage      t

otal_writes 

Sep 19 22:35:05 ce-2 pang[3825]: /mnt/vol1          mounted     active
ce-2          538        32         506        94.02      

0          

Sep 19 22:35:05 ce-2 pang[3825]: /mnt/vol2          mounted     active
ce-2          538        31         506        94.06      

0          

Sep 19 22:35:05 ce-2 pang[3825]: /mnt/vol3          mounted     active
ce-2          538        31         506        94.07      

0          

Sep 19 22:35:05 ce-2 pang[3825]: /mnt/vol4          mounted     active
ce-2          538        32         505        93.87      

0          

Sep 19 22:35:05 ce-2 pang[3825]: /mnt/vol5          mounted     active
ce-2          538        31         507        94.11      

0          

Sep 19 22:35:05 ce-2 pang[3825]: /mnt/vol6          mounted     active
ce-2          538        35         503        93.44      

0          

Sep 19 22:35:05 ce-2 pang[3825]: /mnt/vol7          mounted     active
ce-2          538        32         506        93.98      

0          

Sep 19 22:35:05 ce-2 pang[3825]: /mnt/vol8          mounted     active
ce-2          538        31         507        94.19      

0          

Sep 19 22:35:05 ce-2 pang[3825]: /mnt/vol9          mounted     active
ce-2          538        35         503        93.43      

0          

Sep 19 22:35:05 ce-2 pang[3825]: /mnt/vol10         mounted     active
ce-2          538        30         508        94.34      

0          

Sep 19 22:35:05 ce-2 pang[3825]: /mnt/vol11         mounted     active
ce-2          538        32         506        94.04      

0          

Sep 19 22:35:05 ce-2 pang[3825]: /mnt/vol12         mounted     active
ce-2          538        31         507        94.19      

0          

Sep 19 22:35:05 ce-2 pang[3825]: /mnt/vol13         mounted     active
ce-2          538        34         504        93.60      

0          

Sep 19 22:35:05 ce-2 pang[3825]: /mnt/vol21         mounted_cmdb
active    ce-2          538        35         502        93.

35      0          

Sep 19 22:35:37 ce-2 pang[3825]: detected   major:  too many one direction
sessions 

Sep 19 22:35:55 ce-2 pang[3825]: Volume /dev/sdo3 (id 14) was turned ON 

Sep 19 22:36:44 ce-2 pang[3825]: detected   major:  too many one direction
sessions 

Sep 19 22:36:58 ce-2 pang[3825]: Volume /dev/sdp3 (id 15) was turned ON 

Sep 19 22:37:51 ce-2 pang[3825]: detected   major:  too many one direction
sessions 

Sep 19 22:38:03 ce-2 pang[3825]: Volume /dev/sdq3 (id 16) was turned ON 

Sep 19 22:38:04 ce-1 pang[16220]: PANG version 5.1.1b28_ 

Sep 19 22:38:59 ce-2 pang[3825]: detected   major:  too many one direction
sessions 

Sep 19 22:39:14 ce-2 pang[3825]: Volume /dev/sdr3 (id 17) was turned ON 

Sep 19 22:40:08 ce-2 pang[3825]: detected   major:  too many one direction
sessions 

Sep 19 22:40:18 ce-1 pang[16220]: found volume 19. cmdb full BACKUP path
/mnt/vol19cmdb. 

Sep 19 22:40:18 ce-1 pang[16220]: Set own_cmdb_path is: /opt/pang/bin. 

Sep 19 22:40:22 ce-2 pang[3825]: Volume /dev/sdt3 (id 19) was turned ON 

Sep 19 22:40:25 ce-2 pang[3825]: Volume /dev/sds3 (id 18) was turned ON 

Sep 19 22:41:13 ce-2 pang[3825]: detected   major:  too many one direction
sessions 

Sep 19 22:41:23 ce-1 pang[16220]: stored hash count 4248568, found hash
count 4248568 

Sep 19 22:41:23 ce-1 pang[16220]: stored byte count 7831237746425, found
byte count 12501584374291 

Sep 19 22:42:24 ce-2 pang[3825]: detected   major:  too many one direction
sessions 

Sep 19 22:42:28 ce-2 pang[3825]: volume             state      availability
owner      total      free       used       usage      t

otal_writes 

Sep 19 22:42:28 ce-2 pang[3825]: /mnt/vol1          mounted     active
ce-2          538        31         506        94.08      

0          

Sep 19 22:42:28 ce-2 pang[3825]: /mnt/vol2          mounted     active
ce-2          538        31         506        94.11      

0          

Sep 19 22:42:28 ce-2 pang[3825]: /mnt/vol3          mounted     active
ce-2          538        31         507        94.12      

0          

Sep 19 22:42:28 ce-2 pang[3825]: /mnt/vol4          mounted     active
ce-2          538        32         506        93.95      

0          

Sep 19 22:42:28 ce-2 pang[3825]: /mnt/vol5          mounted     active
ce-2          538        31         507        94.15      

0          

Sep 19 22:42:28 ce-2 pang[3825]: /mnt/vol6          mounted     active
ce-2          538        35         503        93.48      

0          

Sep 19 22:42:28 ce-2 pang[3825]: /mnt/vol7          mounted     active
ce-2          538        32         506        94.03      

0          

Sep 19 22:42:28 ce-2 pang[3825]: /mnt/vol8          mounted     active
ce-2          538        31         507        94.22      

0          

Sep 19 22:42:28 ce-2 pang[3825]: /mnt/vol9          mounted     active
ce-2          538        35         503        93.46      

0          

Sep 19 22:42:28 ce-2 pang[3825]: /mnt/vol10         mounted     active
ce-2          538        30         508        94.39      

0          

Sep 19 22:42:28 ce-2 pang[3825]: /mnt/vol11         mounted     active
ce-2          538        31         506        94.08      

0          

Sep 19 22:42:28 ce-2 pang[3825]: /mnt/vol12         mounted     active
ce-2          538        31         507        94.22      

0          

Sep 19 22:42:28 ce-2 pang[3825]: /mnt/vol13         mounted     active
ce-2          538        34         504        93.64      

0          

Sep 19 22:42:28 ce-2 pang[3825]: /mnt/vol14         mounted     active
ce-2          538        30         508        94.41      

0          

Sep 19 22:42:28 ce-2 pang[3825]: /mnt/vol21         mounted_cmdb
active    ce-2          538        35         503        93.

39      0          

Sep 19 22:43:33 ce-2 pang[3825]: detected   major:  too many one direction
sessions 

Sep 19 22:43:41 ce-2 pang[3825]: volumes are  not mapped!!! still
reallocating resources... 

Sep 19 22:44:02 ce-2 pang[3825]: volumes are  not mapped!!! still
reallocating resources... 

Sep 19 22:44:20 ce-2 pang[3825]: volumes are  mapped and have owners :-)
will send Resources allocated msg!!!
Bug details contain sensitive information and therefore require a Cisco.com account to be viewed.

Bug Details Include

  • Full Description (including symptoms, conditions and workarounds)
  • Status
  • Severity
  • Known Fixed Releases
  • Related Community Discussions
  • Number of Related Support Cases
Bug information is viewable for customers and partners who have a service contract. Registered users can view up to 200 bugs per month without a service contract.