Guest

Preview Tool

Cisco Bug: CSCur56942 - CCE tomcat webui connection timed out issue (when fetching config)

Last Modified

Feb 04, 2017

Products (1)

  • Cisco CNS 2100 Series Intelligence Engine

Known Affected Releases

3.5.4

Description (partial)

Symptom:
Seems we have found some kind of memory leak at CCE's tomcat webui server. The problem is that from time to time (very often) webui doesn't respond on Devices -> View Device -> Groups(1) -> config -> default -> Device* (just pick up one of them to check the config).
Operation will hang at „Please wait, this operation will take some time. Processing the request...?. After couple of seconds the connection time out will show.
After an investigation we have found that simple tomcat restart is solving this issue. We are suppose that there might be memory leak or some misconfiguration of JVM settings for Tomcat process.
Load on the server is ok, memory usage is ok, swap is not in use.

[ktaraszk@lndcicce01 ~]$ free -m
             total       used       free     shared    buffers     cached
Mem:          7873       1747       6125          0        567        392
-/+ buffers/cache:        786       7086
Swap:        10239          0      10239
[ktaraszk@lndcicce01 ~]$ uptime
 23:19:40 up 105 days,  5:47,  1 user,  load average: 0.00, 0.00, 0.00

[ktaraszk@lndcicce01 ~]$ ps aux | grep [j]ava
root     17546  0.0  3.4 1335608 276192 ?      Sl   Oct02  15:23 /opt/ConfigEngine/jdk1.6.0_05/bin/java -Dtomcat -client -Xms1024M -Xmx1024M -Xss192k -XX:NewRatio=6 -XX:MaxPermSize=128m -XX:+HeapDumpOnOutOfMemoryError -XX:+DisableExplicitGC -Djava.endorsed.dirs=/opt/ConfigEngine/tomcat/common/endorsed -classpath /opt/ConfigEngine/jdk1.6.0_05/lib/tools.jar:/opt/ConfigEngine/tomcat/bin/bootstrap.jar:/opt/ConfigEngine/CSCOGroupAdmin/WEB-INF/conf:/opt/ConfigEngine/CSCOcnsie/lib/tibrvj.jar:/opt/ConfigEngine/CSCOcnsie/lib/tibrvjweb.jar:/opt/ConfigEngine/CSCOnsm/conf:/opt/ConfigEngine/CSCOimgw/conf:/opt/ConfigEngine/CSCOcnsie/conf:/opt/ConfigEngine/CSCOcnsie/Templates:/opt/ConfigEngine/CSCOnsmAdmin/WEB-INF/conf:/opt/ConfigEngine/CSCOudi/WEB-INF/conf:/opt/ConfigEngine/CSCOdat/DAT/WEB-INF/conf -Djava.security.manager -Djava.security.policy==/opt/ConfigEngine/tomcat/conf/catalina.policy -Djava.security.auth.login.config==/opt/ConfigEngine/CSCOcnsie/conf/jaas.config -Dcatalina.base=/opt/ConfigEngine/tomcat -Dcatalina.home=/opt/ConfigEngine/tomcat -Djava.io.tmpdir=/opt/ConfigEngine/tomcat/temp -Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.SimpleLog -Dorg.apache.commons.logging.simplelog.defaultlog=error org.apache.catalina.startup.Bootstrap start

Currently this issue is occurs at one of our servers (LNDCICCE01). We didn't restart it yet (tomcat). It's „ready? for your investigation. We captured thread dump and heap dump from same issue at other server. Please find it attached.

In a nuthell: webui is not able to fetch device configuration and show it, tomcat restart solving this issue.

Conditions:
N/A
Bug details contain sensitive information and therefore require a Cisco.com account to be viewed.

Bug Details Include

  • Full Description (including symptoms, conditions and workarounds)
  • Status
  • Severity
  • Known Fixed Releases
  • Related Community Discussions
  • Number of Related Support Cases
Bug information is viewable for customers and partners who have a service contract. Registered users can view up to 200 bugs per month without a service contract.