Guest

Preview Tool

Cisco Bug: CSCvt16423 - Application upgrade from 1.3.1.3 to 1.3.3.0 failing due to pipelineadmin crashing

Last Modified

Sep 30, 2020

Products (1)

  • Cisco DNA Center

Known Affected Releases

DNAC1.3.3.0 DNAC1.3.3.4

Description (partial)

Symptom:
While upgrading Cisco DNA Center's application packages from version 1.3.1.3 to 1.3.3.0, the pipelineadmin service may degrade into the CrashLoopBackOff state.


Events:
  Type     Reason            Age                From                 Message
  ----     ------            ----               ----                 -------
  Warning  FailedScheduling  5m (x3 over 5m)    default-scheduler    0/1 nodes are available: 1 node(s) didn't match pod affinity/anti-affinity, 1 node(s) didn't satisfy existing pods anti-affinity rules.
  Normal   Scheduled         5m                 default-scheduler    Successfully assigned ndp/pipelineadmin-65985d6c77-8kg88 to x.x.x.x
  Normal   Created           4m (x4 over 5m)    kubelet, x.x.x.x  Created container
  Normal   Started           4m (x4 over 5m)    kubelet, x.x.x.x  Started container
  Normal   Pulling           3m (x5 over 5m)    kubelet, x.x.x.x  pulling image "maglev-registry.maglev-system.svc.cluster.local:5000/ndp/pipelineadmin:1.4.1.397"
  Normal   Pulled            3m (x5 over 5m)    kubelet, x.x.x.x  Successfully pulled image "maglev-registry.maglev-system.svc.cluster.local:5000/ndp/pipelineadmin:1.4.1.397"
  Warning  BackOff           34s (x22 over 5m)  kubelet, x.x.x.x  Back-off restarting failed container


| 436 | 2020-02-24 08:39:20 | ERROR | main | 1 | com.cisco.maglev.server.services.ServiceContainer | com.cisco.maglev.server.services.ServiceException: Error while initializing services.
        at com.cisco.maglev.server.services.BaseServiceContainer.init(BaseServiceContainer.java:92)
        at com.cisco.maglev.server.services.CustomServiceContainer.init(CustomServiceContainer.java:46)
        at com.cisco.maglev.server.ServerImpl.init(ServerImpl.java:131)
        at com.cisco.maglev.MaglevServer.main(MaglevServer.java:25)
Caused by: com.cisco.maglev.server.services.ServiceLifeCycleException: Unable to create directory /var/data/ndp/_no_backup_/pipelines
        at com.cisco.ndp.pipeline.admin.AdminService.oninit(AdminService.java:156)
        at com.cisco.maglev.server.services.ServiceImpl.init(ServiceImpl.java:60)

The service fails to initialize due to being "Unable to create directory /var/data/ndp/_no_backup_/pipelines".

Conditions:
This was observed following an upgrade from 1.3.1.3 to 1.3.3.0, in a single node setup.
Bug details contain sensitive information and therefore require a Cisco.com account to be viewed.

Bug Details Include

  • Full Description (including symptoms, conditions and workarounds)
  • Status
  • Severity
  • Known Fixed Releases
  • Related Community Discussions
  • Number of Related Support Cases
Bug information is viewable for customers and partners who have a service contract. Registered users can view up to 200 bugs per month without a service contract.