Guest

Preview Tool

Cisco Bug: CSCvs51942 - Create ACI-CNI Tenant cluster fails to create ACI network and never spawns VMs

Last Modified

Dec 20, 2019

Products (1)

  • Cisco Container Platform

Known Affected Releases

5.0.0

Description (partial)

Symptom:
{"level":"info","ts":1576737512.71857,"logger":"netprofile.aci","msg":"ACI cluster in backend is not ready","name":"ccp-foxconn-cl01","namespace":"default"}
{"level":"error","ts":1576737517.5311747,"logger":"controller.cni","msg":"failed create kubernetes client from secret","name":"ccp-foxconn-cl01-cni","namespace":"default","error":"failed to extract kubeconfig data: fail to get secret: default/ccp-foxconn-cl01-kubeconfig: Secret \"ccp-foxconn-cl01-kubeconfig\" not found","errorVerbose":"Secret \"ccp-foxconn-cl01-kubeconfig\" not found\nfail to get secret: default/ccp-foxconn-cl01-kubeconfig\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/k8s/client.ExtractKubeConfigFromSecret.func1\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/k8s/client/client.go:98\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/k8s/client.NewK8SClient\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/k8s/client/client.go:53\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/controller/cni.(*ReconcileCNI).Reconcile\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/controller/cni/cni_controller.go:133\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:215\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:158\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:134\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/k8s.io/apimachinery/pkg/util/wait.Until\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1337\nfailed to extract kubeconfig data\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/k8s/client.NewK8SClient\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/k8s/client/client.go:55\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/controller/cni.(*ReconcileCNI).Reconcile\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/controller/cni/cni_controller.go:133\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:215\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:158\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:134\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/k8s.io/apimachinery/pkg/util/wait.Until\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1337","stacktrace":"wwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/github.com/go-logr/zapr/zapr.go:128\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/controller/cni.(*ReconcileCNI).failedCreateClient\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/controller/cni/cni_controller.go:211\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/controller/cni.(*ReconcileCNI).Reconcile\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/controller/cni/cni_controller.go:187\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:215\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:158\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:134\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/k8s.io/apimachinery/pkg/util/wait.Until\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88"}

{"level":"error","ts":1576737517.7009702,"logger":"controller.nginxingress","msg":"remote client not ready","name":"ccp-foxconn-cl01-ingress","namespace":"default","error":"failed to build client to remote cluster","errorVerbose":"failed to build client to remote cluster\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/controller/nginxingress.init.ializers\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/controller/nginxingress/config_driver.go:73\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:188\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1337","stacktrace":"wwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/github.com/go-logr/zapr/zapr.go:128\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/controller/nginxingress.(*KubeClientDriver).buildManifestData.func1\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/controller/nginxingress/config_driver.go:121\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/controller/nginxingress.(*KubeClientDriver).buildManifestData\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/controller/nginxingress/config_driver.go:126\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/controller/nginxingress.(*KubeClientDriver).ensureConfig\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/controller/nginxingress/config_driver.go:244\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/controller/nginxingress.(*ReconcileNginxIngress).syncConfig\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/controller/nginxingress/nginxingress_controller.go:248\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/controller/nginxingress.(*ReconcileNginxIngress).Reconcile\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/controller/nginxingress/nginxingress_controller.go:133\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:215\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:158\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:134\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/k8s.io/apimachinery/pkg/util/wait.Until\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88"}


{"level":"error","ts":1576737517.701055,"logger":"controller.nginxingress","msg":"Failed ensureConfig","name":"ccp-foxconn-cl01-ingress","namespace":"default","error":"remote client not ready: failed to build client to remote cluster","errorVerbose":"failed to build client to remote cluster\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/controller/nginxingress.init.ializers\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/controller/nginxingress/config_driver.go:73\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:188\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1337\nremote client not ready\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/controller/nginxingress.(*KubeClientDriver).buildManifestData.func1\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/controller/nginxingress/config_driver.go:122\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/controller/nginxingress.(*KubeClientDriver).buildManifestData\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/controller/nginxingress/config_driver.go:126\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/controller/nginxingress.(*KubeClientDriver).ensureConfig\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/controller/nginxingress/config_driver.go:244\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/controller/nginxingress.(*ReconcileNginxIngress).syncConfig\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/controller/nginxingress/nginxingress_controller.go:248\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/controller/nginxingress.(*ReconcileNginxIngress).Reconcile\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/controller/nginxingress/nginxingress_controller.go:133\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:215\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:158\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:134\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/k8s.io/apimachinery/pkg/util/wait.Until\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1337","stacktrace":"wwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/github.com/go-logr/zapr.(*zapLogger).Error\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/github.com/go-logr/zapr/zapr.go:128\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/controller/nginxingress.(*KubeClientDriver).ensureConfig\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/controller/nginxingress/config_driver.go:246\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/controller/nginxingress.(*ReconcileNginxIngress).syncConfig\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/controller/nginxingress/nginxingress_controller.go:248\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/controller/nginxingress.(*ReconcileNginxIngress).Reconcile\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/pkg/controller/nginxingress/nginxingress_controller.go:133\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:215\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:158\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/k8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:134\nwwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/k8s.io/apimachinery/pkg/util/wait.Until\n\t/go/src/wwwin-github.cisco.com/CPSG/ccp-net-tinker/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88"}

Conditions:
Create ACI-CNI tenant cluster
Bug details contain sensitive information and therefore require a Cisco.com account to be viewed.

Bug Details Include

  • Full Description (including symptoms, conditions and workarounds)
  • Status
  • Severity
  • Known Fixed Releases
  • Related Community Discussions
  • Number of Related Support Cases
Bug information is viewable for customers and partners who have a service contract. Registered users can view up to 200 bugs per month without a service contract.