Open
0 of 1 issue completedDescription
Kuma Version
0.0.0-preview.v7bf53c40b
Describe the bug
When deleting a namespace with an injected mesh, the system continues to process resources even though Kubernetes services and other objects may already be deleted. This results in unnecessary errors and processing. In one case, deleting a namespace with 800 services (2 replicas each) generated 125,000 log lines.
Two types of repeated errors occur:
-
Pod-to-dataplane conversion errors due to missing services:
ERROR discovery.k8s.pod-to-dataplane-converter could not get K8S Service for service tag {"error": "failed to get Service \"kuma-test/fake-service-799\": Service \"fake-service-799\" not found"}
-
DataplaneInsight errors due to namespace termination:
ERROR xds.sink failed to flush DataplaneInsight {"dataplaneid": {"Mesh":"default","Name":"fake-service-736-6b479fb987-zjsxr.kuma-test"}, "error": "failed to create k8s resource: dataplaneinsights.kuma.io \"fake-service-736-6b479fb987-zjsxr\" is forbidden: unable to create new content in namespace kuma-test because it is being terminated"}
To Reproduce
- Create a namespace with an injected mesh.
- Deploy ~1000 services with 2 replicas each.
- Delete the namespace.
- Observe logs attempting to convert pods to dataplanes and flush DataplaneInsights while the namespace is terminating.
Expected behavior
The system should detect namespace termination and avoid processing resources that may already be deleted.
Additional context (optional)
The system should recognize when a namespace is terminating and avoid unnecessary operations like pod-to-dataplane conversion and DataplaneInsight
updates.
Activity