github/kubernetes
1
0
Fork 0
mirror of https://github.com/k3s-io/kubernetes.git synced 2025-07-30 06:54:01 +00:00
Code Issues Projects Releases Wiki Activity

Merge pull request #38203 from mwielgus/fix-labeling

Browse Source 
Automatic merge from submit-queue

Skip not registered nodes in labeling in CA e2e tests

This PR fixes problems with querying for not yet registered nodes. The underlying problem is related to the way the test is written. So we apply labels to the existing nodes, create pods that require N+1 nodes  with the labels and expect a new node to be added. But the new node is created without the labels. As soon as the node is spotted it is labeled. But sometimes it is too late. CA notices that the new node doesn't solve the problem and ask for another, hoping that this time it will get the node with the labels.  The node is added by MIG but it takes a minute or more for the node to start and register in kubernetes. At this moment the labeling is started. The list of nodes to be labeled is taken from MIG. The extra node is there. But it is not in kubernetes yet. So 404 error is returned on labeling attempt and test fails.

This PR filters the list of nodes to be labeled and applies the labels only on the fully registered nodes.

Fixes 404 in #33754

cc: @jszczepkowski @piosz @fgrzadkowski
This commit is contained in:
Kubernetes Submit Queue 2016-12-06 15:37:12 -08:00 committed by GitHub
commit b3ca990ce6
1 changed files with 13 additions and 7 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

20
test/e2e/cluster_size_autoscaling.go
View File

@ -237,13 +237,19 @@ var _ = framework.KubeDescribe("Cluster size autoscaling [Slow]", func() {
// However at this moment we DO WANT it to crash so that we don't check all test runs for the
// rare behavior, but only the broken ones.
}
defer removeLabels(newNodesSet)
By(fmt.Sprintf("Setting labels for new nodes: %v", newNodesSet.List()))
updateNodeLabels(c, newNodesSet, labels, nil)
framework.ExpectNoError(WaitForClusterSizeFunc(f.ClientSet,
func(size int) bool { return size >= nodeCount+1 }, scaleUpTimeout))
By(fmt.Sprintf("New nodes: %v\n", newNodesSet))
registeredNodes := sets.NewString()
for nodeName := range newNodesSet {
node, err := f.ClientSet.Core().Nodes().Get(nodeName)
if err == nil && node != nil {
registeredNodes.Insert(nodeName)
} else {
glog.Errorf("Failed to get node %v: %v", nodeName, err)
}
}
By(fmt.Sprintf("Setting labels for registered new nodes: %v", registeredNodes.List()))
updateNodeLabels(c, registeredNodes, labels, nil)
defer removeLabels(registeredNodes)
framework.ExpectNoError(waitForAllCaPodsReadyInNamespace(f, c))
framework.ExpectNoError(framework.DeleteRCAndPods(f.ClientSet, f.InternalClientset, f.Namespace.Name, "node-selector"))