github/kubernetes
1
0
Fork 0
mirror of https://github.com/k3s-io/kubernetes.git synced 2025-07-23 11:50:44 +00:00
Code Issues Projects Releases Wiki Activity

Added a buffer period in the node performance tests

Browse Source 
The node-kubelet-flaky e2e job that runs the the
`Node Performance Testing [Serial] [Slow] [Flaky]` e2e tests have been
flaking because of inconsistencies on the cpu manager checkpoint file.
This seems to be caused because the checkpoint file is deleted (which is
what needs to happen in order to change the CPU manager policy which is
used for these e2e tests) right after the e2e tests asserts that a pod
does not exist anymore.
However, after a pod is deleted, the CPU manager may still be cleaning
up the resources used by the pod which may result in the checkpoint file
being created.
Whenever this happened, the kubelet would panic if we then try to
subsequently change the CPU manager policy to "static" from "none" or
vice versa (this is done 4 times in these tests).

Signed-off-by: alejandrox1 <alarcj137@gmail.com>
This commit is contained in:
alejandrox1 2020-06-17 15:12:15 -04:00
parent 868efab6f5
commit 9263dd1f02
1 changed files with 9 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
test/e2e_node/node_perf_test.go
View File

@ -81,6 +81,15 @@ var _ = SIGDescribe("Node Performance Testing [Serial] [Slow] [Flaky]", func() {
GracePeriodSeconds: &gp,
}
f.PodClient().DeleteSync(pod.Name, delOpts, framework.DefaultPodDeletionTimeout)
// We are going to give some more time for the CPU manager to do any clean
// up it needs to do now that the pod has been deleted. Otherwise we may
// run into a data race condition in which the PostTestExec function
// deletes the CPU manager's checkpoint file while the CPU manager is still
// doing work and we end with a new checkpoint file after PosttestExec has
// finished. This issues would result in the kubelet panicking after we try
// and set the kubelet config.
time.Sleep(15 * time.Second)
ginkgo.By("running the post test exec from the workload")
err := wl.PostTestExec()
framework.ExpectNoError(err)