Merge pull request #64294 from vishh/shutdown-script

Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Adding a shutdown script that would enable handling preemptible VM terminations gracefully in GCP environment

This PR adds a shutdown script to COS nodes in GCP k8s clusters that will make preemptible nodes sleep for however long they can between the time they receive an ACPI shutdown request and get's terminated.
https://cloud.google.com/compute/docs/instances/preemptible#preemption_process

This will then allow for catching termination signals via GCE metadata APIs and gracefully evict pods in k8s.

xref https://github.com/kubernetes/release/pull/560/
This commit is contained in:
Kubernetes Submit Queue 2018-05-25 22:33:33 -07:00 committed by GitHub
commit 930b3939f1
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
3 changed files with 25 additions and 0 deletions

View File

@ -28,6 +28,7 @@ release_filegroup(
"configure.sh",
"master.yaml",
"node.yaml",
"shutdown.sh",
],
visibility = ["//visibility:public"],
)

View File

@ -28,6 +28,7 @@ function get-node-instance-metadata {
metadata+="gci-update-strategy=${KUBE_TEMP}/gci-update.txt,"
metadata+="gci-ensure-gke-docker=${KUBE_TEMP}/gci-ensure-gke-docker.txt,"
metadata+="gci-docker-version=${KUBE_TEMP}/gci-docker-version.txt,"
metadata+="shutdown-script=${KUBE_ROOT}/cluster/gce/gci/shutdown.sh,"
metadata+="${NODE_EXTRA_METADATA}"
echo "${metadata}"
}

23
cluster/gce/gci/shutdown.sh Executable file
View File

@ -0,0 +1,23 @@
#!/usr/bin/env bash
# Copyright 2016 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# A script that let's gci preemptible nodes gracefully terminate in the event of a VM shutdown.
preemptible=$(curl "http://metadata.google.internal/computeMetadata/v1/instance/scheduling/preemptible" -H "Metadata-Flavor: Google")
if [ ${preemptible} == "TRUE" ]; then
echo "Shutting down! Sleeping for a minute to let the node gracefully terminate"
# https://cloud.google.com/compute/docs/instances/stopping-or-deleting-an-instance#delete_timeout
sleep 30
fi