Merge pull request #2453 from fidencio/wip/kata-deploy-use-stable-and-latest-tags

kata-deploy: Also provide "stable" & "latest" tags
This commit is contained in:
Samuel Ortiz
2021-09-23 13:54:01 +02:00
committed by GitHub
8 changed files with 260 additions and 47 deletions

View File

@@ -100,10 +100,14 @@ jobs:
run: |
# tag the container image we created and push to DockerHub
tag=$(echo $GITHUB_REF | cut -d/ -f3-)
docker tag katadocker/kata-deploy-ci:${{steps.build-and-push-kata-deploy-ci.outputs.PKG_SHA}} katadocker/kata-deploy:${tag}
docker tag quay.io/kata-containers/kata-deploy-ci:${{steps.build-and-push-kata-deploy-ci.outputs.PKG_SHA}} quay.io/kata-containers/kata-deploy:${tag}
docker push katadocker/kata-deploy:${tag}
docker push quay.io/kata-containers/kata-deploy:${tag}
tags=$(tag)
tags+=$([[ "$tag" =~ "alpha"|"rc" ]] && echo "latest" || echo "stable")
for tag in ${tags[@]}; do \
docker tag katadocker/kata-deploy-ci:${{steps.build-and-push-kata-deploy-ci.outputs.PKG_SHA}} katadocker/kata-deploy:${tag} && \
docker tag quay.io/kata-containers/kata-deploy-ci:${{steps.build-and-push-kata-deploy-ci.outputs.PKG_SHA}} quay.io/kata-containers/kata-deploy:${tag} && \
docker push katadocker/kata-deploy:${tag} && \
docker push quay.io/kata-containers/kata-deploy:${tag}; \
done
upload-static-tarball:
needs: kata-deploy

View File

@@ -11,19 +11,40 @@ a node only if it uses either containerd or CRI-O CRI-shims.
### Install Kata on a running Kubernetes cluster
#### Installing the latest image
The latest image refers to pre-release and release candidate content. For stable releases, please, use the "stable" instructions.
```sh
$ cd $GOPATH/src/github.com/kata-containers/kata-containers/tools/packaging/kata-deploy
$ kubectl apply -f kata-rbac/base/kata-rbac.yaml
$ kubectl apply -f kata-deploy/base/kata-deploy.yaml
$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-rbac/base/kata-rbac.yaml
$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-deploy/base/kata-deploy.yaml
```
or on a [k3s](https://k3s.io/) cluster:
#### Installing the stable image
The stable image refers to the last stable releases content.
Note that if you use a tagged version of the repo, the stable image does match that version.
For instance, if you use the 2.2.1 tagged version of the kata-deploy.yaml file, then the version 2.2.1 of the kata runtime will be deployed.
```sh
$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-rbac/base/kata-rbac.yaml
$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-deploy/base/kata-deploy-stable.yaml
```
#### For your [k3s](https://k3s.io/) cluster, do:
```sh
$ GO111MODULE=auto go get github.com/kata-containers/kata-containers
$ cd $GOPATH/src/github.com/kata-containers/kata-containers/tools/packaging/kata-deploy
$ kubectl apply -k kata-deploy/overlays/k3s
```
#### Ensure kata-deploy is ready
```sh
kubectl -n kube-system wait --timeout=10m --for=condition=Ready -l name=kata-deploy pod
```
### Run a sample workload
Workloads specify the runtime they'd like to utilize by setting the appropriate `runtimeClass` object within
@@ -32,8 +53,7 @@ which will ensure the workload is only scheduled on a node that has Kata Contain
`runtimeClass` is a built-in type in Kubernetes. To apply each Kata Containers `runtimeClass`:
```sh
$ cd $GOPATH/src/github.com/kata-containers/kata-containers/tools/packaging/kata-deploy/runtimeclasses
$ kubectl apply -f kata-runtimeClasses.yaml
$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/runtimeclasses/kata-runtimeClasses.yaml
```
The following YAML snippet shows how to specify a workload should use Kata with Cloud Hypervisor:
@@ -66,42 +86,74 @@ spec:
To run an example with `kata-clh`:
```sh
$ cd $GOPATH/src/github.com/kata-containers/kata-containers/tools/packaging/kata-deploy/examples
$ kubectl apply -f test-deploy-kata-clh.yaml
$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/examples/test-deploy-kata-clh.yaml
```
To run an example with `kata-fc`:
```sh
$ cd $GOPATH/src/github.com/kata-containers/kata-containers/tools/packaging/kata-deploy/examples
$ kubectl apply -f test-deploy-kata-fc.yaml
$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/examples/test-deploy-kata-fc.yaml
```
To run an example with `kata-qemu`:
```sh
$ cd $GOPATH/src/github.com/kata-containers/kata-containers/tools/packaging/kata-deploy/examples
$ kubectl apply -f test-deploy-kata-qemu.yaml
$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/examples/test-deploy-kata-qemu.yaml
```
The following removes the test pods:
```sh
$ cd $GOPATH/src/github.com/kata-containers/kata-containers/tools/packaging/kata-deploy/examples
$ kubectl delete -f test-deploy-kata-clh.yaml
$ kubectl delete -f test-deploy-kata-fc.yaml
$ kubectl delete -f test-deploy-kata-qemu.yaml
$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/examples/test-deploy-kata-clh.yaml
$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/examples/test-deploy-kata-fc.yaml
$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/examples/test-deploy-kata-qemu.yaml
```
### Remove Kata from the Kubernetes cluster
#### Removing the latest image
```sh
$ cd $GOPATH/src/github.com/kata-containers/kata-containers/tools/packaging/kata-deploy
$ kubectl delete -f kata-deploy/base/kata-deploy.yaml
$ kubectl apply -f kata-cleanup/base/kata-cleanup.yaml
$ kubectl delete -f kata-cleanup/base/kata-cleanup.yaml
$ kubectl delete -f kata-rbac/base/kata-rbac.yaml
$ kubectl delete -f runtimeclasses/kata-runtimeClasses.yaml
$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-deploy/base/kata-deploy.yaml
$ kubectl -n kube-system wait --timeout=10m --for=delete -l name=kata-deploy pod
```
After ensuring kata-deploy has been deleted, cleanup the cluster:
```sh
$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-cleanup/base/kata-cleanup.yaml
```
The cleanup daemon-set will run a single time, cleaning up the node-label, which makes it difficult to check in an automated fashion.
This process should take, at most, 5 minutes.
After that, let's delete the cleanup daemon-set, the added RBAC and runtime classes:
```sh
$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-cleanup/base/kata-cleanup.yaml
$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-rbac/base/kata-rbac.yaml
$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/runtimeclasses/kata-runtimeClasses.yaml
```
#### Removing the stable image
```sh
$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-deploy/base/kata-deploy-stable.yaml
$ kubectl -n kube-system wait --timeout=10m --for=delete -l name=kata-deploy pod
```
After ensuring kata-deploy has been deleted, cleanup the cluster:
```sh
$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-cleanup/base/kata-cleanup-stabe.yaml
```
The cleanup daemon-set will run a single time, cleaning up the node-label, which makes it difficult to check in an automated fashion.
This process should take, at most, 5 minutes.
After that, let's delete the cleanup daemon-set, the added RBAC and runtime classes:
```sh
$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-cleanup/base/kata-cleanup-stable.yaml
$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-rbac/base/kata-rbac.yaml
$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/runtimeclasses/kata-runtimeClasses.yaml
```
## `kata-deploy` details

View File

@@ -0,0 +1,46 @@
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: kubelet-kata-cleanup
namespace: kube-system
spec:
selector:
matchLabels:
name: kubelet-kata-cleanup
template:
metadata:
labels:
name: kubelet-kata-cleanup
spec:
serviceAccountName: kata-label-node
nodeSelector:
katacontainers.io/kata-runtime: cleanup
containers:
- name: kube-kata-cleanup
image: quay.io/kata-containers/kata-deploy:stable
imagePullPolicy: Always
command: [ "bash", "-c", "/opt/kata-artifacts/scripts/kata-deploy.sh reset" ]
env:
- name: NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
securityContext:
privileged: false
volumeMounts:
- name: dbus
mountPath: /var/run/dbus
- name: systemd
mountPath: /run/systemd
volumes:
- name: dbus
hostPath:
path: /var/run/dbus
- name: systemd
hostPath:
path: /run/systemd
updateStrategy:
rollingUpdate:
maxUnavailable: 1
type: RollingUpdate

View File

@@ -18,7 +18,7 @@ spec:
katacontainers.io/kata-runtime: cleanup
containers:
- name: kube-kata-cleanup
image: quay.io/kata-containers/kata-deploy:2.3.0-alpha0
image: quay.io/kata-containers/kata-deploy:latest
imagePullPolicy: Always
command: [ "bash", "-c", "/opt/kata-artifacts/scripts/kata-deploy.sh reset" ]
env:

View File

@@ -0,0 +1,69 @@
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: kata-deploy
namespace: kube-system
spec:
selector:
matchLabels:
name: kata-deploy
template:
metadata:
labels:
name: kata-deploy
spec:
serviceAccountName: kata-label-node
containers:
- name: kube-kata
image: quay.io/kata-containers/kata-deploy:stable
imagePullPolicy: Always
lifecycle:
preStop:
exec:
command: ["bash", "-c", "/opt/kata-artifacts/scripts/kata-deploy.sh cleanup"]
command: [ "bash", "-c", "/opt/kata-artifacts/scripts/kata-deploy.sh install" ]
env:
- name: NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
securityContext:
privileged: false
volumeMounts:
- name: crio-conf
mountPath: /etc/crio/
- name: containerd-conf
mountPath: /etc/containerd/
- name: kata-artifacts
mountPath: /opt/kata/
- name: dbus
mountPath: /var/run/dbus
- name: systemd
mountPath: /run/systemd
- name: local-bin
mountPath: /usr/local/bin/
volumes:
- name: crio-conf
hostPath:
path: /etc/crio/
- name: containerd-conf
hostPath:
path: /etc/containerd/
- name: kata-artifacts
hostPath:
path: /opt/kata/
type: DirectoryOrCreate
- name: dbus
hostPath:
path: /var/run/dbus
- name: systemd
hostPath:
path: /run/systemd
- name: local-bin
hostPath:
path: /usr/local/bin/
updateStrategy:
rollingUpdate:
maxUnavailable: 1
type: RollingUpdate

View File

@@ -16,7 +16,7 @@ spec:
serviceAccountName: kata-label-node
containers:
- name: kube-kata
image: quay.io/kata-containers/kata-deploy:2.3.0-alpha0
image: quay.io/kata-containers/kata-deploy:latest
imagePullPolicy: Always
lifecycle:
preStop:

View File

@@ -1,19 +1,6 @@
---
kind: RuntimeClass
apiVersion: node.k8s.io/v1beta1
metadata:
name: kata-qemu-virtiofs
handler: kata-qemu-virtiofs
overhead:
podFixed:
memory: "160Mi"
cpu: "250m"
scheduling:
nodeSelector:
katacontainers.io/kata-runtime: "true"
---
kind: RuntimeClass
apiVersion: node.k8s.io/v1beta1
metadata:
name: kata-qemu
handler: kata-qemu

View File

@@ -111,13 +111,68 @@ bump_repo() {
fi
if [ "${repo}" == "kata-containers" ]; then
info "Updating kata-deploy / kata-cleanup image tags"
sed -i "s#quay.io/kata-containers/kata-deploy:${current_version}#quay.io/kata-containers/kata-deploy:${new_version}#g" tools/packaging/kata-deploy/kata-deploy/base/kata-deploy.yaml
sed -i "s#quay.io/kata-containers/kata-deploy:${current_version}#quay.io/kata-containers/kata-deploy:${new_version}#g" tools/packaging/kata-deploy/kata-cleanup/base/kata-cleanup.yaml
git diff
# Here there are 3 scenarios of what we can do, based on
# which branch we're targetting:
#
# 1) [main] ------> [main] NO-OP
# "alpha0" "alpha1"
#
# +----------------+----------------+
# | from | to |
# -----------------+----------------+----------------+
# kata-deploy | "latest" | "latest" |
# -----------------+----------------+----------------+
# kata-deploy-base | "stable | "stable" |
# -----------------+----------------+----------------+
#
#
# 2) [main] ------> [stable] Update kata-deploy and
# "alpha2" "rc0" get rid of kata-deploy-base
#
# +----------------+----------------+
# | from | to |
# -----------------+----------------+----------------+
# kata-deploy | "latest" | "rc0" |
# -----------------+----------------+----------------+
# kata-deploy-base | "stable" | REMOVED |
# -----------------+----------------+----------------+
#
#
# 3) [stable] ------> [stable] Update kata-deploy
# "x.y.z" "x.y.(z+1)"
#
# +----------------+----------------+
# | from | to |
# -----------------+----------------+----------------+
# kata-deploy | "x.y.z" | "x.y.(z+1)" |
# -----------------+----------------+----------------+
# kata-deploy-base | NON-EXISTENT | NON-EXISTENT |
# -----------------+----------------+----------------+
git add tools/packaging/kata-deploy/kata-deploy/base/kata-deploy.yaml
git add tools/packaging/kata-deploy/kata-cleanup/base/kata-cleanup.yaml
info "Updating kata-deploy / kata-cleanup image tags"
if [ "${target_branch}" == "main" ] && [[ "${new_version}" =~ "rc" ]]; then
# case 2)
## change the "latest" tag to the "#{new_version}" one
sed -i "s#quay.io/kata-containers/kata-deploy:latest#quay.io/kata-containers/kata-deploy:${new_version}#g" tools/packaging/kata-deploy/kata-deploy/base/kata-deploy.yaml
sed -i "s#quay.io/kata-containers/kata-deploy:latest#quay.io/kata-containers/kata-deploy:${new_version}#g" tools/packaging/kata-deploy/kata-cleanup/base/kata-cleanup.yaml
git diff
git add tools/packaging/kata-deploy/kata-deploy/base/kata-deploy.yaml
git add tools/packaging/kata-deploy/kata-cleanup/base/kata-cleanup.yaml
## and remove the kata-deploy & kata-cleanup stable yaml files
git rm tools/packaging/kata-deploy/kata-deploy/base/kata-deploy-stable.yaml
git rm tools/packaging/kata-deploy/kata-cleanup/base/kata-cleanup-stable.yaml
elif [[ "${target_branch}" =~ "stable" ]]; then
# case 3)
sed -i "s#quay.io/kata-containers/kata-deploy:${current_version}#quay.io/kata-containers/kata-deploy:${new_version}#g" tools/packaging/kata-deploy/kata-deploy/base/kata-deploy.yaml
sed -i "s#quay.io/kata-containers/kata-deploy:${current_version}#quay.io/kata-containers/kata-deploy:${new_version}#g" tools/packaging/kata-deploy/kata-cleanup/base/kata-cleanup.yaml
git diff
git add tools/packaging/kata-deploy/kata-deploy/base/kata-deploy.yaml
git add tools/packaging/kata-deploy/kata-cleanup/base/kata-cleanup.yaml
fi
fi
info "Creating PR message"