diff --git a/.github/workflows/release.yaml b/.github/workflows/release.yaml index 16799bda4e..418e74a9d8 100644 --- a/.github/workflows/release.yaml +++ b/.github/workflows/release.yaml @@ -100,10 +100,14 @@ jobs: run: | # tag the container image we created and push to DockerHub tag=$(echo $GITHUB_REF | cut -d/ -f3-) - docker tag katadocker/kata-deploy-ci:${{steps.build-and-push-kata-deploy-ci.outputs.PKG_SHA}} katadocker/kata-deploy:${tag} - docker tag quay.io/kata-containers/kata-deploy-ci:${{steps.build-and-push-kata-deploy-ci.outputs.PKG_SHA}} quay.io/kata-containers/kata-deploy:${tag} - docker push katadocker/kata-deploy:${tag} - docker push quay.io/kata-containers/kata-deploy:${tag} + tags=$(tag) + tags+=$([[ "$tag" =~ "alpha"|"rc" ]] && echo "latest" || echo "stable") + for tag in ${tags[@]}; do \ + docker tag katadocker/kata-deploy-ci:${{steps.build-and-push-kata-deploy-ci.outputs.PKG_SHA}} katadocker/kata-deploy:${tag} && \ + docker tag quay.io/kata-containers/kata-deploy-ci:${{steps.build-and-push-kata-deploy-ci.outputs.PKG_SHA}} quay.io/kata-containers/kata-deploy:${tag} && \ + docker push katadocker/kata-deploy:${tag} && \ + docker push quay.io/kata-containers/kata-deploy:${tag}; \ + done upload-static-tarball: needs: kata-deploy diff --git a/tools/packaging/kata-deploy/README.md b/tools/packaging/kata-deploy/README.md index e33fab314f..9f54f0a728 100644 --- a/tools/packaging/kata-deploy/README.md +++ b/tools/packaging/kata-deploy/README.md @@ -11,19 +11,40 @@ a node only if it uses either containerd or CRI-O CRI-shims. ### Install Kata on a running Kubernetes cluster +#### Installing the latest image + +The latest image refers to pre-release and release candidate content. For stable releases, please, use the "stable" instructions. + ```sh -$ cd $GOPATH/src/github.com/kata-containers/kata-containers/tools/packaging/kata-deploy -$ kubectl apply -f kata-rbac/base/kata-rbac.yaml -$ kubectl apply -f kata-deploy/base/kata-deploy.yaml +$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-rbac/base/kata-rbac.yaml +$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-deploy/base/kata-deploy.yaml ``` -or on a [k3s](https://k3s.io/) cluster: +#### Installing the stable image + +The stable image refers to the last stable releases content. + +Note that if you use a tagged version of the repo, the stable image does match that version. +For instance, if you use the 2.2.1 tagged version of the kata-deploy.yaml file, then the version 2.2.1 of the kata runtime will be deployed. ```sh +$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-rbac/base/kata-rbac.yaml +$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-deploy/base/kata-deploy-stable.yaml +``` + +#### For your [k3s](https://k3s.io/) cluster, do: + +```sh +$ GO111MODULE=auto go get github.com/kata-containers/kata-containers $ cd $GOPATH/src/github.com/kata-containers/kata-containers/tools/packaging/kata-deploy $ kubectl apply -k kata-deploy/overlays/k3s ``` +#### Ensure kata-deploy is ready +```sh +kubectl -n kube-system wait --timeout=10m --for=condition=Ready -l name=kata-deploy pod +``` + ### Run a sample workload Workloads specify the runtime they'd like to utilize by setting the appropriate `runtimeClass` object within @@ -32,8 +53,7 @@ which will ensure the workload is only scheduled on a node that has Kata Contain `runtimeClass` is a built-in type in Kubernetes. To apply each Kata Containers `runtimeClass`: ```sh - $ cd $GOPATH/src/github.com/kata-containers/kata-containers/tools/packaging/kata-deploy/runtimeclasses - $ kubectl apply -f kata-runtimeClasses.yaml + $ kubectl apply -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/runtimeclasses/kata-runtimeClasses.yaml ``` The following YAML snippet shows how to specify a workload should use Kata with Cloud Hypervisor: @@ -66,42 +86,74 @@ spec: To run an example with `kata-clh`: ```sh -$ cd $GOPATH/src/github.com/kata-containers/kata-containers/tools/packaging/kata-deploy/examples -$ kubectl apply -f test-deploy-kata-clh.yaml +$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/examples/test-deploy-kata-clh.yaml ``` To run an example with `kata-fc`: ```sh -$ cd $GOPATH/src/github.com/kata-containers/kata-containers/tools/packaging/kata-deploy/examples -$ kubectl apply -f test-deploy-kata-fc.yaml +$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/examples/test-deploy-kata-fc.yaml ``` To run an example with `kata-qemu`: ```sh -$ cd $GOPATH/src/github.com/kata-containers/kata-containers/tools/packaging/kata-deploy/examples -$ kubectl apply -f test-deploy-kata-qemu.yaml +$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/examples/test-deploy-kata-qemu.yaml ``` The following removes the test pods: ```sh -$ cd $GOPATH/src/github.com/kata-containers/kata-containers/tools/packaging/kata-deploy/examples -$ kubectl delete -f test-deploy-kata-clh.yaml -$ kubectl delete -f test-deploy-kata-fc.yaml -$ kubectl delete -f test-deploy-kata-qemu.yaml +$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/examples/test-deploy-kata-clh.yaml +$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/examples/test-deploy-kata-fc.yaml +$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/examples/test-deploy-kata-qemu.yaml ``` ### Remove Kata from the Kubernetes cluster +#### Removing the latest image + ```sh -$ cd $GOPATH/src/github.com/kata-containers/kata-containers/tools/packaging/kata-deploy -$ kubectl delete -f kata-deploy/base/kata-deploy.yaml -$ kubectl apply -f kata-cleanup/base/kata-cleanup.yaml -$ kubectl delete -f kata-cleanup/base/kata-cleanup.yaml -$ kubectl delete -f kata-rbac/base/kata-rbac.yaml -$ kubectl delete -f runtimeclasses/kata-runtimeClasses.yaml +$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-deploy/base/kata-deploy.yaml +$ kubectl -n kube-system wait --timeout=10m --for=delete -l name=kata-deploy pod +``` + +After ensuring kata-deploy has been deleted, cleanup the cluster: +```sh +$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-cleanup/base/kata-cleanup.yaml +``` + +The cleanup daemon-set will run a single time, cleaning up the node-label, which makes it difficult to check in an automated fashion. +This process should take, at most, 5 minutes. + +After that, let's delete the cleanup daemon-set, the added RBAC and runtime classes: + +```sh +$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-cleanup/base/kata-cleanup.yaml +$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-rbac/base/kata-rbac.yaml +$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/runtimeclasses/kata-runtimeClasses.yaml +``` + +#### Removing the stable image + +```sh +$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-deploy/base/kata-deploy-stable.yaml +$ kubectl -n kube-system wait --timeout=10m --for=delete -l name=kata-deploy pod +``` + +After ensuring kata-deploy has been deleted, cleanup the cluster: +```sh +$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-cleanup/base/kata-cleanup-stabe.yaml +``` + +The cleanup daemon-set will run a single time, cleaning up the node-label, which makes it difficult to check in an automated fashion. +This process should take, at most, 5 minutes. + +After that, let's delete the cleanup daemon-set, the added RBAC and runtime classes: +```sh +$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-cleanup/base/kata-cleanup-stable.yaml +$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/kata-rbac/base/kata-rbac.yaml +$ kubectl delete -f https://raw.githubusercontent.com/kata-containers/kata-containers/main/tools/packaging/kata-deploy/runtimeclasses/kata-runtimeClasses.yaml ``` ## `kata-deploy` details diff --git a/tools/packaging/kata-deploy/kata-cleanup/base/kata-cleanup-stable.yaml b/tools/packaging/kata-deploy/kata-cleanup/base/kata-cleanup-stable.yaml new file mode 100644 index 0000000000..f1d9d0a2f9 --- /dev/null +++ b/tools/packaging/kata-deploy/kata-cleanup/base/kata-cleanup-stable.yaml @@ -0,0 +1,46 @@ +--- +apiVersion: apps/v1 +kind: DaemonSet +metadata: + name: kubelet-kata-cleanup + namespace: kube-system +spec: + selector: + matchLabels: + name: kubelet-kata-cleanup + template: + metadata: + labels: + name: kubelet-kata-cleanup + spec: + serviceAccountName: kata-label-node + nodeSelector: + katacontainers.io/kata-runtime: cleanup + containers: + - name: kube-kata-cleanup + image: quay.io/kata-containers/kata-deploy:stable + imagePullPolicy: Always + command: [ "bash", "-c", "/opt/kata-artifacts/scripts/kata-deploy.sh reset" ] + env: + - name: NODE_NAME + valueFrom: + fieldRef: + fieldPath: spec.nodeName + securityContext: + privileged: false + volumeMounts: + - name: dbus + mountPath: /var/run/dbus + - name: systemd + mountPath: /run/systemd + volumes: + - name: dbus + hostPath: + path: /var/run/dbus + - name: systemd + hostPath: + path: /run/systemd + updateStrategy: + rollingUpdate: + maxUnavailable: 1 + type: RollingUpdate diff --git a/tools/packaging/kata-deploy/kata-cleanup/base/kata-cleanup.yaml b/tools/packaging/kata-deploy/kata-cleanup/base/kata-cleanup.yaml index d4aa6d8f23..851e958a4c 100644 --- a/tools/packaging/kata-deploy/kata-cleanup/base/kata-cleanup.yaml +++ b/tools/packaging/kata-deploy/kata-cleanup/base/kata-cleanup.yaml @@ -18,7 +18,7 @@ spec: katacontainers.io/kata-runtime: cleanup containers: - name: kube-kata-cleanup - image: quay.io/kata-containers/kata-deploy:2.3.0-alpha0 + image: quay.io/kata-containers/kata-deploy:latest imagePullPolicy: Always command: [ "bash", "-c", "/opt/kata-artifacts/scripts/kata-deploy.sh reset" ] env: diff --git a/tools/packaging/kata-deploy/kata-deploy/base/kata-deploy-stable.yaml b/tools/packaging/kata-deploy/kata-deploy/base/kata-deploy-stable.yaml new file mode 100644 index 0000000000..346e4c0ee2 --- /dev/null +++ b/tools/packaging/kata-deploy/kata-deploy/base/kata-deploy-stable.yaml @@ -0,0 +1,69 @@ +--- +apiVersion: apps/v1 +kind: DaemonSet +metadata: + name: kata-deploy + namespace: kube-system +spec: + selector: + matchLabels: + name: kata-deploy + template: + metadata: + labels: + name: kata-deploy + spec: + serviceAccountName: kata-label-node + containers: + - name: kube-kata + image: quay.io/kata-containers/kata-deploy:stable + imagePullPolicy: Always + lifecycle: + preStop: + exec: + command: ["bash", "-c", "/opt/kata-artifacts/scripts/kata-deploy.sh cleanup"] + command: [ "bash", "-c", "/opt/kata-artifacts/scripts/kata-deploy.sh install" ] + env: + - name: NODE_NAME + valueFrom: + fieldRef: + fieldPath: spec.nodeName + securityContext: + privileged: false + volumeMounts: + - name: crio-conf + mountPath: /etc/crio/ + - name: containerd-conf + mountPath: /etc/containerd/ + - name: kata-artifacts + mountPath: /opt/kata/ + - name: dbus + mountPath: /var/run/dbus + - name: systemd + mountPath: /run/systemd + - name: local-bin + mountPath: /usr/local/bin/ + volumes: + - name: crio-conf + hostPath: + path: /etc/crio/ + - name: containerd-conf + hostPath: + path: /etc/containerd/ + - name: kata-artifacts + hostPath: + path: /opt/kata/ + type: DirectoryOrCreate + - name: dbus + hostPath: + path: /var/run/dbus + - name: systemd + hostPath: + path: /run/systemd + - name: local-bin + hostPath: + path: /usr/local/bin/ + updateStrategy: + rollingUpdate: + maxUnavailable: 1 + type: RollingUpdate diff --git a/tools/packaging/kata-deploy/kata-deploy/base/kata-deploy.yaml b/tools/packaging/kata-deploy/kata-deploy/base/kata-deploy.yaml index 7a396f0e4d..a03a56b84e 100644 --- a/tools/packaging/kata-deploy/kata-deploy/base/kata-deploy.yaml +++ b/tools/packaging/kata-deploy/kata-deploy/base/kata-deploy.yaml @@ -16,7 +16,7 @@ spec: serviceAccountName: kata-label-node containers: - name: kube-kata - image: quay.io/kata-containers/kata-deploy:2.3.0-alpha0 + image: quay.io/kata-containers/kata-deploy:latest imagePullPolicy: Always lifecycle: preStop: diff --git a/tools/packaging/kata-deploy/runtimeclasses/kata-runtimeClasses.yaml b/tools/packaging/kata-deploy/runtimeclasses/kata-runtimeClasses.yaml index fd8bc858be..9695ba6572 100644 --- a/tools/packaging/kata-deploy/runtimeclasses/kata-runtimeClasses.yaml +++ b/tools/packaging/kata-deploy/runtimeclasses/kata-runtimeClasses.yaml @@ -1,19 +1,6 @@ --- kind: RuntimeClass apiVersion: node.k8s.io/v1beta1 -metadata: - name: kata-qemu-virtiofs -handler: kata-qemu-virtiofs -overhead: - podFixed: - memory: "160Mi" - cpu: "250m" -scheduling: - nodeSelector: - katacontainers.io/kata-runtime: "true" ---- -kind: RuntimeClass -apiVersion: node.k8s.io/v1beta1 metadata: name: kata-qemu handler: kata-qemu diff --git a/tools/packaging/release/update-repository-version.sh b/tools/packaging/release/update-repository-version.sh index d00c623a03..cded448cc0 100755 --- a/tools/packaging/release/update-repository-version.sh +++ b/tools/packaging/release/update-repository-version.sh @@ -111,13 +111,68 @@ bump_repo() { fi if [ "${repo}" == "kata-containers" ]; then - info "Updating kata-deploy / kata-cleanup image tags" - sed -i "s#quay.io/kata-containers/kata-deploy:${current_version}#quay.io/kata-containers/kata-deploy:${new_version}#g" tools/packaging/kata-deploy/kata-deploy/base/kata-deploy.yaml - sed -i "s#quay.io/kata-containers/kata-deploy:${current_version}#quay.io/kata-containers/kata-deploy:${new_version}#g" tools/packaging/kata-deploy/kata-cleanup/base/kata-cleanup.yaml - git diff + # Here there are 3 scenarios of what we can do, based on + # which branch we're targetting: + # + # 1) [main] ------> [main] NO-OP + # "alpha0" "alpha1" + # + # +----------------+----------------+ + # | from | to | + # -----------------+----------------+----------------+ + # kata-deploy | "latest" | "latest" | + # -----------------+----------------+----------------+ + # kata-deploy-base | "stable | "stable" | + # -----------------+----------------+----------------+ + # + # + # 2) [main] ------> [stable] Update kata-deploy and + # "alpha2" "rc0" get rid of kata-deploy-base + # + # +----------------+----------------+ + # | from | to | + # -----------------+----------------+----------------+ + # kata-deploy | "latest" | "rc0" | + # -----------------+----------------+----------------+ + # kata-deploy-base | "stable" | REMOVED | + # -----------------+----------------+----------------+ + # + # + # 3) [stable] ------> [stable] Update kata-deploy + # "x.y.z" "x.y.(z+1)" + # + # +----------------+----------------+ + # | from | to | + # -----------------+----------------+----------------+ + # kata-deploy | "x.y.z" | "x.y.(z+1)" | + # -----------------+----------------+----------------+ + # kata-deploy-base | NON-EXISTENT | NON-EXISTENT | + # -----------------+----------------+----------------+ - git add tools/packaging/kata-deploy/kata-deploy/base/kata-deploy.yaml - git add tools/packaging/kata-deploy/kata-cleanup/base/kata-cleanup.yaml + info "Updating kata-deploy / kata-cleanup image tags" + if [ "${target_branch}" == "main" ] && [[ "${new_version}" =~ "rc" ]]; then + # case 2) + ## change the "latest" tag to the "#{new_version}" one + sed -i "s#quay.io/kata-containers/kata-deploy:latest#quay.io/kata-containers/kata-deploy:${new_version}#g" tools/packaging/kata-deploy/kata-deploy/base/kata-deploy.yaml + sed -i "s#quay.io/kata-containers/kata-deploy:latest#quay.io/kata-containers/kata-deploy:${new_version}#g" tools/packaging/kata-deploy/kata-cleanup/base/kata-cleanup.yaml + + git diff + + git add tools/packaging/kata-deploy/kata-deploy/base/kata-deploy.yaml + git add tools/packaging/kata-deploy/kata-cleanup/base/kata-cleanup.yaml + + ## and remove the kata-deploy & kata-cleanup stable yaml files + git rm tools/packaging/kata-deploy/kata-deploy/base/kata-deploy-stable.yaml + git rm tools/packaging/kata-deploy/kata-cleanup/base/kata-cleanup-stable.yaml + elif [[ "${target_branch}" =~ "stable" ]]; then + # case 3) + sed -i "s#quay.io/kata-containers/kata-deploy:${current_version}#quay.io/kata-containers/kata-deploy:${new_version}#g" tools/packaging/kata-deploy/kata-deploy/base/kata-deploy.yaml + sed -i "s#quay.io/kata-containers/kata-deploy:${current_version}#quay.io/kata-containers/kata-deploy:${new_version}#g" tools/packaging/kata-deploy/kata-cleanup/base/kata-cleanup.yaml + git diff + + git add tools/packaging/kata-deploy/kata-deploy/base/kata-deploy.yaml + git add tools/packaging/kata-deploy/kata-cleanup/base/kata-cleanup.yaml + fi fi info "Creating PR message"