diff --git a/kata-deploy/README.md b/kata-deploy/README.md index 52df13e67b..ebe2292d43 100644 --- a/kata-deploy/README.md +++ b/kata-deploy/README.md @@ -78,49 +78,44 @@ kubectl apply -f kata-deploy.yaml ### Run a sample workload -Untrusted workloads can node-select based on ```kata-containers.io/kata-runtime=true```, and are -run through ```kata-containers.io/kata-runtime``` if they are marked with the appropriate CRIO or containerd -annotation: -``` -CRIO: io.kubernetes.cri-o.TrustedSandbox: "false" -containerd: io.kubernetes.cri.untrusted-workload: "true" -``` -The following is a sample workload for running untrusted on a kata-enabled node: +Workloads which utilize Kata can node-select based on ```kata-containers.io/kata-runtime=true```, and are +run through an applicable runtime if they are marked with the appropriate runtimeClass annotation. + + +The following YAML snippet shows how to specify a workload should use Kata with QEMU: ``` -apiVersion: v1 -kind: Pod -metadata: - name: nginx - annotations: - io.kubernetes.cri-o.TrustedSandbox: "false" - io.kubernetes.cri.untrusted-workload: "true" - labels: - env: test spec: - containers: - - name: nginx - image: nginx - imagePullPolicy: IfNotPresent - nodeSelector: - kata-containers.io/kata-runtime: "true" -``` - -To run: -``` -kubectl apply -f examples/nginx-untrusted.yaml + template: + spec: + runtimeClassName: kata-qemu ``` -Now, you should see the pod start. You can verify that the pod is making use of -```kata-containers.io/kata-runtime``` by comparing the container ID observed with the following: +The following YAML snippet shows how to specify a workload should use Kata with Firecracker: ``` -/opt/kata/bin/kata-containers.io/kata-runtime list -kubectl describe pod nginx-untrusted +spec: + template: + spec: + runtimeClassName: kata-fc ``` -The following removes the test pod: + +To run an example with kata-qemu: + ``` -kubectl delete -f examples/nginx-untrusted.yaml +kubectl apply -f https://raw.githubusercontent.com/kata-containers/packaging/master/kata-deploy/examples/test-deploy-kata-qemu.yaml +``` + +To run an example with kata-fc: + +``` +kubectl apply -f https://raw.githubusercontent.com/kata-containers/packaging/master/kata-deploy/examples/test-deploy-kata-fc.yaml +``` + +The following removes the test pods: +``` +kubectl delete -f https://raw.githubusercontent.com/kata-containers/packaging/master/kata-deploy/examples/test-deploy-kata-qemu.yaml +kubectl delete -f https://raw.githubusercontent.com/kata-containers/packaging/master/kata-deploy/examples/test-deploy-kata-fc.yaml ``` ### Remove Kata from the Kubernetes cluster @@ -135,16 +130,16 @@ kubectl delete -f kata-rbac.yaml ## kata-deploy Details ### Dockerfile - -The Dockerfile used to create the container image deployed in the DaemonSet is provided here. -This image contains all the necessary artifacts for running Kata Containers. +The [Dockerfile](kata-deploy/Dockerfile) used to create the container image deployed in the DaemonSet is provided here. +This image contains all the necessary artifacts for running Kata Containers, all of which are pulled +from the [Kata Containers release page](https://github.com/kata-containers/runtime/releases). Host artifacts: -* kata-containers.io/kata-runtime: pulled from Kata GitHub releases page -* kata-proxy: pulled from Kata GitHub releases page -* kata-shim: pulled from Kata GitHub releases page -* qemu-system-x86_64: statically built and included in this repo, based on Kata's QEMU repo -* qemu/* : supporting binaries required for qemu-system-x86_64 +* kata-containers.io/kata-runtime +* kata-proxy +* kata-shim +* firecracker +* qemu-system-x86_64 and supporting binaries Virtual Machine artifacts: * kata-containers.img: pulled from Kata github releases page @@ -152,24 +147,16 @@ Virtual Machine artifacts: ### Daemonsets and RBAC: -A few daemonsets are introduced for kata-deploy, as well as an RBAC to facilitate +Two daemonsets are introduced for kata-deploy, as well as an RBAC to facilitate applying labels to the nodes. -#### runtime-labeler: +#### Kata installer: kata-deploy -This daemonset creates a label on each node in -the cluster identifying the CRI shim in use. For example, -`kata-containers.io/container-runtime=crio` or `kata-containers.io/container-runtime=containerd.` - -#### CRI-O and containerd kata installer - -Depending on the value of `kata-containers.io/container-runtime` label on the node, either the CRI-O or -containerd kata installation daemonset executes. These daemonsets install -the necessary kata binaries, configuration files, and virtual machine artifacts on +This daemonset installs the necessary kata binaries, configuration files, and virtual machine artifacts on the node. Once installed, the daemonset adds a node label `kata-containers.io/kata-runtime=true` and reconfigures -either CRI-O or containerd to make use of Kata for untrusted workloads. As a final step the daemonset -restarts either CRI-O or containerd and kubelet. Upon deletion, the daemonset removes the kata binaries -and VM artifacts and updates the node label to `kata-containers.io/kata-runtime=cleanup.` +either CRI-O or containerd to register two runtimeClasses: `kata-qemu` (for QEMU isolation) and `kata-fc` (for Firecracker isolation). +As a final step the daemonset restarts either CRI-O or containerd. Upon deletion, the daemonset removes the +Kata binaries and VM artifacts and updates the node label to `kata-containers.io/kata-runtime=cleanup.` ### Kata cleanup: This daemonset runs of the node has the label `kata-containers.io/kata-runtime=cleanup.` These daemonsets removes diff --git a/kata-deploy/examples/test-deploy-kata-fc.yaml b/kata-deploy/examples/test-deploy-kata-fc.yaml new file mode 100644 index 0000000000..498de27ae9 --- /dev/null +++ b/kata-deploy/examples/test-deploy-kata-fc.yaml @@ -0,0 +1,42 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + labels: + run: php-apache-kata-fc + name: php-apache-kata-fc +spec: + replicas: 1 + selector: + matchLabels: + run: php-apache-kata-fc + template: + metadata: + labels: + run: php-apache-kata-fc + spec: + runtimeClassName: kata-fc + containers: + - image: k8s.gcr.io/hpa-example + imagePullPolicy: Always + name: php-apache + ports: + - containerPort: 80 + protocol: TCP + resources: + requests: + cpu: 200m + restartPolicy: Always +--- +apiVersion: v1 +kind: Service +metadata: + name: php-apache-kata-fc +spec: + ports: + - port: 80 + protocol: TCP + targetPort: 80 + selector: + run: php-apache-kata-fc + sessionAffinity: None + type: ClusterIP diff --git a/kata-deploy/examples/test-deploy-kata-qemu.yaml b/kata-deploy/examples/test-deploy-kata-qemu.yaml new file mode 100644 index 0000000000..84fd1bfeda --- /dev/null +++ b/kata-deploy/examples/test-deploy-kata-qemu.yaml @@ -0,0 +1,45 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + labels: + run: php-apache-kata-qemu + name: php-apache-kata-qemu +spec: + replicas: 1 + selector: + matchLabels: + run: php-apache-kata-qemu + template: + metadata: + annotations: + io.kubernetes.cri-o.TrustedSandbox: "false" + io.kubernetes.cri.untrusted-workload: "true" + labels: + run: php-apache-kata-qemu + spec: + runtimeClassName: kata-qemu + containers: + - image: k8s.gcr.io/hpa-example + imagePullPolicy: Always + name: php-apache + ports: + - containerPort: 80 + protocol: TCP + resources: + requests: + cpu: 200m + restartPolicy: Always +--- +apiVersion: v1 +kind: Service +metadata: + name: php-apache-kata-qemu +spec: + ports: + - port: 80 + protocol: TCP + targetPort: 80 + selector: + run: php-apache-kata-qemu + sessionAffinity: None + type: ClusterIP diff --git a/kata-deploy/examples/test-deploy-runc.yaml b/kata-deploy/examples/test-deploy-runc.yaml new file mode 100644 index 0000000000..c7702bc0f5 --- /dev/null +++ b/kata-deploy/examples/test-deploy-runc.yaml @@ -0,0 +1,41 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + labels: + run: php-apache-runc + name: php-apache-runc +spec: + replicas: 1 + selector: + matchLabels: + run: php-apache-runc + template: + metadata: + labels: + run: php-apache-runc + spec: + containers: + - image: k8s.gcr.io/hpa-example + imagePullPolicy: Always + name: php-apache + ports: + - containerPort: 80 + protocol: TCP + resources: + requests: + cpu: 200m + restartPolicy: Always +--- +apiVersion: v1 +kind: Service +metadata: + name: php-apache-runc +spec: + ports: + - port: 80 + protocol: TCP + targetPort: 80 + selector: + run: php-apache-runc + sessionAffinity: None + type: ClusterIP