mirror of
https://github.com/kata-containers/kata-containers.git
synced 2025-07-22 11:13:22 +00:00
Merge pull request #88 from jcvenegas/k8s-cri-contaienrd
how-to: Use K8s "cri" containerd plugin and kata.
This commit is contained in:
Jose Carlos Venegas Munoz
GitHub
commit
68f91f3f59
213
how-to/how-to-use-k8s-with-cri-containerd-and-kata.md
Normal file
213
how-to/how-to-use-k8s-with-cri-containerd-and-kata.md
Normal file
@ -0,0 +1,213 @@
|
|||||||
|
# How to use Kata Containers and CRI (containerd plugin) with Kubernetes
|
||||||
|
|
||||||
|
This document describes how to set up a single-machine Kubernetes cluster.
|
||||||
|
The Kubernetes cluster will use the CRI containerd plugin and Kata Containers to launch untrusted workloads.
|
||||||
|
|
||||||
|
## Requirements
|
||||||
|
- Kubernetes, kubelet, kubeadm
|
||||||
|
- cri-containerd
|
||||||
|
- Kata Containers
|
||||||
|
|
||||||
|
For information about the supported version of these components see
|
||||||
|
Kata Containers [versions.yaml](https://github.com/kata-containers/runtime/blob/master/versions.yaml) file.
|
||||||
|
|
||||||
|
## Install containerd(with CRI plugin enabled)
|
||||||
|
|
||||||
|
Follow the instructions from [CRI installation guide](http://github.com/containerd/cri/blob/master/docs/installation.md)
|
||||||
|
|
||||||
|
<!---
|
||||||
|
```bash
|
||||||
|
# Check if containerd is installed
|
||||||
|
$ command -v containerd
|
||||||
|
```
|
||||||
|
--->
|
||||||
|
|
||||||
|
## Install Kata Containers
|
||||||
|
|
||||||
|
Follow the instructions to [install Kata](https://github.com/kata-containers/documentation/blob/master/install/README.md).
|
||||||
|
|
||||||
|
<!---
|
||||||
|
```bash
|
||||||
|
# Check if kata-runtime is installed
|
||||||
|
$ command -v kata-runtime
|
||||||
|
# Check kata is well configured
|
||||||
|
$ kata-runtime kata-env
|
||||||
|
```
|
||||||
|
--->
|
||||||
|
|
||||||
|
## Install Kubernetes
|
||||||
|
Install Kubernetes in your host. See kubeadm [installation](https://kubernetes.io/docs/tasks/tools/install-kubeadm/)
|
||||||
|
<!---
|
||||||
|
```bash
|
||||||
|
# Check if kubadm is installed
|
||||||
|
$ command -v kubeadm
|
||||||
|
```
|
||||||
|
--->
|
||||||
|
|
||||||
|
### Configure containerd to use Kata Containers
|
||||||
|
|
||||||
|
The CRI containerd plugin support configuration for two runtime types.
|
||||||
|
|
||||||
|
- Default runtime: A runtime that is used by default to run workloads.
|
||||||
|
- Untrusted workload runtime: A runtime that will be used run untrusted workloads.
|
||||||
|
|
||||||
|
#### Define the Kata runtime as `untrusted_workload_runtime`
|
||||||
|
|
||||||
|
Configure the Kata runtime for untrusted workload with the [config option](https://github.com/containerd/cri/blob/v1.0.0-rc.0/docs/config.md)
|
||||||
|
`plugins.cri.containerd.untrusted_workload_runtime`.
|
||||||
|
|
||||||
|
Unless configured otherwise, the default runtime is set to `runc`.
|
||||||
|
```bash
|
||||||
|
# Configure containerd to use Kata as untrusted_workload_runtime
|
||||||
|
$ sudo mkdir -p /etc/containerd/
|
||||||
|
$ cat << EOT | sudo tee /etc/containerd/config.toml
|
||||||
|
[plugins]
|
||||||
|
[plugins.cri.containerd]
|
||||||
|
[plugins.cri.containerd.untrusted_workload_runtime]
|
||||||
|
runtime_type = "io.containerd.runtime.v1.linux"
|
||||||
|
runtime_engine = "/usr/bin/kata-runtime"
|
||||||
|
EOT
|
||||||
|
```
|
||||||
|
|
||||||
|
### Configure Kubelet to use containerd
|
||||||
|
|
||||||
|
In order to allow kubelet use containerd (using CRI interface) configure the service to
|
||||||
|
point to containerd socket.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Configure k8s to use containerd
|
||||||
|
$ sudo mkdir -p /etc/systemd/system/kubelet.service.d/
|
||||||
|
$ cat << EOF | sudo tee /etc/systemd/system/kubelet.service.d/0-containerd.conf
|
||||||
|
[Service]
|
||||||
|
Environment="KUBELET_EXTRA_ARGS=--container-runtime=remote --runtime-request-timeout=15m --container-runtime-endpoint=unix:///run/containerd/containerd.sock"
|
||||||
|
EOF
|
||||||
|
$ sudo systemctl daemon-reload
|
||||||
|
```
|
||||||
|
|
||||||
|
### Optional: Configure proxy
|
||||||
|
|
||||||
|
If you are behind a proxy this script will configure your proxy for docker
|
||||||
|
kubelet and containerd.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Set proxys
|
||||||
|
$ services=(
|
||||||
|
'kubelet'
|
||||||
|
'containerd'
|
||||||
|
'docker'
|
||||||
|
)
|
||||||
|
|
||||||
|
$ for s in "${services[@]}"; do
|
||||||
|
|
||||||
|
service_dir="/etc/systemd/system/${s}.service.d/"
|
||||||
|
sudo mkdir -p ${service_dir}
|
||||||
|
|
||||||
|
cat << EOT | sudo tee "${service_dir}/proxy.conf"
|
||||||
|
[Service]
|
||||||
|
Environment="HTTP_PROXY=${http_proxy}"
|
||||||
|
Environment="HTTPS_PROXY=${https_proxy}"
|
||||||
|
Environment="NO_PROXY=${no_proxy}"
|
||||||
|
EOT
|
||||||
|
done
|
||||||
|
$ sudo systemctl daemon-reload
|
||||||
|
```
|
||||||
|
|
||||||
|
### Start Kubernetes with kubeadm
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Mark sure containerd is up and running
|
||||||
|
$ sudo systemctl restart containerd
|
||||||
|
$ sudo systemctl status containerd
|
||||||
|
|
||||||
|
# Prevent docker iptables rules conflict with k8s pod communication
|
||||||
|
$ sudo iptables -P FORWARD ACCEPT
|
||||||
|
|
||||||
|
# Start cluster using kubeadm
|
||||||
|
$ sudo kubeadm init --skip-preflight-checks \
|
||||||
|
--cri-socket /run/containerd/containerd.sock --pod-network-cidr=10.244.0.0/16
|
||||||
|
|
||||||
|
$ export KUBECONFIG=/etc/kubernetes/admin.conf
|
||||||
|
|
||||||
|
$ sudo -E kubectl get nodes
|
||||||
|
$ sudo -E kubectl get pods
|
||||||
|
```
|
||||||
|
|
||||||
|
### Install a pod network
|
||||||
|
Install a pod network plugin is needed to allow pods communicate with each other.
|
||||||
|
|
||||||
|
Install flannel plugging, by following the instructions in the section *Installing a pod network*
|
||||||
|
from [Using kubeadm to Create a Cluster ](https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/)
|
||||||
|
guide.
|
||||||
|
|
||||||
|
<!---
|
||||||
|
```bash
|
||||||
|
# Install a pod network using flannel
|
||||||
|
# There is not a programmatic way to know last what flannel commit use
|
||||||
|
# See https://github.com/coreos/flannel/issues/995
|
||||||
|
$ sudo -E kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
|
||||||
|
```
|
||||||
|
--->
|
||||||
|
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# wait for pod network
|
||||||
|
$ timeout_dns=0
|
||||||
|
$ until [ "$timeout_dns" -eq "420" ]; do
|
||||||
|
if sudo -E kubectl get pods --all-namespaces | grep dns | grep Running; then
|
||||||
|
break
|
||||||
|
fi
|
||||||
|
sleep 1s
|
||||||
|
((timeout_dns+=1))
|
||||||
|
done
|
||||||
|
|
||||||
|
# check pod network is running
|
||||||
|
$ sudo -E kubectl get pods --all-namespaces | grep dns | grep Running && echo "OK" || ( echo "FAIL" && false )
|
||||||
|
```
|
||||||
|
|
||||||
|
### Allow run pods in master node
|
||||||
|
|
||||||
|
By default, the cluster will not schedule pods in the master node to allow that run:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# allow master node run pods
|
||||||
|
$ sudo -E kubectl taint nodes --all node-role.kubernetes.io/master-
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
### Create a unstrusted pod using Kata Containers
|
||||||
|
|
||||||
|
By default, all pods are created with the default runtime configured in CRI containerd plugin.
|
||||||
|
If a pod has the `io.kubernetes.cri.untrusted-workload annotation` set as
|
||||||
|
`"true"`, the CRI plugin will run the pod with the Kata Containers runtime.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Create untrusted pod configuration
|
||||||
|
$ cat << EOT | tee nginx-untrusted.yaml
|
||||||
|
apiVersion: v1
|
||||||
|
kind: Pod
|
||||||
|
metadata:
|
||||||
|
name: nginx-untrusted
|
||||||
|
annotations:
|
||||||
|
io.kubernetes.cri.untrusted-workload: "true"
|
||||||
|
spec:
|
||||||
|
containers:
|
||||||
|
- name: nginx
|
||||||
|
image: nginx
|
||||||
|
|
||||||
|
EOT
|
||||||
|
|
||||||
|
# Create untrusted pod
|
||||||
|
$ sudo -E kubectl apply -f nginx-untrusted.yaml
|
||||||
|
|
||||||
|
# Check pod is running
|
||||||
|
$ sudo -E kubectl get pods
|
||||||
|
|
||||||
|
# Check qemu is running
|
||||||
|
$ ps aux | grep qemu
|
||||||
|
```
|
||||||
|
### Delete created pod
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Delete pod
|
||||||
|
$ sudo -E kubectl delete -f nginx-untrusted.yaml
|
||||||
|
```
|
||||||
Loading…
Reference in New Issue
Block a user