github/kata-containers
1
0
Fork 1
mirror of https://github.com/kata-containers/kata-containers.git synced 2025-07-12 14:48:13 +00:00
Code Issues Projects Releases Wiki Activity

Merge pull request #251 from jodh-intel/improve-k8s+cri-containerd-howto

Browse Source 
docs: Clean up k8s with cri-containerd howto
This commit is contained in:
Graham Whaley 2018-10-04 13:23:41 +01:00 committed by GitHub
commit ec9f9d428e
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 208 additions and 163 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

371
how-to/how-to-use-k8s-with-cri-containerd-and-kata.md
View File

@ -1,64 +1,82 @@
# How to use Kata Containers and CRI (containerd plugin) with Kubernetes
This document describes how to set up a single-machine Kubernetes cluster.
The Kubernetes cluster will use the CRI containerd plugin and Kata Containers to launch untrusted workloads.
* [Requirements](#requirements)
* [Install containerd with CRI plugin enabled](#install-containerd-with-cri-plugin-enabled)
* [Install Kata Containers](#install-kata-containers)
* [Install Kubernetes](#install-kubernetes)
* [Configure containerd to use Kata Containers](#configure-containerd-to-use-kata-containers)
* [Define the Kata runtime as the untrusted workload runtime](#define-the-kata-runtime-as-the-untrusted-workload-runtime)
* [Configure Kubelet to use containerd](#configure-kubelet-to-use-containerd)
* [Configure proxy - OPTIONAL](#configure-proxy---optional)
* [Start Kubernetes](#start-kubernetes)
* [Install a Pod Network](#install-a-pod-network)
* [Allow pods to run in the master node](#allow-pods-to-run-in-the-master-node)
* [Create an unstrusted pod using Kata Containers](#create-an-unstrusted-pod-using-kata-containers)
* [Delete created pod](#delete-created-pod)
This document describes how to set up a single-machine Kubernetes (k8s) cluster.
The Kubernetes cluster will use the
[CRI containerd plugin](https://github.com/containerd/cri) and
[Kata Containers](https://katacontainers.io) to launch untrusted workloads.
## Requirements
## Requirements
- Kubernetes, kubelet, kubeadm
- cri-containerd
- Kata Containers
For information about the supported version of these components see
Kata Containers [versions.yaml](https://github.com/kata-containers/runtime/blob/master/versions.yaml) file.
> **Note:** For information about the supported versions of these components,
> see the Kata Containers
> [versions.yaml](https://github.com/kata-containers/runtime/blob/master/versions.yaml)
> file.
## Install containerd(with CRI plugin enabled)
## Install containerd with CRI plugin enabled
Follow the instructions from [CRI installation guide](http://github.com/containerd/cri/blob/master/docs/installation.md)
- Follow the instructions from the
[CRI installation guide](http://github.com/containerd/cri/blob/master/docs/installation.md).
<!---
```bash
# Check if containerd is installed
$ command -v containerd
```
--->
- Check if `containerd` is now available
```bash
$ command -v containerd
```
## Install Kata Containers
## Install Kata Containers
Follow the instructions to [install Kata](https://github.com/kata-containers/documentation/blob/master/install/README.md).
<!---
```bash
# Check if kata-runtime is installed
$ command -v kata-runtime
# Check kata is well configured
$ kata-runtime kata-env
```
--->
Follow the instructions to
[install Kata Containers](https://github.com/kata-containers/documentation/blob/master/install/README.md).
## Install Kubernetes
Install Kubernetes in your host. See kubeadm [installation](https://kubernetes.io/docs/tasks/tools/install-kubeadm/)
<!---
- Follow the instructions for
[kubeadm installation](https://kubernetes.io/docs/setup/independent/install-kubeadm/).
- Check `kubeadm` is now available
```bash
$ command -v kubeadm
```
## Configure containerd to use Kata Containers
The CRI `containerd` plugin supports configuration for two runtime types.
- **Default runtime:**
A runtime that is used by default to run workloads.
- **Untrusted workload runtime:**
A runtime that will be used to run untrusted workloads. This is appropriate
for workloads that require a higher degree of security isolation.
#### Define the Kata runtime as the untrusted workload runtime
Configure `containerd` to use the Kata runtime to run untrusted workloads by
setting the `plugins.cri.containerd.untrusted_workload_runtime`
[config option](https://github.com/containerd/cri/blob/v1.0.0-rc.0/docs/config.md):
```bash
# Check if kubadm is installed
$ command -v kubeadm
```
--->
### Configure containerd to use Kata Containers
The CRI containerd plugin support configuration for two runtime types.
- Default runtime: A runtime that is used by default to run workloads.
- Untrusted workload runtime: A runtime that will be used run untrusted workloads.
#### Define the Kata runtime as `untrusted_workload_runtime`
Configure the Kata runtime for untrusted workload with the [config option](https://github.com/containerd/cri/blob/v1.0.0-rc.0/docs/config.md)
`plugins.cri.containerd.untrusted_workload_runtime`.
Unless configured otherwise, the default runtime is set to `runc`.
```bash
# Configure containerd to use Kata as untrusted_workload_runtime
$ sudo mkdir -p /etc/containerd/
$ cat << EOT | sudo tee /etc/containerd/config.toml
[plugins]
@ -69,145 +87,172 @@ $ cat << EOT | sudo tee /etc/containerd/config.toml
EOT
```
### Configure Kubelet to use containerd
> **Note:** Unless configured otherwise, the default runtime is set to `runc`.
In order to allow kubelet use containerd (using CRI interface) configure the service to
point to containerd socket.
## Configure Kubelet to use containerd
In order to allow kubelet to use containerd (using the CRI interface), configure the service to point to the `containerd` socket.
- Configure Kubernetes to use `containerd`
```bash
$ sudo mkdir -p /etc/systemd/system/kubelet.service.d/
$ cat << EOF | sudo tee /etc/systemd/system/kubelet.service.d/0-containerd.conf
[Service]
Environment="KUBELET_EXTRA_ARGS=--container-runtime=remote --runtime-request-timeout=15m --container-runtime-endpoint=unix:///run/containerd/containerd.sock"
EOF
```
- Inform systemd about the new configuration
```bash
$ sudo systemctl daemon-reload
```
## Configure proxy - OPTIONAL
If you are behind a proxy, use the following script to configure your proxy for docker, kubelet, and containerd:
```bash
# Configure k8s to use containerd
$ sudo mkdir -p /etc/systemd/system/kubelet.service.d/
$ cat << EOF | sudo tee /etc/systemd/system/kubelet.service.d/0-containerd.conf
[Service]
Environment="KUBELET_EXTRA_ARGS=--container-runtime=remote --runtime-request-timeout=15m --container-runtime-endpoint=unix:///run/containerd/containerd.sock"
EOF
$ sudo systemctl daemon-reload
```
$ services="
kubelet
containerd
docker
"
### Optional: Configure proxy
$ for service in ${services}; do
If you are behind a proxy this script will configure your proxy for docker
kubelet and containerd.
service_dir="/etc/systemd/system/${service}.service.d/"
sudo mkdir -p ${service_dir}
```bash
# Set proxys
$ services=(
'kubelet'
'containerd'
'docker'
)
$ for s in "${services[@]}"; do
service_dir="/etc/systemd/system/${s}.service.d/"
sudo mkdir -p ${service_dir}
cat << EOT | sudo tee "${service_dir}/proxy.conf"
cat << EOT | sudo tee "${service_dir}/proxy.conf"
[Service]
Environment="HTTP_PROXY=${http_proxy}"
Environment="HTTPS_PROXY=${https_proxy}"
Environment="NO_PROXY=${no_proxy}"
EOT
done
$ sudo systemctl daemon-reload
```
### Start Kubernetes with kubeadm
## Start Kubernetes
- Make sure `containerd` is up and running
```bash
$ sudo systemctl restart containerd
$ sudo systemctl status containerd
```
- Prevent conflicts between `docker` iptables (packet filtering) rules and k8s pod communication
If Docker is installed on the node, it is necessary to modify the rule
below. See https://github.com/kubernetes/kubernetes/issues/40182 for further
details.
```bash
$ sudo iptables -P FORWARD ACCEPT
```
- Start cluster using `kubeadm`
```bash
$ sudo kubeadm init --skip-preflight-checks --cri-socket /run/containerd/containerd.sock --pod-network-cidr=10.244.0.0/16
$ export KUBECONFIG=/etc/kubernetes/admin.conf
$ sudo -E kubectl get nodes
$ sudo -E kubectl get pods
```
## Install a Pod Network
A pod network plugin is needed to allow pods to communicate with each other.
- Install the `flannel` plugin by following the
[Using kubeadm to Create a Cluster](https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/#instructions)
guide, starting from the **Installing a pod network** section.
- Create a pod network using flannel
> **Note:** There is no known way to determine programmatically the best version (commit) to use.
> See https://github.com/coreos/flannel/issues/995.
```bash
$ sudo -E kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
```
- Wait for the pod network to become available
```bash
# number of seconds to wait for pod network to become available
$ timeout_dns=420
$ while [ "$timeout_dns" -gt 0 ]; do
if sudo -E kubectl get pods --all-namespaces | grep dns | grep Running; then
break
fi
sleep 1s
((timeout_dns--))
done
```
- Check the pod network is running
```bash
$ sudo -E kubectl get pods --all-namespaces | grep dns | grep Running && echo "OK" || ( echo "FAIL" && false )
```
## Allow pods to run in the master node
By default, the cluster will not schedule pods in the master node. To enable master node scheduling:
```bash
# Mark sure containerd is up and running
$ sudo systemctl restart containerd
$ sudo systemctl status containerd
# Prevent docker iptables rules conflict with k8s pod communication
$ sudo iptables -P FORWARD ACCEPT
# Start cluster using kubeadm
$ sudo kubeadm init --skip-preflight-checks \
--cri-socket /run/containerd/containerd.sock --pod-network-cidr=10.244.0.0/16
$ export KUBECONFIG=/etc/kubernetes/admin.conf
$ sudo -E kubectl get nodes
$ sudo -E kubectl get pods
```
### Install a pod network
Install a pod network plugin is needed to allow pods communicate with each other.
Install flannel plugging, by following the instructions in the section *Installing a pod network*
from [Using kubeadm to Create a Cluster ](https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/)
guide.
<!---
```bash
# Install a pod network using flannel
# There is not a programmatic way to know last what flannel commit use
# See https://github.com/coreos/flannel/issues/995
$ sudo -E kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
```
--->
```bash
# wait for pod network
$ timeout_dns=0
$ until [ "$timeout_dns" -eq "420" ]; do
if sudo -E kubectl get pods --all-namespaces | grep dns | grep Running; then
break
fi
sleep 1s
((timeout_dns+=1))
done
# check pod network is running
$ sudo -E kubectl get pods --all-namespaces | grep dns | grep Running && echo "OK" || ( echo "FAIL" && false )
```
### Allow run pods in master node
By default, the cluster will not schedule pods in the master node to allow that run:
```bash
# allow master node run pods
$ sudo -E kubectl taint nodes --all node-role.kubernetes.io/master-
```
### Create a unstrusted pod using Kata Containers
## Create an unstrusted pod using Kata Containers
By default, all pods are created with the default runtime configured in CRI containerd plugin.
If a pod has the `io.kubernetes.cri.untrusted-workload annotation` set as
`"true"`, the CRI plugin will run the pod with the Kata Containers runtime.
If a pod has the `io.kubernetes.cri.untrusted-workload` annotation set to `"true"`, the CRI plugin runs the pod with the
[Kata Containers runtime](https://github.com/kata-containers/runtime/blob/master/README.md).
- Create an untrusted pod configuration
```bash
$ cat << EOT | tee nginx-untrusted.yaml
apiVersion: v1
kind: Pod
metadata:
name: nginx-untrusted
annotations:
io.kubernetes.cri.untrusted-workload: "true"
spec:
containers:
- name: nginx
image: nginx
EOT
```
- Create an untrusted pod
```bash
$ sudo -E kubectl apply -f nginx-untrusted.yaml
```
- Check pod is running
```bash
$ sudo -E kubectl get pods
```
- Check hypervisor is running
```bash
$ ps aux | grep qemu
```
## Delete created pod
```bash
# Create untrusted pod configuration
$ cat << EOT | tee nginx-untrusted.yaml
apiVersion: v1
kind: Pod
metadata:
name: nginx-untrusted
annotations:
io.kubernetes.cri.untrusted-workload: "true"
spec:
containers:
- name: nginx
image: nginx
EOT
# Create untrusted pod
$ sudo -E kubectl apply -f nginx-untrusted.yaml
# Check pod is running
$ sudo -E kubectl get pods
# Check qemu is running
$ ps aux | grep qemu
```
### Delete created pod
```bash
# Delete pod
$ sudo -E kubectl delete -f nginx-untrusted.yaml
$ sudo -E kubectl delete -f nginx-untrusted.yaml
```