mirror of
https://github.com/kata-containers/kata-containers.git
synced 2025-06-27 07:48:55 +00:00
Merge pull request #541 from marcov/howto-move
how-to: move k8s guide into a dedicate how-to md
This commit is contained in:
GabyCT
GitHub
commit
c017e2519d
@ -31,13 +31,6 @@
|
||||
* [Update the Docker systemd unit file](#update-the-docker-systemd-unit-file)
|
||||
* [Create a container using Kata](#create-a-container-using-kata)
|
||||
* [Run Kata Containers with Kubernetes](#run-kata-containers-with-kubernetes)
|
||||
* [Install a CRI implementation](#install-a-cri-implementation)
|
||||
* [CRI-O](#cri-o)
|
||||
* [containerd with CRI plugin](#containerd-with-cri-plugin)
|
||||
* [Install Kubernetes](#install-kubernetes)
|
||||
* [Configure for CRI-O](#configure-for-cri-o)
|
||||
* [Configure for containerd](#configure-for-containerd)
|
||||
* [Run a Kubernetes pod with Kata Containers](#run-a-kubernetes-pod-with-kata-containers)
|
||||
* [Troubleshoot Kata Containers](#troubleshoot-kata-containers)
|
||||
* [Appendices](#appendices)
|
||||
* [Checking Docker default runtime](#checking-docker-default-runtime)
|
||||
@ -392,166 +385,7 @@ $ sudo docker run -ti --runtime kata-runtime busybox sh
|
||||
```
|
||||
|
||||
# Run Kata Containers with Kubernetes
|
||||
|
||||
Now that Kata Containers is installed on your system, you need some
|
||||
extra components to make this work with Kubernetes.
|
||||
|
||||
## Install a CRI implementation
|
||||
|
||||
Kata Containers runtime is an OCI compatible runtime and cannot directly
|
||||
interact with the CRI API level. For this reason we rely on a CRI
|
||||
implementation to translate CRI into OCI. There are two supported ways
|
||||
called [CRI-O](https://github.com/kubernetes-incubator/cri-o) and
|
||||
[CRI-containerd](https://github.com/containerd/cri). It is up to you to
|
||||
choose the one that you want, but you have to pick one. After choosing
|
||||
either CRI-O or CRI-containerd, you must make the appropriate changes
|
||||
to ensure it relies on the Kata Containers runtime.
|
||||
|
||||
As of Kata Containers 1.5, using `shimv2` with containerd 1.2.0 or above is the preferred
|
||||
way to run Kata Containers with Kubernetes ([see the howto](https://github.com/kata-containers/documentation/blob/master/how-to/how-to-use-k8s-with-cri-containerd-and-kata.md#configure-containerd-to-use-kata-containers)).
|
||||
The CRI-O will catch up soon.
|
||||
|
||||
### CRI-O
|
||||
|
||||
If you select CRI-O, follow the "CRI-O Tutorial" instructions
|
||||
[here](https://github.com/kubernetes-incubator/cri-o/blob/master/tutorial.md)
|
||||
to properly install it.
|
||||
|
||||
Once you have installed CRI-O, you need to modify the CRI-O configuration
|
||||
with information about different container runtimes. By default, we choose
|
||||
`runc`, but in this case we also specify Kata Containers runtime to run
|
||||
__untrusted__ workloads. In other words, this defines an alternative runtime
|
||||
to be used when the workload cannot be trusted and a higher level of security
|
||||
is required. An additional flag can be used to let CRI-O know if a workload
|
||||
should be considered _trusted_ or _untrusted_ by default.
|
||||
For further details, see the documentation
|
||||
[here](https://github.com/kata-containers/documentation/blob/master/design/architecture.md#mixing-vm-based-and-namespace-based-runtimes).
|
||||
|
||||
Additionally, we need CRI-O to perform the network namespace management.
|
||||
Otherwise, when the VM starts the network will not be available.
|
||||
|
||||
The following is an example of how to modify the `/etc/crio/crio.conf` file
|
||||
in order to apply the previous explanations, and therefore get Kata Containers
|
||||
runtime to invoke by CRI-O.
|
||||
|
||||
```toml
|
||||
# The "crio.runtime" table contains settings pertaining to the OCI
|
||||
# runtime used and options for how to set up and manage the OCI runtime.
|
||||
[crio.runtime]
|
||||
manage_network_ns_lifecycle = true
|
||||
|
||||
# runtime is the OCI compatible runtime used for trusted container workloads.
|
||||
# This is a mandatory setting as this runtime will be the default one
|
||||
# and will also be used for untrusted container workloads if
|
||||
# runtime_untrusted_workload is not set.
|
||||
runtime = "/usr/bin/runc"
|
||||
|
||||
# runtime_untrusted_workload is the OCI compatible runtime used for untrusted
|
||||
# container workloads. This is an optional setting, except if
|
||||
# default_container_trust is set to "untrusted".
|
||||
runtime_untrusted_workload = "/usr/bin/kata-runtime"
|
||||
|
||||
# default_workload_trust is the default level of trust crio puts in container
|
||||
# workloads. It can either be "trusted" or "untrusted", and the default
|
||||
# is "trusted".
|
||||
# Containers can be run through different container runtimes, depending on
|
||||
# the trust hints we receive from kubelet:
|
||||
# - If kubelet tags a container workload as untrusted, crio will try first to
|
||||
# run it through the untrusted container workload runtime. If it is not set,
|
||||
# crio will use the trusted runtime.
|
||||
# - If kubelet does not provide any information about the container workload trust
|
||||
# level, the selected runtime will depend on the default_container_trust setting.
|
||||
# If it is set to "untrusted", then all containers except for the host privileged
|
||||
# ones, will be run by the runtime_untrusted_workload runtime. Host privileged
|
||||
# containers are by definition trusted and will always use the trusted container
|
||||
# runtime. If default_container_trust is set to "trusted", crio will use the trusted
|
||||
# container runtime for all containers.
|
||||
default_workload_trust = "untrusted"
|
||||
|
||||
```
|
||||
|
||||
Restart CRI-O to take changes into account
|
||||
```
|
||||
$ sudo systemctl restart crio
|
||||
```
|
||||
|
||||
### containerd with CRI plugin
|
||||
|
||||
If you select containerd with `cri` plugin, follow the "Getting Started for Developers"
|
||||
instructions [here](https://github.com/containerd/cri#getting-started-for-developers)
|
||||
to properly install it.
|
||||
|
||||
To customize containerd to select Kata Containers runtime, follow our
|
||||
"Configure containerd to use Kata Containers" internal documentation
|
||||
[here](https://github.com/kata-containers/documentation/blob/master/how-to/how-to-use-k8s-with-cri-containerd-and-kata.md#configure-containerd-to-use-kata-containers).
|
||||
|
||||
## Install Kubernetes
|
||||
|
||||
Depending on what your needs are and what you expect to do with Kubernetes,
|
||||
please refer to the following
|
||||
[documentation](https://kubernetes.io/docs/setup/) to install it correctly.
|
||||
|
||||
Kubernetes talks with CRI implementations through a `container-runtime-endpoint`,
|
||||
also called CRI socket. This socket path is different depending on which CRI
|
||||
implementation you chose, and the Kubelet service has to be updated accordingly.
|
||||
|
||||
### Configure for CRI-O
|
||||
|
||||
`/etc/systemd/system/kubelet.service.d/0-crio.conf`
|
||||
```
|
||||
[Service]
|
||||
Environment="KUBELET_EXTRA_ARGS=--container-runtime=remote --runtime-request-timeout=15m --container-runtime-endpoint=unix:///var/run/crio/crio.sock"
|
||||
```
|
||||
|
||||
### Configure for containerd
|
||||
|
||||
`/etc/systemd/system/kubelet.service.d/0-cri-containerd.conf`
|
||||
```
|
||||
[Service]
|
||||
Environment="KUBELET_EXTRA_ARGS=--container-runtime=remote --runtime-request-timeout=15m --container-runtime-endpoint=unix:///run/containerd/containerd.sock"
|
||||
```
|
||||
For more information about containerd see the "Configure Kubelet to use containerd"
|
||||
documentation [here](https://github.com/kata-containers/documentation/blob/master/how-to/how-to-use-k8s-with-cri-containerd-and-kata.md#configure-kubelet-to-use-containerd).
|
||||
|
||||
## Run a Kubernetes pod with Kata Containers
|
||||
|
||||
After you update your Kubelet service based on the CRI implementation you
|
||||
are using, reload and restart Kubelet. Then, start your cluster:
|
||||
```bash
|
||||
$ sudo systemctl daemon-reload
|
||||
$ sudo systemctl restart kubelet
|
||||
|
||||
# If using CRI-O
|
||||
$ sudo kubeadm init --skip-preflight-checks --cri-socket /var/run/crio/crio.sock --pod-network-cidr=10.244.0.0/16
|
||||
|
||||
# If using CRI-containerd
|
||||
$ sudo kubeadm init --skip-preflight-checks --cri-socket /run/containerd/containerd.sock --pod-network-cidr=10.244.0.0/16
|
||||
|
||||
$ export KUBECONFIG=/etc/kubernetes/admin.conf
|
||||
```
|
||||
|
||||
You can force Kubelet to use Kata Containers by adding some `untrusted`
|
||||
annotation to your pod configuration. In our case, this ensures Kata
|
||||
Containers is the selected runtime to run the described workload.
|
||||
|
||||
`nginx-untrusted.yaml`
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: nginx-untrusted
|
||||
annotations:
|
||||
io.kubernetes.cri.untrusted-workload: "true"
|
||||
spec:
|
||||
containers:
|
||||
- name: nginx
|
||||
image: nginx
|
||||
```
|
||||
|
||||
Next, you run your pod:
|
||||
```
|
||||
$ sudo -E kubectl apply -f nginx-untrusted.yaml
|
||||
```
|
||||
Refer to to the [Run Kata Containers with Kubernetes](how-to/run-kata-with-k8s.md) how-to guide.
|
||||
|
||||
# Troubleshoot Kata Containers
|
||||
|
||||
@ -606,7 +440,7 @@ the following steps, which assume the use of a rootfs image.
|
||||
> **Note:** The following debug console instructions assume a systemd-based guest
|
||||
> O/S image. This means you must create a rootfs for a distro that supports systemd.
|
||||
> Currently, all distros supported by [osbuilder](https://github.com/kata-containers/osbuilder) support systemd
|
||||
> except for Alpine Linux.
|
||||
> except for Alpine Linux.
|
||||
>
|
||||
> Look for `INIT_PROCESS=systemd` in the `config.sh` osbuilder rootfs config file
|
||||
> to verify an osbuilder distro supports systemd for the distro you want to build rootfs for.
|
||||
|
||||
@ -1,11 +1,22 @@
|
||||
# Howto guides
|
||||
# Howto Guides
|
||||
|
||||
* [Howto Guides](#howto-guides)
|
||||
* [Kubernetes Integration](#kubernetes-integration)
|
||||
* [Hypervisors Integration](#hypervisors-integration)
|
||||
* [Advanced Topics](#advanced-topics)
|
||||
|
||||
## Kubernetes Integration
|
||||
- [Run Kata Containers with Kubernetes](run-kata-with-k8s.md)
|
||||
- [How to use Kata Containers and Containerd](containerd-kata.md)
|
||||
- [How to use Kata Containers and CRI (containerd plugin) with Kubernetes](how-to-use-k8s-with-cri-containerd-and-kata.md)
|
||||
- [How to use Kata Containers with virtio-fs](how-to-use-virtio-fs-with-kata.md)
|
||||
- [Kata Containers and service mesh for Kubernetes](service-mesh.md)
|
||||
|
||||
## Hypervisors Integration
|
||||
- [Kata Containers with Firecracker](https://github.com/kata-containers/documentation/wiki/Initial-release-of-Kata-Containers-with-Firecracker-support)
|
||||
- [Kata Containers with NEMU](how-to-use-kata-containers-with-nemu.md)
|
||||
|
||||
## Advanced Topics
|
||||
- [How to use Kata Containers with virtio-fs](how-to-use-virtio-fs-with-kata.md)
|
||||
- [Setting Sysctls with Kata](how-to-use-sysctls-with-kata.md)
|
||||
- [What Is VMCache and How To Enable It](what-is-vm-cache-and-how-do-I-use-it.md)
|
||||
- [What Is VM Templating and How To Enable It](what-is-vm-templating-and-how-do-I-use-it.md)
|
||||
|
||||
197
how-to/run-kata-with-k8s.md
Normal file
197
how-to/run-kata-with-k8s.md
Normal file
@ -0,0 +1,197 @@
|
||||
# Run Kata Containers with Kubernetes
|
||||
|
||||
* [Run Kata Containers with Kubernetes](#run-kata-containers-with-kubernetes)
|
||||
* [Prerequisites](#prerequisites)
|
||||
* [Install a CRI implementation](#install-a-cri-implementation)
|
||||
* [CRI-O](#cri-o)
|
||||
* [Kubernetes Runtime Class (CRI-O v1.12 )](#kubernetes-runtime-class-cri-o-v112)
|
||||
* [Untrusted annotation (until CRI-O v1.12)](#untrusted-annotation-until-cri-o-v112)
|
||||
* [Network namespace management](#network-namespace-management)
|
||||
* [containerd with CRI plugin](#containerd-with-cri-plugin)
|
||||
* [Install Kubernetes](#install-kubernetes)
|
||||
* [Configure for CRI-O](#configure-for-cri-o)
|
||||
* [Configure for containerd](#configure-for-containerd)
|
||||
* [Run a Kubernetes pod with Kata Containers](#run-a-kubernetes-pod-with-kata-containers)
|
||||
|
||||
## Prerequisites
|
||||
This guide requires Kata Containers available on your system, install-able by following [this guide](https://github.com/kata-containers/documentation/blob/master/install/README.md).
|
||||
|
||||
## Install a CRI implementation
|
||||
|
||||
Kubernetes CRI (Container Runtime Interface) implementations allow using any
|
||||
OCI-compatible runtime with Kubernetes, such as the Kata Containers runtime.
|
||||
|
||||
Kata Containers support both the [CRI-O](https://github.com/kubernetes-incubator/cri-o) and
|
||||
[CRI-containerd](https://github.com/containerd/cri) CRI implementations.
|
||||
|
||||
After choosing one CRI implementation, you must make the appropriate configuration
|
||||
to ensure it integrates with Kata Containers.
|
||||
|
||||
Kata Containers 1.5 introduced the `shimv2` for containerd 1.2.0, reducing the components
|
||||
required to spawn pods and containers, and this is the preferred way to run Kata Containers with Kubernetes ([as documented here](https://github.com/kata-containers/documentation/blob/master/how-to/how-to-use-k8s-with-cri-containerd-and-kata.md#configure-containerd-to-use-kata-containers)).
|
||||
|
||||
An equivalent shim implementation for CRI-O is planned.
|
||||
|
||||
### CRI-O
|
||||
For CRI-O installation instructions, refer to the [CRI-O Tutorial](https://github.com/kubernetes-incubator/cri-o/blob/master/tutorial.md) page.
|
||||
|
||||
The following sections show how to set up the CRI-O configuration file (default path: `/etc/crio/crio.conf`) for Kata.
|
||||
|
||||
Unless otherwise stated, all the following settings are specific to the `crio.runtime` table:
|
||||
```toml
|
||||
# The "crio.runtime" table contains settings pertaining to the OCI
|
||||
# runtime used and options for how to set up and manage the OCI runtime.
|
||||
[crio.runtime]
|
||||
```
|
||||
A comprehensive documentation of the configuration file can be found [here](https://github.com/cri-o/cri-o/blob/master/docs/crio.conf.5.md).
|
||||
|
||||
> **Note**: After any change to this file, the CRI-O daemon have to be restarted with:
|
||||
>````
|
||||
>$ sudo systemctl restart crio
|
||||
>````
|
||||
|
||||
#### Kubernetes Runtime Class (CRI-O v1.12+)
|
||||
The [Kubernetes Runtime Class](https://kubernetes.io/docs/concepts/containers/runtime-class/)
|
||||
is the preferred way of specifying the container runtime configuration to run a Pod's containers.
|
||||
To use this feature, Kata must added as a runtime handler with:
|
||||
|
||||
```toml
|
||||
[crio.runtime.runtimes.kata-runtime]
|
||||
runtime_path = "/usr/bin/kata-runtime"
|
||||
runtime_type = "oci"
|
||||
```
|
||||
|
||||
You can also add multiple entries to specify alternatives hypervisors, e.g.:
|
||||
```toml
|
||||
[crio.runtime.runtimes.kata-qemu]
|
||||
runtime_path = "/usr/bin/kata-runtime"
|
||||
runtime_type = "oci"
|
||||
|
||||
[crio.runtime.runtimes.kata-fc]
|
||||
runtime_path = "/usr/bin/kata-runtime"
|
||||
runtime_type = "oci"
|
||||
```
|
||||
|
||||
#### Untrusted annotation (until CRI-O v1.12)
|
||||
The untrusted annotation is used to specify a runtime for __untrusted__ workloads, i.e.
|
||||
a runtime to be used when the workload cannot be trusted and a higher level of security
|
||||
is required. An additional flag can be used to let CRI-O know if a workload
|
||||
should be considered _trusted_ or _untrusted_ by default.
|
||||
For further details, see the documentation
|
||||
[here](https://github.com/kata-containers/documentation/blob/master/design/architecture.md#mixing-vm-based-and-namespace-based-runtimes).
|
||||
|
||||
```toml
|
||||
# runtime is the OCI compatible runtime used for trusted container workloads.
|
||||
# This is a mandatory setting as this runtime will be the default one
|
||||
# and will also be used for untrusted container workloads if
|
||||
# runtime_untrusted_workload is not set.
|
||||
runtime = "/usr/bin/runc"
|
||||
|
||||
# runtime_untrusted_workload is the OCI compatible runtime used for untrusted
|
||||
# container workloads. This is an optional setting, except if
|
||||
# default_container_trust is set to "untrusted".
|
||||
runtime_untrusted_workload = "/usr/bin/kata-runtime"
|
||||
|
||||
# default_workload_trust is the default level of trust crio puts in container
|
||||
# workloads. It can either be "trusted" or "untrusted", and the default
|
||||
# is "trusted".
|
||||
# Containers can be run through different container runtimes, depending on
|
||||
# the trust hints we receive from kubelet:
|
||||
# - If kubelet tags a container workload as untrusted, crio will try first to
|
||||
# run it through the untrusted container workload runtime. If it is not set,
|
||||
# crio will use the trusted runtime.
|
||||
# - If kubelet does not provide any information about the container workload trust
|
||||
# level, the selected runtime will depend on the default_container_trust setting.
|
||||
# If it is set to "untrusted", then all containers except for the host privileged
|
||||
# ones, will be run by the runtime_untrusted_workload runtime. Host privileged
|
||||
# containers are by definition trusted and will always use the trusted container
|
||||
# runtime. If default_container_trust is set to "trusted", crio will use the trusted
|
||||
# container runtime for all containers.
|
||||
default_workload_trust = "untrusted"
|
||||
```
|
||||
|
||||
#### Network namespace management
|
||||
To enable networking for the workloads run by Kata, CRI-O needs to be configured to
|
||||
manage network namespaces, by setting the following key to `true`:
|
||||
```toml
|
||||
manage_network_ns_lifecycle = true
|
||||
```
|
||||
|
||||
### containerd with CRI plugin
|
||||
|
||||
If you select containerd with `cri` plugin, follow the "Getting Started for Developers"
|
||||
instructions [here](https://github.com/containerd/cri#getting-started-for-developers)
|
||||
to properly install it.
|
||||
|
||||
To customize containerd to select Kata Containers runtime, follow our
|
||||
"Configure containerd to use Kata Containers" internal documentation
|
||||
[here](https://github.com/kata-containers/documentation/blob/master/how-to/how-to-use-k8s-with-cri-containerd-and-kata.md#configure-containerd-to-use-kata-containers).
|
||||
|
||||
## Install Kubernetes
|
||||
|
||||
Depending on what your needs are and what you expect to do with Kubernetes,
|
||||
please refer to the following
|
||||
[documentation](https://kubernetes.io/docs/setup/) to install it correctly.
|
||||
|
||||
Kubernetes talks with CRI implementations through a `container-runtime-endpoint`,
|
||||
also called CRI socket. This socket path is different depending on which CRI
|
||||
implementation you chose, and the Kubelet service has to be updated accordingly.
|
||||
|
||||
### Configure for CRI-O
|
||||
|
||||
`/etc/systemd/system/kubelet.service.d/0-crio.conf`
|
||||
```
|
||||
[Service]
|
||||
Environment="KUBELET_EXTRA_ARGS=--container-runtime=remote --runtime-request-timeout=15m --container-runtime-endpoint=unix:///var/run/crio/crio.sock"
|
||||
```
|
||||
|
||||
### Configure for containerd
|
||||
|
||||
`/etc/systemd/system/kubelet.service.d/0-cri-containerd.conf`
|
||||
```
|
||||
[Service]
|
||||
Environment="KUBELET_EXTRA_ARGS=--container-runtime=remote --runtime-request-timeout=15m --container-runtime-endpoint=unix:///run/containerd/containerd.sock"
|
||||
```
|
||||
For more information about containerd see the "Configure Kubelet to use containerd"
|
||||
documentation [here](https://github.com/kata-containers/documentation/blob/master/how-to/how-to-use-k8s-with-cri-containerd-and-kata.md#configure-kubelet-to-use-containerd).
|
||||
|
||||
## Run a Kubernetes pod with Kata Containers
|
||||
|
||||
After you update your Kubelet service based on the CRI implementation you
|
||||
are using, reload and restart Kubelet. Then, start your cluster:
|
||||
```bash
|
||||
$ sudo systemctl daemon-reload
|
||||
$ sudo systemctl restart kubelet
|
||||
|
||||
# If using CRI-O
|
||||
$ sudo kubeadm init --skip-preflight-checks --cri-socket /var/run/crio/crio.sock --pod-network-cidr=10.244.0.0/16
|
||||
|
||||
# If using CRI-containerd
|
||||
$ sudo kubeadm init --skip-preflight-checks --cri-socket /run/containerd/containerd.sock --pod-network-cidr=10.244.0.0/16
|
||||
|
||||
$ export KUBECONFIG=/etc/kubernetes/admin.conf
|
||||
```
|
||||
|
||||
You can force Kubelet to use Kata Containers by adding some `untrusted`
|
||||
annotation to your pod configuration. In our case, this ensures Kata
|
||||
Containers is the selected runtime to run the described workload.
|
||||
|
||||
`nginx-untrusted.yaml`
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: nginx-untrusted
|
||||
annotations:
|
||||
io.kubernetes.cri.untrusted-workload: "true"
|
||||
spec:
|
||||
containers:
|
||||
- name: nginx
|
||||
image: nginx
|
||||
```
|
||||
|
||||
Next, you run your pod:
|
||||
```
|
||||
$ sudo -E kubectl apply -f nginx-untrusted.yaml
|
||||
```
|
||||
|
||||
Loading…
Reference in New Issue
Block a user