diff --git a/docs/devel/container-runtime-interface.md b/docs/devel/container-runtime-interface.md new file mode 100644 index 00000000000..596fc808c47 --- /dev/null +++ b/docs/devel/container-runtime-interface.md @@ -0,0 +1,123 @@ +# CRI: the Container Runtime Interface + +## What is CRI? + +CRI (_Container Runtime Interface_) consists of a +[protobuf API](../../pkg/kubelet/api/v1alpha1/runtime/api.proto), +specifications/requirements (to-be-added), +and [libraries] (https://github.com/kubernetes/kubernetes/tree/master/pkg/kubelet/server/streaming) +for container runtimes to integrate with kubelet on a node. CRI is currently in Alpha. + +In the future, we plan to add more developer tools such as the CRI validation +tests. + + +## Why develop CRI? + +Prior to the existence of CRI, container runtimes (e.g., `docker`, `rkt`) were +integrated with kubelet through implementing an internal, high-level interface +in kubelet. The entrance barrier for runtimes was high because the integration +required understanding the internals of kubelet and contributing to the main +Kubernetes repository. More importantly, this would not scale because every new +addition incurs a significant maintenance overhead in the main kubernetes +repository. + +Kubernetes aims to be extensible. CRI is one small, yet important step to enable +pluggable container runtimes and build a healthier ecosystem. + +## How to use CRI? + +1. Start the image and runtime services on your node. You can have a single + service acting as both image and runtime services. +2. Set the kubelet flags + - Pass the unix socket(s) to which your services listen to kubelet: + `--container-runtime-endpoint` and `--image-service-endpoint`. + - Enable CRI in kubelet by`--experimental-cri=true`). + - Use the "remote" runtime by `--container-runtime=remote`. + +Please see the [Status Update](#status-update) section for known issues for +each release. + +Note that CRI is still in its early stages. We are actively incorporating +feedback from early developers to improve the API. Developers should expect +occasional API breaking changes. + +## Does Kubelet use CRI today? + +No, but we are working on it. + +The first step is to switch kubelet to integrate with Docker via CRI by +default. The current [Docker CRI implementation](https://github.com/kubernetes/kubernetes/blob/release-1.5/pkg/kubelet/dockershim) +already passes most end-to-end tests, and has mandatory PR builders to prevent +regressions. While we are expanding the test coverage gradually, it is +difficult to test on all combinations of OS distributions, platforms, and +plugins. There are also many experimental or even undocumented features relied +upon by some users. We would like to **encourage the community to help test +this Docker-CRI integration and report bugs and/or missing features** to +smooth the transition in the near future. Please file a Github issue and +include @kubernetes/sig-node for any CRI problem. + +### How to test the new Docker CRI integration? + +Start kubelet with the following flags: + - Use the Docker container runtime by `--container-runtime=docker`(the default). + - Enable CRI in kubelet by`--experimental-cri=true`. + +Please also see the [known issues](#docker-cri-1.5-known-issues) before trying +out. + + +## Design docs and proposals + +We plan to add CRI specifications/requirements in the near future. For now, +these proposals and design docs are the best sources to understand CRI +besides discussions on Github issues. + + - [Original proposal](https://github.com/kubernetes/kubernetes/blob/release-1.5/docs/proposals/container-runtime-interface-v1.md) + - [Exec/attach/port-forward streaming requests](https://docs.google.com/document/d/1OE_QoInPlVCK9rMAx9aybRmgFiVjHpJCHI9LrfdNM_s/edit?usp=sharing) + - [Container stdout/stderr logs](https://github.com/kubernetes/kubernetes/blob/release-1.5/docs/proposals/kubelet-cri-logging.md) + - Networking: The CRI runtime handles network plugins and the + setup/teardown of the pod sandbox. + + +## Work-In-Progress CRI runtimes + + - [cri-o](https://github.com/kubernetes-incubator/cri-o) + - [rktlet](https://github.com/kubernetes-incubator/rktlet) + - [frakti](https://github.com/kubernetes/frakti) + + +## [Status update](#status-update) + +### Kubernetes v1.5 release (CRI v1alpha1) + + - [v1alpha1 version](https://github.com/kubernetes/kubernetes/blob/release-1.5/pkg/kubelet/api/v1alpha1/runtime/api.proto) of CRI is released. + + +#### [CRI known issues](#cri-1.5-known-issues): + + - Container metrics are not defined yet in CRI ([#27097](https://github.com/kubernetes/kubernetes/issues/27097)). + - CRI may not be compatible with other experimental features (e.g., Seccomp) + - Streaming server needs to be further productionized: + - Authentication: [#36666](https://github.com/kubernetes/kubernetes/issues/36666) + - Avoid including user data in the redirect URL: [#36187](https://github.com/kubernetes/kubernetes/issues/36187) + + +#### [Docker CRI integration known issues](#docker-cri-1.5-known-issues) + + - Docker compatibility: Support only Docker v1.11 and v1.12. + - Network: Does not support host port and bandwidth shaping + [#35457](https://github.com/kubernetes/kubernetes/issues/35457) + - Exec/attach/port-forward (streaming requests): Does not support `nsenter` + as the exec handler (`--exec-handler=nsenter`). Also see + (#cri-1.5-known-issues) for limitations on CRI streaming. + +## Contacts + + - Email: sig-node (kubernetes-sig-node@googlegroups.com) + - Slack: https://kubernetes.slack.com/messages/sig-node + + + +[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/devel/container-runtime-interface.md?pixel)]() +