From 72b35ce7d7d14e995d3b67941c849aeb2760bf92 Mon Sep 17 00:00:00 2001 From: Zihong Zheng Date: Fri, 24 Feb 2017 16:42:41 -0800 Subject: [PATCH] Updates READMEs regarding the new behavior of addon-manager --- cluster/addons/README.md | 63 +++++++++----------------- cluster/addons/addon-manager/README.md | 34 +++++++++++--- 2 files changed, 49 insertions(+), 48 deletions(-) diff --git a/cluster/addons/README.md b/cluster/addons/README.md index 52590882459..7c305cedcb3 100644 --- a/cluster/addons/README.md +++ b/cluster/addons/README.md @@ -1,53 +1,34 @@ # Cluster add-ons +## Overview + Cluster add-ons are resources like Services and Deployments (with pods) that are shipped with the Kubernetes binaries and are considered an inherent part of the -Kubernetes clusters. The add-ons are visible through the API (they can be listed using -`kubectl`), but direct manipulation of these objects through Apiserver is discouraged -because the system will bring them back to the original state, in particular: -- If an add-on is deleted, it will be recreated automatically. -- If an add-on is updated through Apiserver, it will be reconfigured to the state given by -the supplied fields in the initial config. +Kubernetes clusters. -On the cluster, the add-ons are kept in `/etc/kubernetes/addons` on the master node, in -yaml / json files. The addon manager periodically `kubectl apply`s the contents of this -directory. Any legit modification would be reflected on the API objects accordingly. -Particularly, rolling-update for deployments is now supported. +There are currently two classes of add-ons: +- Add-ons that will be reconciled. +- Add-ons that will be created if they don't exist. -Each add-on must specify the following label: `kubernetes.io/cluster-service: true`. -Config files that do not define this label will be ignored. For those resources -exist in `kube-system` namespace but not in `/etc/kubernetes/addons`, addon manager -will attempt to remove them if they are attached with this label. Currently the other -usage of `kubernetes.io/cluster-service` is for `kubectl cluster-info` command to recognize -these cluster services. +More details could be found in [addon-manager/README.md](addon-manager/README.md). -The suggested naming for most types of resources is just `` (with no version -number) because we do not expect the resource name to change. But resources like `Pod` -, `ReplicationController` and `DaemonSet` are exceptional. As `Pod` updates may not change -fields other than `containers[*].image` or `spec.activeDeadlineSeconds` and may not add or -remove containers, it may not be sufficient during a major update. For `ReplicationController`, -most of the modifications would be legit, but the underlying pods would not got re-created -automatically. `DaemonSet` has similar problem as the `ReplicationController`. In these -cases, the suggested naming is `-`. When version changes, the system will -delete the old one and create the new one (order not guaranteed). +## Cooperating Horizontal / Vertical Auto-Scaling with "reconcile class addons" -# Add-on update procedure +"Reconcile" class addons will be periodically reconciled to the original state given +by the initial config. In order to make Horizontal / Vertical Auto-scaling functional, +the related fields in config should be left unset. More specifically, leave `replicas` +in `ReplicationController` / `Deployment` / `ReplicaSet` unset for Horizontal Scaling, +leave `resources` for container unset for Vertical Scaling. The periodic reconcile +won't clobbered these fields, hence they could be managed by Horizontal / Vertical +Auto-scaler. -To update add-ons, just update the contents of `/etc/kubernetes/addons` -directory with the desired definition of add-ons. Then the system will take care -of: +## Add-on naming -- Removing objects from the API server whose manifest was removed. -- Creating objects from new manifests -- Updating objects whose fields are legally changed. - -# Cooperating with Horizontal / Vertical Auto-Scaling - -As all cluster add-ons will be reconciled to the original state given by the initial config. -In order to make Horizontal / Vertical Auto-scaling functional, the related fields in config should -be left unset. More specifically, leave `replicas` in `ReplicationController` / `Deployment` -/ `ReplicaSet` unset for Horizontal Scaling, and leave `resources` for container unset for Vertical -Scaling. The periodical update won't include these specs, which will be managed by Horizontal / Vertical - Auto-scaler. +The suggested naming for most of the resources is `` (with no version number). +Though resources like `Pod`, `ReplicationController` and `DaemonSet` are exceptional. +It would be hard to update `Pod` because many fields in `Pod` are immutable. For +`ReplicationController` and `DaemonSet`, in-place update may not trigger the underlying +pods to be re-created. You probably need to change their names during update to trigger +a complete deletion and creation. [![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/cluster/addons/README.md?pixel)]() diff --git a/cluster/addons/addon-manager/README.md b/cluster/addons/addon-manager/README.md index e2fb4413b77..b9da7039ba4 100644 --- a/cluster/addons/addon-manager/README.md +++ b/cluster/addons/addon-manager/README.md @@ -1,15 +1,35 @@ -### addon-manager +### Addon-manager -The `addon-manager` periodically `kubectl apply`s the Kubernetes manifest in the `/etc/kubernetes/addons` directory, -and handles any added / updated / deleted addon. +addon-manager manages two classes of addons with given template files. +- Addons with label `addonmanager.kubernetes.io/mode=Reconcile` will be periodically +reconciled. Direct manipulation to these addons through apiserver is discouraged because +addon-manager will bring them back to the original state. In particular: + - Addon will be re-created if it is deleted. + - Addon will be reconfigured to the state given by the supplied fields in the template + file periodically. + - Addon will be deleted when its manifest file is deleted. +- Addons with label `addonmanager.kubernetes.io/mode=EnsureExists` will be checked for +existence only. Users can edit these addons as they want. In particular: + - Addon will only be created/re-created with the given template file when there is no + instance of the resource with that name. + - Addon will not be deleted when the manifest file is deleted. -It supports all types of resource. -The requirement is to label them with `kubernetes.io/cluster-service: "true"`. - -The `addon-manager` is built for multiple architectures. +Notes: +- Label `kubernetes.io/cluster-service=true` is deprecated (only for Addon Manager). +In future release (after one year), Addon Manager may not respect it anymore. Addons +have this label but without `addonmanager.kubernetes.io/mode=EnsureExists` will be +treated as "reconcile class addons" for now. +- Resources under $ADDON_PATH (default `/etc/kubernetes/addons/`) needs to have either one +of these two labels. Meanwhile namespaced resources need to be in `kube-system` namespace. +Otherwise it will be omitted. +- The above label and namespace rule does not stand for `/opt/namespace.yaml` and +resources under `/etc/kubernetes/admission-controls/`. addon-manager will attempt to +create them regardless during startup. #### How to release +The `addon-manager` is built for multiple architectures. + 1. Change something in the source 2. Bump `VERSION` in the `Makefile` 3. Bump `KUBECTL_VERSION` in the `Makefile` if required