From 05ce0506f87f727fbadf0fc753a7af7c6675038a Mon Sep 17 00:00:00 2001 From: Zihong Zheng Date: Mon, 17 Oct 2016 13:06:21 -0700 Subject: [PATCH] Updated addon manager READMEs --- cluster/addons/README.md | 88 +++++++++++--------------- cluster/addons/addon-manager/README.md | 7 +- 2 files changed, 41 insertions(+), 54 deletions(-) diff --git a/cluster/addons/README.md b/cluster/addons/README.md index d604932a9c3..03e059ac029 100644 --- a/cluster/addons/README.md +++ b/cluster/addons/README.md @@ -1,66 +1,54 @@ # Cluster add-ons -Cluster add-ons are Services and Replication Controllers (with pods) that are +Cluster add-ons are resources like Services and Deployments (with pods) that are shipped with the Kubernetes binaries and are considered an inherent part of the -Kubernetes clusters. The add-ons are visible through the API (they can be listed -using ```kubectl```), but manipulation of these objects is discouraged because -the system will bring them back to the original state, in particular: -* if an add-on is stopped, it will be restarted automatically -* if an add-on is rolling-updated (for Replication Controllers), the system will stop the new version and - start the old one again (or perform rolling update to the old version, in the - future). +Kubernetes clusters. The add-ons are visible through the API (they can be listed using +`kubectl`), but direct manipulation of these objects through Apiserver is discouraged +because the system will bring them back to the original state, in particular: +- If an add-on is deleted, it will be recreated automatically. +- If an add-on is updated through Apiserver, it will be reconfigured to the state given by +the supplied fields in the initial config. Though it is fine to modify a field that was +unspecified. -On the cluster, the add-ons are kept in ```/etc/kubernetes/addons``` on the master node, in yaml files -(json is not supported at the moment). A system daemon periodically checks if -the contents of this directory is consistent with the add-on objects on the API -server. If any difference is spotted, the system updates the API objects -accordingly. (Limitation: for now, the system compares only the names of objects -in the directory and on the API server. So changes in parameters may not be -noticed). So the only persistent way to make changes in add-ons is to update the -manifests on the master server. But still, users are discouraged to do it -on their own - they should rather wait for a new release of -Kubernetes that will also contain new versions of add-ons. +On the cluster, the add-ons are kept in `/etc/kubernetes/addons` on the master node, in +yaml / json files. The addon manager periodically `kubectl apply`s the contents of this +directory. Any legit modification would be reflected on the API objects accordingly. +Particularly, rolling-update for deployments is now supported. -Each add-on must specify the following label: ```kubernetes.io/cluster-service: true```. -Yaml files that do not define this label will be ignored. +Each add-on must specify the following label: `kubernetes.io/cluster-service: true`. +Config files that do not define this label will be ignored. For those resources +exist in `kube-system` namespace but not in `/etc/kubernetes/addons`, addon manager +will attempt to remove them if they are attached with this label. Currently the other +usage of `kubernetes.io/cluster-service` is for `kubectl cluster-info` command to recognize +these cluster services. -The naming convention for Replication Controllers is -```-```, where `````` is the same in consecutive -versions and `````` changes when the component is updated -(`````` must not contain ```-```). For instance, -```heapster-controller-v1``` and ```heapster-controller-12``` are the -same controllers with two different versions, while ```heapster-controller-v1``` -and ```heapster-newcontroller-12``` are treated as two different applications. -When a new version of a Replication Controller add-on is found, the system will -stop the old (current) replication controller and start the new one -(in the future, rolling update will be performed). - -For services, the naming scheme is just `````` (with empty version number) -because we do not expect the service name to change in consecutive versions (and -rolling-update of services does not exist). +The suggested naming for most types of resources is just `` (with no version +number) because we do not expect the resource name to change. But resources like `Pod` +, `ReplicationController` and `DaemonSet` are exceptional. As `Pod` updates may not change +fields other than `containers[*].image` or `spec.activeDeadlineSeconds` and may not add or +remove containers, it may not be sufficient during a major update. For `ReplicationController`, +most of the modifications would be legit, but the underlying pods would not got re-created +automatically. `DaemonSet` has similar problem as the `ReplicationController`. In these +cases, the suggested naming is `-`. When version changes, the system will +delete the old one and create the new one (order not guaranteed). # Add-on update procedure -To update add-ons, just update the contents of ```/etc/kubernetes/addons``` +To update add-ons, just update the contents of `/etc/kubernetes/addons` directory with the desired definition of add-ons. Then the system will take care of: -1. Removing the objects from the API server whose manifest was removed. - 1. This is done for add-ons in the system that do not have a manifest file with the - same basename -1. Creating objects from new manifests - 1. This is done for manifests that do not correspond to existing API objects - with the same basename -1. Updating objects whose basename is the same, but whose versions changed. - 1. The update is currently performed by removing the old object and creating - the new one. In the future, rolling update of replication controllers will - be implemented to keep the add-on services up and running during update of add-on - pods. - 1. Note that this cannot happen for Services as their version is always empty. - -Note that in order to run the updator script, python is required on the machine. -For OS distros that don't have python installed, a python container will be used. +- Removing objects from the API server whose manifest was removed. +- Creating objects from new manifests +- Updating objects whose fields are legally changed. +# Cooperating with Horizontal / Vertical Auto-Scaling +As all cluster add-ons will be reconciled to the original state given by the initial config. +In order to make Horizontal / Vertical Auto-scaling functional, the related fields in config should +be left unset. More specifically, leave `replicas` in `ReplicationController` / `Deployment` +/ `ReplicaSet` unset for Horizontal Scaling, and leave `resources` for container unset for Vertical +Scaling. The periodical update won't include these specs, which will be managed by Horizontal / Vertical + Auto-scaler. [![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/cluster/addons/README.md?pixel)]() diff --git a/cluster/addons/addon-manager/README.md b/cluster/addons/addon-manager/README.md index 6451941f4c3..bd7c96cfb37 100644 --- a/cluster/addons/addon-manager/README.md +++ b/cluster/addons/addon-manager/README.md @@ -1,10 +1,9 @@ ### addon-manager -The `addon-manager` periodically checks for Kubernetes manifest changes in the `/etc/kubernetes/addons` directory, -and when there's a new or changed addon, the `addon-manager` automatically `kubectl create`s it. +The `addon-manager` periodically `kubectl apply`s the Kubernetes manifest in the `/etc/kubernetes/addons` directory, +and handles any added / updated / deleted addon. -It supports `ReplicationControllers`, `Deployments`, `DaemonSets`, `ConfigMaps`, `Services`, `PersistentVolumes` and -`PersistentVolumeClaims`. +It supports all types of resource. The `addon-manager` is built for multiple architectures.