From 1a21ffdd0ab9d2f114d07264c4829d4ef4661e7f Mon Sep 17 00:00:00 2001 From: Tim Hockin Date: Mon, 8 Sep 2014 12:58:44 -0700 Subject: [PATCH 1/5] Revisit identifiers spec --- docs/identifiers.md | 85 ++++++++++++++++++++------------------------- 1 file changed, 38 insertions(+), 47 deletions(-) diff --git a/docs/identifiers.md b/docs/identifiers.md index e368ab2085f..dc017fffdca 100644 --- a/docs/identifiers.md +++ b/docs/identifiers.md @@ -1,14 +1,15 @@ # Identifiers and Names in Kubernetes -A summarization of the goals and recommendations for identifiers and names in Kubernetes. Described in [GitHub issue #199](https://github.com/GoogleCloudPlatform/kubernetes/issues/199). +A summarization of the goals and recommendations for identifiers in Kubernetes. Described in [GitHub issue #199](https://github.com/GoogleCloudPlatform/kubernetes/issues/199). + ## Definitions -identifier -: An opaque machine generated value guaranteed to be unique in a certain space +uid +: An opaque system-generated value guaranteed to be unique in time and space; intended to distinguish between historical occurrences of similar entities. name -: A human readable string intended to help an end user distinguish between similar but distinct entities +: A string guaranteed to be unique within a given scope at a particular time; used in resource URLs; provided by clients at creation time and encouraged to be human friendly; intended to facilitate creation idempotence and space-uniqueness of singleton objects, distinguish distinct entities, and reference particular entities across operations. [rfc1035](http://www.ietf.org/rfc/rfc1035.txt)/[rfc1123](http://www.ietf.org/rfc/rfc1123.txt) label (DNS_LABEL) : An alphanumeric (a-z, A-Z, and 0-9) string, with a maximum length of 63 characters, with the '-' character allowed anywhere except the first or last character, suitable for use as a hostname or segment in a domain name @@ -19,68 +20,58 @@ name [rfc4122](http://www.ietf.org/rfc/rfc4122.txt) universally unique identifier (UUID) : A 128 bit generated value that is extremely unlikely to collide across time and space and requires no central coordination -## Objectives for names and identifiers -1) Uniquely identify an instance of a pod on the apiserver and on the kubelet +## Objectives for names and uids -2) Uniquely identify an instance of a container within a pod on the apiserver and on the kubelet +1) Uniquely identify (via a uid) an object across space and time -3) Uniquely identify a single execution of a container in time for logging or reporting +2) Uniquely name (via a Name) an object across space -4) The structure of a pod specification should stay largely the same throughout the entire system +3) Provide human-friendly names in API operations and/or configuration files -5) Provide human-friendly, memorable, semantically meaningful, short-ish references in container and pod operations +4) Allow idempotent creation of API resources (#148) and enforcement of space-uniqueness of singleton objects -6) Provide predictable container and pod references in operations and/or configuration files - -7) Allow idempotent creation of API resources (#148) - -8) Allow DNS names to be automatically generated for individual containers or pods (#146) +5) Allow DNS names to be automatically generated for some objects +FIXME: Should this be more agnostic to resource type, and talk about pod as a particular case? ## Design -1) Each apiserver has a Namespace string (a DNS_SUBDOMAIN) that is unique across all apiservers that share its configured minions. +1) Each apiserver must be assigned a Namespace string (a DNS_SUBDOMAIN). + 1) must be non-empty and unique across all apiservers that share minions Example: "k8s.example.com" -2) Each pod instance on an apiserver has a PodName string (a DNS_SUBDOMAIN) which is and unique within the Namespace. - 1) If not specified by the client, the apiserver will assign this identifier +2) When an object is created on an apiserver, a Name string (a DNS_SUBDOMAIN) must be provided. + 1) must be non-empty and unique within the apiserver's Namespace + 2) enables idempotent and space-unique creation + 1) generating random names will defeat idempotentcy + 3) other parts of the system (e.g. replication controller) may join strings (e.g. a base name and a random suffic) to create a unique Name Example: "guestbook.user" + Example: "backend-x4eb1" -3) Each pod instance on an apiserver has a PodFullName (a DNS_SUBDOMAIN) string which is derived from a combination of the Namespace and Name strings. - 1) If the joined Namespace and PodName is too long for a DNS_SUBDOMAIN, the apiserver must transform it to fit, while still being unique - Example: "guestbook.user.k8s.example.com" +FIXME: final debate on having master default a name. Alternative: set "autosetName"=true +FIXME: how long can + be? We previously had FullName, making it the apiserver's problem to truncate long names to DNS_DOMAIN len. -4) Each pod instance on an apiserver has a PodID (a UUID) that is unique across space and time - 1) If not specified by the client, the apiserver will assign this identifier - 2) This identifier will persist for the lifetime of the pod, even if the pod is stopped and started or moved across hosts +3) Upon acceptance at the apiserver, a pod is assigned a uid (a UUID). + 1) must be non-empty and unique across space and time Example: "01234567-89ab-cdef-0123-456789abcdef" -5) Each container within a pod has a ContainerName string (a DNS_LABEL) that is unique within that pod - 1) This name must be specified by the client or the apiserver will reject the pod +4) Each container within a pod must have a Name string (a DNS_LABEL). + 1) must be non-empty and unique within the pod Example: "frontend" -6) Each pod instance on a kubelet has a PodNamespace string (a DNS_SUBDOMAIN) - 1) This corresponds to the apiserver's Namespace string - 2) If not specified, the kubelet will assign this name to a deterministic value which is likely to be unique across all sources on the host - Example: "k8s.example.com" - Example: "file-f4231812554558a718a01ca942782d81" +5) When a pod is bound to a node, the node is told the pod's uid. + 1) if not provided, the kubelet will generate one + 2) provides for pods from node-local config files -7) Each pod instance on a kubelet has a PodName string (a DNS_SUBDOMAIN) which is unique within the source Namespace - 1) This corresponds to the apiserver's PodName string - 2) If not specified, the kubelet will assign this name to a deterministic value - Example: "frontend" +6) When a pod is bound to a node, the node is told the pod's Namespace, and Name. + 1) if Namespace is not provided, the kubelet will generate one + 2) generated Namespaces must be deterministic + 3) provides a cluster-wide space-unique name + Example: Namespace="k8s.example.com" Name="guestbook.user" + Example: Namespace="k8s.example.com" Name="backend-x4eb1" + Example: Namespace="file-f4231812554558a718a01ca942782d81" Name="cadvisor" -8) When starting an instance of a pod on a kubelet, a PodInstanceID (a UUID) will be assigned to that pod instance - 1) If not specified, the kubelet will assign this identifier - 2) If the pod is restarted, it must retain the PodInstanceID it previously had - 3) If the pod is stopped and a new instance with the same PodNamespace and PodName is started, it must be assigned a new PodInstanceID - 4) If the pod is moved across hosts, it must be assigned a new PodInstanceID - Example: "01234567-89ab-cdef-0123-456789abcdef" - -9) The kubelet may use the PodNamespace, PodName, PodID, and PodInstanceID to produce a docker container name (--name) - Example: "01234567-89ab-cdef-0123-456789abcdef_frontend_k8s.example.com" - -10) Each run of a container within a pod will be assigned a ContainerAttemptID (string) that is unique across time. - 1) This corresponds to Docker container IDs +7) Each run of a container within a pod will be assigned an AttemptID (string) that is unique across time. + 1) corresponds to Docker's container ID Example: "77af4d6b9913e693e8d0b4b294fa62ade6054e6b2f1ffb617ac955dd63fb0182" From b6eb7cc979fdba7c86aee1d650ca50f7b6e2fbe4 Mon Sep 17 00:00:00 2001 From: Tim Hockin Date: Mon, 8 Sep 2014 15:38:11 -0700 Subject: [PATCH 2/5] Simplify pod namespaces --- docs/identifiers.md | 21 ++++++++------------- 1 file changed, 8 insertions(+), 13 deletions(-) diff --git a/docs/identifiers.md b/docs/identifiers.md index dc017fffdca..6efd613d647 100644 --- a/docs/identifiers.md +++ b/docs/identifiers.md @@ -37,12 +37,8 @@ name FIXME: Should this be more agnostic to resource type, and talk about pod as a particular case? ## Design -1) Each apiserver must be assigned a Namespace string (a DNS_SUBDOMAIN). - 1) must be non-empty and unique across all apiservers that share minions - Example: "k8s.example.com" - -2) When an object is created on an apiserver, a Name string (a DNS_SUBDOMAIN) must be provided. - 1) must be non-empty and unique within the apiserver's Namespace +1) When an object is created on an apiserver, a Name string (a DNS_SUBDOMAIN) must be provided. + 1) must be non-empty and unique within the apiserver 2) enables idempotent and space-unique creation 1) generating random names will defeat idempotentcy 3) other parts of the system (e.g. replication controller) may join strings (e.g. a base name and a random suffic) to create a unique Name @@ -50,23 +46,22 @@ FIXME: Should this be more agnostic to resource type, and talk about pod as a pa Example: "backend-x4eb1" FIXME: final debate on having master default a name. Alternative: set "autosetName"=true -FIXME: how long can + be? We previously had FullName, making it the apiserver's problem to truncate long names to DNS_DOMAIN len. -3) Upon acceptance at the apiserver, a pod is assigned a uid (a UUID). +2) Upon acceptance at the apiserver, a pod is assigned a uid (a UUID). 1) must be non-empty and unique across space and time Example: "01234567-89ab-cdef-0123-456789abcdef" -4) Each container within a pod must have a Name string (a DNS_LABEL). +3) Each container within a pod must have a Name string (a DNS_LABEL). 1) must be non-empty and unique within the pod Example: "frontend" -5) When a pod is bound to a node, the node is told the pod's uid. +4) When a pod is bound to a node, the node is told the pod's uid. 1) if not provided, the kubelet will generate one 2) provides for pods from node-local config files -6) When a pod is bound to a node, the node is told the pod's Namespace, and Name. - 1) if Namespace is not provided, the kubelet will generate one - 2) generated Namespaces must be deterministic +6) When a pod is bound to a node, the node is told the pod's Name. + 1) kubelet will namespace pods from distinct sources (e.g. files vs apiserver) + 2) namespaces must be deterministic 3) provides a cluster-wide space-unique name Example: Namespace="k8s.example.com" Name="guestbook.user" Example: Namespace="k8s.example.com" Name="backend-x4eb1" From 1d9cff4a87a8e952448b841e4f8bc8018cfdafe0 Mon Sep 17 00:00:00 2001 From: Tim Hockin Date: Tue, 9 Sep 2014 09:57:02 -0700 Subject: [PATCH 3/5] Make pods a case study in identifiers.md --- docs/identifiers.md | 96 ++++++++++++++++++++++++++++----------------- 1 file changed, 61 insertions(+), 35 deletions(-) diff --git a/docs/identifiers.md b/docs/identifiers.md index 6efd613d647..07bc9391554 100644 --- a/docs/identifiers.md +++ b/docs/identifiers.md @@ -5,11 +5,11 @@ A summarization of the goals and recommendations for identifiers in Kubernetes. ## Definitions -uid -: An opaque system-generated value guaranteed to be unique in time and space; intended to distinguish between historical occurrences of similar entities. +UID +: A non-empty, opaque, system-generated value guaranteed to be unique in time and space; intended to distinguish between historical occurrences of similar entities. -name -: A string guaranteed to be unique within a given scope at a particular time; used in resource URLs; provided by clients at creation time and encouraged to be human friendly; intended to facilitate creation idempotence and space-uniqueness of singleton objects, distinguish distinct entities, and reference particular entities across operations. +Name +: A non-empty string guaranteed to be unique within a given scope at a particular time; used in resource URLs; provided by clients at creation time and encouraged to be human friendly; intended to facilitate creation idempotence and space-uniqueness of singleton objects, distinguish distinct entities, and reference particular entities across operations. [rfc1035](http://www.ietf.org/rfc/rfc1035.txt)/[rfc1123](http://www.ietf.org/rfc/rfc1123.txt) label (DNS_LABEL) : An alphanumeric (a-z, A-Z, and 0-9) string, with a maximum length of 63 characters, with the '-' character allowed anywhere except the first or last character, suitable for use as a hostname or segment in a domain name @@ -21,52 +21,78 @@ name : A 128 bit generated value that is extremely unlikely to collide across time and space and requires no central coordination -## Objectives for names and uids +## Objectives for names and UIDs -1) Uniquely identify (via a uid) an object across space and time +1. Uniquely identify (via a UID) an object across space and time -2) Uniquely name (via a Name) an object across space +2. Uniquely name (via a name) an object across space -3) Provide human-friendly names in API operations and/or configuration files +3. Provide human-friendly names in API operations and/or configuration files -4) Allow idempotent creation of API resources (#148) and enforcement of space-uniqueness of singleton objects +4. Allow idempotent creation of API resources (#148) and enforcement of space-uniqueness of singleton objects -5) Allow DNS names to be automatically generated for some objects +5. Allow DNS names to be automatically generated for some objects -FIXME: Should this be more agnostic to resource type, and talk about pod as a particular case? -## Design +## General design -1) When an object is created on an apiserver, a Name string (a DNS_SUBDOMAIN) must be provided. - 1) must be non-empty and unique within the apiserver - 2) enables idempotent and space-unique creation - 1) generating random names will defeat idempotentcy - 3) other parts of the system (e.g. replication controller) may join strings (e.g. a base name and a random suffic) to create a unique Name +1. When an object is created via an api, a Name string (a DNS_SUBDOMAIN) must be provided. + 1. must be non-empty and unique within the apiserver + 2. enables idempotent and space-unique creation + 1. generating random names will defeat idempotentcy + 3. parts of the system (e.g. replication controller) may join strings (e.g. a base name and a random suffix) to create a unique Name Example: "guestbook.user" Example: "backend-x4eb1" FIXME: final debate on having master default a name. Alternative: set "autosetName"=true -2) Upon acceptance at the apiserver, a pod is assigned a uid (a UUID). - 1) must be non-empty and unique across space and time +2. Upon acceptance of an object via an api, the object is assigned a UID (a UUID). + 1. must be non-empty and unique across space and time Example: "01234567-89ab-cdef-0123-456789abcdef" -3) Each container within a pod must have a Name string (a DNS_LABEL). - 1) must be non-empty and unique within the pod - Example: "frontend" -4) When a pod is bound to a node, the node is told the pod's uid. - 1) if not provided, the kubelet will generate one - 2) provides for pods from node-local config files +## Case study: Scheduling a pod -6) When a pod is bound to a node, the node is told the pod's Name. - 1) kubelet will namespace pods from distinct sources (e.g. files vs apiserver) - 2) namespaces must be deterministic - 3) provides a cluster-wide space-unique name - Example: Namespace="k8s.example.com" Name="guestbook.user" - Example: Namespace="k8s.example.com" Name="backend-x4eb1" - Example: Namespace="file-f4231812554558a718a01ca942782d81" Name="cadvisor" +Pods can be placed onto a particular node in a number of ways. This case +study demonstrates how the above design can be applied to satisfy the +objectives. -7) Each run of a container within a pod will be assigned an AttemptID (string) that is unique across time. - 1) corresponds to Docker's container ID - Example: "77af4d6b9913e693e8d0b4b294fa62ade6054e6b2f1ffb617ac955dd63fb0182" +### A pod scheduled by a user through the apiserver + +1. A user submits a pod named "guestbook" to the apiserver. + +2. The apiserver validates the input. + 1. The pod name must be space-unique within the apiserver. + 2. Each container within the pod has a name which must be space-unique within the pod. + +3. The pod is accepted and a UID is assigned. + +4. The pod is bound to a node. + 1. The kubelet on the node is passed the pod's UID and name. + +5. Kubelet validates the input. + +6. Kubelet further namespaces the pod name with information about the source of the pod. + 1. E.g. Namespace="api.k8s.example.com" + +7. Kubelet runs the pod. + 1. Each container is started up with enough metadata to distinguish the pod from whence it came. + 2. Each attempt to run a container is assigned a UID (a string) that is unique across time. + 1. This may correspond to Docker's container ID. + +### A pod placed by a config file on the node + +1. A config file is stored on the node, containing a pod named "cadvisor" with no UID. + +2. Kubelet validates the input. + 1. Since UID is not provided, kubelet generates one. + +3. Kubelet further namespaces the pod name with information about the source of the pod. + 1. The generated namespace should be deterministic and cluster-unique for the source. + 1. E.g. a hash of the hostname and file path. + 2. E.g. Namespace="file-f4231812554558a718a01ca942782d81" + +4. Kubelet runs the pod. + 1. Each container is started up with enough metadata to distinguish the pod from whence it came. + 2. Each attempt to run a container is assigned a UID (a string) that is unique across time. + 1. This may correspond to Docker's container ID. From 1eeed78a3886f6ea594e25b9ca23eebb093674ee Mon Sep 17 00:00:00 2001 From: Tim Hockin Date: Tue, 9 Sep 2014 23:05:38 -0700 Subject: [PATCH 4/5] Document auto-set names --- docs/identifiers.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/identifiers.md b/docs/identifiers.md index 07bc9391554..10bc3c35296 100644 --- a/docs/identifiers.md +++ b/docs/identifiers.md @@ -40,12 +40,11 @@ Name 1. must be non-empty and unique within the apiserver 2. enables idempotent and space-unique creation 1. generating random names will defeat idempotentcy + 2. for situations where generating a name is impractical, some or all objects may support a param to auto-generate a name 3. parts of the system (e.g. replication controller) may join strings (e.g. a base name and a random suffix) to create a unique Name Example: "guestbook.user" Example: "backend-x4eb1" -FIXME: final debate on having master default a name. Alternative: set "autosetName"=true - 2. Upon acceptance of an object via an api, the object is assigned a UID (a UUID). 1. must be non-empty and unique across space and time Example: "01234567-89ab-cdef-0123-456789abcdef" From d4b6f7ab3c5499a564a13cb8ca9f2645c12b7e7b Mon Sep 17 00:00:00 2001 From: Tim Hockin Date: Thu, 11 Sep 2014 11:04:54 -0700 Subject: [PATCH 5/5] Re-add namespaces --- docs/identifiers.md | 49 +++++++++++++++++++-------------------------- 1 file changed, 21 insertions(+), 28 deletions(-) diff --git a/docs/identifiers.md b/docs/identifiers.md index 10bc3c35296..1c0660c69c1 100644 --- a/docs/identifiers.md +++ b/docs/identifiers.md @@ -36,18 +36,14 @@ Name ## General design -1. When an object is created via an api, a Name string (a DNS_SUBDOMAIN) must be provided. - 1. must be non-empty and unique within the apiserver - 2. enables idempotent and space-unique creation - 1. generating random names will defeat idempotentcy - 2. for situations where generating a name is impractical, some or all objects may support a param to auto-generate a name - 3. parts of the system (e.g. replication controller) may join strings (e.g. a base name and a random suffix) to create a unique Name - Example: "guestbook.user" - Example: "backend-x4eb1" +1. When an object is created via an API, a Name string (a DNS_SUBDOMAIN) must be specified. Name must be non-empty and unique within the apiserver. This enables idempotent and space-unique creation operations. Parts of the system (e.g. replication controller) may join strings (e.g. a base name and a random suffix) to create a unique Name. For situations where generating a name is impractical, some or all objects may support a param to auto-generate a name. Generating random names will defeat idempotency. + * Examples: "guestbook.user", "backend-x4eb1" -2. Upon acceptance of an object via an api, the object is assigned a UID (a UUID). - 1. must be non-empty and unique across space and time - Example: "01234567-89ab-cdef-0123-456789abcdef" +2. When an object is created via an api, a Namespace string (a DNS_SUBDOMAIN? format TBD via #1114) may be specified. Depending on the API receiver, namespaces might be validated (e.g. apiserver might ensure that the namespace actually exists). If a namespace is not specified, one will be assigned by the API receiver. This assignment policy might vary across API receivers (e.g. apiserver might have a default, kubelet might generate something semi-random). + * Example: "api.k8s.example.com" + +3. Upon acceptance of an object via an API, the object is assigned a UID (a UUID). UID must be non-empty and unique across space and time. + * Example: "01234567-89ab-cdef-0123-456789abcdef" ## Case study: Scheduling a pod @@ -58,40 +54,37 @@ objectives. ### A pod scheduled by a user through the apiserver -1. A user submits a pod named "guestbook" to the apiserver. +1. A user submits a pod with Namespace="" and Name="guestbook" to the apiserver. 2. The apiserver validates the input. - 1. The pod name must be space-unique within the apiserver. - 2. Each container within the pod has a name which must be space-unique within the pod. + 1. A default Namespace is assigned. + 2. The pod name must be space-unique within the Namespace. + 3. Each container within the pod has a name which must be space-unique within the pod. -3. The pod is accepted and a UID is assigned. +3. The pod is accepted. + 1. A new UID is assigned. 4. The pod is bound to a node. - 1. The kubelet on the node is passed the pod's UID and name. + 1. The kubelet on the node is passed the pod's UID, Namespace, and Name. 5. Kubelet validates the input. -6. Kubelet further namespaces the pod name with information about the source of the pod. - 1. E.g. Namespace="api.k8s.example.com" - -7. Kubelet runs the pod. +6. Kubelet runs the pod. 1. Each container is started up with enough metadata to distinguish the pod from whence it came. 2. Each attempt to run a container is assigned a UID (a string) that is unique across time. - 1. This may correspond to Docker's container ID. + * This may correspond to Docker's container ID. ### A pod placed by a config file on the node -1. A config file is stored on the node, containing a pod named "cadvisor" with no UID. +1. A config file is stored on the node, containing a pod with UID="", Namespace="", and Name="cadvisor". 2. Kubelet validates the input. 1. Since UID is not provided, kubelet generates one. + 2. Since Namespace is not provided, kubelet generates one. + 1. The generated namespace should be deterministic and cluster-unique for the source, such as a hash of the hostname and file path. + * E.g. Namespace="file-f4231812554558a718a01ca942782d81" -3. Kubelet further namespaces the pod name with information about the source of the pod. - 1. The generated namespace should be deterministic and cluster-unique for the source. - 1. E.g. a hash of the hostname and file path. - 2. E.g. Namespace="file-f4231812554558a718a01ca942782d81" - -4. Kubelet runs the pod. +3. Kubelet runs the pod. 1. Each container is started up with enough metadata to distinguish the pod from whence it came. 2. Each attempt to run a container is assigned a UID (a string) that is unique across time. 1. This may correspond to Docker's container ID.