From 7f1970a22d7fc76db81fa5d05c2f0c565cbcd361 Mon Sep 17 00:00:00 2001 From: Eric Tune Date: Fri, 17 Jul 2015 16:29:29 -0700 Subject: [PATCH] Improve resource quota doc. --- docs/admin/etcd.md | 1 + docs/admin/resource-quota.md | 87 +++++++++++++++++++++++++----------- 2 files changed, 62 insertions(+), 26 deletions(-) diff --git a/docs/admin/etcd.md b/docs/admin/etcd.md index dc315fc36c8..f37827e84fe 100644 --- a/docs/admin/etcd.md +++ b/docs/admin/etcd.md @@ -30,6 +30,7 @@ Documentation for other releases can be found at + # etcd [etcd](https://coreos.com/etcd/docs/2.0.12/) is a highly-available key value diff --git a/docs/admin/resource-quota.md b/docs/admin/resource-quota.md index 21bec1eb75b..c70698205fb 100644 --- a/docs/admin/resource-quota.md +++ b/docs/admin/resource-quota.md @@ -31,12 +31,35 @@ Documentation for other releases can be found at -# Administering Resource Quotas +# Resource Quotas -Kubernetes can limit both the number of objects created in a namespace, and the -total amount of resources requested by pods in a namespace. This facilitates -sharing of a single Kubernetes cluster by several teams or tenants, each in -a namespace. +When several users or teams share a cluster with a fixed number of nodes, +there is a concern that one team could use more than its fair share of resources. + +Resource quotas are a tool for administrators to address this concern. Resource quotas +work like this: +- Different teams work in different namespaces. Currently this is voluntary, but + support for making this mandatory via ACLs is planned. +- Users put [compute resource limits](../user-guide/compute-resources.md) on their pods. +- The administrator creates a Resource Quota for each namespace. +- If users exceed the limits specified in the Resource Quota in a namespace, they are blocked + from creating more pods in that namespace. The POST of the pod will fail with HTTP status + code `403 FORBIDDEN`. +- If quota is enabled in a namespace and the user does not specify limits on the pod for each + of the resources for which quota is enabled, then the POST of the pod will fail with HTTP + status code `403 FORBIDDEN`. Hint: Use the LimitRange admission controller to force default + values of limits before the quota is checked to avoid this problem. + +Examples of policies that could be created using namespaces and quotas are: +- In a cluster with a capacity of 32 GiB RAM, and 16 cores, let team A use 20 Gib and 10 cores, + let B use 10GiB and 4 cores, and hold 2GiB and 2 cores in reserve for future allocation. +- Limit the "testing" namespace to using 1 core and 1GiB RAM. Let the "production" namespace + use any amount. + +In the case where the total capacity of the cell is less than the sum of the quotas of the namespaces, +there may be contention for resources. This is handled on a first-come-first-served basis. + +Neither contention nor changes to quota will affect already-running pods. ## Enabling Resource Quota @@ -48,7 +71,24 @@ Resource Quota is enforced in a particular namespace when there is a `ResourceQuota` object in that namespace. There should be at most one `ResourceQuota` object in a namespace. -See [ResourceQuota design doc](../design/admission_control_resource_quota.md) for more information. +## Compute Resource Quota + +The total sum of [compute resources](../user-guide/compute-resources.md) requested by pods +in a namespace can be limited. The following compute resource types are supported: + +| ResourceName | Description | +| ------------ | ----------- | +| cpu | Total cpu limits of containers | +| memory | Total memory limits of containers +| `example.com/customresource` | Total of `resources.limits."example.com/customresource"` of containers | + +For example, `cpu` quota sums up the `resources.limits.cpu` fields of every +container of every pod in the namespace, and enforces a maximum on that sum. + +Any resource that is not part of core Kubernetes must follow the resource naming convention prescribed by Kubernetes. + +This means the resource must have a fully-qualified name (i.e. mycompany.org/shinynewresource) + ## Object Count Quota @@ -67,23 +107,9 @@ are supported: For example, `pods` quota counts and enforces a maximum on the number of `pods` created in a single namespace. -## Compute Resource Quota - -The total number of objects of a given type can be restricted. The following types -are supported: - -| ResourceName | Description | -| ------------ | ----------- | -| cpu | Total cpu limits of containers | -| memory | Total memory usage limits of containers -| `example.com/customresource` | Total of `resources.limits."example.com/customresource"` of containers | - -For example, `cpu` quota sums up the `resources.limits.cpu` fields of every -container of every pod in the namespace, and enforces a maximum on that sum. - -Any resource that is not part of core Kubernetes must follow the resource naming convention prescribed by Kubernetes. - -This means the resource must have a fully-qualified name (i.e. mycompany.org/shinynewresource) +You might want to set a pods quota on a namespace +to avoid the case where a user creates many small pods and exhausts the cluster's +supply of Pod IPs. ## Viewing and Setting Quotas @@ -128,22 +154,31 @@ services 3 5 ## Quota and Cluster Capacity -Resource Quota objects are independent of the Cluster Capacity. They are -expressed in absolute units. +Resource Quota objects are independent of the Cluster Capacity. They are +expressed in absolute units. So, if you add nodes to your cluster, this does *not* +automatically give each namespace the ability to consume more resources. Sometimes more complex policies may be desired, such as: - proportionally divide total cluster resources among several teams. - allow each tenant to grow resource usage as needed, but have a generous limit to prevent accidental resource exhaustion. + - detect demand from one namespace, add nodes, and increase quota. Such policies could be implemented using ResourceQuota as a building-block, by writing a 'controller' which watches the quota usage and adjusts the quota -hard limits of each namespace. +hard limits of each namespace according to other signals. + +Note that resource quota divides up aggregate cluster resources, but it creates no +restrictions around nodes: pods from several namespaces may run on the same node. ## Example See a [detailed example for how to use resource quota](../user-guide/resourcequota/). +## Read More + +See [ResourceQuota design doc](../design/admission_control_resource_quota.md) for more information. + [![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/admin/resource-quota.md?pixel)]()