github/kubernetes
1
0
Fork 0
mirror of https://github.com/k3s-io/kubernetes.git synced 2025-08-04 18:00:08 +00:00
Code Issues Projects Releases Wiki Activity

Merge pull request #3057 from derekwaynecarr/resource_controller_proposal

Browse Source 
Admission control plugins: LimitRanger and ResourceQuota
This commit is contained in:
Eric Tune 2015-01-26 10:46:20 -08:00
commit 4c2d554c73
2 changed files with 274 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

122
docs/design/admission_control_limit_range.md Normal file
View File

@ -0,0 +1,122 @@
# Admission control plugin: LimitRanger
## Background
This document proposes a system for enforcing min/max limits per resource as part of admission control.
## Model Changes
A new resource, **LimitRange**, is introduced to enumerate min/max limits for a resource type scoped to a
Kubernetes namespace.
```
const (
// Limit that applies to all pods in a namespace
LimitTypePod string = "Pod"
// Limit that applies to all containers in a namespace
LimitTypeContainer string = "Container"
)
// LimitRangeItem defines a min/max usage limit for any resource that matches on kind
type LimitRangeItem struct {
// Type of resource that this limit applies to
Type string `json:"type,omitempty"`
// Max usage constraints on this kind by resource name
Max ResourceList `json:"max,omitempty"`
// Min usage constraints on this kind by resource name
Min ResourceList `json:"min,omitempty"`
}
// LimitRangeSpec defines a min/max usage limit for resources that match on kind
type LimitRangeSpec struct {
// Limits is the list of LimitRangeItem objects that are enforced
Limits []LimitRangeItem `json:"limits"`
}
// LimitRange sets resource usage limits for each kind of resource in a Namespace
type LimitRange struct {
TypeMeta `json:",inline"`
ObjectMeta `json:"metadata,omitempty"`
// Spec defines the limits enforced
Spec LimitRangeSpec `json:"spec,omitempty"`
}
// LimitRangeList is a list of LimitRange items.
type LimitRangeList struct {
TypeMeta `json:",inline"`
ListMeta `json:"metadata,omitempty"`
// Items is a list of LimitRange objects
Items []LimitRange `json:"items"`
}
```
## AdmissionControl plugin: LimitRanger
The **LimitRanger** plug-in introspects all incoming admission requests.
It makes decisions by evaluating the incoming object against all defined **LimitRange** objects in the request context namespace.
The following min/max limits are imposed:
**Type: Container**
| ResourceName | Description |
| ------------ | ----------- |
| cpu | Min/Max amount of cpu per container |
| memory | Min/Max amount of memory per container |
**Type: Pod**
| ResourceName | Description |
| ------------ | ----------- |
| cpu | Min/Max amount of cpu per pod |
| memory | Min/Max amount of memory per pod |
If the incoming object would cause a violation of the enumerated constraints, the request is denied with a set of
messages explaining what constraints were the source of the denial.
If a constraint is not enumerated by a **LimitRange** it is not tracked.
## kube-apiserver
The server is updated to be aware of **LimitRange** objects.
The constraints are only enforced if the kube-apiserver is started as follows:
```
$ kube-apiserver -admission_control=LimitRanger
```
## kubectl
kubectl is modified to support the **LimitRange** resource.
```kubectl describe``` provides a human-readable output of limits.
For example,
```
$ kubectl namespace myspace
$ kubectl create -f examples/limitrange/limit-range.json
$ kubectl get limits
NAME
limits
$ kubectl describe limits limits
Name: limits
Type Resource Min Max
---- -------- --- ---
Pod memory 1Mi 1Gi
Pod cpu 250m 2
Container cpu 250m 2
Container memory 1Mi 1Gi
```
## Future Enhancements: Define limits for a particular pod or container.
In the current proposal, the **LimitRangeItem** matches purely on **LimitRangeItem.Type**
It is expected we will want to define limits for particular pods or containers by name/uid and label/field selector.
To make a **LimitRangeItem** more restrictive, we will intend to add these additional restrictions at a future point in time.

152
docs/design/admission_control_resource_quota.md Normal file
View File

@ -0,0 +1,152 @@
# Admission control plugin: ResourceQuota
## Background
This document proposes a system for enforcing hard resource usage limits per namespace as part of admission control.
## Model Changes
A new resource, **ResourceQuota**, is introduced to enumerate hard resource limits in a Kubernetes namespace.
A new resource, **ResourceQuotaUsage**, is introduced to support atomic updates of a **ResourceQuota** status.
```
// The following identify resource constants for Kubernetes object types
const (
// Pods, number
ResourcePods ResourceName = "pods"
// Services, number
ResourceServices ResourceName = "services"
// ReplicationControllers, number
ResourceReplicationControllers ResourceName = "replicationcontrollers"
// ResourceQuotas, number
ResourceQuotas ResourceName = "resourcequotas"
)
// ResourceQuotaSpec defines the desired hard limits to enforce for Quota
type ResourceQuotaSpec struct {
// Hard is the set of desired hard limits for each named resource
Hard ResourceList `json:"hard,omitempty"`
}
// ResourceQuotaStatus defines the enforced hard limits and observed use
type ResourceQuotaStatus struct {
// Hard is the set of enforced hard limits for each named resource
Hard ResourceList `json:"hard,omitempty"`
// Used is the current observed total usage of the resource in the namespace
Used ResourceList `json:"used,omitempty"`
}
// ResourceQuota sets aggregate quota restrictions enforced per namespace
type ResourceQuota struct {
TypeMeta `json:",inline"`
ObjectMeta `json:"metadata,omitempty"`
// Spec defines the desired quota
Spec ResourceQuotaSpec `json:"spec,omitempty"`
// Status defines the actual enforced quota and its current usage
Status ResourceQuotaStatus `json:"status,omitempty"`
}
// ResourceQuotaUsage captures system observed quota status per namespace
// It is used to enforce atomic updates of a backing ResourceQuota.Status field in storage
type ResourceQuotaUsage struct {
TypeMeta `json:",inline"`
ObjectMeta `json:"metadata,omitempty"`
// Status defines the actual enforced quota and its current usage
Status ResourceQuotaStatus `json:"status,omitempty"`
}
// ResourceQuotaList is a list of ResourceQuota items
type ResourceQuotaList struct {
TypeMeta `json:",inline"`
ListMeta `json:"metadata,omitempty"`
// Items is a list of ResourceQuota objects
Items []ResourceQuota `json:"items"`
}
```
## AdmissionControl plugin: ResourceQuota
The **ResourceQuota** plug-in introspects all incoming admission requests.
It makes decisions by evaluating the incoming object against all defined **ResourceQuota.Status.Hard** resource limits in the request
namespace. If acceptance of the resource would cause the total usage of a named resource to exceed its hard limit, the request is denied.
The following resource limits are imposed as part of core Kubernetes at the namespace level:
| ResourceName | Description |
| ------------ | ----------- |
| cpu | Total cpu usage |
| memory | Total memory usage |
| pods | Total number of pods |
| services | Total number of services |
| replicationcontrollers | Total number of replication controllers |
| resourcequotas | Total number of resource quotas |
Any resource that is not part of core Kubernetes must follow the resource naming convention prescribed by Kubernetes.
This means the resource must have a fully-qualified name (i.e. mycompany.org/shinynewresource)
If the incoming request does not cause the total usage to exceed any of the enumerated hard resource limits, the plug-in will post a
**ResourceQuotaUsage** document to the server to atomically update the observed usage based on the previously read
**ResourceQuota.ResourceVersion**. This keeps incremental usage atomically consistent, but does introduce a bottleneck (intentionally)
into the system.
To optimize system performance, it is encouraged that all resource quotas are tracked on the same **ResourceQuota** document. As a result,
its encouraged to actually impose a cap on the total number of individual quotas that are tracked in the **Namespace** to 1 by explicitly
capping it in **ResourceQuota** document.
## kube-apiserver
The server is updated to be aware of **ResourceQuota** objects.
The quota is only enforced if the kube-apiserver is started as follows:
```
$ kube-apiserver -admission_control=ResourceQuota
```
## kube-controller-manager
A new controller is defined that runs a synch loop to calculate quota usage across the namespace.
**ResourceQuota** usage is only calculated if a namespace has a **ResourceQuota** object.
If the observed usage is different than the recorded usage, the controller sends a **ResourceQuotaUsage** resource
to the server to atomically update.
The synchronization loop frequency will control how quickly DELETE actions are recorded in the system and usage is ticked down.
To optimize the synchronization loop, this controller will WATCH on Pod resources to track DELETE events, and in response, recalculate
usage. This is because a Pod deletion will have the most impact on observed cpu and memory usage in the system, and we anticipate
this being the resource most closely running at the prescribed quota limits.
## kubectl
kubectl is modified to support the **ResourceQuota** resource.
```kubectl describe``` provides a human-readable output of quota.
For example,
```
$ kubectl namespace myspace
$ kubectl create -f examples/resourcequota/resource-quota.json
$ kubectl get quota
NAME
myquota
$ kubectl describe quota myquota
Name: myquota
Resource Used Hard
-------- ---- ----
cpu 100m 20
memory 0 1.5Gb
pods 1 10
replicationControllers 1 10
services 2 3
```