From 2a8cc9a3a7726d9efe11b083d1cc4d63ed070f4b Mon Sep 17 00:00:00 2001 From: Eric Tune Date: Wed, 13 Aug 2014 16:36:37 -0700 Subject: [PATCH 1/8] Access.md added. --- docs/access.md | 353 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 353 insertions(+) create mode 100644 docs/access.md diff --git a/docs/access.md b/docs/access.md new file mode 100644 index 00000000000..252d24c574f --- /dev/null +++ b/docs/access.md @@ -0,0 +1,353 @@ +# K8s Identity and Access Management Sketch + +This document suggests a direction for identity and access management in the Kubernetes system. + + +## Background + +High level goals are: + - Have a plan for how identity, authentication, and authorization will fit in to the API. + - Have a plan for partitioning resources within a cluster between independent organizational units. + - Ease integration with existing enterprise and hosted scenarios. + +### Actors +Each of these can act as normal users or attackers. + - End users (e.g. visitors to a website which is served by programs which are run on kubernetes) + - k8s users (e.g. people who create k8s API objects like Pods) + - k8s insiders (e.g. people who setup k8s binaries, or others with special access to machines and networks.) + +### Threats +Both intentional attacks and accidental use of privilege are concerns. + +For both cases it may be useful to think about these categories differently: + - Application Path - attack by sending network messages from the internet to the IP/port of any application running on k8s. May exploit weakness in application or misconfiguration of k8s. + - K8s API Path - attack by sending network messages to any k8s infrastructure network endpoint. Attacker may have k8s user identity. + - Insider Path - attack on k8s system components. Attacker may have privileged access to networks, machines or k8s software and data. Software errors in k8s system components and administrator error are some types of threat in this category. + +This document is primarily concerned with K8s API paths, and secondarily with Internal paths. The Application path also needs to be secure, but is not the focus of this document. + +### Assets to protect + +End user assets: + - Personal information like private messages, or images uploaded by end users + - web server logs + +K8s user assets: + - end-user assets of each k8s user + - things private to the K8s app, like: + - credentials for accessing other services (docker private repos, storage services, facebook, etc) + - SSL certificates for web servers + - proprietary data and code + +K8s Cluster assets: + - Assets of each k8s-user + - certificates which identify the k8s machines and servers to users. + - cluster resources (the value of k8s computing resources). + +This document is primarily about protecting k8s user assets and K8s cluster assets from other k8s users and k8s insiders. + +### Usage environments +Cluster in Small organization: + - k8s insiders may be the same people as k8s users. + - few k8s insiders. + - prefer ease of use to fine-grained access control/precise accounting, etc. + - Product requirement that it be easy for potential users to try out setting up a simple cluster. + +Cluster in Large organization: + - k8s insiders typically distinct people from k8s users. may need to divide k8s insider access by roles. + - k8s users need to be protected from each other. + - auditing of k8s user and k8s insider actions important. + - flexible accurate usage accounting and resource controls important. + - Lots of automated access to APIs. + - Need to integrate with existing enterprise directory, authentication, accounting, auditing, and security policy infrastructure. + +Org-run cluster: + - organization that runs k8s master components is same as the org that runs apps on k8s. + - Minions may be on-premises VMs or physical machines; Cloud VMs; or a mix. + +Hosted cluster: + - Offering K8s API as a service, or offering a Paas or Saas built on K8s + - May already offer web services, and need to integrate with existing customer account concept, and existing authentication, accounting, auditing, and security policy infrastructure. + - May want to leverage K8s user accounts and accounting to manage their user accounts (not a priority to support this use case.) + - Precise and accurate accounting of resources needed. + +K8s ecosystem services: + - There may be companies that want to offer their existing services (Build, CI, A/B-test, release automation, etc) for use with K8s. There should be some story for this case. + +Pods configs should be largely portable between Org-run and hosted configurations. + + +# Design +Related discussion: https://github.com/GoogleCloudPlatform/kubernetes/issues/443 + +This doc describes two security profiles: + - Simple profile: like single-user mode. Make it easy to evaluate k8s without lots of configuring accounts and policies. Protects from unauthorized users, but does not partition authorized users. + - Enterprise profile: Provide mechanisms needed for large numbers of users. Defence in depth. Should integrate with existing enterprise security infrastructure. + +K8s distribution should include templates of config, and documentation, for simple and enterprise profiles. System should be flexible enough for knowledgeable users to create intermediate profiles, but K8s developers should only reason about those two Profiles, not a matrix. + +Features in this doc are divided into "Initial Feature", and "Improvements". Initial features would be candidates for version 1.00. + +## Identity +###userAccount +K8s will have a `userAccount` API object. +- `userAccount` has a name which is a string. +- `userAccount` is not the same as the unix username of processes +- 'userAccount` objects have a name. The name should allow relatively free-form strings (at least flexible enough to hold most email addresses) to aid integration with existing enterprise auth systems, or existing account identifiers in hosting systems. +- `userAccount` API objects can have labels + +Initial Features: +- `userAccount` object is immutable +- there is no superuser `userAccount` +- `userAccount` objects are statically populated in the k8s API store by reading a config file. Only a k8s admin can do this. +- `userAccount` can have a default `project`. If API call does not specify a `project`, the default `project` for that caller is assumed. +- `userAccount` may access multiple projects. + +Improvements: +- Have API calls to create and delete `userAccount`s. +- Make `userAccount` part of a separate API group from core k8s objects like `pod`. Facilitates plugging in alternate Access Management. + +Simple Profile: + - single `userAccount`, used by all users. + +Enterprise Profile: + - every human user has own `userAccount`. + - `userAccount`s have labels that indicate both membership in groups, and ability to act in certain roles. + - each service using the API has own `userAccount` too. (e.g. `scheduler`, `repcontroller`) + - list of users may be synced in from existing enterprise directory service. + - issue: deleting username still being referenced by pod, etc. + +###Unix accounts +A `userAccount` is not a Unix user account. The fact that a pod is started by a `userAccount` does not mean that the processes in that pod's containers run as a Unix user with a corresponding name or identity. + +Initially: +- The unix accounts available in a container, and used by the processes running in a container are those that are provided by the combination of the base operating system and the Docker manifest. +- Kubernetes doesn't enforce any relation between `userAccount` and unix accounts. + +Improvements: +- Kubelet allocates disjoint blocks of root-namespace uids for each container. This may provide some defense-in-depth against container escapes. (https://github.com/docker/docker/pull/4572) +- requires docker to integrate user namespace support, and deciding what getpwnam() does for these uids. +- any features that help users avoid use of privileged containers (https://github.com/GoogleCloudPlatform/kubernetes/issues/391) + +###project +K8s will have a have a `project` API object. + +Initial Features: +- `project` object is immutable. +- `project` objects are statically populated in the k8s API store by reading a config file. Only admin can do this. +- In order to allow using `project` name as namespace for objects under that `project`, and to ensure the compound names are still DNS-compatible names, `project` names must be DNS label format. + +Improvements: +- have API calls to create and delete `project` objects. +- sync in from enterprise directory service. + +Most API objects have an associated `project`: +- pods have a `project`. +- `project`s don't have a `project`, nor do `userAccount`s + - or else they belong to a special global `project`? +- An object's `project` cannot be changed after creation. + +In the Enterprise Profile: + - a `userAccount` may have permission to access several `project`s. API calls need to specify a `project`. Client libs may fill this in using a stored preference. + +In the Simple Profile: + - There is a single default `project`. No need to specify `project` when making an API call. + +Project versus userAccount vs Labels: +- `userAccount`s are intended for audit logging, and to define who has access to `project`s. +- `labels` (see docs/labels.md) should be used to distinguish pods, users, and other objects that cooperate towards a common goal but are different in some way, such as version, or responsibilities. +- `project`s provide a namespace for objects. + + +how is `project` selected when making a REST call? +- query parameter +- default to only project if there is only one. +- subdomain, e.g. http://myproject.kubernetes.example.com. nginx proxy can translate that to a query parameter + +## Authentication + +Initially: +- Use bearer tokens to authenticate users. +- Each API request must include a "Authorization: Bearer " header. +- A token identifies a particular `userAccount`. +- Administrator utility generates tokens at cluster setup. +- Tokens are long lived. +- Tokens are just long random strings base64 encoded. No content. + +Improvements: +- make it harder for the wrong party to use the token. + - shorter lived tokens + - use tokens that are bound to the channel between the client and the api server + +Things to consider for Improved implementation: +- JWT http://tools.ietf.org/html/draft-ietf-oauth-json-web-token-13 + - Possible library: http://code.google.com/p/goauth2/oauth/jwt + - Use with endpoint that generates short-term bearer token. +- tokens that are bound to the channel between the client and the api server + - http://www.ietf.org/proceedings/90/slides/slides-90-uta-0.pdf + - http://ww.browserauth.net + +Where to do authentication. Either: +- Authenticate in reverse proxy (nginx). Proxy either rejects invalid token or appends authorized_user identifier to the request before passing to APIserver. +- Apiserver checks token. + +Considerations: +- In some arrangements, the proxy might terminate SSL, and use a CA-signed certificate, while the APIserver might just use a self-signed or organization-signed certificate. +- some large orgs will already have existing SaaS web services (e.g. storage, VMs). Some will already have a gateway that handles auth for multiple services. If K8s does auth in the proxy, then those orgs can just replace the proxy with their existing web service gateway. +- nginx is more stable than k8s code. Prefer to put secrets (tokens, SSL cert.) in lower-touch place. +- Admins configuring k8s for enterprise use are more likely to know how to config a proxy than to modify Go code of apiserver. + +Based on above considerations, auth should happen in the proxy. + +Implementation options: +- Simplest: a list of encoded tokens is stored in nginx config file as a map. a rewrite rule (http://nginx.org/en/docs/http/ngx_http_rewrite_module.html) adds authorized_user if Auth header is in the is in the map, else returns a 401 Unauthorized. +- nginx can defer auth to a separate auth server. This could aid integration with existing enterprise auth solutions. e.g.: + - https://github.com/algermissen/nginx-dlg-auth/blob/master/README.md + - http://nginx.org/en/docs/http/ngx_http_auth_request_module.html +- nginx can have lua plugin: https://github.com/openresty/lua-nginx-module +- User can provide own proxy, other than nginx. + + +## Authorization + +Authorization is done by the APIserver. + +Authorization is done using Policy objects. + + +### Policy objects +Policy objects are API objects. They express http://en.wikipedia.org/wiki/Attribute_Based_Access_Control + +Simple Profile: +- one Policy object that allows the single `userAccount` to CRUD objects in the single `project`. + +Enterprise Profile: +- Many policy objects in each of many projects. +- Tools and services that wrap policy creation interface to enforce meta policies, do template expansions, report, etc. + + + +Initial Features: +- Policy object is immutable +- Policy objects are statically populated in the k8s API store by reading a config file. Only admin can do this. +- Just a few policies per `project` which list which users can create objects, which can just view, them, etc. +- Objects are created with reference to these default policies. + +Improvements: +- Have API calls to create and delete and modify Policy objects. These would be in a separate API group from core k8s APIs. This allows for replacing the k8s authorization service with an alternate implementation, and to centralize policies that might apply to services other than k8s. +- Ability to change policy for an object. +- Ability to create an object with a non-default policy effective immediately at creation time. +- Ability to defer policy object checking to a policy server. +- Authorization tokens to authorize entities without a `userAccount`. + + +### Policy object format + +Policy Object: +```go +type policies map[string]policy + +type policy { + project string, // ref to project of this Policy, to namespace the name. + name string, // name within the project name of the Policy + a PolicyType, + s Subject, + v Verb, + o Object + expires string // RFC3339 +} +type PolicyType string +const { + ALLOW PolicyType = "ALLOW", + // Later DENY, etc. +} + +type Subject string // Serialized label selector +type Verb string +const { + GET Verb = "GET", + // ... other HTTP methods. + ANY Verb = "ANY" // any http method + CREATE Verb = "CREATE" // PUT or POST + // ... +} +type Object { + exact string // any resource with exactly this path + // OR + prefix string // any resource with this path prefix (after removing "/api//") + // OR + where string // serialized label selector. +} +``` + +Ideally, lists of policy objects would have relatively concise and readable YAML forms, such as: +``` +{name: bob_can_read_pods, a: ALLOW, s: user.name is bob@example.com, v: GET, prefix: /pods} +{name: admins_can_delete_pods, a: ALLOW, s: user.role is admin, v: DELETE, prefix: /pods} +{name: tmp1234, a: ALLOW, s: user.name is "some.agent", v: POST, prefix: /pods/somepod, expires: 2014-08-13 16:21:42-07:00 } +``` + +Requests that don't match at least one ALLOW are not allowed. +TODO: define DENY, and other operations and their precedence. + +Delegation can be implemented by writing new narrowly tailored policies. +TODO: example of policy to delegate pod creation from a podTemplate (see https://github.com/GoogleCloudPlatform/kubernetes/issues/170). + +### Architecture for Authorization +When the APIserver receives a new request, it passes the +the `userAccount`, http method, and http path to an `Authorize() method`. + +In a simple implementation, the Authorize() module: + - runs in the APIserver + - searches all policy objects for a match. + - updates its cache when new Policy is added. + +In alternate implementations, it may: + - have indexes to speed matching. (Maybe this can share code with Label Queries.) + - defer to a centralized auth server for the enterprise. + + +### Labels +Initially, IIUC, labels are strings and not API objects in their own right. +Eventually, labels may have policies or namespaces which restrict application of certain labels. + + +## Accounting + +The API should have a `quota` concept (see (https://github.com/GoogleCloudPlatform/kubernetes/issues/442 +). A quota object relates a project (and optionally a label selector) to a maximum quantity of resources that may be used (see resources.md). + +Initially: +- a `quota` object is immutable. +- for hosted k8s systems that do billing, Project is recommended level for billing accounts. +- Every object that consumes resources should have a `project` so that Resource usage stats are roll-up-able to `project`. +- admin sets quota objects by writing a config file. + +Improvements: +- allow one project to charge the quota for one or more other projects. This would be controlled by a policy which allows changing a billing_project= label on an object. +- allow quota to be set by project owners for (project x label) combinations (e.g. let "webserver" project use 100 cores, but to prevent accidents, don't allow "webserver" project and "instance=test" use more than 10 cores. +- tools to help write consistent quota config files based on number of minions, historical project usages, QoS needs, etc. +- way for admin to incrementally adjust Quota objects. + +Simple profile: + - a single `project` with infinite resource limits. + +Enterprise profile: + - multiple projects + +Issues: +- need for locking or "eventual consistency" when multiple apiserver goroutines are accessing the object store and handling pod creations. + + +## Audit Logging + +API actions can be logged. + +Initial implementation: +- All API calls logged to nginx logs. + +Improvements: +- API server does logging instead. +- Policies to drop logging for high rate trusted API calls, or by users performing audit or other sensitive functions. + + From 49450fcebdd5bae7bcc3406950956e0e60b3a57a Mon Sep 17 00:00:00 2001 From: Eric Tune Date: Thu, 21 Aug 2014 13:41:42 -0700 Subject: [PATCH 2/8] Minor changes and fixes. --- docs/access.md | 34 ++++++++++++++-------------------- 1 file changed, 14 insertions(+), 20 deletions(-) diff --git a/docs/access.md b/docs/access.md index 252d24c574f..ba70d1fbce8 100644 --- a/docs/access.md +++ b/docs/access.md @@ -41,8 +41,8 @@ K8s user assets: K8s Cluster assets: - Assets of each k8s-user - - certificates which identify the k8s machines and servers to users. - - cluster resources (the value of k8s computing resources). + - Certificates or secrets which identify the k8s machines and servers to users. + - The value of k8s cluster computing resources (cpu, memory, etc). This document is primarily about protecting k8s user assets and K8s cluster assets from other k8s users and k8s insiders. @@ -69,7 +69,7 @@ Hosted cluster: - Offering K8s API as a service, or offering a Paas or Saas built on K8s - May already offer web services, and need to integrate with existing customer account concept, and existing authentication, accounting, auditing, and security policy infrastructure. - May want to leverage K8s user accounts and accounting to manage their user accounts (not a priority to support this use case.) - - Precise and accurate accounting of resources needed. + - Precise and accurate accounting of resources needed. Resource controls needed for hard limits (users given limited slice of data) and soft limits (users can grow up to some limit and then be expanded). K8s ecosystem services: - There may be companies that want to offer their existing services (Build, CI, A/B-test, release automation, etc) for use with K8s. There should be some story for this case. @@ -78,7 +78,9 @@ Pods configs should be largely portable between Org-run and hosted configuration # Design -Related discussion: https://github.com/GoogleCloudPlatform/kubernetes/issues/443 +Related discussion: +- https://github.com/GoogleCloudPlatform/kubernetes/issues/442 +- https://github.com/GoogleCloudPlatform/kubernetes/issues/443 This doc describes two security profiles: - Simple profile: like single-user mode. Make it easy to evaluate k8s without lots of configuring accounts and policies. Protects from unauthorized users, but does not partition authorized users. @@ -104,7 +106,6 @@ Initial Features: - `userAccount` may access multiple projects. Improvements: -- Have API calls to create and delete `userAccount`s. - Make `userAccount` part of a separate API group from core k8s objects like `pod`. Facilitates plugging in alternate Access Management. Simple Profile: @@ -114,8 +115,7 @@ Enterprise Profile: - every human user has own `userAccount`. - `userAccount`s have labels that indicate both membership in groups, and ability to act in certain roles. - each service using the API has own `userAccount` too. (e.g. `scheduler`, `repcontroller`) - - list of users may be synced in from existing enterprise directory service. - - issue: deleting username still being referenced by pod, etc. + - automated jobs to denormalize the ldap group info into the local system list of users into the k8s userAccount file. ###Unix accounts A `userAccount` is not a Unix user account. The fact that a pod is started by a `userAccount` does not mean that the processes in that pod's containers run as a Unix user with a corresponding name or identity. @@ -139,7 +139,6 @@ Initial Features: Improvements: - have API calls to create and delete `project` objects. -- sync in from enterprise directory service. Most API objects have an associated `project`: - pods have a `project`. @@ -160,9 +159,12 @@ Project versus userAccount vs Labels: how is `project` selected when making a REST call? -- query parameter - default to only project if there is only one. +- query parameter - subdomain, e.g. http://myproject.kubernetes.example.com. nginx proxy can translate that to a query parameter + - Subdomains have potential scaling limits. If using project names to identify the domain, you have to defend against profane or vanity names, and probably support blacklist limits on new project names. In non-hosted environments is less of an issue. Requires that the apiservers be tied into DNS, which is onerous to configure in some environments. Nice in some contexts. +- offering a project API scope /projects// +- global access via globally unique id (where supported, which isn't very consistent today) /pods/ ## Authentication @@ -185,28 +187,20 @@ Things to consider for Improved implementation: - Use with endpoint that generates short-term bearer token. - tokens that are bound to the channel between the client and the api server - http://www.ietf.org/proceedings/90/slides/slides-90-uta-0.pdf - - http://ww.browserauth.net + - http://www.browserauth.net Where to do authentication. Either: -- Authenticate in reverse proxy (nginx). Proxy either rejects invalid token or appends authorized_user identifier to the request before passing to APIserver. +- Authenticate in reverse proxy (currently nginx, could easily be apache+mod_auth). Proxy either rejects invalid token or appends authorized_user identifier to the request before passing to APIserver. - Apiserver checks token. Considerations: - In some arrangements, the proxy might terminate SSL, and use a CA-signed certificate, while the APIserver might just use a self-signed or organization-signed certificate. - some large orgs will already have existing SaaS web services (e.g. storage, VMs). Some will already have a gateway that handles auth for multiple services. If K8s does auth in the proxy, then those orgs can just replace the proxy with their existing web service gateway. -- nginx is more stable than k8s code. Prefer to put secrets (tokens, SSL cert.) in lower-touch place. +- Apache or nginx is more stable than k8s code. Prefer to put secrets (tokens, SSL cert.) in lower-touch place. - Admins configuring k8s for enterprise use are more likely to know how to config a proxy than to modify Go code of apiserver. Based on above considerations, auth should happen in the proxy. -Implementation options: -- Simplest: a list of encoded tokens is stored in nginx config file as a map. a rewrite rule (http://nginx.org/en/docs/http/ngx_http_rewrite_module.html) adds authorized_user if Auth header is in the is in the map, else returns a 401 Unauthorized. -- nginx can defer auth to a separate auth server. This could aid integration with existing enterprise auth solutions. e.g.: - - https://github.com/algermissen/nginx-dlg-auth/blob/master/README.md - - http://nginx.org/en/docs/http/ngx_http_auth_request_module.html -- nginx can have lua plugin: https://github.com/openresty/lua-nginx-module -- User can provide own proxy, other than nginx. - ## Authorization From 6409ccbaa8971359239a63d161b31533cb0836dd Mon Sep 17 00:00:00 2001 From: Eric Tune Date: Fri, 22 Aug 2014 15:02:33 -0700 Subject: [PATCH 3/8] Redefine Actors; use consistently; Capitalize k8s. --- docs/access.md | 74 ++++++++++++++++++++++++++------------------------ 1 file changed, 38 insertions(+), 36 deletions(-) diff --git a/docs/access.md b/docs/access.md index ba70d1fbce8..5ebfab82fe4 100644 --- a/docs/access.md +++ b/docs/access.md @@ -12,64 +12,66 @@ High level goals are: ### Actors Each of these can act as normal users or attackers. - - End users (e.g. visitors to a website which is served by programs which are run on kubernetes) - - k8s users (e.g. people who create k8s API objects like Pods) - - k8s insiders (e.g. people who setup k8s binaries, or others with special access to machines and networks.) + - External Users: People who are accessing applications running on K8s (e.g. a web site served by webserver running in a container on K8s), but who do not have K8s API access. + - K8s Users : People who access the K8s API (e.g. create K8s API objects like Pods) + - K8s Project Admins: People who manage access for some K8s Users + - K8s Cluster Admins: People who control the machines, networks, or binaries that comprise a K8s cluster. + - K8s Admin means K8s Cluster Admins and K8s Project Admins taken together. ### Threats Both intentional attacks and accidental use of privilege are concerns. For both cases it may be useful to think about these categories differently: - - Application Path - attack by sending network messages from the internet to the IP/port of any application running on k8s. May exploit weakness in application or misconfiguration of k8s. - - K8s API Path - attack by sending network messages to any k8s infrastructure network endpoint. Attacker may have k8s user identity. - - Insider Path - attack on k8s system components. Attacker may have privileged access to networks, machines or k8s software and data. Software errors in k8s system components and administrator error are some types of threat in this category. + - Application Path - attack by sending network messages from the internet to the IP/port of any application running on K8s. May exploit weakness in application or misconfiguration of K8s. + - K8s API Path - attack by sending network messages to any K8s API endpoint. + - Insider Path - attack on K8s system components. Attacker may have privileged access to networks, machines or K8s software and data. Software errors in K8s system components and administrator error are some types of threat in this category. This document is primarily concerned with K8s API paths, and secondarily with Internal paths. The Application path also needs to be secure, but is not the focus of this document. ### Assets to protect -End user assets: - - Personal information like private messages, or images uploaded by end users +External User assets: + - Personal information like private messages, or images uploaded by External Users - web server logs -K8s user assets: - - end-user assets of each k8s user +K8s User assets: + - External User assets of each K8s User - things private to the K8s app, like: - credentials for accessing other services (docker private repos, storage services, facebook, etc) - SSL certificates for web servers - proprietary data and code K8s Cluster assets: - - Assets of each k8s-user - - Certificates or secrets which identify the k8s machines and servers to users. - - The value of k8s cluster computing resources (cpu, memory, etc). + - Assets of each K8s User + - Machine Certificates or secrets. + - The value of K8s cluster computing resources (cpu, memory, etc). -This document is primarily about protecting k8s user assets and K8s cluster assets from other k8s users and k8s insiders. +This document is primarily about protecting K8s User assets and K8s cluster assets from other K8s Users and K8s Project and Cluster Admins. ### Usage environments Cluster in Small organization: - - k8s insiders may be the same people as k8s users. - - few k8s insiders. + - K8s Admins may be the same people as K8s Users. + - few K8s Admins. - prefer ease of use to fine-grained access control/precise accounting, etc. - - Product requirement that it be easy for potential users to try out setting up a simple cluster. + - Product requirement that it be easy for potential K8s Cluster Admin to try out setting up a simple cluster. Cluster in Large organization: - - k8s insiders typically distinct people from k8s users. may need to divide k8s insider access by roles. - - k8s users need to be protected from each other. - - auditing of k8s user and k8s insider actions important. + - K8s Admins typically distinct people from K8s Users. May need to divide K8s Cluster Admin access by roles. + - K8s Users need to be protected from each other. + - Auditing of K8s User and K8s Admin actions important. - flexible accurate usage accounting and resource controls important. - Lots of automated access to APIs. - Need to integrate with existing enterprise directory, authentication, accounting, auditing, and security policy infrastructure. Org-run cluster: - - organization that runs k8s master components is same as the org that runs apps on k8s. + - organization that runs K8s master components is same as the org that runs apps on K8s. - Minions may be on-premises VMs or physical machines; Cloud VMs; or a mix. Hosted cluster: - Offering K8s API as a service, or offering a Paas or Saas built on K8s - May already offer web services, and need to integrate with existing customer account concept, and existing authentication, accounting, auditing, and security policy infrastructure. - - May want to leverage K8s user accounts and accounting to manage their user accounts (not a priority to support this use case.) - - Precise and accurate accounting of resources needed. Resource controls needed for hard limits (users given limited slice of data) and soft limits (users can grow up to some limit and then be expanded). + - May want to leverage K8s User accounts and accounting to manage their User accounts (not a priority to support this use case.) + - Precise and accurate accounting of resources needed. Resource controls needed for hard limits (Users given limited slice of data) and soft limits (Users can grow up to some limit and then be expanded). K8s ecosystem services: - There may be companies that want to offer their existing services (Build, CI, A/B-test, release automation, etc) for use with K8s. There should be some story for this case. @@ -83,7 +85,7 @@ Related discussion: - https://github.com/GoogleCloudPlatform/kubernetes/issues/443 This doc describes two security profiles: - - Simple profile: like single-user mode. Make it easy to evaluate k8s without lots of configuring accounts and policies. Protects from unauthorized users, but does not partition authorized users. + - Simple profile: like single-user mode. Make it easy to evaluate K8s without lots of configuring accounts and policies. Protects from unauthorized users, but does not partition authorized users. - Enterprise profile: Provide mechanisms needed for large numbers of users. Defence in depth. Should integrate with existing enterprise security infrastructure. K8s distribution should include templates of config, and documentation, for simple and enterprise profiles. System should be flexible enough for knowledgeable users to create intermediate profiles, but K8s developers should only reason about those two Profiles, not a matrix. @@ -101,21 +103,21 @@ K8s will have a `userAccount` API object. Initial Features: - `userAccount` object is immutable - there is no superuser `userAccount` -- `userAccount` objects are statically populated in the k8s API store by reading a config file. Only a k8s admin can do this. +- `userAccount` objects are statically populated in the K8s API store by reading a config file. Only a K8s Cluster Admin can do this. - `userAccount` can have a default `project`. If API call does not specify a `project`, the default `project` for that caller is assumed. - `userAccount` may access multiple projects. Improvements: -- Make `userAccount` part of a separate API group from core k8s objects like `pod`. Facilitates plugging in alternate Access Management. +- Make `userAccount` part of a separate API group from core K8s objects like `pod`. Facilitates plugging in alternate Access Management. Simple Profile: - - single `userAccount`, used by all users. + - single `userAccount`, used by all K8s Users and Project Admins. Enterprise Profile: - every human user has own `userAccount`. - `userAccount`s have labels that indicate both membership in groups, and ability to act in certain roles. - each service using the API has own `userAccount` too. (e.g. `scheduler`, `repcontroller`) - - automated jobs to denormalize the ldap group info into the local system list of users into the k8s userAccount file. + - automated jobs to denormalize the ldap group info into the local system list of users into the K8s userAccount file. ###Unix accounts A `userAccount` is not a Unix user account. The fact that a pod is started by a `userAccount` does not mean that the processes in that pod's containers run as a Unix user with a corresponding name or identity. @@ -134,7 +136,7 @@ K8s will have a have a `project` API object. Initial Features: - `project` object is immutable. -- `project` objects are statically populated in the k8s API store by reading a config file. Only admin can do this. +- `project` objects are statically populated in the K8s API store by reading a config file. Only a K8s Cluster Admin can do this. - In order to allow using `project` name as namespace for objects under that `project`, and to ensure the compound names are still DNS-compatible names, `project` names must be DNS label format. Improvements: @@ -196,8 +198,8 @@ Where to do authentication. Either: Considerations: - In some arrangements, the proxy might terminate SSL, and use a CA-signed certificate, while the APIserver might just use a self-signed or organization-signed certificate. - some large orgs will already have existing SaaS web services (e.g. storage, VMs). Some will already have a gateway that handles auth for multiple services. If K8s does auth in the proxy, then those orgs can just replace the proxy with their existing web service gateway. -- Apache or nginx is more stable than k8s code. Prefer to put secrets (tokens, SSL cert.) in lower-touch place. -- Admins configuring k8s for enterprise use are more likely to know how to config a proxy than to modify Go code of apiserver. +- Apache or nginx is more stable than K8s code. Prefer to put secrets (tokens, SSL cert.) in lower-touch place. +- Admins configuring K8s for enterprise use are more likely to know how to config a proxy than to modify Go code of apiserver. Based on above considerations, auth should happen in the proxy. @@ -223,12 +225,12 @@ Enterprise Profile: Initial Features: - Policy object is immutable -- Policy objects are statically populated in the k8s API store by reading a config file. Only admin can do this. +- Policy objects are statically populated in the K8s API store by reading a config file. Only a K8s Cluster Admin can do this. - Just a few policies per `project` which list which users can create objects, which can just view, them, etc. - Objects are created with reference to these default policies. Improvements: -- Have API calls to create and delete and modify Policy objects. These would be in a separate API group from core k8s APIs. This allows for replacing the k8s authorization service with an alternate implementation, and to centralize policies that might apply to services other than k8s. +- Have API calls to create and delete and modify Policy objects. These would be in a separate API group from core K8s APIs. This allows for replacing the K8s authorization service with an alternate implementation, and to centralize policies that might apply to services other than K8s. - Ability to change policy for an object. - Ability to create an object with a non-default policy effective immediately at creation time. - Ability to defer policy object checking to a policy server. @@ -313,15 +315,15 @@ The API should have a `quota` concept (see (https://github.com/GoogleCloudPlatfo Initially: - a `quota` object is immutable. -- for hosted k8s systems that do billing, Project is recommended level for billing accounts. +- for hosted K8s systems that do billing, Project is recommended level for billing accounts. - Every object that consumes resources should have a `project` so that Resource usage stats are roll-up-able to `project`. -- admin sets quota objects by writing a config file. +- K8s Cluster Admin sets quota objects by writing a config file. Improvements: - allow one project to charge the quota for one or more other projects. This would be controlled by a policy which allows changing a billing_project= label on an object. - allow quota to be set by project owners for (project x label) combinations (e.g. let "webserver" project use 100 cores, but to prevent accidents, don't allow "webserver" project and "instance=test" use more than 10 cores. - tools to help write consistent quota config files based on number of minions, historical project usages, QoS needs, etc. -- way for admin to incrementally adjust Quota objects. +- way for K8s Cluster Admin to incrementally adjust Quota objects. Simple profile: - a single `project` with infinite resource limits. From 53a88c145021238b67a265b78d040ad17b2d2ae9 Mon Sep 17 00:00:00 2001 From: Eric Tune Date: Fri, 22 Aug 2014 16:20:33 -0700 Subject: [PATCH 4/8] Defer details of Authorization to a separate doc. --- docs/access.md | 112 +++++++------------------------------------------ 1 file changed, 15 insertions(+), 97 deletions(-) diff --git a/docs/access.md b/docs/access.md index 5ebfab82fe4..32a7adc249e 100644 --- a/docs/access.md +++ b/docs/access.md @@ -206,107 +206,25 @@ Based on above considerations, auth should happen in the proxy. ## Authorization -Authorization is done by the APIserver. +K8s authorization should: +- Allow for a range of maturity levels, from single-user for those test driving the system, to integration with existing to enterprise authorization systems. +- Allow for centralized management of users and policies. In some organizations, this will mean that the definition of users and access policies needs to reside on a system other than k8s and encompass other web services (such as a storage service). +- Allow processes running in K8s Pods to take on identity, and to allow narrow scoping of permissions for those identities in order to limit damage from software faults. +- Have Authorization Policies exposed as API objects so that a single config file can create or delete Pods, Controllers, Services, and the identities and policies for those Pods and Controllers. +- Be separate as much as practical from Authentication, to allow Authentication methods to change over time and space, without impacting Authorization policies. -Authorization is done using Policy objects. +K8s will implement a relatively simple +[Attribute-Based Access Control][http://en.wikipedia.org/wiki/Attribute_Based_Access_Control] model. +The model will be described in more detail in a forthcoming document. The model +- Be less complex than XACML +- Be easily recognizable to those familiar with Amazon IAM Policies. +- Have a subset/aliases/defaults which allow it to be used in a way comfortable to those users more familiar with Role-Based Access Control. +Authorization policy is set by creating a set of Policy objects. -### Policy objects -Policy objects are API objects. They express http://en.wikipedia.org/wiki/Attribute_Based_Access_Control +The API Server will be the Enforcement Point for Policy. For each API call that it receives, it will construct the Attributes needed to evaluate the policy (what user is making the call, what resource they are accessing, what they are trying to do that resource, etc) and pass those attribytes to a Decision Point. The Decision Point code evaluates the Attributes against all the Policies and allows or denys the API call. The system will be modular enough that the Decision Point code can either be linked into the APIserver binary, or be another service that the apiserver calls for each Decision (with appropriate time-limited caching as needed for performance). -Simple Profile: -- one Policy object that allows the single `userAccount` to CRUD objects in the single `project`. - -Enterprise Profile: -- Many policy objects in each of many projects. -- Tools and services that wrap policy creation interface to enforce meta policies, do template expansions, report, etc. - - - -Initial Features: -- Policy object is immutable -- Policy objects are statically populated in the K8s API store by reading a config file. Only a K8s Cluster Admin can do this. -- Just a few policies per `project` which list which users can create objects, which can just view, them, etc. -- Objects are created with reference to these default policies. - -Improvements: -- Have API calls to create and delete and modify Policy objects. These would be in a separate API group from core K8s APIs. This allows for replacing the K8s authorization service with an alternate implementation, and to centralize policies that might apply to services other than K8s. -- Ability to change policy for an object. -- Ability to create an object with a non-default policy effective immediately at creation time. -- Ability to defer policy object checking to a policy server. -- Authorization tokens to authorize entities without a `userAccount`. - - -### Policy object format - -Policy Object: -```go -type policies map[string]policy - -type policy { - project string, // ref to project of this Policy, to namespace the name. - name string, // name within the project name of the Policy - a PolicyType, - s Subject, - v Verb, - o Object - expires string // RFC3339 -} -type PolicyType string -const { - ALLOW PolicyType = "ALLOW", - // Later DENY, etc. -} - -type Subject string // Serialized label selector -type Verb string -const { - GET Verb = "GET", - // ... other HTTP methods. - ANY Verb = "ANY" // any http method - CREATE Verb = "CREATE" // PUT or POST - // ... -} -type Object { - exact string // any resource with exactly this path - // OR - prefix string // any resource with this path prefix (after removing "/api//") - // OR - where string // serialized label selector. -} -``` - -Ideally, lists of policy objects would have relatively concise and readable YAML forms, such as: -``` -{name: bob_can_read_pods, a: ALLOW, s: user.name is bob@example.com, v: GET, prefix: /pods} -{name: admins_can_delete_pods, a: ALLOW, s: user.role is admin, v: DELETE, prefix: /pods} -{name: tmp1234, a: ALLOW, s: user.name is "some.agent", v: POST, prefix: /pods/somepod, expires: 2014-08-13 16:21:42-07:00 } -``` - -Requests that don't match at least one ALLOW are not allowed. -TODO: define DENY, and other operations and their precedence. - -Delegation can be implemented by writing new narrowly tailored policies. -TODO: example of policy to delegate pod creation from a podTemplate (see https://github.com/GoogleCloudPlatform/kubernetes/issues/170). - -### Architecture for Authorization -When the APIserver receives a new request, it passes the -the `userAccount`, http method, and http path to an `Authorize() method`. - -In a simple implementation, the Authorize() module: - - runs in the APIserver - - searches all policy objects for a match. - - updates its cache when new Policy is added. - -In alternate implementations, it may: - - have indexes to speed matching. (Maybe this can share code with Label Queries.) - - defer to a centralized auth server for the enterprise. - - -### Labels -Initially, IIUC, labels are strings and not API objects in their own right. -Eventually, labels may have policies or namespaces which restrict application of certain labels. - +Policy objects may be applicable only to a single project; K8s Project Admins would be able to create those as needed. Other Policy objects may be applicable to all projects; a K8s Cluster Admin might create those in order to authorize a new type of controller to be used by all projects, or to make a K8s User into a K8s Project Admin.) ## Accounting From f476202ce45496ca49050e8f7ca4faacd49a9b2f Mon Sep 17 00:00:00 2001 From: Eric Tune Date: Mon, 25 Aug 2014 14:54:44 -0700 Subject: [PATCH 5/8] Separate userAccount and auth-method. Add UID. --- docs/access.md | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-) diff --git a/docs/access.md b/docs/access.md index 32a7adc249e..a9139c887b4 100644 --- a/docs/access.md +++ b/docs/access.md @@ -95,23 +95,30 @@ Features in this doc are divided into "Initial Feature", and "Improvements". I ## Identity ###userAccount K8s will have a `userAccount` API object. -- `userAccount` has a name which is a string. -- `userAccount` is not the same as the unix username of processes -- 'userAccount` objects have a name. The name should allow relatively free-form strings (at least flexible enough to hold most email addresses) to aid integration with existing enterprise auth systems, or existing account identifiers in hosting systems. +- `userAccount` has a UID which is immutable. This is used to associate users with objects and to record actions in audit logs. +- `userAccount` has a name which is a string and human readable and unique among userAccounts. It is used to refer to users in Policies, to ensure that the Policies are human readable. It can be changed only when there are no Policy objects or other objects which refer to that name. An email address is a suggested format for this field. +- `userAccount` is not related to the unix username of processes in Pods created by that userAccount. - `userAccount` API objects can have labels +The system may associate one or more Authentication Methods with a +`userAccount` (but they are not formally part of the userAccount object.) +In a simple deployment, the authentication method for a +user might be an authentication token which is verified by a K8s server. In a +more complex deployment, the authentication might be delegated to +another system which is trusted by the K8s API to authenticate users, but where +the authentication details are unknown to K8s. + Initial Features: -- `userAccount` object is immutable - there is no superuser `userAccount` - `userAccount` objects are statically populated in the K8s API store by reading a config file. Only a K8s Cluster Admin can do this. - `userAccount` can have a default `project`. If API call does not specify a `project`, the default `project` for that caller is assumed. -- `userAccount` may access multiple projects. +- `userAccount` is global. A single human with access to multiple projects is recommended to only have one userAccount. Improvements: - Make `userAccount` part of a separate API group from core K8s objects like `pod`. Facilitates plugging in alternate Access Management. Simple Profile: - - single `userAccount`, used by all K8s Users and Project Admins. + - single `userAccount`, used by all K8s Users and Project Admins. One access token shared by all. Enterprise Profile: - every human user has own `userAccount`. From 371e7020b56bd307e3944c8ffdd0293ed9320009 Mon Sep 17 00:00:00 2001 From: Eric Tune Date: Wed, 27 Aug 2014 13:01:43 -0700 Subject: [PATCH 6/8] More Goals and less impl details of Authn. Expanded discussion of projects in the REST API based on feedback. --- docs/access.md | 63 +++++++++++++++++++++++++------------------------- 1 file changed, 31 insertions(+), 32 deletions(-) diff --git a/docs/access.md b/docs/access.md index a9139c887b4..a92a3126863 100644 --- a/docs/access.md +++ b/docs/access.md @@ -162,54 +162,53 @@ In the Simple Profile: - There is a single default `project`. No need to specify `project` when making an API call. Project versus userAccount vs Labels: -- `userAccount`s are intended for audit logging, and to define who has access to `project`s. +- `userAccount`s are intended for audit logging (both name and UID should be logged), and to define who has access to `project`s. - `labels` (see docs/labels.md) should be used to distinguish pods, users, and other objects that cooperate towards a common goal but are different in some way, such as version, or responsibilities. - `project`s provide a namespace for objects. -how is `project` selected when making a REST call? -- default to only project if there is only one. +*To Be Determined* +How is project selected when making a REST call? +- subdomain, e.g. http://myproject.kubernetes.example.com + - In hosted environment, may need to avoid profane or vanity names. + - Requires that the apiservers be tied into DNS, which is onerous to configure in some environments. Nice in some contexts. - query parameter -- subdomain, e.g. http://myproject.kubernetes.example.com. nginx proxy can translate that to a query parameter - - Subdomains have potential scaling limits. If using project names to identify the domain, you have to defend against profane or vanity names, and probably support blacklist limits on new project names. In non-hosted environments is less of an issue. Requires that the apiservers be tied into DNS, which is onerous to configure in some environments. Nice in some contexts. -- offering a project API scope /projects// -- global access via globally unique id (where supported, which isn't very consistent today) /pods/ + - e.g. `/pods/?project=` + - tedious to add to every request, but necessary to disambiguate object names +- project in resource name + - e.g. `/projects//` ## Authentication +Goals for K8s authentication: +- Include a built-in authentication system with no configuration required to use in single-user mode, and little configuration required to add several user accounts, and no https proxy required. +- Allow for authentication to be handled by a system external to Kubernetes, to allow integration with existing to enterprise authorization systems. The kubernetes project itself should avoid taking contributions of multiple authorization schemes. Instead, a trusted proxy in front of the apiserver can be used to authenticate users. + - For organizations whose security requirements only allow FIPS compliant implementations (e.g. apache) for authentication. + - So the proxy can terminate SSL, and isolate the CA-signed certificate from less trusted, higher-touch APIserver. + - For organizations that already have existing SaaS web services (e.g. storage, VMs) and want a common authentication portal. +- Avoid mixing authentication and authorization, so that authorization policies be centrally managed, and to allow changes in authentication methods without affecting authorization code. + Initially: -- Use bearer tokens to authenticate users. -- Each API request must include a "Authorization: Bearer " header. -- A token identifies a particular `userAccount`. +- Tokens used to authenticate a user. +- Long lived tokens identify a particular `userAccount`. - Administrator utility generates tokens at cluster setup. -- Tokens are long lived. -- Tokens are just long random strings base64 encoded. No content. +- OAuth2.0 Bearer tokens protocol, http://tools.ietf.org/html/rfc6750 +- No scopes for tokens. Authorization happens in the API server +- Tokens dynamically generated by apiserver to identify pods which are making API calls. +- Tokens checked in a module of the APIserver. +- Authentication in apiserver can be disabled by flag, to allow testing without authorization enabled, and to allow use of an authenticating proxy. In this mode, a query parameter or header added by the proxy will identify the caller. Improvements: -- make it harder for the wrong party to use the token. - - shorter lived tokens - - use tokens that are bound to the channel between the client and the api server +- Refresh of tokens. +- SSH keys to access inside containers. -Things to consider for Improved implementation: -- JWT http://tools.ietf.org/html/draft-ietf-oauth-json-web-token-13 - - Possible library: http://code.google.com/p/goauth2/oauth/jwt - - Use with endpoint that generates short-term bearer token. -- tokens that are bound to the channel between the client and the api server +To be considered for subsequent versions: +- Fuller use of OAuth (http://tools.ietf.org/html/rfc6749) +- Scoped tokens. +- Tokens that are bound to the channel between the client and the api server - http://www.ietf.org/proceedings/90/slides/slides-90-uta-0.pdf - http://www.browserauth.net -Where to do authentication. Either: -- Authenticate in reverse proxy (currently nginx, could easily be apache+mod_auth). Proxy either rejects invalid token or appends authorized_user identifier to the request before passing to APIserver. -- Apiserver checks token. - -Considerations: -- In some arrangements, the proxy might terminate SSL, and use a CA-signed certificate, while the APIserver might just use a self-signed or organization-signed certificate. -- some large orgs will already have existing SaaS web services (e.g. storage, VMs). Some will already have a gateway that handles auth for multiple services. If K8s does auth in the proxy, then those orgs can just replace the proxy with their existing web service gateway. -- Apache or nginx is more stable than K8s code. Prefer to put secrets (tokens, SSL cert.) in lower-touch place. -- Admins configuring K8s for enterprise use are more likely to know how to config a proxy than to modify Go code of apiserver. - -Based on above considerations, auth should happen in the proxy. - ## Authorization From 57ca9eefc82bf52471e6b268c899f2edf793a983 Mon Sep 17 00:00:00 2001 From: Eric Tune Date: Fri, 5 Sep 2014 13:59:15 -0700 Subject: [PATCH 7/8] Rename project to namespace and reference #1114. --- docs/access.md | 63 +++++--------- docs/authorization.md | 198 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 219 insertions(+), 42 deletions(-) create mode 100644 docs/authorization.md diff --git a/docs/access.md b/docs/access.md index a92a3126863..0627f52e58f 100644 --- a/docs/access.md +++ b/docs/access.md @@ -111,8 +111,8 @@ the authentication details are unknown to K8s. Initial Features: - there is no superuser `userAccount` - `userAccount` objects are statically populated in the K8s API store by reading a config file. Only a K8s Cluster Admin can do this. -- `userAccount` can have a default `project`. If API call does not specify a `project`, the default `project` for that caller is assumed. -- `userAccount` is global. A single human with access to multiple projects is recommended to only have one userAccount. +- `userAccount` can have a default `namespace`. If API call does not specify a `namespace`, the default `namespace` for that caller is assumed. +- `userAccount` is global. A single human with access to multiple namespaces is recommended to only have one userAccount. Improvements: - Make `userAccount` part of a separate API group from core K8s objects like `pod`. Facilitates plugging in alternate Access Management. @@ -138,51 +138,29 @@ Improvements: - requires docker to integrate user namespace support, and deciding what getpwnam() does for these uids. - any features that help users avoid use of privileged containers (https://github.com/GoogleCloudPlatform/kubernetes/issues/391) -###project -K8s will have a have a `project` API object. +###Namespaces +K8s will have a have a `namespace` API object. It is similar to a Google Compute Engine `project`. It provides a namespace for objects created by a group of people co-operating together, preventing name collisions with non-cooperating groups. It also serves as a reference point for authorization policies. -Initial Features: -- `project` object is immutable. -- `project` objects are statically populated in the K8s API store by reading a config file. Only a K8s Cluster Admin can do this. -- In order to allow using `project` name as namespace for objects under that `project`, and to ensure the compound names are still DNS-compatible names, `project` names must be DNS label format. - -Improvements: -- have API calls to create and delete `project` objects. - -Most API objects have an associated `project`: -- pods have a `project`. -- `project`s don't have a `project`, nor do `userAccount`s - - or else they belong to a special global `project`? -- An object's `project` cannot be changed after creation. +Namespaces are described in [namespace.md](https://github.com/GoogleCloudPlatform/kubernetes/blob/master/docs/namespaces.md), +or will be once [#1114](https://github.com/GoogleCloudPlatform/kubernetes/pull/1114) is merged. In the Enterprise Profile: - - a `userAccount` may have permission to access several `project`s. API calls need to specify a `project`. Client libs may fill this in using a stored preference. + - a `userAccount` may have permission to access several `namespace`s. In the Simple Profile: - - There is a single default `project`. No need to specify `project` when making an API call. + - There is a single `namespace` used by the single user. -Project versus userAccount vs Labels: -- `userAccount`s are intended for audit logging (both name and UID should be logged), and to define who has access to `project`s. +Namespaces versus userAccount vs Labels: +- `userAccount`s are intended for audit logging (both name and UID should be logged), and to define who has access to `namespace`s. - `labels` (see docs/labels.md) should be used to distinguish pods, users, and other objects that cooperate towards a common goal but are different in some way, such as version, or responsibilities. -- `project`s provide a namespace for objects. +- `namespace`s prevent name collisions between uncoordinated groups of people, and provide a place to attach common policies for co-operating groups of people. -*To Be Determined* -How is project selected when making a REST call? -- subdomain, e.g. http://myproject.kubernetes.example.com - - In hosted environment, may need to avoid profane or vanity names. - - Requires that the apiservers be tied into DNS, which is onerous to configure in some environments. Nice in some contexts. -- query parameter - - e.g. `/pods/?project=` - - tedious to add to every request, but necessary to disambiguate object names -- project in resource name - - e.g. `/projects//` - ## Authentication Goals for K8s authentication: - Include a built-in authentication system with no configuration required to use in single-user mode, and little configuration required to add several user accounts, and no https proxy required. -- Allow for authentication to be handled by a system external to Kubernetes, to allow integration with existing to enterprise authorization systems. The kubernetes project itself should avoid taking contributions of multiple authorization schemes. Instead, a trusted proxy in front of the apiserver can be used to authenticate users. +- Allow for authentication to be handled by a system external to Kubernetes, to allow integration with existing to enterprise authorization systems. The kubernetes namespace itself should avoid taking contributions of multiple authorization schemes. Instead, a trusted proxy in front of the apiserver can be used to authenticate users. - For organizations whose security requirements only allow FIPS compliant implementations (e.g. apache) for authentication. - So the proxy can terminate SSL, and isolate the CA-signed certificate from less trusted, higher-touch APIserver. - For organizations that already have existing SaaS web services (e.g. storage, VMs) and want a common authentication portal. @@ -230,30 +208,31 @@ Authorization policy is set by creating a set of Policy objects. The API Server will be the Enforcement Point for Policy. For each API call that it receives, it will construct the Attributes needed to evaluate the policy (what user is making the call, what resource they are accessing, what they are trying to do that resource, etc) and pass those attribytes to a Decision Point. The Decision Point code evaluates the Attributes against all the Policies and allows or denys the API call. The system will be modular enough that the Decision Point code can either be linked into the APIserver binary, or be another service that the apiserver calls for each Decision (with appropriate time-limited caching as needed for performance). -Policy objects may be applicable only to a single project; K8s Project Admins would be able to create those as needed. Other Policy objects may be applicable to all projects; a K8s Cluster Admin might create those in order to authorize a new type of controller to be used by all projects, or to make a K8s User into a K8s Project Admin.) +Policy objects may be applicable only to a single namespace or to all namespaces; K8s Project Admins would be able to create those as needed. Other Policy objects may be applicable to all namespaces; a K8s Cluster Admin might create those in order to authorize a new type of controller to be used by all namespaces, or to make a K8s User into a K8s Project Admin.) + ## Accounting The API should have a `quota` concept (see (https://github.com/GoogleCloudPlatform/kubernetes/issues/442 -). A quota object relates a project (and optionally a label selector) to a maximum quantity of resources that may be used (see resources.md). +). A quota object relates a namespace (and optionally a label selector) to a maximum quantity of resources that may be used (see resources.md). Initially: - a `quota` object is immutable. - for hosted K8s systems that do billing, Project is recommended level for billing accounts. -- Every object that consumes resources should have a `project` so that Resource usage stats are roll-up-able to `project`. +- Every object that consumes resources should have a `namespace` so that Resource usage stats are roll-up-able to `namespace`. - K8s Cluster Admin sets quota objects by writing a config file. Improvements: -- allow one project to charge the quota for one or more other projects. This would be controlled by a policy which allows changing a billing_project= label on an object. -- allow quota to be set by project owners for (project x label) combinations (e.g. let "webserver" project use 100 cores, but to prevent accidents, don't allow "webserver" project and "instance=test" use more than 10 cores. -- tools to help write consistent quota config files based on number of minions, historical project usages, QoS needs, etc. +- allow one namespace to charge the quota for one or more other namespaces. This would be controlled by a policy which allows changing a billing_namespace= label on an object. +- allow quota to be set by namespace owners for (namespace x label) combinations (e.g. let "webserver" namespace use 100 cores, but to prevent accidents, don't allow "webserver" namespace and "instance=test" use more than 10 cores. +- tools to help write consistent quota config files based on number of minions, historical namespace usages, QoS needs, etc. - way for K8s Cluster Admin to incrementally adjust Quota objects. Simple profile: - - a single `project` with infinite resource limits. + - a single `namespace` with infinite resource limits. Enterprise profile: - - multiple projects + - multiple namespaces each with their own limits. Issues: - need for locking or "eventual consistency" when multiple apiserver goroutines are accessing the object store and handling pod creations. diff --git a/docs/authorization.md b/docs/authorization.md new file mode 100644 index 00000000000..87cda60492b --- /dev/null +++ b/docs/authorization.md @@ -0,0 +1,198 @@ +Policies should be able to constriain on kind of object. +more about the context for a policy eval call. +predefined labels in the selector or other policy language primitive for object type (spoons moving away from labelselector such as VanillaST (gmail syntax). + +namespace for attributes. +namespace for other modules? + +### Policy objects +Policy objects are API objects. They express http://en.wikipedia.org/wiki/Attribute_Based_Access_Control + +Simple Profile: +- one Policy object that allows the single `userAccount` to CRUD objects in the single `project`. + +Enterprise Profile: +- Many policy objects in each of many projects. +- Tools and services that wrap policy creation interface to enforce meta policies, do template expansions, report, etc. + + + +Initial Features: +- Policy object is immutable +- Policy objects are statically populated in the K8s API store by reading a config file. Only a K8s Cluster Admin can do this. +- Just a few policies per `project` which list which users can create objects, which can just view, them, etc. +- Objects are created with reference to these default policies. + +Improvements: +- Have API calls to create and delete and modify Policy objects. These would be in a separate API group from core K8s APIs. This allows for replacing the K8s authorization service with an alternate implementation, and to centralize policies that might apply to services other than K8s. +- Ability to change policy for an object. +- Ability to create an object with a non-default policy effective immediately at creation time. +- Ability to defer policy object checking to a policy server. +- Authorization tokens to authorize entities without a `userAccount`. + + +### Policy object format + +Policy Object: +```go +type policies map[string]policy + +type policy { + project string, // ref to project of this Policy, to namespace the name. + name string, // name within the project name of the Policy + a PolicyType, + s Subject, + v Verb, + o Object + expires string // RFC3339 +} +type PolicyType string +const { + ALLOW PolicyType = "ALLOW", + // Later DENY, etc. +} + +type Subject string // Serialized label selector +type Verb string +const { + GET Verb = "GET", + // ... other HTTP methods. + ANY Verb = "ANY" // any http method + CREATE Verb = "CREATE" // PUT or POST + // ... +} +type Object { + exact string // any resource with exactly this path + // OR + prefix string // any resource with this path prefix (after removing "/api//") + // OR + where string // serialized label selector. +} +``` + +Ideally, lists of policy objects would have relatively concise and readable YAML forms, such as: +``` +{name: bob_can_read_pods, a: ALLOW, s: user.name is bob@example.com, v: GET, prefix: /pods} +{name: admins_can_delete_pods, a: ALLOW, s: user.role is admin, v: DELETE, prefix: /pods} +{name: tmp1234, a: ALLOW, s: user.name is "some.agent", v: POST, prefix: /pods/somepod, expires: 2014-08-13 16:21:42-07:00 } +``` + +Requests that don't match at least one ALLOW are not allowed. +TODO: define DENY, and other operations and their precedence. + +Delegation can be implemented by writing new narrowly tailored policies. +TODO: example of policy to delegate pod creation from a podTemplate (see https://github.com/GoogleCloudPlatform/kubernetes/issues/170). + +### Architecture for Authorization +When the APIserver receives a new request, it passes the +the `userAccount`, http method, and http path to an `Authorize() method`. + +In a simple implementation, the Authorize() module: + - runs in the APIserver + - searches all policy objects for a match. + - updates its cache when new Policy is added. + +In alternate implementations, it may: + - have indexes to speed matching. (Maybe this can share code with Label Queries.) + - defer to a centralized auth server for the enterprise. + + +### Labels +Initially, IIUC, labels are strings and not API objects in their own right. +Eventually, labels may have policies or namespaces which restrict application of certain labels. + + +### Use cases: + +only allow this pod to be started as this role if it was from a trusted build process. +Seems easy enough to add with labels and centralized policy store. +Seems like a PITA to implement and understand with three-legged oauth flow. + +flows for adding three users + +policy is immutible how to add subject. + +inherit vs override + +Make users have one of (admin, edit, view) + +i +- Allow cluster admin power to create or remove any object. +- Allow project admin power to create or remove any object in the project +- Allow project admin power to add or edit any policies scoped to that project +-Create users with the 3 key roles (simulate RBAC) +- View capabilities by user +- View allowed actions by object +- control who has restart capability on a pod. +8. Can we define a Policy Template as a global resource? Can we associate a Policy Template(s) with a Project Template to make initial setup easier? +9. A policy object should have timestamps. +Typical scenario: +I have a global administrator for my OpenShift deployment that needs to access each Project. +Each project is managed by a team where each member may have particular roles with varying rights. +System view: +We have a Policy object that denotes global admin access added to each Project. +Policy{ + project: "the project", + name:"openshift_admin", + prefix: "/", + s:"group=openshift_admin" + v: "ANY" + o: { + prefix: "/" + } +} +Option 1: +For each unique role, we would need to have a Policy object, but in this case our Policy subject would need to use an OR in the label selector that enumerated each person or group. This could get awkward to manage. If a policy object has timestamps, harder to reconcile when a person was added or removed from the project from this model. +Option 2: +For each person or group, we have a dedicated Policy object. Simpler to enumerate, more resources to compare potentially. Easier to audit when a person was added or removed from data model. +Issues: +There is a note that "Policy object is immutable, and is statically populated by admin" +It sounds like there is a need for a policy template, but don't you need to edit a policy in order to modify subjects to grow access? + +### Blah blah blah + +Have Account concept? +Cross-project permissions? + +Per-api-group Actions +which are initially defined. +plugins can register their own actions. + +Amazon: +{ + "Version": "2012-10-17", + "Statement": [{ + "Effect": "Allow", + "Action": "s3:ListBucket", + "Resource": "arn:aws:s3:::example_bucket" + }] +} +http://docs.aws.amazon.com/IAM/latest/UserGuide/PoliciesOverview.html + +User creation after cluster exists: +create user object +create userCredentials and attach to user. +add labels to user or put in groups. + + +### Blah +What is plan for verbs per resource type? +Seems like Client already defines these. + +No-one can do anything that is not allowed. + +### AWS CloudFormation + + +wget https://raw.githubusercontent.com/marceldegraaf/blog-coreos-1/master/stack.yml +ruby -r json -r yaml -e "yaml = YAML.load(File.read('./stack.yml')); print yaml.to_json" > stack.json + + +Check it out. + +Check out how. +### Concepts: +Enforcement point (reads a API call, adds attributes, and calls Decision Point) +Attribute Sources (adds attributes) +Decision Piont (evals policies) + From 224c29e51d3c60fda4e793ed0f38c844f635f00a Mon Sep 17 00:00:00 2001 From: Eric Tune Date: Fri, 5 Sep 2014 14:27:47 -0700 Subject: [PATCH 8/8] Removed file not ready to share. --- docs/authorization.md | 198 ------------------------------------------ 1 file changed, 198 deletions(-) delete mode 100644 docs/authorization.md diff --git a/docs/authorization.md b/docs/authorization.md deleted file mode 100644 index 87cda60492b..00000000000 --- a/docs/authorization.md +++ /dev/null @@ -1,198 +0,0 @@ -Policies should be able to constriain on kind of object. -more about the context for a policy eval call. -predefined labels in the selector or other policy language primitive for object type (spoons moving away from labelselector such as VanillaST (gmail syntax). - -namespace for attributes. -namespace for other modules? - -### Policy objects -Policy objects are API objects. They express http://en.wikipedia.org/wiki/Attribute_Based_Access_Control - -Simple Profile: -- one Policy object that allows the single `userAccount` to CRUD objects in the single `project`. - -Enterprise Profile: -- Many policy objects in each of many projects. -- Tools and services that wrap policy creation interface to enforce meta policies, do template expansions, report, etc. - - - -Initial Features: -- Policy object is immutable -- Policy objects are statically populated in the K8s API store by reading a config file. Only a K8s Cluster Admin can do this. -- Just a few policies per `project` which list which users can create objects, which can just view, them, etc. -- Objects are created with reference to these default policies. - -Improvements: -- Have API calls to create and delete and modify Policy objects. These would be in a separate API group from core K8s APIs. This allows for replacing the K8s authorization service with an alternate implementation, and to centralize policies that might apply to services other than K8s. -- Ability to change policy for an object. -- Ability to create an object with a non-default policy effective immediately at creation time. -- Ability to defer policy object checking to a policy server. -- Authorization tokens to authorize entities without a `userAccount`. - - -### Policy object format - -Policy Object: -```go -type policies map[string]policy - -type policy { - project string, // ref to project of this Policy, to namespace the name. - name string, // name within the project name of the Policy - a PolicyType, - s Subject, - v Verb, - o Object - expires string // RFC3339 -} -type PolicyType string -const { - ALLOW PolicyType = "ALLOW", - // Later DENY, etc. -} - -type Subject string // Serialized label selector -type Verb string -const { - GET Verb = "GET", - // ... other HTTP methods. - ANY Verb = "ANY" // any http method - CREATE Verb = "CREATE" // PUT or POST - // ... -} -type Object { - exact string // any resource with exactly this path - // OR - prefix string // any resource with this path prefix (after removing "/api//") - // OR - where string // serialized label selector. -} -``` - -Ideally, lists of policy objects would have relatively concise and readable YAML forms, such as: -``` -{name: bob_can_read_pods, a: ALLOW, s: user.name is bob@example.com, v: GET, prefix: /pods} -{name: admins_can_delete_pods, a: ALLOW, s: user.role is admin, v: DELETE, prefix: /pods} -{name: tmp1234, a: ALLOW, s: user.name is "some.agent", v: POST, prefix: /pods/somepod, expires: 2014-08-13 16:21:42-07:00 } -``` - -Requests that don't match at least one ALLOW are not allowed. -TODO: define DENY, and other operations and their precedence. - -Delegation can be implemented by writing new narrowly tailored policies. -TODO: example of policy to delegate pod creation from a podTemplate (see https://github.com/GoogleCloudPlatform/kubernetes/issues/170). - -### Architecture for Authorization -When the APIserver receives a new request, it passes the -the `userAccount`, http method, and http path to an `Authorize() method`. - -In a simple implementation, the Authorize() module: - - runs in the APIserver - - searches all policy objects for a match. - - updates its cache when new Policy is added. - -In alternate implementations, it may: - - have indexes to speed matching. (Maybe this can share code with Label Queries.) - - defer to a centralized auth server for the enterprise. - - -### Labels -Initially, IIUC, labels are strings and not API objects in their own right. -Eventually, labels may have policies or namespaces which restrict application of certain labels. - - -### Use cases: - -only allow this pod to be started as this role if it was from a trusted build process. -Seems easy enough to add with labels and centralized policy store. -Seems like a PITA to implement and understand with three-legged oauth flow. - -flows for adding three users - -policy is immutible how to add subject. - -inherit vs override - -Make users have one of (admin, edit, view) - -i -- Allow cluster admin power to create or remove any object. -- Allow project admin power to create or remove any object in the project -- Allow project admin power to add or edit any policies scoped to that project --Create users with the 3 key roles (simulate RBAC) -- View capabilities by user -- View allowed actions by object -- control who has restart capability on a pod. -8. Can we define a Policy Template as a global resource? Can we associate a Policy Template(s) with a Project Template to make initial setup easier? -9. A policy object should have timestamps. -Typical scenario: -I have a global administrator for my OpenShift deployment that needs to access each Project. -Each project is managed by a team where each member may have particular roles with varying rights. -System view: -We have a Policy object that denotes global admin access added to each Project. -Policy{ - project: "the project", - name:"openshift_admin", - prefix: "/", - s:"group=openshift_admin" - v: "ANY" - o: { - prefix: "/" - } -} -Option 1: -For each unique role, we would need to have a Policy object, but in this case our Policy subject would need to use an OR in the label selector that enumerated each person or group. This could get awkward to manage. If a policy object has timestamps, harder to reconcile when a person was added or removed from the project from this model. -Option 2: -For each person or group, we have a dedicated Policy object. Simpler to enumerate, more resources to compare potentially. Easier to audit when a person was added or removed from data model. -Issues: -There is a note that "Policy object is immutable, and is statically populated by admin" -It sounds like there is a need for a policy template, but don't you need to edit a policy in order to modify subjects to grow access? - -### Blah blah blah - -Have Account concept? -Cross-project permissions? - -Per-api-group Actions -which are initially defined. -plugins can register their own actions. - -Amazon: -{ - "Version": "2012-10-17", - "Statement": [{ - "Effect": "Allow", - "Action": "s3:ListBucket", - "Resource": "arn:aws:s3:::example_bucket" - }] -} -http://docs.aws.amazon.com/IAM/latest/UserGuide/PoliciesOverview.html - -User creation after cluster exists: -create user object -create userCredentials and attach to user. -add labels to user or put in groups. - - -### Blah -What is plan for verbs per resource type? -Seems like Client already defines these. - -No-one can do anything that is not allowed. - -### AWS CloudFormation - - -wget https://raw.githubusercontent.com/marceldegraaf/blog-coreos-1/master/stack.yml -ruby -r json -r yaml -e "yaml = YAML.load(File.read('./stack.yml')); print yaml.to_json" > stack.json - - -Check it out. - -Check out how. -### Concepts: -Enforcement point (reads a API call, adds attributes, and calls Decision Point) -Attribute Sources (adds attributes) -Decision Piont (evals policies) -