First commit

.gitignore

@@ -0,0 +1,17 @@
+# OSX leaves these everywhere on SMB shares
+._*
+
+# Eclipse files
+.classpath
+.project
+.settings/**
+
+# This is where the result of the go build goes
+/target/**
+/target
+
+# This is where we stage releases
+/release/**
+
+# Emacs save files
+*~

LICENSE

@@ -0,0 +1,202 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

README.md

@@ -0,0 +1,128 @@
+# Kubernetes
+
+Kubernetes is an open source reference implementation of container cluster management.
+
+## Getting started on Google Compute Engine
+
+### Prerequisites
+
+1. You need a Google Cloud Platform account with billing enabled.  Visit http://cloud.google.com/console for more details
+2. You must have Go installed: [www.golang.org](http://www.golang.org)
+3. Ensure that your `gcloud` components are up-to-date by running `gcloud components update`.
+4. Get the Kubernetes source:  `git clone https://github.com/GoogleCloudPlatform/kubernetes.git`
+
+### Setup
+```
+cd kubernetes
+./src/scripts/dev-build-and-up.sh
+```
+
+### Running a container (simple version)
+```
+cd kubernetes
+./src/scripts/build-go.sh
+./src/scripts/cloudcfg.sh -p 8080:80 run dockerfile/nginx 2 myNginx
+```
+
+This will spin up two containers running Nginx mapping port 80 to 8080.
+
+To stop the container:
+```
+./src/scripts/cloudcfg.sh stop myNginx
+```
+
+To delete the container:
+```
+./src/scripts/cloudcfg.sh rm myNginx
+```
+
+### Running a container (more complete version)
+```
+cd kubernetes
+./src/scripts/cloudcfg.sh -c examples/task.json create /tasks
+```
+
+Where task.json contains something like:
+```
+{
+  "ID": "nginx",
+  "desiredState": {
+    "image": "dockerfile/nginx",
+    "networkPorts": [{
+      "containerPort": 80,
+      "hostPort": 8080
+    }]
+  },
+  "labels": {
+    "name": "foo"
+  }
+}
+```
+
+Look in the ```examples/``` for more examples
+
+### Tearing down the cluster
+```
+cd kubernetes
+./src/scripts/kube-down.sh
+```
+
+## Development
+
+### Hooks
+```
+# Before committing any changes, please link/copy these hooks into your .git
+# directory. This will keep you from accidentally committing non-gofmt'd
+# go code.
+cd kubernetes
+ln -s "../../hooks/prepare-commit-msg" .git/hooks/prepare-commit-msg
+ln -s "../../hooks/commit-msg" .git/hooks/commit-msg
+```
+
+### Unit tests
+```
+cd kubernetes
+./src/scripts/test-go.sh
+```
+
+### Coverage
+```
+cd kubernetes
+go tool cover -html=target/c.out
+```
+
+### Integration tests
+```
+# You need an etcd somewhere in your path.
+# To get from head:
+go get github.com/coreos/etcd
+go install github.com/coreos/etcd
+sudo ln -s "$GOPATH/bin/etcd" /usr/bin/etcd
+# Or just use the packaged one:
+sudo ln -s "$REPO_ROOT/target/bin/etcd" /usr/bin/etcd
+```
+
+```
+cd kubernetes
+./src/scripts/integration-test.sh
+```
+
+### Keeping your development fork in sync
+One time after cloning your forked repo:
+```
+git remote add upstream https://github.com/GoogleCloudPlatform/kubernetes.git
+```
+
+Then each time you want to sync to upstream:
+```
+git fetch upstream
+git rebase upstream/master
+```
+
+### Regenerating the documentation
+Install [nodejs](http://nodejs.org/download/), [npm](https://www.npmjs.org/), and
+[raml2html](https://github.com/kevinrenskers/raml2html), then run:
+```
+cd kubernetes/api
+raml2html kubernetes.raml > kubernetes.html
+```

api/doc/controller-schema.json

@@ -0,0 +1,50 @@
+{
+  "$schema": "http://json-schema.org/draft-03/schema",
+  "type": "object",
+  "required": false,
+  "description": "A replicationController resource. A replicationController helps to create and manage a set of tasks. It acts as a factory to create new tasks based on a template. It ensures that there are a specific number of tasks running. If fewer tasks are running than `replicas` then the needed tasks are generated using `taskTemplate`. If more tasks are running than `replicas`, then excess tasks are deleted.",
+  "properties": {
+    "kind": {
+      "type": "string",
+      "required": false
+    },
+    "id": {
+      "type": "string",
+      "required": false
+    },
+    "creationTimestamp": {
+      "type": "string",
+      "required": false
+    },
+    "selfLink": {
+      "type": "string",
+      "required": false
+    },
+    "desiredState": {
+      "type": "object",
+      "required": false,
+      "description": "The desired configuration of the replicationController",
+      "properties": {
+        "replicas": {
+          "type": "number",
+          "required": false,
+          "description": "Number of tasks desired in the set"
+        },
+        "replicasInSet": {
+          "type": "object",
+          "required": false,
+          "description": "Required labels used to identify tasks in the set"
+        },
+        "taskTemplate": {
+          "type": "object",
+          "required": false,
+          "description": "Template from which to create new tasks, as necessary. Identical to task schema."
+        }
+      }
+    },
+    "labels": {
+      "type": "object",
+      "required": false
+    }
+  }
+}

api/doc/service-schema.json

@@ -0,0 +1,36 @@
+{
+  "$schema": "http://json-schema.org/draft-03/schema",
+  "type": "object",
+  "required": false,
+  "description": "A service resource.",
+  "properties": {
+    "kind": {
+      "type": "string",
+      "required": false
+    },
+    "id": {
+      "type": "string",
+      "required": false
+    },
+    "creationTimestamp": {
+      "type": "string",
+      "required": false
+    },
+    "selfLink": {
+      "type": "string",
+      "required": false
+    },
+    "name": {
+      "type": "string",
+      "required": false
+    },
+    "port": {
+      "type": "number",
+      "required": false
+    },
+    "labels": {
+      "type": "object",
+      "required": false
+    }
+  }
+}

api/doc/task-schema.json

@@ -0,0 +1,87 @@
+{
+  "$schema": "http://json-schema.org/draft-03/schema",
+  "type": "object",
+  "required": false,
+  "description": "Task resource. A task corresponds to a colocated group of [Docker containers](http://docker.io).",
+  "properties": {
+    "kind": {
+      "type": "string",
+      "required": false
+    },
+    "id": {
+      "type": "string",
+      "required": false
+    },
+    "creationTimestamp": {
+      "type": "string",
+      "required": false
+    },
+    "selfLink": {
+      "type": "string",
+      "required": false
+    },
+    "desiredState": {
+      "type": "object",
+      "required": false,
+      "description": "The desired configuration of the task",
+      "properties": {
+        "manifest": {
+          "type": "object",
+          "required": false,
+          "description": "Manifest describing group of [Docker containers](http://docker.io); compatible with format used by [Google Cloud Platform's container-vm images](https://developers.google.com/compute/docs/containers)"
+        },
+        "status": {
+          "type": "string",
+          "required": false,
+          "description": ""
+        },
+        "host": {
+          "type": "string",
+          "required": false,
+          "description": ""
+        },
+        "hostIP": {
+          "type": "string",
+          "required": false,
+          "description": ""
+        },
+        "info": {
+          "type": "object",
+          "required": false,
+          "description": ""
+        }
+      }
+    },
+    "currentState": {
+      "type": "object",
+      "required": false,
+      "description": "The current configuration and status of the task. Fields in common with desiredState have the same meaning.",
+      "properties": {
+        "manifest": {
+          "type": "object",
+          "required": false
+        },
+        "status": {
+          "type": "string",
+          "required": false
+        },
+        "host": {
+          "type": "string",
+          "required": false
+        },
+        "hostIP": {
+          "type": "string",
+          "required": false
+        },
+        "info": {
+          "type": "object",
+          "required": false
+        }
+      }
+    },
+    "labels": {
+      "type": "object",
+      "required": false
+    }
+  }
+}

api/examples/controller-list.json

@@ -0,0 +1,30 @@
+{
+    "items": [
+        {
+            "id": "testRun",
+            "desiredState": {
+                "replicas": 2,
+                "replicasInSet": {
+                    "name": "testRun"
+                },
+                "taskTemplate": {
+                    "desiredState": {
+                        "image": "dockerfile/nginx",
+                        "networkPorts": [
+                            {
+                                "hostPort": 8080,
+                                "containerPort": 80
+                            }
+                        ]
+                    },
+                    "labels": {
+                        "name": "testRun"
+                    }
+                }
+            },
+            "labels": {
+                "name": "testRun"
+            }
+        }
+    ]
+}

api/examples/controller.json

@@ -0,0 +1,18 @@
+  {
+    "id": "nginxController",
+    "desiredState": {
+      "replicas": 2,
+      "replicasInSet": {"name": "nginx"},
+      "taskTemplate": {
+        "desiredState": {
+           "manifest": {
+             "containers": [{
+               "image": "dockerfile/nginx",
+               "ports": [{"containerPort": 80, "hostPort": 8080}]
+             }]
+           }
+         },
+         "labels": {"name": "nginx"}
+        }},
+    "labels": {"name": "nginx"}
+  }

api/examples/service-list.json

@@ -0,0 +1,19 @@
+{
+    "items": [
+        {
+            "id": "example1",
+            "port": 8000,
+            "labels": {
+                "name": "nginx"
+            }
+        },
+        {
+            "id": "example2",
+            "port": 8080,
+            "labels": {
+                "env": "prod",
+                "name": "jetty"
+            }
+        }
+    ]
+}

api/examples/service.json

@@ -0,0 +1,7 @@
+{
+  "id": "example2",
+  "port": 8000,
+  "labels": {
+     "name": "nginx"
+  }
+}

api/examples/task-list.json

@@ -0,0 +1,46 @@
+{
+    "items": [
+        {
+            "id": "my-task-1",
+            "labels": {
+                "name": "testRun",
+                "replicationController": "testRun"
+            },
+            "desiredState": {
+              "manifest": {
+                "containers": [{
+                  "image": "dockerfile/nginx",
+                  "ports": [{
+                    "hostPort": 8080,
+                    "containerPort": 80
+                  }]
+                }
+              }
+            },
+            "currentState": {
+                "host": "host-1"
+            }
+        },
+        {
+            "id": "my-task-2",
+            "labels": {
+                "name": "testRun",
+                "replicationController": "testRun"
+            },
+            "desiredState": {
+              "manifest": {
+                "containers": [{
+                  "image": "dockerfile/nginx",
+                  "ports": [{
+                    "hostPort": 8080,
+                    "containerPort": 80
+                  }]
+                }
+              }
+            },
+            "currentState": {
+                "host": "host-2"
+            }
+        }
+    ]
+}

api/examples/task.json

@@ -0,0 +1,18 @@
+{
+  "id": "php",
+  "desiredState": {
+    "manifest": {
+      "containers": [{
+        "image": "dockerfile/nginx",
+        "ports": [{
+          "containerPort": 80,
+          "hostPort": 8080
+        }]
+      }]
+    }
+  },
+  "labels": {
+    "name": "foo"
+  }
+}
+    

api/kubernetes.raml

@@ -0,0 +1,200 @@
+#%RAML 0.8
+baseUri: http://server/api/{version}
+title: Kubernetes
+version: v1beta1
+mediaType: application/json
+documentation:
+ - title: Overview
+   content: |
+     The Kubernetes API currently manages 3 main resources: `tasks`,
+     `replicationControllers`, and `services`. Tasks correspond to
+     colocated groups of [Docker containers](http://docker.io) with
+     shared volumes, as supported by [Google Cloud Platform's
+     container-vm
+     images](https://developers.google.com/compute/docs/containers).
+     Singleton tasks can be created directly via the `/tasks`
+     endpoint. Sets of tasks may created, maintained, and scaled using
+     replicationControllers.  Services create load-balanced targets
+     for sets of tasks.
+
+ - title: Resource identifiers
+   content: |
+     Each resource has a string `id` and list of key-value
+     `labels`. The `id` is generated by the system and is guaranteed
+     to be unique in space and time across all resources.  `labels`
+     is a map of string (key) to string (value). Each resource may
+     have at most one label with a particular key. Individual labels
+     are used to specify identifying metadata that can be used to
+     define sets of resources by specifying required labels. Examples
+     of typical task label keys include `stage`, `service`, `name`,
+     `tier`, `partition`, and `track`, but you are free to develop
+     your own conventions.
+
+ - title: Creation semantics
+   content: |
+     Creation is currently not idempotent. We plan to add a
+     modification token to each resource. A unique value for the token
+     should be provided by the user during creation. If the user
+     specifies a duplicate token at creation time, the system should
+     return an error with a pointer to the exiting resource with that
+     token. In this way a user can deterministically recover from a
+     dropped connection during a resource creation request.
+
+ - title: Update semantics
+   content: |
+     Custom verbs are minimized and are used only for 'edge triggered'
+     actions such as a reboot. Resource descriptions are generally set
+     up with `desiredState` for the user provided parameters and
+     `currentState` for the actual system state. While consistent
+     terminology is used across these two stanzas they do not match
+     member for member.
+
+     When a new version of a resource is PUT the `desiredState` is
+     updated and available immediately. Over time the system will work
+     to bring the `currentState` into line with the `desiredState`. The
+     system will drive toward the most recent `desiredState` regardless
+     of previous versions of that stanza. In other words, if a value
+     is changed from 2 to 5 in one PUT and then back down to 3 in
+     another PUT the system isn't required to 'touch base' at 5 before
+     making 3 the `currentState`.
+
+     When doing an update, we assume that the entire `desiredState`
+     stanza is specified. If a field is omitted it is assumed that the
+     user is looking to delete that field. It is viable for a user to
+     GET the resource, modify what they like in the `desiredState` or
+     labels stanzas and then PUT it back. If the `currentState` is
+     included in the PUT it will be silently ignored.
+
+     While currently unspecified, it is intended that concurrent
+     modification should be accomplished with optimistic locking of
+     resources. We plan to add a modification token to each resource. If
+     this is included with the PUT operation the system will verify
+     that there haven't been other successful mutations to the
+     resource during a read/modify/write cycle. The correct client
+     action at this point is to GET the resource again, apply the
+     changes afresh and try submitting again.
+
+     Note that updates currently only work for replicationControllers
+     and services, but not for tasks. Label updates have not yet been
+     implemented, either.
+
+/tasks:
+  get:
+    description: List all tasks on this cluster
+    responses:
+      200:
+        body:
+          application/json:
+            example: !include examples/task-list.json
+  post:
+    description: Create a new task. currentState is ignored if present.
+    body:
+      json/application:
+        schema: !include doc/task-schema.json
+        example: !include examples/task.json
+  
+  /{taskId}:
+    get:
+      description: Get a specific task
+      responses:
+        200:
+          body:
+            application/json:
+              example: !include examples/task.json
+    put:
+      description: Update a task
+      body:
+        json/application:
+          schema: !include doc/task-schema.json
+          example: !include examples/task.json
+    delete:
+      description: Delete a specific task
+      responses:
+        200:
+          body:
+            application/json:
+              example: |
+                { 
+                  "success": true
+                }
+
+/replicationControllers:
+  get:
+    description: List all replicationControllers on this cluster
+    responses:
+      200:
+        body:
+          application/json:
+            example: !include examples/controller-list.json
+  post:
+    description: Create a new controller. currentState is ignored if present.
+    body:
+      json/application:
+        schema: !include doc/controller-schema.json
+        example: !include examples/controller.json
+  
+  /{controllerId}:
+    get:
+      description: Get a specific controller
+      responses:
+        200:
+          body:
+            application/json:
+              example: !include examples/controller.json
+    put:
+      description: Update a controller
+      body:
+        json/application:
+          schema: !include doc/controller-schema.json
+          example: !include examples/controller.json
+    delete:
+      description: Delete a specific controller
+      responses:
+        200:
+          body:
+            application/json:
+              example: |
+                { 
+                  "success": true
+                }
+
+/services:
+  get:
+    description: List all services on this cluster
+    responses:
+      200:
+        body:
+          application/json:
+            example: !include examples/service-list.json
+  post:
+    description: Create a new service
+    body:
+      json/application:
+        schema: !include doc/service-schema.json
+        example: !include examples/service.json
+  
+  /{serviceId}:
+    get:
+      description: Get a specific service
+      responses:
+        200:
+          body:
+            application/json:
+              example: !include examples/service.json
+    put:
+      description: Update a service
+      body:
+        json/application:
+          schema: !include doc/service-schema.json
+          example: !include examples/service.json
+    delete:
+      description: Delete a specific service
+      responses:
+        200:
+          body:
+            application/json:
+              example: |
+                { 
+                  "success": true
+                }
+

cmd/apiserver/apiserver.go

@@ -0,0 +1,94 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+// apiserver is the main api server and master for the cluster.
+// it is responsible for serving the cluster management API.
+package main
+
+import (
+	"flag"
+	"fmt"
+	"log"
+	"net/http"
+	"time"
+
+	"github.com/coreos/go-etcd/etcd"
+
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/apiserver"
+	kube_client "github.com/GoogleCloudPlatform/kubernetes/pkg/client"
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/registry"
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/util"
+)
+
+var (
+	port                        = flag.Uint("port", 8080, "The port to listen on.  Default 8080.")
+	address                     = flag.String("address", "127.0.0.1", "The address on the local server to listen to. Default 127.0.0.1")
+	apiPrefix                   = flag.String("api_prefix", "/api/v1beta1", "The prefix for API requests on the server. Default '/api/v1beta1'")
+	etcdServerList, machineList util.StringList
+)
+
+func init() {
+	flag.Var(&etcdServerList, "etcd_servers", "Servers for the etcd (http://ip:port), comma separated")
+	flag.Var(&machineList, "machines", "List of machines to schedule onto, comma separated.")
+}
+
+func main() {
+	flag.Parse()
+
+	if len(machineList) == 0 {
+		log.Fatal("No machines specified!")
+	}
+
+	var (
+		taskRegistry       registry.TaskRegistry
+		controllerRegistry registry.ControllerRegistry
+		serviceRegistry    registry.ServiceRegistry
+	)
+
+	if len(etcdServerList) > 0 {
+		log.Printf("Creating etcd client pointing to %v", etcdServerList)
+		etcdClient := etcd.NewClient(etcdServerList)
+		taskRegistry = registry.MakeEtcdRegistry(etcdClient, machineList)
+		controllerRegistry = registry.MakeEtcdRegistry(etcdClient, machineList)
+		serviceRegistry = registry.MakeEtcdRegistry(etcdClient, machineList)
+	} else {
+		taskRegistry = registry.MakeMemoryRegistry()
+		controllerRegistry = registry.MakeMemoryRegistry()
+		serviceRegistry = registry.MakeMemoryRegistry()
+	}
+
+	containerInfo := &kube_client.HTTPContainerInfo{
+		Client: http.DefaultClient,
+		Port:   10250,
+	}
+
+	storage := map[string]apiserver.RESTStorage{
+		"tasks":                  registry.MakeTaskRegistryStorage(taskRegistry, containerInfo, registry.MakeFirstFitScheduler(machineList, taskRegistry)),
+		"replicationControllers": registry.MakeControllerRegistryStorage(controllerRegistry),
+		"services":               registry.MakeServiceRegistryStorage(serviceRegistry),
+	}
+
+	endpoints := registry.MakeEndpointController(serviceRegistry, taskRegistry)
+	go util.Forever(func() { endpoints.SyncServiceEndpoints() }, time.Second*10)
+
+	s := &http.Server{
+		Addr:           fmt.Sprintf("%s:%d", *address, *port),
+		Handler:        apiserver.New(storage, *apiPrefix),
+		ReadTimeout:    10 * time.Second,
+		WriteTimeout:   10 * time.Second,
+		MaxHeaderBytes: 1 << 20,
+	}
+	log.Fatal(s.ListenAndServe())
+}

cmd/cloudcfg/cloudcfg.go

@@ -0,0 +1,126 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+package main
+
+import (
+	"flag"
+	"fmt"
+	"log"
+	"net/http"
+	"os"
+	"strconv"
+	"time"
+
+	kube_client "github.com/GoogleCloudPlatform/kubernetes/pkg/client"
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/cloudcfg"
+)
+
+const APP_VERSION = "0.1"
+
+// The flag package provides a default help printer via -h switch
+var versionFlag *bool = flag.Bool("v", false, "Print the version number.")
+var httpServer *string = flag.String("h", "", "The host to connect to.")
+var config *string = flag.String("c", "", "Path to the config file.")
+var labelQuery *string = flag.String("l", "", "Label query to use for listing")
+var updatePeriod *time.Duration = flag.Duration("u", 60*time.Second, "Update interarrival in seconds")
+var portSpec *string = flag.String("p", "", "The port spec, comma-separated list of <external>:<internal>,...")
+var servicePort *int = flag.Int("s", -1, "If positive, create and run a corresponding service on this port, only used with 'run'")
+var authConfig *string = flag.String("auth", os.Getenv("HOME")+"/.kubernetes_auth", "Path to the auth info file.  If missing, prompt the user")
+
+func usage() {
+	log.Fatal("Usage: cloudcfg -h <host> [-c config/file.json] [-p <hostPort>:<containerPort>,..., <hostPort-n>:<containerPort-n> <method> <path>")
+}
+
+// CloudCfg command line tool.
+func main() {
+	flag.Parse() // Scan the arguments list
+
+	if *versionFlag {
+		fmt.Println("Version:", APP_VERSION)
+		os.Exit(0)
+	}
+
+	if len(flag.Args()) < 2 {
+		usage()
+	}
+	method := flag.Arg(0)
+	url := *httpServer + "/api/v1beta1" + flag.Arg(1)
+	var request *http.Request
+	var err error
+
+	auth, err := cloudcfg.LoadAuthInfo(*authConfig)
+	if err != nil {
+		log.Fatalf("Error loading auth: %#v", err)
+	}
+
+	if method == "get" || method == "list" {
+		if len(*labelQuery) > 0 && method == "list" {
+			url = url + "?labels=" + *labelQuery
+		}
+		request, err = http.NewRequest("GET", url, nil)
+	} else if method == "delete" {
+		request, err = http.NewRequest("DELETE", url, nil)
+	} else if method == "create" {
+		request, err = cloudcfg.RequestWithBody(*config, url, "POST")
+	} else if method == "update" {
+		request, err = cloudcfg.RequestWithBody(*config, url, "PUT")
+	} else if method == "rollingupdate" {
+		client := &kube_client.Client{
+			Host: *httpServer,
+			Auth: &auth,
+		}
+		cloudcfg.Update(flag.Arg(1), client, *updatePeriod)
+	} else if method == "run" {
+		args := flag.Args()
+		if len(args) < 4 {
+			log.Fatal("usage: cloudcfg -h <host> run <image> <replicas> <name>")
+		}
+		image := args[1]
+		replicas, err := strconv.Atoi(args[2])
+		name := args[3]
+		if err != nil {
+			log.Fatalf("Error parsing replicas: %#v", err)
+		}
+		err = cloudcfg.RunController(image, name, replicas, kube_client.Client{Host: *httpServer, Auth: &auth}, *portSpec, *servicePort)
+		if err != nil {
+			log.Fatalf("Error: %#v", err)
+		}
+		return
+	} else if method == "stop" {
+		err = cloudcfg.StopController(flag.Arg(1), kube_client.Client{Host: *httpServer, Auth: &auth})
+		if err != nil {
+			log.Fatalf("Error: %#v", err)
+		}
+		return
+	} else if method == "rm" {
+		err = cloudcfg.DeleteController(flag.Arg(1), kube_client.Client{Host: *httpServer, Auth: &auth})
+		if err != nil {
+			log.Fatalf("Error: %#v", err)
+		}
+		return
+	} else {
+		log.Fatalf("Unknown command: %s", method)
+	}
+	if err != nil {
+		log.Fatalf("Error: %#v", err)
+	}
+	var body string
+	body, err = cloudcfg.DoRequest(request, auth.User, auth.Password)
+	if err != nil {
+		log.Fatalf("Error: %#v", err)
+	}
+	fmt.Println(body)
+}

cmd/controller-manager/controller-manager.go

@@ -0,0 +1,58 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+// The controller manager is responsible for monitoring replication controllers, and creating corresponding
+// tasks to achieve the desired state.  It listens for new controllers in etcd, and it sends requests to the
+// master to create/delete tasks.
+//
+// TODO: Refactor the etcd watch code so that it is a pluggable interface.
+package main
+
+import (
+	"flag"
+	"log"
+	"os"
+	"time"
+
+	kube_client "github.com/GoogleCloudPlatform/kubernetes/pkg/client"
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/registry"
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/util"
+	"github.com/coreos/go-etcd/etcd"
+)
+
+var (
+	etcd_servers = flag.String("etcd_servers", "", "Servers for the etcd (http://ip:port).")
+	master       = flag.String("master", "", "The address of the Kubernetes API server")
+)
+
+func main() {
+	flag.Parse()
+
+	if len(*etcd_servers) == 0 || len(*master) == 0 {
+		log.Fatal("usage: controller-manager -etcd_servers <servers> -master <master>")
+	}
+
+	// Set up logger for etcd client
+	etcd.SetLogger(log.New(os.Stderr, "etcd ", log.LstdFlags))
+
+	controllerManager := registry.MakeReplicationManager(etcd.NewClient([]string{*etcd_servers}),
+		kube_client.Client{
+			Host: "http://" + *master,
+		})
+
+	go util.Forever(func() { controllerManager.Synchronize() }, 20*time.Second)
+	go util.Forever(func() { controllerManager.WatchControllers() }, 20*time.Second)
+	select {}
+}

cmd/integration/integration.go

@@ -0,0 +1,87 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+// A basic integration test for the service.
+// Assumes that there is a pre-existing etcd server running on localhost.
+package main
+
+import (
+	"encoding/json"
+	"io/ioutil"
+	"log"
+	"net/http/httptest"
+	"time"
+
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/api"
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/apiserver"
+	kube_client "github.com/GoogleCloudPlatform/kubernetes/pkg/client"
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/registry"
+	"github.com/coreos/go-etcd/etcd"
+)
+
+func main() {
+
+	// Setup
+	servers := []string{"http://localhost:4001"}
+	log.Printf("Creating etcd client pointing to %v", servers)
+	etcdClient := etcd.NewClient(servers)
+	machineList := []string{"machine"}
+
+	reg := registry.MakeEtcdRegistry(etcdClient, machineList)
+
+	apiserver := apiserver.New(map[string]apiserver.RESTStorage{
+		"tasks":                  registry.MakeTaskRegistryStorage(reg, &kube_client.FakeContainerInfo{}, registry.MakeRoundRobinScheduler(machineList)),
+		"replicationControllers": registry.MakeControllerRegistryStorage(reg),
+	}, "/api/v1beta1")
+	server := httptest.NewServer(apiserver)
+
+	controllerManager := registry.MakeReplicationManager(etcd.NewClient(servers),
+		kube_client.Client{
+			Host: server.URL,
+		})
+
+	go controllerManager.Synchronize()
+	go controllerManager.WatchControllers()
+
+	// Ok. we're good to go.
+	log.Printf("API Server started on %s", server.URL)
+	// Wait for the synchronization threads to come up.
+	time.Sleep(time.Second * 10)
+
+	kubeClient := kube_client.Client{
+		Host: server.URL,
+	}
+	data, err := ioutil.ReadFile("api/examples/controller.json")
+	if err != nil {
+		log.Fatalf("Unexpected error: %#v", err)
+	}
+	var controllerRequest api.ReplicationController
+	if err = json.Unmarshal(data, &controllerRequest); err != nil {
+		log.Fatalf("Unexpected error: %#v", err)
+	}
+
+	if _, err = kubeClient.CreateReplicationController(controllerRequest); err != nil {
+		log.Fatalf("Unexpected error: %#v", err)
+	}
+	// Give the controllers some time to actually create the tasks
+	time.Sleep(time.Second * 10)
+
+	// Validate that they're truly up.
+	tasks, err := kubeClient.ListTasks(nil)
+	if err != nil || len(tasks.Items) != 2 {
+		log.Fatal("FAILED")
+	}
+	log.Printf("OK")
+}

cmd/kubelet/kubelet.go

@@ -0,0 +1,67 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+// The kubelet binary is responsible for maintaining a set of containers on a particular host VM.
+// It sync's data from both configuration file as well as from a quorum of etcd servers.
+// It then queries Docker to see what is currently running.  It synchronizes the configuration data,
+// with the running set of containers by starting or stopping Docker containers.
+package main
+
+import (
+	"flag"
+	"log"
+	"math/rand"
+	"os"
+	"time"
+
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/kubelet"
+	"github.com/coreos/go-etcd/etcd"
+	"github.com/fsouza/go-dockerclient"
+)
+
+var (
+	file               = flag.String("config", "", "Path to the config file")
+	etcd_servers       = flag.String("etcd_servers", "", "Url of etcd servers in the cluster")
+	syncFrequency      = flag.Duration("sync_frequency", 10*time.Second, "Max seconds between synchronizing running containers and config")
+	fileCheckFrequency = flag.Duration("file_check_frequency", 20*time.Second, "Seconds between checking file for new data")
+	httpCheckFrequency = flag.Duration("http_check_frequency", 20*time.Second, "Seconds between checking http for new data")
+	manifest_url       = flag.String("manifest_url", "", "URL for accessing the container manifest")
+	address            = flag.String("address", "127.0.0.1", "The address for the info server to serve on")
+	port               = flag.Uint("port", 10250, "The port for the info server to serve on")
+)
+
+const dockerBinary = "/usr/bin/docker"
+
+func main() {
+	flag.Parse()
+	rand.Seed(time.Now().UTC().UnixNano())
+
+	// Set up logger for etcd client
+	etcd.SetLogger(log.New(os.Stderr, "etcd ", log.LstdFlags))
+
+	endpoint := "unix:///var/run/docker.sock"
+	dockerClient, err := docker.NewClient(endpoint)
+	if err != nil {
+		log.Fatal("Couldn't connnect to docker.")
+	}
+
+	my_kubelet := kubelet.Kubelet{
+		DockerClient:       dockerClient,
+		FileCheckFrequency: *fileCheckFrequency,
+		SyncFrequency:      *syncFrequency,
+		HTTPCheckFrequency: *httpCheckFrequency,
+	}
+	my_kubelet.RunKubelet(*file, *manifest_url, *etcd_servers, *address, *port)
+}

cmd/proxy/proxy.go

@@ -0,0 +1,64 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+package main
+
+import (
+	"flag"
+	"log"
+	"os"
+
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/proxy"
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/proxy/config"
+	"github.com/coreos/go-etcd/etcd"
+)
+
+var (
+	config_file  = flag.String("configfile", "/tmp/proxy_config", "Configuration file for the proxy")
+	etcd_servers = flag.String("etcd_servers", "http://10.240.10.57:4001", "Servers for the etcd cluster (http://ip:port).")
+)
+
+func main() {
+	flag.Parse()
+
+	// Set up logger for etcd client
+	etcd.SetLogger(log.New(os.Stderr, "etcd ", log.LstdFlags))
+
+	log.Printf("Using configuration file %s and etcd_servers %s", *config_file, *etcd_servers)
+
+	proxyConfig := config.NewServiceConfig()
+
+	// Create a configuration source that handles configuration from etcd.
+	etcdClient := etcd.NewClient([]string{*etcd_servers})
+	config.NewConfigSourceEtcd(etcdClient,
+		proxyConfig.GetServiceConfigurationChannel("etcd"),
+		proxyConfig.GetEndpointsConfigurationChannel("etcd"))
+
+	// And create a configuration source that reads from a local file
+	config.NewConfigSourceFile(*config_file,
+		proxyConfig.GetServiceConfigurationChannel("file"),
+		proxyConfig.GetEndpointsConfigurationChannel("file"))
+
+	loadBalancer := proxy.NewLoadBalancerRR()
+	proxier := proxy.NewProxier(loadBalancer)
+	// Wire proxier to handle changes to services
+	proxyConfig.RegisterServiceHandler(proxier)
+	// And wire loadBalancer to handle changes to endpoints to services
+	proxyConfig.RegisterEndpointsHandler(loadBalancer)
+
+	// Just loop forever for now...
+	select {}
+
+}

examples/guestbook/frontend-controller.json

@@ -0,0 +1,18 @@
+  {
+    "id": "frontendController",
+    "desiredState": {
+      "replicas": 3,
+      "replicasInSet": {"name": "frontend"},
+      "taskTemplate": {
+        "desiredState": {
+           "manifest": {
+             "containers": [{
+               "image": "brendanburns/php-redis",
+               "ports": [{"containerPort": 80, "hostPort": 8080}]
+             }]
+           }
+         },
+         "labels": {"name": "frontend"}
+        }},
+    "labels": {"name": "frontend"}
+  }

examples/guestbook/guestbook.md

@@ -0,0 +1,222 @@
+## GuestBook example
+
+This example shows how to build a simple multi-tier web application using Kubernetes and Docker.
+
+The example combines a web frontend, a redis master for storage and a replicated set of redis slaves.
+
+### Step Zero: Prerequisites
+This example assumes that you have forked the repository and turned up a Kubernetes cluster.
+
+
+### Step One: Turn up the redis master.
+
+Create a file named redis-master.json, this file is describes a single task, which runs a redis key-value server in a container.
+
+```javascript
+{
+  "id": "redis-master-2",
+  "desiredState": {
+    "manifest": {
+      "containers": [{
+        "name": "master",
+        "image": "dockerfile/redis",
+        "ports": [{
+          "containerPort": 6379,
+          "hostPort": 6379 
+        }]
+      }]
+    }
+  },
+  "labels": {
+    "name": "redis-master"
+  }
+}
+```
+
+Once you have that task file, you can create the redis task in your Kubernetes cluster using the cloudcfg cli:
+
+```shell
+./src/scripts/cloudcfg.sh -c redis-master.json create /tasks
+```
+
+Once that's up you can list the tasks in the cluster, to verify that the master is running:
+
+```shell
+./src/scripts/cloudcfg.sh list /tasks
+```
+
+You should see a single redis master task.  It will also display the machine that the task is running on.  If you ssh to that machine, you can run
+```shell
+sudo docker ps
+```
+
+And see the actual task.  (note that initial ```docker pull``` may take a few minutes, depending on network conditions.
+
+### Step Two: Turn up the master service.
+A Kubernetes 'service' is named load balancer that proxies traffic to one or more containers.  The services in a Kubernetes cluster are discoverable inside other containers via environment variables.  Services find the containers to load balance based on task labels.  The task that you created in Step One has the label "name=redis-master", so the corresponding service is defined by that label.  Create a file named redis-master-service.json that contains:
+
+```javascript
+{
+  "id": "redismaster",
+  "port": 10000,
+  "labels": {
+    "name": "redis-master"
+  }
+}
+```
+
+Once you have that service description, you can create the service with the cloudcfg cli:
+
+```shell
+./src/scripts/cloudcfg.sh -c redis-master-service create /services
+```
+
+Once created, the service proxy on each minion is configured to set up a proxy on the specified port (in this case port 10000).
+
+### Step Three: Turn up the replicated slave service.
+Although the redis master is a single task, the redis read slaves are a 'replicated' task, in Kubernetes, a replication controller is responsible for managing multiple instances of a replicated task.  Create a file named redis-slave-controller.json that contains:
+
+```javascript
+  {
+    "id": "redisSlaveController",
+    "desiredState": {
+      "replicas": 2,
+      "replicasInSet": {"name": "redis-slave"},
+      "taskTemplate": {
+        "desiredState": {
+           "manifest": {
+             "containers": [{
+               "image": "brendanburns/redis-slave",
+               "ports": [{"containerPort": 6379, "hostPort": 6380}]
+             }]
+           }
+         },
+         "labels": {"name": "redis-slave"}
+        }},
+    "labels": {"name": "redis-slave"}
+  }
+```
+
+Then you can create the service by running:
+
+```shell
+./src/scripts/cloudcfg.sh -c redis-slave-controller.json create /replicationControllers
+```
+
+The redis slave configures itself by looking for the Kubernetes service environment variables in the container environment.  In particular, the redis slave is started with the following command:
+
+```shell
+redis-server --slaveof $SERVICE_HOST $REDISMASTER_SERVICE_PORT
+```
+
+Once that's up you can list the tasks in the cluster, to verify that the master and slaves are running:
+
+```shell
+./src/scripts/cloudcfg.sh list /tasks
+```
+
+You should see a single redis master task, and two redis slave tasks.
+
+### Step Four: Create the redis slave service.
+
+Just like the master, we want to have a service to proxy connections to the read slaves.  In this case, in addition to discovery, the slave service provides transparent load balancing to clients.  As before, create a service specification:
+
+```javascript
+{
+  "id": "redisslave",
+  "port": 10001,
+  "labels": {
+    "name": "redis-slave"
+  }
+}
+```
+
+This time the label query for the service is 'name=redis-slave'
+
+Now that you have created the service specification, create it in your cluster with the cloudcfg cli:
+
+```shell
+./src/scripts/cloudcfg.sh -c redis-slave-service.json create /services
+```
+
+### Step Five: Create the frontend service.
+
+This is a simple PHP server that is configured to talk to both the slave and master services depdending on if the request is a read or a write.  It exposes a simple AJAX interface, and serves an angular based U/X.  Like the redis read slaves it is a replicated service instantiated by a replication controller.  Create a file named frontend-controller.json:
+
+```javascript
+  {
+    "id": "frontendController",
+    "desiredState": {
+      "replicas": 3,
+      "replicasInSet": {"name": "frontend"},
+      "taskTemplate": {
+        "desiredState": {
+           "manifest": {
+             "containers": [{
+               "image": "brendanburns/php-redis",
+               "ports": [{"containerPort": 80, "hostPort": 8080}]
+             }]
+           }
+         },
+         "labels": {"name": "frontend"}
+        }},
+    "labels": {"name": "frontend"}
+  }
+```
+
+With this file, you can turn up your frontend with:
+
+```shell
+./src/scripts/cloudcfg.sh -c frontend-controller.json create /replicationControllers
+```
+
+Once that's up you can list the tasks in the cluster, to verify that the master, slaves and frontends are running:
+
+```shell
+./src/scripts/cloudcfg.sh list /tasks
+```
+
+You should see a single redis master task, two redis slave and three frontend tasks.
+
+The code for the PHP service looks like this:
+```php
+<?
+
+set_include_path('.:/usr/share/php:/usr/share/pear:/vendor/predis');
+
+error_reporting(E_ALL);
+ini_set('display_errors', 1);
+
+require 'predis/autoload.php';
+
+if (isset($_GET['cmd']) === true) {
+  header('Content-Type: application/json');
+  if ($_GET['cmd'] == 'set') {
+    $client = new Predis\Client([
+      'scheme' => 'tcp',
+      'host'   => getenv('SERVICE_HOST'),
+      'port'   => getenv('REDISMASTER_SERVICE_PORT'),
+    ]);
+    $client->set($_GET['key'], $_GET['value']);
+    print('{"message": "Updated"}');
+  } else {
+    $read_port = getenv('REDISMASTER_SERVICE_PORT');
+
+    if (isset($_ENV['REDISSLAVE_SERVICE_PORT'])) {
+      $read_port = getenv('REDISSLAVE_SERVICE_PORT');
+    }
+    $client = new Predis\Client([
+      'scheme' => 'tcp',
+      'host'   => getenv('SERVICE_HOST'),
+      'port'   => $read_port,
+    ]);
+
+    $value = $client->get($_GET['key']);
+    print('{"data": "' . $value . '"}');
+  }
+} else {
+  phpinfo();
+} ?>
+```
+
+To play with the service itself, find the name of a frontend, grab the external IP of that host from the Google Cloud Console, and visit http://&lt;host-ip&gt;:8080, note you may need to open the firewall for port 8080 using the console or the gcloud tool.

examples/guestbook/index.php

@@ -0,0 +1,37 @@
+<?
+
+set_include_path('.:/usr/share/php:/usr/share/pear:/vendor/predis');
+
+error_reporting(E_ALL);
+ini_set('display_errors', 1);
+
+require 'predis/autoload.php';
+
+if (isset($_GET['cmd']) === true) {
+  header('Content-Type: application/json');
+  if ($_GET['cmd'] == 'set') {
+    $client = new Predis\Client([
+      'scheme' => 'tcp',
+      'host'   => getenv('SERVICE_HOST'),
+      'port'   => getenv('REDISMASTER_SERVICE_PORT'),
+    ]);
+    $client->set($_GET['key'], $_GET['value']);
+    print('{"message": "Updated"}');
+  } else {
+    $read_port = getenv('REDISMASTER_SERVICE_PORT');
+
+    if (isset($_ENV['REDISSLAVE_SERVICE_PORT'])) {
+      $read_port = getenv('REDISSLAVE_SERVICE_PORT');
+    }
+    $client = new Predis\Client([
+      'scheme' => 'tcp',
+      'host'   => getenv('SERVICE_HOST'),
+      'port'   => $read_port,
+    ]);
+
+    $value = $client->get($_GET['key']);
+    print('{"data": "' . $value . '"}');
+  }
+} else {
+  phpinfo();
+} ?>

examples/guestbook/redis-master-service.json

@@ -0,0 +1,7 @@
+{
+  "id": "redismaster",
+  "port": 10000,
+  "labels": {
+    "name": "redis-master"
+  }
+}

examples/guestbook/redis-master.json

@@ -0,0 +1,19 @@
+{
+  "id": "redis-master-2",
+  "desiredState": {
+    "manifest": {
+      "containers": [{
+        "name": "master",
+        "image": "dockerfile/redis",
+        "ports": [{
+          "containerPort": 6379,
+          "hostPort": 6379 
+        }]
+      }]
+    }
+  },
+  "labels": {
+    "name": "redis-master"
+  }
+}
+    

examples/guestbook/redis-slave-controller.json

@@ -0,0 +1,18 @@
+  {
+    "id": "redisSlaveController",
+    "desiredState": {
+      "replicas": 2,
+      "replicasInSet": {"name": "redisslave"},
+      "taskTemplate": {
+        "desiredState": {
+           "manifest": {
+             "containers": [{
+               "image": "brendanburns/redis-slave",
+               "ports": [{"containerPort": 6379, "hostPort": 6380}]
+             }]
+           }
+         },
+         "labels": {"name": "redisslave"}
+        }},
+    "labels": {"name": "redisslave"}
+  }

examples/guestbook/redis-slave-service.json

@@ -0,0 +1,7 @@
+{
+  "id": "redisslave",
+  "port": 10001,
+  "labels": {
+    "name": "redisslave"
+  }
+}

hooks/commit-msg

@@ -0,0 +1,10 @@
+#!/bin/bash
+
+if [[ "$(grep -c "# then delete this line" $1)" == "1" ]]; then
+  echo "Unresolved gofmt errors. Aborting commit."
+  echo "The message of your attempted commit was:"
+  cat $1
+  exit 1
+fi
+
+exit 0

hooks/prepare-commit-msg

@@ -0,0 +1,17 @@
+#!/bin/bash
+
+errors=0
+for file in $(git diff --cached --name-only | grep "\.go"); do
+  diff="$(gofmt -d "${file}")"
+  if [[ -n "$diff" ]]; then
+    echo "# *** ERROR: *** File ${file} has not been gofmt'd." >> $1
+    errors=1
+  fi
+done
+
+if [[ $errors == "1" ]]; then
+  echo "# To fix these errors, run gofmt -w <file>." >> $1
+  echo "# If you want to commit in spite of these format errors," >> $1
+  echo "# then delete this line. Otherwise, your commit will be" >> $1
+  echo "# aborted." >> $1
+fi

pkg/api/types.go

@@ -0,0 +1,149 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+// Package api includes all types used to communicate between the various
+// parts of the Kubernetes system.
+package api
+
+// ContainerManifest corresponds to the Container Manifest format, documented at:
+// https://developers.google.com/compute/docs/containers#container_manifest
+// This is used as the representation of Kubernete's workloads.
+type ContainerManifest struct {
+	Version    string      `yaml:"version" json:"version"`
+	Volumes    []Volume    `yaml:"volumes" json:"volumes"`
+	Containers []Container `yaml:"containers" json:"containers"`
+	Id         string      `yaml:"id,omitempty" json:"id,omitempty"`
+}
+
+type Volume struct {
+	Name string `yaml:"name" json:"name"`
+}
+
+type Port struct {
+	Name          string `yaml:"name,omitempty" json:"name,omitempty"`
+	HostPort      int    `yaml:"hostPort,omitempty" json:"hostPort,omitempty"`
+	ContainerPort int    `yaml:"containerPort,omitempty" json:"containerPort,omitempty"`
+	Protocol      string `yaml:"protocol,omitempty" json:"protocol,omitempty"`
+}
+
+type VolumeMount struct {
+	Name      string `yaml:"name,omitempty" json:"name,omitempty"`
+	ReadOnly  bool   `yaml:"readOnly,omitempty" json:"readOnly,omitempty"`
+	MountPath string `yaml:"mountPath,omitempty" json:"mountPath,omitempty"`
+}
+
+type EnvVar struct {
+	Name  string `yaml:"name,omitempty" json:"name,omitempty"`
+	Value string `yaml:"value,omitempty" json:"value,omitempty"`
+}
+
+// Container represents a single container that is expected to be run on the host.
+type Container struct {
+	Name         string        `yaml:"name,omitempty" json:"name,omitempty"`
+	Image        string        `yaml:"image,omitempty" json:"image,omitempty"`
+	Command      string        `yaml:"command,omitempty" json:"command,omitempty"`
+	WorkingDir   string        `yaml:"workingDir,omitempty" json:"workingDir,omitempty"`
+	Ports        []Port        `yaml:"ports,omitempty" json:"ports,omitempty"`
+	Env          []EnvVar      `yaml:"env,omitempty" json:"env,omitempty"`
+	Memory       int           `yaml:"memory,omitempty" json:"memory,omitempty"`
+	CPU          int           `yaml:"cpu,omitempty" json:"cpu,omitempty"`
+	VolumeMounts []VolumeMount `yaml:"volumeMounts,omitempty" json:"volumeMounts,omitempty"`
+}
+
+// Event is the representation of an event logged to etcd backends
+type Event struct {
+	Event     string             `json:"event,omitempty"`
+	Manifest  *ContainerManifest `json:"manifest,omitempty"`
+	Container *Container         `json:"container,omitempty"`
+	Timestamp int64              `json:"timestamp"`
+}
+
+// The below types are used by kube_client and api_server.
+
+// JSONBase is shared by all objects sent to, or returned from the client
+type JSONBase struct {
+	Kind              string `json:"kind,omitempty" yaml:"kind,omitempty"`
+	ID                string `json:"id,omitempty" yaml:"id,omitempty"`
+	CreationTimestamp string `json:"creationTimestamp,omitempty" yaml:"creationTimestamp,omitempty"`
+	SelfLink          string `json:"selfLink,omitempty" yaml:"selfLink,omitempty"`
+}
+
+// TaskState is the state of a task, used as either input (desired state) or output (current state)
+type TaskState struct {
+	Manifest ContainerManifest `json:"manifest,omitempty" yaml:"manifest,omitempty"`
+	Status   string            `json:"status,omitempty" yaml:"status,omitempty"`
+	Host     string            `json:"host,omitempty" yaml:"host,omitempty"`
+	HostIP   string            `json:"hostIP,omitempty" yaml:"hostIP,omitempty"`
+	Info     interface{}       `json:"info,omitempty" yaml:"info,omitempty"`
+}
+
+type TaskList struct {
+	JSONBase
+	Items []Task `json:"items" yaml:"items,omitempty"`
+}
+
+// Task is a single task, used as either input (create, update) or as output (list, get)
+type Task struct {
+	JSONBase
+	Labels       map[string]string `json:"labels,omitempty" yaml:"labels,omitempty"`
+	DesiredState TaskState         `json:"desiredState,omitempty" yaml:"desiredState,omitempty"`
+	CurrentState TaskState         `json:"currentState,omitempty" yaml:"currentState,omitempty"`
+}
+
+// ReplicationControllerState is the state of a replication controller, either input (create, update) or as output (list, get)
+type ReplicationControllerState struct {
+	Replicas      int               `json:"replicas" yaml:"replicas"`
+	ReplicasInSet map[string]string `json:"replicasInSet,omitempty" yaml:"replicasInSet,omitempty"`
+	TaskTemplate  TaskTemplate      `json:"taskTemplate,omitempty" yaml:"taskTemplate,omitempty"`
+}
+
+type ReplicationControllerList struct {
+	JSONBase
+	Items []ReplicationController `json:"items,omitempty" yaml:"items,omitempty"`
+}
+
+// ReplicationController represents the configuration of a replication controller
+type ReplicationController struct {
+	JSONBase
+	DesiredState ReplicationControllerState `json:"desiredState,omitempty" yaml:"desiredState,omitempty"`
+	Labels       map[string]string          `json:"labels,omitempty" yaml:"labels,omitempty"`
+}
+
+// TaskTemplate holds the information used for creating tasks
+type TaskTemplate struct {
+	DesiredState TaskState         `json:"desiredState,omitempty" yaml:"desiredState,omitempty"`
+	Labels       map[string]string `json:"labels,omitempty" yaml:"labels,omitempty"`
+}
+
+// ServiceList holds a list of services
+type ServiceList struct {
+	Items []Service `json:"items" yaml:"items"`
+}
+
+// Defines a service abstraction by a name (for example, mysql) consisting of local port
+// (for example 3306) that the proxy listens on, and the labels that define the service.
+type Service struct {
+	JSONBase
+	Port   int               `json:"port,omitempty" yaml:"port,omitempty"`
+	Labels map[string]string `json:"labels,omitempty" yaml:"labels,omitempty"`
+}
+
+// Defines the endpoints that implement the actual service, for example:
+// Name: "mysql", Endpoints: ["10.10.1.1:1909", "10.10.2.2:8834"]
+type Endpoints struct {
+	Name      string
+	Endpoints []string
+}

pkg/apiserver/api_server.go

@@ -0,0 +1,209 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+// Package apiserver is ...
+package apiserver
+
+import (
+	"encoding/json"
+	"fmt"
+	"io/ioutil"
+	"log"
+	"net/http"
+	"net/url"
+	"strings"
+)
+
+// RESTStorage is a generic interface for RESTful storage services
+type RESTStorage interface {
+	List(*url.URL) (interface{}, error)
+	Get(id string) (interface{}, error)
+	Delete(id string) error
+	Extract(body string) (interface{}, error)
+	Create(interface{}) error
+	Update(interface{}) error
+}
+
+// Status is a return value for calls that don't return other objects
+type Status struct {
+	success bool
+}
+
+// ApiServer is an HTTPHandler that delegates to RESTStorage objects.
+// It handles URLs of the form:
+// ${prefix}/${storage_key}[/${object_name}]
+// Where 'prefix' is an arbitrary string, and 'storage_key' points to a RESTStorage object stored in storage.
+//
+// TODO: consider migrating this to go-restful which is a more full-featured version of the same thing.
+type ApiServer struct {
+	prefix  string
+	storage map[string]RESTStorage
+}
+
+// New creates a new ApiServer object.
+// 'storage' contains a map of handlers.
+// 'prefix' is the hosting path prefix.
+func New(storage map[string]RESTStorage, prefix string) *ApiServer {
+	return &ApiServer{
+		storage: storage,
+		prefix:  prefix,
+	}
+}
+
+func (server *ApiServer) handleIndex(w http.ResponseWriter) {
+	w.WriteHeader(http.StatusOK)
+	// TODO: serve this out of a file?
+	data := "<html><body>Welcome to Kubernetes</body></html>"
+	fmt.Fprint(w, data)
+}
+
+// HTTP Handler interface
+func (server *ApiServer) ServeHTTP(w http.ResponseWriter, req *http.Request) {
+	log.Printf("%s %s", req.Method, req.RequestURI)
+	url, err := url.ParseRequestURI(req.RequestURI)
+	if err != nil {
+		server.error(err, w)
+		return
+	}
+	if url.Path == "/index.html" || url.Path == "/" || url.Path == "" {
+		server.handleIndex(w)
+		return
+	}
+	if !strings.HasPrefix(url.Path, server.prefix) {
+		server.notFound(req, w)
+		return
+	}
+	requestParts := strings.Split(url.Path[len(server.prefix):], "/")[1:]
+	if len(requestParts) < 1 {
+		server.notFound(req, w)
+		return
+	}
+	storage := server.storage[requestParts[0]]
+	if storage == nil {
+		server.notFound(req, w)
+		return
+	} else {
+		server.handleREST(requestParts, url, req, w, storage)
+	}
+}
+
+func (server *ApiServer) notFound(req *http.Request, w http.ResponseWriter) {
+	w.WriteHeader(404)
+	fmt.Fprintf(w, "Not Found: %#v", req)
+}
+
+func (server *ApiServer) write(statusCode int, object interface{}, w http.ResponseWriter) {
+	w.WriteHeader(statusCode)
+	output, err := json.MarshalIndent(object, "", "    ")
+	if err != nil {
+		server.error(err, w)
+		return
+	}
+	w.Write(output)
+}
+
+func (server *ApiServer) error(err error, w http.ResponseWriter) {
+	w.WriteHeader(500)
+	fmt.Fprintf(w, "Internal Error: %#v", err)
+}
+
+func (server *ApiServer) readBody(req *http.Request) (string, error) {
+	defer req.Body.Close()
+	body, err := ioutil.ReadAll(req.Body)
+	return string(body), err
+}
+
+func (server *ApiServer) handleREST(parts []string, url *url.URL, req *http.Request, w http.ResponseWriter, storage RESTStorage) {
+	switch req.Method {
+	case "GET":
+		switch len(parts) {
+		case 1:
+			controllers, err := storage.List(url)
+			if err != nil {
+				server.error(err, w)
+				return
+			}
+			server.write(200, controllers, w)
+		case 2:
+			task, err := storage.Get(parts[1])
+			if err != nil {
+				server.error(err, w)
+				return
+			}
+			if task == nil {
+				server.notFound(req, w)
+				return
+			}
+			server.write(200, task, w)
+		default:
+			server.notFound(req, w)
+		}
+		return
+	case "POST":
+		if len(parts) != 1 {
+			server.notFound(req, w)
+			return
+		}
+		body, err := server.readBody(req)
+		if err != nil {
+			server.error(err, w)
+			return
+		}
+		obj, err := storage.Extract(body)
+		if err != nil {
+			server.error(err, w)
+			return
+		}
+		storage.Create(obj)
+		server.write(200, obj, w)
+		return
+	case "DELETE":
+		if len(parts) != 2 {
+			server.notFound(req, w)
+			return
+		}
+		err := storage.Delete(parts[1])
+		if err != nil {
+			server.error(err, w)
+			return
+		}
+		server.write(200, Status{success: true}, w)
+		return
+	case "PUT":
+		if len(parts) != 2 {
+			server.notFound(req, w)
+			return
+		}
+		body, err := server.readBody(req)
+		if err != nil {
+			server.error(err, w)
+		}
+		obj, err := storage.Extract(body)
+		if err != nil {
+			server.error(err, w)
+			return
+		}
+		err = storage.Update(obj)
+		if err != nil {
+			server.error(err, w)
+			return
+		}
+		server.write(200, obj, w)
+		return
+	default:
+		server.notFound(req, w)
+	}
+}

pkg/apiserver/api_server_test.go

@@ -0,0 +1,282 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+package apiserver
+
+import (
+	"bytes"
+	"encoding/json"
+	"fmt"
+	"io/ioutil"
+	"net/http"
+	"net/http/httptest"
+	"net/url"
+	"reflect"
+	"testing"
+)
+
+// TODO: This doesn't reduce typing enough to make it worth the less readable errors. Remove.
+func expectNoError(t *testing.T, err error) {
+	if err != nil {
+		t.Errorf("Unexpected error: %#v", err)
+	}
+}
+
+type Simple struct {
+	Name string
+}
+
+type SimpleList struct {
+	Items []Simple
+}
+
+type SimpleRESTStorage struct {
+	err     error
+	list    []Simple
+	item    Simple
+	deleted string
+	updated Simple
+}
+
+func (storage *SimpleRESTStorage) List(*url.URL) (interface{}, error) {
+	result := SimpleList{
+		Items: storage.list,
+	}
+	return result, storage.err
+}
+
+func (storage *SimpleRESTStorage) Get(id string) (interface{}, error) {
+	return storage.item, storage.err
+}
+
+func (storage *SimpleRESTStorage) Delete(id string) error {
+	storage.deleted = id
+	return storage.err
+}
+
+func (storage *SimpleRESTStorage) Extract(body string) (interface{}, error) {
+	var item Simple
+	json.Unmarshal([]byte(body), &item)
+	return item, storage.err
+}
+
+func (storage *SimpleRESTStorage) Create(interface{}) error {
+	return storage.err
+}
+
+func (storage *SimpleRESTStorage) Update(object interface{}) error {
+	storage.updated = object.(Simple)
+	return storage.err
+}
+
+func extractBody(response *http.Response, object interface{}) (string, error) {
+	defer response.Body.Close()
+	body, err := ioutil.ReadAll(response.Body)
+	if err != nil {
+		return string(body), err
+	}
+	err = json.Unmarshal(body, object)
+	return string(body), err
+}
+
+func TestSimpleList(t *testing.T) {
+	storage := map[string]RESTStorage{}
+	simpleStorage := SimpleRESTStorage{}
+	storage["simple"] = &simpleStorage
+	handler := New(storage, "/prefix/version")
+	server := httptest.NewServer(handler)
+
+	resp, err := http.Get(server.URL + "/prefix/version/simple")
+	expectNoError(t, err)
+
+	if resp.StatusCode != 200 {
+		t.Errorf("Unexpected status: %d, Expected: %d, %#v", resp.StatusCode, 200, resp)
+	}
+}
+
+func TestErrorList(t *testing.T) {
+	storage := map[string]RESTStorage{}
+	simpleStorage := SimpleRESTStorage{
+		err: fmt.Errorf("Test Error"),
+	}
+	storage["simple"] = &simpleStorage
+	handler := New(storage, "/prefix/version")
+	server := httptest.NewServer(handler)
+
+	resp, err := http.Get(server.URL + "/prefix/version/simple")
+	expectNoError(t, err)
+
+	if resp.StatusCode != 500 {
+		t.Errorf("Unexpected status: %d, Expected: %d, %#v", resp.StatusCode, 200, resp)
+	}
+}
+
+func TestNonEmptyList(t *testing.T) {
+	storage := map[string]RESTStorage{}
+	simpleStorage := SimpleRESTStorage{
+		list: []Simple{
+			Simple{
+				Name: "foo",
+			},
+		},
+	}
+	storage["simple"] = &simpleStorage
+	handler := New(storage, "/prefix/version")
+	server := httptest.NewServer(handler)
+
+	resp, err := http.Get(server.URL + "/prefix/version/simple")
+	expectNoError(t, err)
+
+	if resp.StatusCode != 200 {
+		t.Errorf("Unexpected status: %d, Expected: %d, %#v", resp.StatusCode, 200, resp)
+	}
+
+	var listOut SimpleList
+	body, err := extractBody(resp, &listOut)
+	if len(listOut.Items) != 1 {
+		t.Errorf("Unexpected response: %#v", listOut)
+	}
+	if listOut.Items[0].Name != simpleStorage.list[0].Name {
+		t.Errorf("Unexpected data: %#v, %s", listOut.Items[0], string(body))
+	}
+}
+
+func TestGet(t *testing.T) {
+	storage := map[string]RESTStorage{}
+	simpleStorage := SimpleRESTStorage{
+		item: Simple{
+			Name: "foo",
+		},
+	}
+	storage["simple"] = &simpleStorage
+	handler := New(storage, "/prefix/version")
+	server := httptest.NewServer(handler)
+
+	resp, err := http.Get(server.URL + "/prefix/version/simple/id")
+	var itemOut Simple
+	body, err := extractBody(resp, &itemOut)
+	expectNoError(t, err)
+	if itemOut.Name != simpleStorage.item.Name {
+		t.Errorf("Unexpected data: %#v, expected %#v (%s)", itemOut, simpleStorage.item, string(body))
+	}
+}
+
+func TestDelete(t *testing.T) {
+	storage := map[string]RESTStorage{}
+	simpleStorage := SimpleRESTStorage{}
+	ID := "id"
+	storage["simple"] = &simpleStorage
+	handler := New(storage, "/prefix/version")
+	server := httptest.NewServer(handler)
+
+	client := http.Client{}
+	request, err := http.NewRequest("DELETE", server.URL+"/prefix/version/simple/"+ID, nil)
+	_, err = client.Do(request)
+	expectNoError(t, err)
+	if simpleStorage.deleted != ID {
+		t.Errorf("Unexpected delete: %s, expected %s (%s)", simpleStorage.deleted, ID)
+	}
+}
+
+func TestUpdate(t *testing.T) {
+	storage := map[string]RESTStorage{}
+	simpleStorage := SimpleRESTStorage{}
+	ID := "id"
+	storage["simple"] = &simpleStorage
+	handler := New(storage, "/prefix/version")
+	server := httptest.NewServer(handler)
+
+	item := Simple{
+		Name: "bar",
+	}
+	body, err := json.Marshal(item)
+	expectNoError(t, err)
+	client := http.Client{}
+	request, err := http.NewRequest("PUT", server.URL+"/prefix/version/simple/"+ID, bytes.NewReader(body))
+	_, err = client.Do(request)
+	expectNoError(t, err)
+	if simpleStorage.updated.Name != item.Name {
+		t.Errorf("Unexpected update value %#v, expected %#v.", simpleStorage.updated, item)
+	}
+}
+
+func TestBadPath(t *testing.T) {
+	handler := New(map[string]RESTStorage{}, "/prefix/version")
+	server := httptest.NewServer(handler)
+	client := http.Client{}
+
+	request, err := http.NewRequest("GET", server.URL+"/foobar", nil)
+	expectNoError(t, err)
+	response, err := client.Do(request)
+	expectNoError(t, err)
+	if response.StatusCode != 404 {
+		t.Errorf("Unexpected response %#v", response)
+	}
+}
+
+func TestMissingPath(t *testing.T) {
+	handler := New(map[string]RESTStorage{}, "/prefix/version")
+	server := httptest.NewServer(handler)
+	client := http.Client{}
+
+	request, err := http.NewRequest("GET", server.URL+"/prefix/version", nil)
+	expectNoError(t, err)
+	response, err := client.Do(request)
+	expectNoError(t, err)
+	if response.StatusCode != 404 {
+		t.Errorf("Unexpected response %#v", response)
+	}
+}
+
+func TestMissingStorage(t *testing.T) {
+	handler := New(map[string]RESTStorage{
+		"foo": &SimpleRESTStorage{},
+	}, "/prefix/version")
+	server := httptest.NewServer(handler)
+	client := http.Client{}
+
+	request, err := http.NewRequest("GET", server.URL+"/prefix/version/foobar", nil)
+	expectNoError(t, err)
+	response, err := client.Do(request)
+	expectNoError(t, err)
+	if response.StatusCode != 404 {
+		t.Errorf("Unexpected response %#v", response)
+	}
+}
+
+func TestCreate(t *testing.T) {
+	handler := New(map[string]RESTStorage{
+		"foo": &SimpleRESTStorage{},
+	}, "/prefix/version")
+	server := httptest.NewServer(handler)
+	client := http.Client{}
+
+	simple := Simple{Name: "foo"}
+	data, _ := json.Marshal(simple)
+	request, err := http.NewRequest("POST", server.URL+"/prefix/version/foo", bytes.NewBuffer(data))
+	expectNoError(t, err)
+	response, err := client.Do(request)
+	expectNoError(t, err)
+	if response.StatusCode != 200 {
+		t.Errorf("Unexpected response %#v", response)
+	}
+
+	var itemOut Simple
+	body, err := extractBody(response, &itemOut)
+	expectNoError(t, err)
+	if !reflect.DeepEqual(itemOut, simple) {
+		t.Errorf("Unexpected data: %#v, expected %#v (%s)", itemOut, simple, string(body))
+	}
+}

pkg/client/client.go

@@ -0,0 +1,251 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+// A client for the Kubernetes cluster management API
+// There are three fundamental objects
+//   Task - A single running container
+//   TaskForce - A set of co-scheduled Task(s)
+//   ReplicationController - A manager for replicating TaskForces
+package client
+
+import (
+	"bytes"
+	"crypto/tls"
+	"encoding/json"
+	"fmt"
+	"io"
+	"io/ioutil"
+	"log"
+	"net/http"
+	"net/url"
+	"strings"
+
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/api"
+)
+
+// ClientInterface holds the methods for clients of Kubenetes, an interface to allow mock testing
+type ClientInterface interface {
+	ListTasks(labelQuery map[string]string) (api.TaskList, error)
+	GetTask(name string) (api.Task, error)
+	DeleteTask(name string) error
+	CreateTask(api.Task) (api.Task, error)
+	UpdateTask(api.Task) (api.Task, error)
+
+	GetReplicationController(name string) (api.ReplicationController, error)
+	CreateReplicationController(api.ReplicationController) (api.ReplicationController, error)
+	UpdateReplicationController(api.ReplicationController) (api.ReplicationController, error)
+	DeleteReplicationController(string) error
+
+	GetService(name string) (api.Service, error)
+	CreateService(api.Service) (api.Service, error)
+	UpdateService(api.Service) (api.Service, error)
+	DeleteService(string) error
+}
+
+// AuthInfo is used to store authorization information
+type AuthInfo struct {
+	User     string
+	Password string
+}
+
+// Client is the actual implementation of a Kubernetes client.
+// Host is the http://... base for the URL
+type Client struct {
+	Host       string
+	Auth       *AuthInfo
+	httpClient *http.Client
+}
+
+// Underlying base implementation of performing a request.
+// method is the HTTP method (e.g. "GET")
+// path is the path on the host to hit
+// requestBody is the body of the request. Can be nil.
+// target the interface to marshal the JSON response into.  Can be nil.
+func (client Client) rawRequest(method, path string, requestBody io.Reader, target interface{}) ([]byte, error) {
+	request, err := http.NewRequest(method, client.makeURL(path), requestBody)
+	if err != nil {
+		return []byte{}, err
+	}
+	if client.Auth != nil {
+		request.SetBasicAuth(client.Auth.User, client.Auth.Password)
+	}
+	tr := &http.Transport{
+		TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
+	}
+	var httpClient *http.Client
+	if client.httpClient != nil {
+		httpClient = client.httpClient
+	} else {
+		httpClient = &http.Client{Transport: tr}
+	}
+	response, err := httpClient.Do(request)
+	if err != nil {
+		return nil, err
+	}
+	if response.StatusCode != 200 {
+		return nil, fmt.Errorf("request [%s %s] failed (%d) %s", method, client.makeURL(path), response.StatusCode, response.Status)
+	}
+	defer response.Body.Close()
+	body, err := ioutil.ReadAll(response.Body)
+	if err != nil {
+		return body, err
+	}
+	if target != nil {
+		err = json.Unmarshal(body, target)
+	}
+	if err != nil {
+		log.Printf("Failed to parse: %s\n", string(body))
+		// FIXME: no need to return err here?
+	}
+	return body, err
+}
+
+func (client Client) makeURL(path string) string {
+	return client.Host + "/api/v1beta1/" + path
+}
+
+func EncodeLabelQuery(labelQuery map[string]string) string {
+	query := make([]string, 0, len(labelQuery))
+	for key, value := range labelQuery {
+		query = append(query, key+"="+value)
+	}
+	return url.QueryEscape(strings.Join(query, ","))
+}
+
+func DecodeLabelQuery(labelQuery string) map[string]string {
+	result := map[string]string{}
+	if len(labelQuery) == 0 {
+		return result
+	}
+	parts := strings.Split(labelQuery, ",")
+	for _, part := range parts {
+		pieces := strings.Split(part, "=")
+		if len(pieces) == 2 {
+			result[pieces[0]] = pieces[1]
+		} else {
+			log.Printf("Invalid label query: %s", labelQuery)
+		}
+	}
+	return result
+}
+
+// ListTasks takes a label query, and returns the list of tasks that match that query
+func (client Client) ListTasks(labelQuery map[string]string) (api.TaskList, error) {
+	path := "tasks"
+	if labelQuery != nil && len(labelQuery) > 0 {
+		path += "?labels=" + EncodeLabelQuery(labelQuery)
+	}
+	var result api.TaskList
+	_, err := client.rawRequest("GET", path, nil, &result)
+	return result, err
+}
+
+// GetTask takes the name of the task, and returns the corresponding Task object, and an error if it occurs
+func (client Client) GetTask(name string) (api.Task, error) {
+	var result api.Task
+	_, err := client.rawRequest("GET", "tasks/"+name, nil, &result)
+	return result, err
+}
+
+// DeleteTask takes the name of the task, and returns an error if one occurs
+func (client Client) DeleteTask(name string) error {
+	_, err := client.rawRequest("DELETE", "tasks/"+name, nil, nil)
+	return err
+}
+
+// CreateTask takes the representation of a task.  Returns the server's representation of the task, and an error, if it occurs
+func (client Client) CreateTask(task api.Task) (api.Task, error) {
+	var result api.Task
+	body, err := json.Marshal(task)
+	if err == nil {
+		_, err = client.rawRequest("POST", "tasks", bytes.NewBuffer(body), &result)
+	}
+	return result, err
+}
+
+// UpdateTask takes the representation of a task to update.  Returns the server's representation of the task, and an error, if it occurs
+func (client Client) UpdateTask(task api.Task) (api.Task, error) {
+	var result api.Task
+	body, err := json.Marshal(task)
+	if err == nil {
+		_, err = client.rawRequest("PUT", "tasks/"+task.ID, bytes.NewBuffer(body), &result)
+	}
+	return result, err
+}
+
+// GetReplicationController returns information about a particular replication controller
+func (client Client) GetReplicationController(name string) (api.ReplicationController, error) {
+	var result api.ReplicationController
+	_, err := client.rawRequest("GET", "replicationControllers/"+name, nil, &result)
+	return result, err
+}
+
+// CreateReplicationController creates a new replication controller
+func (client Client) CreateReplicationController(controller api.ReplicationController) (api.ReplicationController, error) {
+	var result api.ReplicationController
+	body, err := json.Marshal(controller)
+	if err == nil {
+		_, err = client.rawRequest("POST", "replicationControllers", bytes.NewBuffer(body), &result)
+	}
+	return result, err
+}
+
+// UpdateReplicationController updates an existing replication controller
+func (client Client) UpdateReplicationController(controller api.ReplicationController) (api.ReplicationController, error) {
+	var result api.ReplicationController
+	body, err := json.Marshal(controller)
+	if err == nil {
+		_, err = client.rawRequest("PUT", "replicationControllers/"+controller.ID, bytes.NewBuffer(body), &result)
+	}
+	return result, err
+}
+
+func (client Client) DeleteReplicationController(name string) error {
+	_, err := client.rawRequest("DELETE", "replicationControllers/"+name, nil, nil)
+	return err
+}
+
+// GetReplicationController returns information about a particular replication controller
+func (client Client) GetService(name string) (api.Service, error) {
+	var result api.Service
+	_, err := client.rawRequest("GET", "services/"+name, nil, &result)
+	return result, err
+}
+
+// CreateReplicationController creates a new replication controller
+func (client Client) CreateService(svc api.Service) (api.Service, error) {
+	var result api.Service
+	body, err := json.Marshal(svc)
+	if err == nil {
+		_, err = client.rawRequest("POST", "services", bytes.NewBuffer(body), &result)
+	}
+	return result, err
+}
+
+// UpdateReplicationController updates an existing replication controller
+func (client Client) UpdateService(svc api.Service) (api.Service, error) {
+	var result api.Service
+	body, err := json.Marshal(svc)
+	if err == nil {
+		_, err = client.rawRequest("PUT", "services/"+svc.ID, bytes.NewBuffer(body), &result)
+	}
+	return result, err
+}
+
+func (client Client) DeleteService(name string) error {
+	_, err := client.rawRequest("DELETE", "services/"+name, nil, nil)
+	return err
+}

pkg/client/client_test.go

@@ -0,0 +1,391 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+package client
+
+import (
+	"encoding/json"
+	"net/http/httptest"
+	"net/url"
+	"reflect"
+	"testing"
+
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/api"
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/util"
+)
+
+// TODO: This doesn't reduce typing enough to make it worth the less readable errors. Remove.
+func expectNoError(t *testing.T, err error) {
+	if err != nil {
+		t.Errorf("Unexpected error: %#v", err)
+	}
+}
+
+// TODO: Move this to a common place, it's needed in multiple tests.
+var apiPath = "/api/v1beta1"
+
+func makeUrl(suffix string) string {
+	return apiPath + suffix
+}
+
+func TestListEmptyTasks(t *testing.T) {
+	fakeHandler := util.FakeHandler{
+		StatusCode:   200,
+		ResponseBody: `{ "items": []}`,
+	}
+	testServer := httptest.NewTLSServer(&fakeHandler)
+	client := Client{
+		Host: testServer.URL,
+	}
+	taskList, err := client.ListTasks(nil)
+	fakeHandler.ValidateRequest(t, makeUrl("/tasks"), "GET", nil)
+	if err != nil {
+		t.Errorf("Unexpected error in listing tasks: %#v", err)
+	}
+	if len(taskList.Items) != 0 {
+		t.Errorf("Unexpected items in task list: %#v", taskList)
+	}
+	testServer.Close()
+}
+
+func TestListTasks(t *testing.T) {
+	expectedTaskList := api.TaskList{
+		Items: []api.Task{
+			api.Task{
+				CurrentState: api.TaskState{
+					Status: "Foobar",
+				},
+				Labels: map[string]string{
+					"foo":  "bar",
+					"name": "baz",
+				},
+			},
+		},
+	}
+	body, _ := json.Marshal(expectedTaskList)
+	fakeHandler := util.FakeHandler{
+		StatusCode:   200,
+		ResponseBody: string(body),
+	}
+	testServer := httptest.NewTLSServer(&fakeHandler)
+	client := Client{
+		Host: testServer.URL,
+	}
+	receivedTaskList, err := client.ListTasks(nil)
+	fakeHandler.ValidateRequest(t, makeUrl("/tasks"), "GET", nil)
+	if err != nil {
+		t.Errorf("Unexpected error in listing tasks: %#v", err)
+	}
+	if !reflect.DeepEqual(expectedTaskList, receivedTaskList) {
+		t.Errorf("Unexpected task list: %#v\nvs.\n%#v", receivedTaskList, expectedTaskList)
+	}
+	testServer.Close()
+}
+
+func TestListTasksLabels(t *testing.T) {
+	expectedTaskList := api.TaskList{
+		Items: []api.Task{
+			api.Task{
+				CurrentState: api.TaskState{
+					Status: "Foobar",
+				},
+				Labels: map[string]string{
+					"foo":  "bar",
+					"name": "baz",
+				},
+			},
+		},
+	}
+	body, _ := json.Marshal(expectedTaskList)
+	fakeHandler := util.FakeHandler{
+		StatusCode:   200,
+		ResponseBody: string(body),
+	}
+	testServer := httptest.NewTLSServer(&fakeHandler)
+	client := Client{
+		Host: testServer.URL,
+	}
+	query := map[string]string{"foo": "bar", "name": "baz"}
+	receivedTaskList, err := client.ListTasks(query)
+	fakeHandler.ValidateRequest(t, makeUrl("/tasks"), "GET", nil)
+	queryString := fakeHandler.RequestReceived.URL.Query().Get("labels")
+	queryString, _ = url.QueryUnescape(queryString)
+	// TODO(bburns) : This assumes some ordering in serialization that might not always
+	// be true, parse it into a map.
+	if queryString != "foo=bar,name=baz" {
+		t.Errorf("Unexpected label query: %s", queryString)
+	}
+	if err != nil {
+		t.Errorf("Unexpected error in listing tasks: %#v", err)
+	}
+	if !reflect.DeepEqual(expectedTaskList, receivedTaskList) {
+		t.Errorf("Unexpected task list: %#v\nvs.\n%#v", receivedTaskList, expectedTaskList)
+	}
+	testServer.Close()
+}
+
+func TestGetTask(t *testing.T) {
+	expectedTask := api.Task{
+		CurrentState: api.TaskState{
+			Status: "Foobar",
+		},
+		Labels: map[string]string{
+			"foo":  "bar",
+			"name": "baz",
+		},
+	}
+	body, _ := json.Marshal(expectedTask)
+	fakeHandler := util.FakeHandler{
+		StatusCode:   200,
+		ResponseBody: string(body),
+	}
+	testServer := httptest.NewTLSServer(&fakeHandler)
+	client := Client{
+		Host: testServer.URL,
+	}
+	receivedTask, err := client.GetTask("foo")
+	fakeHandler.ValidateRequest(t, makeUrl("/tasks/foo"), "GET", nil)
+	if err != nil {
+		t.Errorf("Unexpected error: %#v", err)
+	}
+	if !reflect.DeepEqual(expectedTask, receivedTask) {
+		t.Errorf("Received task: %#v\n doesn't match expected task: %#v", receivedTask, expectedTask)
+	}
+	testServer.Close()
+}
+
+func TestDeleteTask(t *testing.T) {
+	fakeHandler := util.FakeHandler{
+		StatusCode:   200,
+		ResponseBody: `{"success": true}`,
+	}
+	testServer := httptest.NewTLSServer(&fakeHandler)
+	client := Client{
+		Host: testServer.URL,
+	}
+	err := client.DeleteTask("foo")
+	fakeHandler.ValidateRequest(t, makeUrl("/tasks/foo"), "DELETE", nil)
+	if err != nil {
+		t.Errorf("Unexpected error: %#v", err)
+	}
+	testServer.Close()
+}
+
+func TestCreateTask(t *testing.T) {
+	requestTask := api.Task{
+		CurrentState: api.TaskState{
+			Status: "Foobar",
+		},
+		Labels: map[string]string{
+			"foo":  "bar",
+			"name": "baz",
+		},
+	}
+	body, _ := json.Marshal(requestTask)
+	fakeHandler := util.FakeHandler{
+		StatusCode:   200,
+		ResponseBody: string(body),
+	}
+	testServer := httptest.NewTLSServer(&fakeHandler)
+	client := Client{
+		Host: testServer.URL,
+	}
+	receivedTask, err := client.CreateTask(requestTask)
+	fakeHandler.ValidateRequest(t, makeUrl("/tasks"), "POST", nil)
+	if err != nil {
+		t.Errorf("Unexpected error: %#v", err)
+	}
+	if !reflect.DeepEqual(requestTask, receivedTask) {
+		t.Errorf("Received task: %#v\n doesn't match expected task: %#v", receivedTask, requestTask)
+	}
+	testServer.Close()
+}
+
+func TestUpdateTask(t *testing.T) {
+	requestTask := api.Task{
+		JSONBase: api.JSONBase{ID: "foo"},
+		CurrentState: api.TaskState{
+			Status: "Foobar",
+		},
+		Labels: map[string]string{
+			"foo":  "bar",
+			"name": "baz",
+		},
+	}
+	body, _ := json.Marshal(requestTask)
+	fakeHandler := util.FakeHandler{
+		StatusCode:   200,
+		ResponseBody: string(body),
+	}
+	testServer := httptest.NewTLSServer(&fakeHandler)
+	client := Client{
+		Host: testServer.URL,
+	}
+	receivedTask, err := client.UpdateTask(requestTask)
+	fakeHandler.ValidateRequest(t, makeUrl("/tasks/foo"), "PUT", nil)
+	if err != nil {
+		t.Errorf("Unexpected error: %#v", err)
+	}
+	expectEqual(t, requestTask, receivedTask)
+	testServer.Close()
+}
+
+func expectEqual(t *testing.T, expected, observed interface{}) {
+	if !reflect.DeepEqual(expected, observed) {
+		t.Errorf("Unexpected inequality.  Expected: %#v Observed: %#v", expected, observed)
+	}
+}
+
+func TestEncodeDecodeLabelQuery(t *testing.T) {
+	queryIn := map[string]string{
+		"foo": "bar",
+		"baz": "blah",
+	}
+	queryString, _ := url.QueryUnescape(EncodeLabelQuery(queryIn))
+	queryOut := DecodeLabelQuery(queryString)
+	expectEqual(t, queryIn, queryOut)
+}
+
+func TestDecodeEmpty(t *testing.T) {
+	query := DecodeLabelQuery("")
+	if len(query) != 0 {
+		t.Errorf("Unexpected query: %#v", query)
+	}
+}
+
+func TestDecodeBad(t *testing.T) {
+	query := DecodeLabelQuery("foo")
+	if len(query) != 0 {
+		t.Errorf("Unexpected query: %#v", query)
+	}
+}
+
+func TestGetController(t *testing.T) {
+	expectedController := api.ReplicationController{
+		JSONBase: api.JSONBase{
+			ID: "foo",
+		},
+		DesiredState: api.ReplicationControllerState{
+			Replicas: 2,
+		},
+		Labels: map[string]string{
+			"foo":  "bar",
+			"name": "baz",
+		},
+	}
+	body, _ := json.Marshal(expectedController)
+	fakeHandler := util.FakeHandler{
+		StatusCode:   200,
+		ResponseBody: string(body),
+	}
+	testServer := httptest.NewTLSServer(&fakeHandler)
+	client := Client{
+		Host: testServer.URL,
+	}
+	receivedController, err := client.GetReplicationController("foo")
+	expectNoError(t, err)
+	if !reflect.DeepEqual(expectedController, receivedController) {
+		t.Errorf("Unexpected controller, expected: %#v, received %#v", expectedController, receivedController)
+	}
+	fakeHandler.ValidateRequest(t, makeUrl("/replicationControllers/foo"), "GET", nil)
+	testServer.Close()
+}
+
+func TestUpdateController(t *testing.T) {
+	expectedController := api.ReplicationController{
+		JSONBase: api.JSONBase{
+			ID: "foo",
+		},
+		DesiredState: api.ReplicationControllerState{
+			Replicas: 2,
+		},
+		Labels: map[string]string{
+			"foo":  "bar",
+			"name": "baz",
+		},
+	}
+	body, _ := json.Marshal(expectedController)
+	fakeHandler := util.FakeHandler{
+		StatusCode:   200,
+		ResponseBody: string(body),
+	}
+	testServer := httptest.NewTLSServer(&fakeHandler)
+	client := Client{
+		Host: testServer.URL,
+	}
+	receivedController, err := client.UpdateReplicationController(api.ReplicationController{
+		JSONBase: api.JSONBase{
+			ID: "foo",
+		},
+	})
+	expectNoError(t, err)
+	if !reflect.DeepEqual(expectedController, receivedController) {
+		t.Errorf("Unexpected controller, expected: %#v, received %#v", expectedController, receivedController)
+	}
+	fakeHandler.ValidateRequest(t, makeUrl("/replicationControllers/foo"), "PUT", nil)
+	testServer.Close()
+}
+
+func TestDeleteController(t *testing.T) {
+	fakeHandler := util.FakeHandler{
+		StatusCode:   200,
+		ResponseBody: `{"success": true}`,
+	}
+	testServer := httptest.NewTLSServer(&fakeHandler)
+	client := Client{
+		Host: testServer.URL,
+	}
+	err := client.DeleteReplicationController("foo")
+	fakeHandler.ValidateRequest(t, makeUrl("/replicationControllers/foo"), "DELETE", nil)
+	if err != nil {
+		t.Errorf("Unexpected error: %#v", err)
+	}
+	testServer.Close()
+}
+
+func TestCreateController(t *testing.T) {
+	expectedController := api.ReplicationController{
+		JSONBase: api.JSONBase{
+			ID: "foo",
+		},
+		DesiredState: api.ReplicationControllerState{
+			Replicas: 2,
+		},
+		Labels: map[string]string{
+			"foo":  "bar",
+			"name": "baz",
+		},
+	}
+	body, _ := json.Marshal(expectedController)
+	fakeHandler := util.FakeHandler{
+		StatusCode:   200,
+		ResponseBody: string(body),
+	}
+	testServer := httptest.NewTLSServer(&fakeHandler)
+	client := Client{
+		Host: testServer.URL,
+	}
+	receivedController, err := client.CreateReplicationController(api.ReplicationController{
+		JSONBase: api.JSONBase{
+			ID: "foo",
+		},
+	})
+	expectNoError(t, err)
+	if !reflect.DeepEqual(expectedController, receivedController) {
+		t.Errorf("Unexpected controller, expected: %#v, received %#v", expectedController, receivedController)
+	}
+	fakeHandler.ValidateRequest(t, makeUrl("/replicationControllers"), "POST", nil)
+	testServer.Close()
+}

pkg/client/container_info.go

@@ -0,0 +1,61 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+package client
+
+import (
+	"encoding/json"
+	"fmt"
+	"io/ioutil"
+	"net/http"
+)
+
+type ContainerInfo interface {
+	GetContainerInfo(host, name string) (interface{}, error)
+}
+
+type HTTPContainerInfo struct {
+	Client *http.Client
+	Port   uint
+}
+
+func (c *HTTPContainerInfo) GetContainerInfo(host, name string) (interface{}, error) {
+	request, err := http.NewRequest("GET", fmt.Sprintf("http://%s:%d/containerInfo?container=%s", host, c.Port, name), nil)
+	if err != nil {
+		return nil, err
+	}
+	response, err := c.Client.Do(request)
+	if err != nil {
+		return nil, err
+	}
+	defer response.Body.Close()
+	body, err := ioutil.ReadAll(response.Body)
+	if err != nil {
+		return nil, err
+	}
+	var data interface{}
+	err = json.Unmarshal(body, &data)
+	return data, err
+}
+
+// Useful for testing.
+type FakeContainerInfo struct {
+	data interface{}
+	err  error
+}
+
+func (c *FakeContainerInfo) GetContainerInfo(host, name string) (interface{}, error) {
+	return c.data, c.err
+}

pkg/client/container_info_test.go

@@ -0,0 +1,54 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+package client
+
+import (
+	"encoding/json"
+	"net/http"
+	"net/http/httptest"
+	"net/url"
+	"strconv"
+	"strings"
+	"testing"
+
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/util"
+)
+
+func TestHTTPContainerInfo(t *testing.T) {
+	body := `{"items":[]}`
+	fakeHandler := util.FakeHandler{
+		StatusCode:   200,
+		ResponseBody: body,
+	}
+	testServer := httptest.NewServer(&fakeHandler)
+
+	hostUrl, err := url.Parse(testServer.URL)
+	expectNoError(t, err)
+	parts := strings.Split(hostUrl.Host, ":")
+
+	port, err := strconv.Atoi(parts[1])
+	expectNoError(t, err)
+	containerInfo := &HTTPContainerInfo{
+		Client: http.DefaultClient,
+		Port:   uint(port),
+	}
+	data, err := containerInfo.GetContainerInfo(parts[0], "foo")
+	expectNoError(t, err)
+	dataString, _ := json.Marshal(data)
+	if string(dataString) != body {
+		t.Errorf("Unexpected response.  Expected: %s, received %s", body, string(dataString))
+	}
+}

pkg/cloudcfg/cloudcfg.go

@@ -0,0 +1,254 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+// Package cloudcfg is ...
+package cloudcfg
+
+import (
+	"bytes"
+	"crypto/tls"
+	"encoding/json"
+	"fmt"
+	"io/ioutil"
+	"log"
+	"net/http"
+	"os"
+	"strconv"
+	"strings"
+	"time"
+
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/api"
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/client"
+	"gopkg.in/v1/yaml"
+)
+
+func promptForString(field string) string {
+	fmt.Printf("Please enter %s: ", field)
+	var result string
+	fmt.Scan(&result)
+	return result
+}
+
+// Parse an AuthInfo object from a file path
+func LoadAuthInfo(path string) (client.AuthInfo, error) {
+	var auth client.AuthInfo
+	if _, err := os.Stat(path); os.IsNotExist(err) {
+		auth.User = promptForString("Username")
+		auth.Password = promptForString("Password")
+		data, err := json.Marshal(auth)
+		if err != nil {
+			return auth, err
+		}
+		err = ioutil.WriteFile(path, data, 0600)
+		return auth, err
+	}
+	data, err := ioutil.ReadFile(path)
+	if err != nil {
+		return auth, err
+	}
+	err = json.Unmarshal(data, &auth)
+	return auth, err
+}
+
+// Perform a rolling update of a collection of tasks.
+// 'name' points to a replication controller.
+// 'client' is used for updating tasks.
+// 'updatePeriod' is the time between task updates.
+func Update(name string, client client.ClientInterface, updatePeriod time.Duration) error {
+	controller, err := client.GetReplicationController(name)
+	if err != nil {
+		return err
+	}
+	labels := controller.DesiredState.ReplicasInSet
+
+	taskList, err := client.ListTasks(labels)
+	if err != nil {
+		return err
+	}
+	for _, task := range taskList.Items {
+		_, err = client.UpdateTask(task)
+		if err != nil {
+			return err
+		}
+		time.Sleep(updatePeriod)
+	}
+	return nil
+}
+
+// RequestWithBody is a helper method that creates an HTTP request with the specified url, method
+// and a body read from 'configFile'
+// FIXME: need to be public API?
+func RequestWithBody(configFile, url, method string) (*http.Request, error) {
+	if len(configFile) == 0 {
+		return nil, fmt.Errorf("empty config file.")
+	}
+	data, err := ioutil.ReadFile(configFile)
+	if err != nil {
+		return nil, err
+	}
+	return RequestWithBodyData(data, url, method)
+}
+
+// RequestWithBodyData is a helper method that creates an HTTP request with the specified url, method
+// and body data
+// FIXME: need to be public API?
+func RequestWithBodyData(data []byte, url, method string) (*http.Request, error) {
+	request, err := http.NewRequest(method, url, bytes.NewBuffer(data))
+	request.ContentLength = int64(len(data))
+	return request, err
+}
+
+// Execute a request, adds authentication, and HTTPS cert ignoring.
+// TODO: Make this stuff optional
+// FIXME: need to be public API?
+func DoRequest(request *http.Request, user, password string) (string, error) {
+	request.SetBasicAuth(user, password)
+	tr := &http.Transport{
+		TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
+	}
+	client := &http.Client{Transport: tr}
+	response, err := client.Do(request)
+	if err != nil {
+		return "", err
+	}
+	defer response.Body.Close()
+	body, err := ioutil.ReadAll(response.Body)
+	return string(body), err
+}
+
+// StopController stops a controller named 'name' by setting replicas to zero
+func StopController(name string, client client.ClientInterface) error {
+	controller, err := client.GetReplicationController(name)
+	if err != nil {
+		return err
+	}
+	controller.DesiredState.Replicas = 0
+	controllerOut, err := client.UpdateReplicationController(controller)
+	if err != nil {
+		return err
+	}
+	data, err := yaml.Marshal(controllerOut)
+	if err != nil {
+		return err
+	}
+	fmt.Print(string(data))
+	return nil
+}
+
+func makePorts(spec string) []api.Port {
+	parts := strings.Split(spec, ",")
+	var result []api.Port
+	for _, part := range parts {
+		pieces := strings.Split(part, ":")
+		if len(pieces) != 2 {
+			log.Printf("Bad port spec: %s", part)
+			continue
+		}
+		host, err := strconv.Atoi(pieces[0])
+		if err != nil {
+			log.Printf("Host part is not integer: %s %v", pieces[0], err)
+			continue
+		}
+		container, err := strconv.Atoi(pieces[1])
+		if err != nil {
+			log.Printf("Container part is not integer: %s %v", pieces[1], err)
+			continue
+		}
+		result = append(result, api.Port{ContainerPort: container, HostPort: host})
+	}
+	return result
+}
+
+// RunController creates a new replication controller named 'name' which creates 'replicas' tasks running 'image'
+func RunController(image, name string, replicas int, client client.ClientInterface, portSpec string, servicePort int) error {
+	controller := api.ReplicationController{
+		JSONBase: api.JSONBase{
+			ID: name,
+		},
+		DesiredState: api.ReplicationControllerState{
+			Replicas: replicas,
+			ReplicasInSet: map[string]string{
+				"name": name,
+			},
+			TaskTemplate: api.TaskTemplate{
+				DesiredState: api.TaskState{
+					Manifest: api.ContainerManifest{
+						Containers: []api.Container{
+							api.Container{
+								Image: image,
+								Ports: makePorts(portSpec),
+							},
+						},
+					},
+				},
+				Labels: map[string]string{
+					"name": name,
+				},
+			},
+		},
+		Labels: map[string]string{
+			"name": name,
+		},
+	}
+
+	controllerOut, err := client.CreateReplicationController(controller)
+	if err != nil {
+		return err
+	}
+	data, err := yaml.Marshal(controllerOut)
+	if err != nil {
+		return err
+	}
+	fmt.Print(string(data))
+
+	if servicePort > 0 {
+		svc, err := createService(name, servicePort, client)
+		if err != nil {
+			return err
+		}
+		data, err = yaml.Marshal(svc)
+		if err != nil {
+			return err
+		}
+		fmt.Printf(string(data))
+	}
+	return nil
+}
+
+func createService(name string, port int, client client.ClientInterface) (api.Service, error) {
+	svc := api.Service{
+		JSONBase: api.JSONBase{ID: name},
+		Port:     port,
+		Labels: map[string]string{
+			"name": name,
+		},
+	}
+	svc, err := client.CreateService(svc)
+	return svc, err
+}
+
+// DeleteController deletes a replication controller named 'name', requires that the controller
+// already be stopped
+func DeleteController(name string, client client.ClientInterface) error {
+	controller, err := client.GetReplicationController(name)
+	if err != nil {
+		return err
+	}
+	if controller.DesiredState.Replicas != 0 {
+		return fmt.Errorf("controller has non-zero replicas (%d)", controller.DesiredState.Replicas)
+	}
+	return client.DeleteReplicationController(name)
+}

pkg/cloudcfg/cloudcfg_test.go

@@ -0,0 +1,308 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+package cloudcfg
+
+import (
+	"encoding/json"
+	"io/ioutil"
+	"net/http"
+	"net/http/httptest"
+	"testing"
+
+	. "github.com/GoogleCloudPlatform/kubernetes/pkg/api"
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/util"
+)
+
+// TODO: This doesn't reduce typing enough to make it worth the less readable errors. Remove.
+func expectNoError(t *testing.T, err error) {
+	if err != nil {
+		t.Errorf("Unexpected error: %#v", err)
+	}
+}
+
+type Action struct {
+	action string
+	value  interface{}
+}
+
+type FakeKubeClient struct {
+	actions []Action
+	tasks   TaskList
+	ctrl    ReplicationController
+}
+
+func (client *FakeKubeClient) ListTasks(labelQuery map[string]string) (TaskList, error) {
+	client.actions = append(client.actions, Action{action: "list-tasks"})
+	return client.tasks, nil
+}
+
+func (client *FakeKubeClient) GetTask(name string) (Task, error) {
+	client.actions = append(client.actions, Action{action: "get-task", value: name})
+	return Task{}, nil
+}
+
+func (client *FakeKubeClient) DeleteTask(name string) error {
+	client.actions = append(client.actions, Action{action: "delete-task", value: name})
+	return nil
+}
+
+func (client *FakeKubeClient) CreateTask(task Task) (Task, error) {
+	client.actions = append(client.actions, Action{action: "create-task"})
+	return Task{}, nil
+}
+
+func (client *FakeKubeClient) UpdateTask(task Task) (Task, error) {
+	client.actions = append(client.actions, Action{action: "update-task", value: task.ID})
+	return Task{}, nil
+}
+
+func (client *FakeKubeClient) GetReplicationController(name string) (ReplicationController, error) {
+	client.actions = append(client.actions, Action{action: "get-controller", value: name})
+	return client.ctrl, nil
+}
+
+func (client *FakeKubeClient) CreateReplicationController(controller ReplicationController) (ReplicationController, error) {
+	client.actions = append(client.actions, Action{action: "create-controller", value: controller})
+	return ReplicationController{}, nil
+}
+
+func (client *FakeKubeClient) UpdateReplicationController(controller ReplicationController) (ReplicationController, error) {
+	client.actions = append(client.actions, Action{action: "update-controller", value: controller})
+	return ReplicationController{}, nil
+}
+
+func (client *FakeKubeClient) DeleteReplicationController(controller string) error {
+	client.actions = append(client.actions, Action{action: "delete-controller", value: controller})
+	return nil
+}
+
+func (client *FakeKubeClient) GetService(name string) (Service, error) {
+	client.actions = append(client.actions, Action{action: "get-controller", value: name})
+	return Service{}, nil
+}
+
+func (client *FakeKubeClient) CreateService(controller Service) (Service, error) {
+	client.actions = append(client.actions, Action{action: "create-service", value: controller})
+	return Service{}, nil
+}
+
+func (client *FakeKubeClient) UpdateService(controller Service) (Service, error) {
+	client.actions = append(client.actions, Action{action: "update-service", value: controller})
+	return Service{}, nil
+}
+
+func (client *FakeKubeClient) DeleteService(controller string) error {
+	client.actions = append(client.actions, Action{action: "delete-service", value: controller})
+	return nil
+}
+
+func validateAction(expectedAction, actualAction Action, t *testing.T) {
+	if expectedAction != actualAction {
+		t.Errorf("Unexpected action: %#v, expected: %#v", actualAction, expectedAction)
+	}
+}
+
+func TestUpdateWithTasks(t *testing.T) {
+	client := FakeKubeClient{
+		tasks: TaskList{
+			Items: []Task{
+				Task{JSONBase: JSONBase{ID: "task-1"}},
+				Task{JSONBase: JSONBase{ID: "task-2"}},
+			},
+		},
+	}
+	Update("foo", &client, 0)
+	if len(client.actions) != 4 {
+		t.Errorf("Unexpected action list %#v", client.actions)
+	}
+	validateAction(Action{action: "get-controller", value: "foo"}, client.actions[0], t)
+	validateAction(Action{action: "list-tasks"}, client.actions[1], t)
+	validateAction(Action{action: "update-task", value: "task-1"}, client.actions[2], t)
+	validateAction(Action{action: "update-task", value: "task-2"}, client.actions[3], t)
+}
+
+func TestUpdateNoTasks(t *testing.T) {
+	client := FakeKubeClient{}
+	Update("foo", &client, 0)
+	if len(client.actions) != 2 {
+		t.Errorf("Unexpected action list %#v", client.actions)
+	}
+	validateAction(Action{action: "get-controller", value: "foo"}, client.actions[0], t)
+	validateAction(Action{action: "list-tasks"}, client.actions[1], t)
+}
+
+func TestDoRequest(t *testing.T) {
+	expectedBody := `{ "items": []}`
+	fakeHandler := util.FakeHandler{
+		StatusCode:   200,
+		ResponseBody: expectedBody,
+	}
+	testServer := httptest.NewTLSServer(&fakeHandler)
+	request, _ := http.NewRequest("GET", testServer.URL+"/foo/bar", nil)
+	body, err := DoRequest(request, "user", "pass")
+	if request.Header["Authorization"] == nil {
+		t.Errorf("Request is missing authorization header: %#v", *request)
+	}
+	if err != nil {
+		t.Error("Unexpected error")
+	}
+	if body != expectedBody {
+		t.Errorf("Expected body: '%s', saw: '%s'", expectedBody, body)
+	}
+	fakeHandler.ValidateRequest(t, "/foo/bar", "GET", &fakeHandler.ResponseBody)
+}
+
+func TestRunController(t *testing.T) {
+	fakeClient := FakeKubeClient{}
+	name := "name"
+	image := "foo/bar"
+	replicas := 3
+	RunController(image, name, replicas, &fakeClient, "8080:80", -1)
+	if len(fakeClient.actions) != 1 || fakeClient.actions[0].action != "create-controller" {
+		t.Errorf("Unexpected actions: %#v", fakeClient.actions)
+	}
+	controller := fakeClient.actions[0].value.(ReplicationController)
+	if controller.ID != name ||
+		controller.DesiredState.Replicas != replicas ||
+		controller.DesiredState.TaskTemplate.DesiredState.Manifest.Containers[0].Image != image {
+		t.Errorf("Unexpected controller: %#v", controller)
+	}
+}
+
+func TestRunControllerWithService(t *testing.T) {
+	fakeClient := FakeKubeClient{}
+	name := "name"
+	image := "foo/bar"
+	replicas := 3
+	RunController(image, name, replicas, &fakeClient, "", 8000)
+	if len(fakeClient.actions) != 2 ||
+		fakeClient.actions[0].action != "create-controller" ||
+		fakeClient.actions[1].action != "create-service" {
+		t.Errorf("Unexpected actions: %#v", fakeClient.actions)
+	}
+	controller := fakeClient.actions[0].value.(ReplicationController)
+	if controller.ID != name ||
+		controller.DesiredState.Replicas != replicas ||
+		controller.DesiredState.TaskTemplate.DesiredState.Manifest.Containers[0].Image != image {
+		t.Errorf("Unexpected controller: %#v", controller)
+	}
+}
+
+func TestStopController(t *testing.T) {
+	fakeClient := FakeKubeClient{}
+	name := "name"
+	StopController(name, &fakeClient)
+	if len(fakeClient.actions) != 2 {
+		t.Errorf("Unexpected actions: %#v", fakeClient.actions)
+	}
+	if fakeClient.actions[0].action != "get-controller" ||
+		fakeClient.actions[0].value.(string) != name {
+		t.Errorf("Unexpected action: %#v", fakeClient.actions[0])
+	}
+	controller := fakeClient.actions[1].value.(ReplicationController)
+	if fakeClient.actions[1].action != "update-controller" ||
+		controller.DesiredState.Replicas != 0 {
+		t.Errorf("Unexpected action: %#v", fakeClient.actions[1])
+	}
+}
+
+func TestCloudCfgDeleteController(t *testing.T) {
+	fakeClient := FakeKubeClient{}
+	name := "name"
+	err := DeleteController(name, &fakeClient)
+	expectNoError(t, err)
+	if len(fakeClient.actions) != 2 {
+		t.Errorf("Unexpected actions: %#v", fakeClient.actions)
+	}
+	if fakeClient.actions[0].action != "get-controller" ||
+		fakeClient.actions[0].value.(string) != name {
+		t.Errorf("Unexpected action: %#v", fakeClient.actions[0])
+	}
+	if fakeClient.actions[1].action != "delete-controller" ||
+		fakeClient.actions[1].value.(string) != name {
+		t.Errorf("Unexpected action: %#v", fakeClient.actions[1])
+	}
+}
+
+func TestCloudCfgDeleteControllerWithReplicas(t *testing.T) {
+	fakeClient := FakeKubeClient{
+		ctrl: ReplicationController{
+			DesiredState: ReplicationControllerState{
+				Replicas: 2,
+			},
+		},
+	}
+	name := "name"
+	err := DeleteController(name, &fakeClient)
+	if len(fakeClient.actions) != 1 {
+		t.Errorf("Unexpected actions: %#v", fakeClient.actions)
+	}
+	if fakeClient.actions[0].action != "get-controller" ||
+		fakeClient.actions[0].value.(string) != name {
+		t.Errorf("Unexpected action: %#v", fakeClient.actions[0])
+	}
+	if err == nil {
+		t.Errorf("Unexpected non-error.")
+	}
+}
+
+func TestRequestWithBodyNoSuchFile(t *testing.T) {
+	request, err := RequestWithBody("non/existent/file.json", "http://www.google.com", "GET")
+	if request != nil {
+		t.Error("Unexpected non-nil result")
+	}
+	if err == nil {
+		t.Error("Unexpected non-error")
+	}
+}
+
+func TestRequestWithBody(t *testing.T) {
+	file, err := ioutil.TempFile("", "foo")
+	expectNoError(t, err)
+	data, err := json.Marshal(Task{JSONBase: JSONBase{ID: "foo"}})
+	expectNoError(t, err)
+	_, err = file.Write(data)
+	expectNoError(t, err)
+	request, err := RequestWithBody(file.Name(), "http://www.google.com", "GET")
+	if request == nil {
+		t.Error("Unexpected nil result")
+	}
+	if err != nil {
+		t.Errorf("Unexpected error: %#v")
+	}
+	dataOut, err := ioutil.ReadAll(request.Body)
+	expectNoError(t, err)
+	if string(data) != string(dataOut) {
+		t.Errorf("Mismatched data. Expected %s, got %s", data, dataOut)
+	}
+}
+
+func validatePort(t *testing.T, p Port, external int, internal int) {
+	if p.HostPort != external || p.ContainerPort != internal {
+		t.Errorf("Unexpected port: %#v != (%d, %d)", p, external, internal)
+	}
+}
+
+func TestMakePorts(t *testing.T) {
+	ports := makePorts("8080:80,8081:8081,443:444")
+	if len(ports) != 3 {
+		t.Errorf("Unexpected ports: %#v", ports)
+	}
+
+	validatePort(t, ports[0], 8080, 80)
+	validatePort(t, ports[1], 8081, 8081)
+	validatePort(t, ports[2], 443, 444)
+}

pkg/kubelet/kubelet.go

@@ -0,0 +1,598 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+// Package kubelet is ...
+package kubelet
+
+import (
+	"bytes"
+	"encoding/json"
+	"fmt"
+	"io/ioutil"
+	"log"
+	"math/rand"
+	"net/http"
+	"os/exec"
+	"strconv"
+	"strings"
+	"sync"
+	"time"
+
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/api"
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/registry"
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/util"
+	"github.com/coreos/go-etcd/etcd"
+	"github.com/fsouza/go-dockerclient"
+	"gopkg.in/v1/yaml"
+)
+
+// State, sub object of the Docker JSON data
+type State struct {
+	Running bool
+}
+
+// The structured representation of the JSON object returned by Docker inspect
+type DockerContainerData struct {
+	state State
+}
+
+// Interface for testability
+type DockerInterface interface {
+	ListContainers(options docker.ListContainersOptions) ([]docker.APIContainers, error)
+	InspectContainer(id string) (*docker.Container, error)
+	CreateContainer(docker.CreateContainerOptions) (*docker.Container, error)
+	StartContainer(id string, hostConfig *docker.HostConfig) error
+	StopContainer(id string, timeout uint) error
+}
+
+// The main kubelet implementation
+type Kubelet struct {
+	Client             registry.EtcdClient
+	DockerClient       DockerInterface
+	FileCheckFrequency time.Duration
+	SyncFrequency      time.Duration
+	HTTPCheckFrequency time.Duration
+	pullLock           sync.Mutex
+}
+
+// Starts background goroutines. If file, manifest_url, or address are empty,
+// they are not watched. Never returns.
+func (sl *Kubelet) RunKubelet(file, manifest_url, etcd_servers, address string, port uint) {
+	fileChannel := make(chan api.ContainerManifest)
+	etcdChannel := make(chan []api.ContainerManifest)
+	httpChannel := make(chan api.ContainerManifest)
+	serverChannel := make(chan api.ContainerManifest)
+
+	go util.Forever(func() { sl.WatchFile(file, fileChannel) }, 20*time.Second)
+	if manifest_url != "" {
+		go util.Forever(func() { sl.WatchHTTP(manifest_url, httpChannel) }, 20*time.Second)
+	}
+	if etcd_servers != "" {
+		servers := []string{etcd_servers}
+		log.Printf("Creating etcd client pointing to %v", servers)
+		sl.Client = etcd.NewClient(servers)
+		go util.Forever(func() { sl.SyncAndSetupEtcdWatch(etcdChannel) }, 20*time.Second)
+	}
+	if address != "" {
+		log.Printf("Starting to listen on %s:%d", address, port)
+		handler := KubeletServer{
+			Kubelet:       sl,
+			UpdateChannel: serverChannel,
+		}
+		s := &http.Server{
+			// TODO: This is broken if address is an ipv6 address.
+			Addr:           fmt.Sprintf("%s:%d", address, port),
+			Handler:        &handler,
+			ReadTimeout:    10 * time.Second,
+			WriteTimeout:   10 * time.Second,
+			MaxHeaderBytes: 1 << 20,
+		}
+		go util.Forever(func() { s.ListenAndServe() }, 0)
+	}
+	sl.RunSyncLoop(etcdChannel, fileChannel, serverChannel, httpChannel, sl)
+}
+
+// Interface implemented by Kubelet, for testability
+type SyncHandler interface {
+	SyncManifests([]api.ContainerManifest) error
+}
+
+// Log an event to the etcd backend.
+func (sl *Kubelet) LogEvent(event *api.Event) error {
+	if sl.Client == nil {
+		return fmt.Errorf("no etcd client connection.")
+	}
+	event.Timestamp = time.Now().Unix()
+	data, err := json.Marshal(event)
+	if err != nil {
+		return err
+	}
+
+	var response *etcd.Response
+	response, err = sl.Client.AddChild(fmt.Sprintf("/events/%s", event.Container.Name), string(data), 60*60*48 /* 2 days */)
+	// TODO(bburns) : examine response here.
+	if err != nil {
+		log.Printf("Error writing event: %s\n", err)
+		if response != nil {
+			log.Printf("Response was: %#v\n", *response)
+		}
+	}
+	return err
+}
+
+// Does this container exist on this host? Returns true if so, and the name under which the container is running.
+// Returns an error if one occurs.
+func (sl *Kubelet) ContainerExists(manifest *api.ContainerManifest, container *api.Container) (exists bool, foundName string, err error) {
+	containers, err := sl.ListContainers()
+	if err != nil {
+		return false, "", err
+	}
+	for _, name := range containers {
+		manifestId, containerName := dockerNameToManifestAndContainer(name)
+		if manifestId == manifest.Id && containerName == container.Name {
+			// TODO(bburns) : This leads to an extra list.  Convert this to use the returned ID and a straight call
+			// to inspect
+			data, err := sl.GetContainerByName(name)
+			return data != nil, name, err
+		}
+	}
+	return false, "", nil
+}
+
+func (sl *Kubelet) GetContainerID(name string) (string, error) {
+	containerList, err := sl.DockerClient.ListContainers(docker.ListContainersOptions{})
+	if err != nil {
+		return "", err
+	}
+	for _, value := range containerList {
+		if strings.Contains(value.Names[0], name) {
+			return value.ID, nil
+		}
+	}
+	return "", fmt.Errorf("couldn't find name: %s", name)
+}
+
+// Get a container by name.
+// returns the container data from Docker, or an error if one exists.
+func (sl *Kubelet) GetContainerByName(name string) (*docker.Container, error) {
+	id, err := sl.GetContainerID(name)
+	if err != nil {
+		return nil, err
+	}
+	return sl.DockerClient.InspectContainer(id)
+}
+
+func (sl *Kubelet) ListContainers() ([]string, error) {
+	result := []string{}
+	containerList, err := sl.DockerClient.ListContainers(docker.ListContainersOptions{})
+	if err != nil {
+		return result, err
+	}
+	for _, value := range containerList {
+		result = append(result, value.Names[0])
+	}
+	return result, err
+}
+
+func (sl *Kubelet) pullImage(image string) error {
+	sl.pullLock.Lock()
+	defer sl.pullLock.Unlock()
+	cmd := exec.Command("docker", "pull", image)
+	err := cmd.Start()
+	if err != nil {
+		return err
+	}
+	return cmd.Wait()
+}
+
+// Converts "-" to "_-_" and "_" to "___" so that we can use "--" to meaningfully separate parts of a docker name.
+func escapeDash(in string) (out string) {
+	out = strings.Replace(in, "_", "___", -1)
+	out = strings.Replace(out, "-", "_-_", -1)
+	return
+}
+
+// Reverses the transformation of escapeDash.
+func unescapeDash(in string) (out string) {
+	out = strings.Replace(in, "_-_", "-", -1)
+	out = strings.Replace(out, "___", "_", -1)
+	return
+}
+
+// Creates a name which can be reversed to identify both manifest id and container name.
+func manifestAndContainerToDockerName(manifest *api.ContainerManifest, container *api.Container) string {
+	// Note, manifest.Id could be blank.
+	return fmt.Sprintf("%s--%s--%x", escapeDash(container.Name), escapeDash(manifest.Id), rand.Uint32())
+}
+
+// Upacks a container name, returning the manifest id and container name we would have used to
+// construct the docker name. If the docker name isn't one we created, we may return empty strings.
+func dockerNameToManifestAndContainer(name string) (manifestId, containerName string) {
+	// For some reason docker appears to be appending '/' to names.
+	// If its there, strip it.
+	if name[0] == '/' {
+		name = name[1:]
+	}
+	parts := strings.Split(name, "--")
+	if len(parts) > 0 {
+		containerName = unescapeDash(parts[0])
+	}
+	if len(parts) > 1 {
+		manifestId = unescapeDash(parts[1])
+	}
+	return
+}
+
+func (sl *Kubelet) RunContainer(manifest *api.ContainerManifest, container *api.Container) (name string, err error) {
+	err = sl.pullImage(container.Image)
+	if err != nil {
+		return "", err
+	}
+
+	name = manifestAndContainerToDockerName(manifest, container)
+	envVariables := []string{}
+	for _, value := range container.Env {
+		envVariables = append(envVariables, fmt.Sprintf("%s=%s", value.Name, value.Value))
+	}
+
+	volumes := map[string]struct{}{}
+	binds := []string{}
+	for _, volume := range container.VolumeMounts {
+		volumes[volume.MountPath] = struct{}{}
+		basePath := "/exports/" + volume.Name + ":" + volume.MountPath
+		if volume.ReadOnly {
+			basePath += ":ro"
+		}
+		binds = append(binds, basePath)
+	}
+
+	exposedPorts := map[docker.Port]struct{}{}
+	portBindings := map[docker.Port][]docker.PortBinding{}
+	for _, port := range container.Ports {
+		interiorPort := port.ContainerPort
+		exteriorPort := port.HostPort
+		// Some of this port stuff is under-documented voodoo.
+		// See http://stackoverflow.com/questions/20428302/binding-a-port-to-a-host-interface-using-the-rest-api
+		dockerPort := docker.Port(strconv.Itoa(interiorPort) + "/tcp")
+		exposedPorts[dockerPort] = struct{}{}
+		portBindings[dockerPort] = []docker.PortBinding{
+			docker.PortBinding{
+				HostPort: strconv.Itoa(exteriorPort),
+			},
+		}
+	}
+	var cmdList []string
+	if len(container.Command) > 0 {
+		cmdList = strings.Split(container.Command, " ")
+	}
+	opts := docker.CreateContainerOptions{
+		Name: name,
+		Config: &docker.Config{
+			Image:        container.Image,
+			ExposedPorts: exposedPorts,
+			Env:          envVariables,
+			Volumes:      volumes,
+			WorkingDir:   container.WorkingDir,
+			Cmd:          cmdList,
+		},
+	}
+	dockerContainer, err := sl.DockerClient.CreateContainer(opts)
+	if err != nil {
+		return "", err
+	}
+	return name, sl.DockerClient.StartContainer(dockerContainer.ID, &docker.HostConfig{
+		PortBindings: portBindings,
+		Binds:        binds,
+	})
+}
+
+func (sl *Kubelet) KillContainer(name string) error {
+	id, err := sl.GetContainerID(name)
+	if err != nil {
+		return err
+	}
+	err = sl.DockerClient.StopContainer(id, 10)
+	manifestId, containerName := dockerNameToManifestAndContainer(name)
+	sl.LogEvent(&api.Event{
+		Event: "STOP",
+		Manifest: &api.ContainerManifest{
+			Id: manifestId,
+		},
+		Container: &api.Container{
+			Name: containerName,
+		},
+	})
+
+	return err
+}
+
+// Watch a file for changes to the set of tasks that should run on this Kubelet
+// This function loops forever and is intended to be run as a goroutine
+func (sl *Kubelet) WatchFile(file string, changeChannel chan<- api.ContainerManifest) {
+	var lastData []byte
+	for {
+		time.Sleep(sl.FileCheckFrequency)
+		var manifest api.ContainerManifest
+		data, err := ioutil.ReadFile(file)
+		if err != nil {
+			log.Printf("Couldn't read file: %s : %v", file, err)
+			continue
+		}
+		if err = sl.ExtractYAMLData(data, &manifest); err != nil {
+			continue
+		}
+		if !bytes.Equal(lastData, data) {
+			lastData = data
+			// Ok, we have a valid configuration, send to channel for
+			// rejiggering.
+			changeChannel <- manifest
+			continue
+		}
+	}
+}
+
+// Watch an HTTP endpoint for changes to the set of tasks that should run on this Kubelet
+// This function runs forever and is intended to be run as a goroutine
+func (sl *Kubelet) WatchHTTP(url string, changeChannel chan<- api.ContainerManifest) {
+	var lastData []byte
+	client := &http.Client{}
+	for {
+		time.Sleep(sl.HTTPCheckFrequency)
+		var config api.ContainerManifest
+		data, err := sl.SyncHTTP(client, url, &config)
+		log.Printf("Containers: %#v", config)
+		if err != nil {
+			log.Printf("Error syncing HTTP: %#v", err)
+			continue
+		}
+		if !bytes.Equal(lastData, data) {
+			lastData = data
+			changeChannel <- config
+			continue
+		}
+	}
+}
+
+// SyncHTTP reads from url a yaml manifest and populates config. Returns the
+// raw bytes, if something was read. Returns an error if something goes wrong.
+// 'client' is used to execute the request, to allow caching of clients.
+func (sl *Kubelet) SyncHTTP(client *http.Client, url string, config *api.ContainerManifest) ([]byte, error) {
+	request, err := http.NewRequest("GET", url, nil)
+	if err != nil {
+		return nil, err
+	}
+	response, err := client.Do(request)
+	if err != nil {
+		return nil, err
+	}
+	defer response.Body.Close()
+	body, err := ioutil.ReadAll(response.Body)
+	if err != nil {
+		return nil, err
+	}
+	if err = sl.ExtractYAMLData(body, &config); err != nil {
+		return body, err
+	}
+	return body, nil
+}
+
+// Take an etcd Response object, and turn it into a structured list of containers
+// Return a list of containers, or an error if one occurs.
+func (sl *Kubelet) ResponseToManifests(response *etcd.Response) ([]api.ContainerManifest, error) {
+	if response.Node == nil || len(response.Node.Value) == 0 {
+		return nil, fmt.Errorf("no nodes field: %#v", response)
+	}
+	var manifests []api.ContainerManifest
+	err := sl.ExtractYAMLData([]byte(response.Node.Value), &manifests)
+	return manifests, err
+}
+
+func (sl *Kubelet) getKubeletStateFromEtcd(key string, changeChannel chan<- []api.ContainerManifest) error {
+	response, err := sl.Client.Get(key+"/kubelet", true, false)
+	if err != nil {
+		log.Printf("Error on get on %s: %#v", key, err)
+		switch err.(type) {
+		case *etcd.EtcdError:
+			etcdError := err.(*etcd.EtcdError)
+			if etcdError.ErrorCode == 100 {
+				return nil
+			}
+		}
+		return err
+	}
+	manifests, err := sl.ResponseToManifests(response)
+	if err != nil {
+		log.Printf("Error parsing response (%#v): %s", response, err)
+		return err
+	}
+	log.Printf("Got initial state from etcd: %+v", manifests)
+	changeChannel <- manifests
+	return nil
+}
+
+// Sync with etcd, and set up an etcd watch for new configurations
+// The channel to send new configurations across
+// This function loops forever and is intended to be run in a go routine.
+func (sl *Kubelet) SyncAndSetupEtcdWatch(changeChannel chan<- []api.ContainerManifest) {
+	hostname, err := exec.Command("hostname", "-f").Output()
+	if err != nil {
+		log.Printf("Couldn't determine hostname : %v", err)
+		return
+	}
+	key := "/registry/hosts/" + strings.TrimSpace(string(hostname))
+	// First fetch the initial configuration (watch only gives changes...)
+	for {
+		err = sl.getKubeletStateFromEtcd(key, changeChannel)
+		if err == nil {
+			// We got a successful response, etcd is up, set up the watch.
+			break
+		}
+		time.Sleep(30 * time.Second)
+	}
+
+	done := make(chan bool)
+	go util.Forever(func() { sl.TimeoutWatch(done) }, 0)
+	for {
+		// The etcd client will close the watch channel when it exits.  So we need
+		// to create and service a new one every time.
+		watchChannel := make(chan *etcd.Response)
+		// We don't push this through Forever because if it dies, we just do it again in 30 secs.
+		// anyway.
+		go sl.WatchEtcd(watchChannel, changeChannel)
+
+		sl.getKubeletStateFromEtcd(key, changeChannel)
+		log.Printf("Setting up a watch for configuration changes in etcd for %s", key)
+		sl.Client.Watch(key, 0, true, watchChannel, done)
+	}
+}
+
+// Timeout the watch after 30 seconds
+func (sl *Kubelet) TimeoutWatch(done chan bool) {
+	t := time.Tick(30 * time.Second)
+	for _ = range t {
+		done <- true
+	}
+}
+
+// Extract data from YAML file into a list of containers.
+func (sl *Kubelet) ExtractYAMLData(buf []byte, output interface{}) error {
+	err := yaml.Unmarshal(buf, output)
+	if err != nil {
+		log.Printf("Couldn't unmarshal configuration: %v", err)
+		return err
+	}
+	return nil
+}
+
+// Watch etcd for changes, receives config objects from the etcd client watch.
+// This function loops forever and is intended to be run as a goroutine.
+func (sl *Kubelet) WatchEtcd(watchChannel <-chan *etcd.Response, changeChannel chan<- []api.ContainerManifest) {
+	defer util.HandleCrash()
+	for {
+		watchResponse := <-watchChannel
+		log.Printf("Got change: %#v", watchResponse)
+
+		// This means the channel has been closed.
+		if watchResponse == nil {
+			return
+		}
+
+		if watchResponse.Node == nil || len(watchResponse.Node.Value) == 0 {
+			log.Printf("No nodes field: %#v", watchResponse)
+			if watchResponse.Node != nil {
+				log.Printf("Node: %#v", watchResponse.Node)
+			}
+		}
+		log.Printf("Got data: %v", watchResponse.Node.Value)
+		var manifests []api.ContainerManifest
+		if err := sl.ExtractYAMLData([]byte(watchResponse.Node.Value), &manifests); err != nil {
+			continue
+		}
+		// Ok, we have a valid configuration, send to channel for
+		// rejiggering.
+		changeChannel <- manifests
+	}
+}
+
+// Sync the configured list of containers (desired state) with the host current state
+func (sl *Kubelet) SyncManifests(config []api.ContainerManifest) error {
+	log.Printf("Desired:%#v", config)
+	var err error
+	desired := map[string]bool{}
+	for _, manifest := range config {
+		for _, element := range manifest.Containers {
+			var exists bool
+			exists, actualName, err := sl.ContainerExists(&manifest, &element)
+			if err != nil {
+				log.Printf("Error detecting container: %#v skipping.", err)
+				continue
+			}
+			if !exists {
+				log.Printf("%#v doesn't exist, creating", element)
+				actualName, err = sl.RunContainer(&manifest, &element)
+				// For some reason, list gives back names that start with '/'
+				actualName = "/" + actualName
+
+				if err != nil {
+					// TODO(bburns) : Perhaps blacklist a container after N failures?
+					log.Printf("Error creating container: %#v", err)
+					desired[actualName] = true
+					continue
+				}
+			} else {
+				log.Printf("%#v exists as %v", element.Name, actualName)
+			}
+			desired[actualName] = true
+		}
+	}
+	existingContainers, _ := sl.ListContainers()
+	log.Printf("Existing:\n%#v Desired: %#v", existingContainers, desired)
+	for _, container := range existingContainers {
+		if !desired[container] {
+			log.Printf("Killing: %s", container)
+			err = sl.KillContainer(container)
+			if err != nil {
+				log.Printf("Error killing container: %#v", err)
+			}
+		}
+	}
+	return err
+}
+
+// runSyncLoop is the main loop for processing changes. It watches for changes from
+// four channels (file, etcd, server, and http) and creates a union of the two. For
+// any new change seen, will run a sync against desired state and running state. If
+// no changes are seen to the configuration, will synchronize the last known desired
+// state every sync_frequency seconds.
+// Never returns.
+func (sl *Kubelet) RunSyncLoop(etcdChannel <-chan []api.ContainerManifest, fileChannel, serverChannel, httpChannel <-chan api.ContainerManifest, handler SyncHandler) {
+	var lastFile, lastEtcd, lastHttp, lastServer []api.ContainerManifest
+	for {
+		select {
+		case manifest := <-fileChannel:
+			log.Printf("Got new manifest from file... %v", manifest)
+			lastFile = []api.ContainerManifest{manifest}
+		case manifests := <-etcdChannel:
+			log.Printf("Got new configuration from etcd... %v", manifests)
+			lastEtcd = manifests
+		case manifest := <-httpChannel:
+			log.Printf("Got new manifest from external http... %v", manifest)
+			lastHttp = []api.ContainerManifest{manifest}
+		case manifest := <-serverChannel:
+			log.Printf("Got new manifest from our server... %v", manifest)
+			lastServer = []api.ContainerManifest{manifest}
+		case <-time.After(sl.SyncFrequency):
+		}
+
+		manifests := append([]api.ContainerManifest{}, lastFile...)
+		manifests = append(manifests, lastEtcd...)
+		manifests = append(manifests, lastHttp...)
+		manifests = append(manifests, lastServer...)
+		err := handler.SyncManifests(manifests)
+		if err != nil {
+			log.Printf("Couldn't sync containers : %#v", err)
+		}
+	}
+}
+
+func (sl *Kubelet) GetContainerInfo(name string) (string, error) {
+	info, err := sl.DockerClient.InspectContainer(name)
+	if err != nil {
+		return "{}", err
+	}
+	data, err := json.Marshal(info)
+	return string(data), err
+}

pkg/kubelet/kubelet_server.go

@@ -0,0 +1,80 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+package kubelet
+
+import (
+	"fmt"
+	"io/ioutil"
+	"net/http"
+	"net/url"
+
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/api"
+	"gopkg.in/v1/yaml"
+)
+
+type KubeletServer struct {
+	Kubelet       *Kubelet
+	UpdateChannel chan api.ContainerManifest
+}
+
+func (s *KubeletServer) error(w http.ResponseWriter, err error) {
+	w.WriteHeader(http.StatusInternalServerError)
+	fmt.Fprintf(w, "Internal Error: %#v", err)
+}
+
+func (s *KubeletServer) ServeHTTP(w http.ResponseWriter, req *http.Request) {
+	u, err := url.ParseRequestURI(req.RequestURI)
+	if err != nil {
+		s.error(w, err)
+		return
+	}
+	switch {
+	case u.Path == "/container":
+		defer req.Body.Close()
+		data, err := ioutil.ReadAll(req.Body)
+		if err != nil {
+			s.error(w, err)
+			return
+		}
+		var manifest api.ContainerManifest
+		err = yaml.Unmarshal(data, &manifest)
+		if err != nil {
+			s.error(w, err)
+			return
+		}
+		s.UpdateChannel <- manifest
+	case u.Path == "/containerInfo":
+		container := u.Query().Get("container")
+		if len(container) == 0 {
+			w.WriteHeader(http.StatusBadRequest)
+			fmt.Fprint(w, "Missing container query arg.")
+			return
+		}
+		id, err := s.Kubelet.GetContainerID(container)
+		body, err := s.Kubelet.GetContainerInfo(id)
+		if err != nil {
+			w.WriteHeader(http.StatusInternalServerError)
+			fmt.Fprintf(w, "Internal Error: %#v", err)
+			return
+		}
+		w.Header().Add("Content-type", "application/json")
+		w.WriteHeader(http.StatusOK)
+		fmt.Fprint(w, body)
+	default:
+		w.WriteHeader(http.StatusNotFound)
+		fmt.Fprint(w, "Not found.")
+	}
+}

pkg/kubelet/kubelet_test.go

@@ -0,0 +1,562 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+package kubelet
+
+import (
+	"encoding/json"
+	"fmt"
+	"net/http"
+	"net/http/httptest"
+	"sync"
+	"testing"
+
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/api"
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/registry"
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/util"
+	"github.com/coreos/go-etcd/etcd"
+	"github.com/fsouza/go-dockerclient"
+)
+
+// TODO: This doesn't reduce typing enough to make it worth the less readable errors. Remove.
+func expectNoError(t *testing.T, err error) {
+	if err != nil {
+		t.Errorf("Unexpected error: %#v", err)
+	}
+}
+
+// These are used for testing extract json (below)
+type TestData struct {
+	Value  string
+	Number int
+}
+
+type TestObject struct {
+	Name string
+	Data TestData
+}
+
+func verifyStringEquals(t *testing.T, actual, expected string) {
+	if actual != expected {
+		t.Errorf("Verification failed.  Expected: %s, Found %s", expected, actual)
+	}
+}
+
+func verifyIntEquals(t *testing.T, actual, expected int) {
+	if actual != expected {
+		t.Errorf("Verification failed.  Expected: %d, Found %d", expected, actual)
+	}
+}
+
+func verifyNoError(t *testing.T, e error) {
+	if e != nil {
+		t.Errorf("Expected no error, found %#v", e)
+	}
+}
+
+func verifyError(t *testing.T, e error) {
+	if e == nil {
+		t.Errorf("Expected error, found nil")
+	}
+}
+
+func TestExtractJSON(t *testing.T) {
+	obj := TestObject{}
+	kubelet := Kubelet{}
+	data := `{ "name": "foo", "data": { "value": "bar", "number": 10 } }`
+	kubelet.ExtractYAMLData([]byte(data), &obj)
+
+	verifyStringEquals(t, obj.Name, "foo")
+	verifyStringEquals(t, obj.Data.Value, "bar")
+	verifyIntEquals(t, obj.Data.Number, 10)
+}
+
+type FakeDockerClient struct {
+	containerList []docker.APIContainers
+	container     *docker.Container
+	err           error
+	called        []string
+}
+
+func (f *FakeDockerClient) clearCalls() {
+	f.called = []string{}
+}
+
+func (f *FakeDockerClient) appendCall(call string) {
+	f.called = append(f.called, call)
+}
+
+func (f *FakeDockerClient) ListContainers(options docker.ListContainersOptions) ([]docker.APIContainers, error) {
+	f.appendCall("list")
+	return f.containerList, f.err
+}
+
+func (f *FakeDockerClient) InspectContainer(id string) (*docker.Container, error) {
+	f.appendCall("inspect")
+	return f.container, f.err
+}
+
+func (f *FakeDockerClient) CreateContainer(docker.CreateContainerOptions) (*docker.Container, error) {
+	f.appendCall("create")
+	return nil, nil
+}
+
+func (f *FakeDockerClient) StartContainer(id string, hostConfig *docker.HostConfig) error {
+	f.appendCall("start")
+	return nil
+}
+
+func (f *FakeDockerClient) StopContainer(id string, timeout uint) error {
+	f.appendCall("stop")
+	return nil
+}
+
+func verifyCalls(t *testing.T, fakeDocker FakeDockerClient, calls []string) {
+	verifyStringArrayEquals(t, fakeDocker.called, calls)
+}
+
+func verifyStringArrayEquals(t *testing.T, actual, expected []string) {
+	invalid := len(actual) != len(expected)
+	for ix, value := range actual {
+		if expected[ix] != value {
+			invalid = true
+		}
+	}
+	if invalid {
+		t.Errorf("Expected: %#v, Actual: %#v", expected, actual)
+	}
+}
+
+func verifyPackUnpack(t *testing.T, manifestId, containerName string) {
+	name := manifestAndContainerToDockerName(
+		&api.ContainerManifest{Id: manifestId},
+		&api.Container{Name: containerName},
+	)
+	returnedManifestId, returnedContainerName := dockerNameToManifestAndContainer(name)
+	if manifestId != returnedManifestId || containerName != returnedContainerName {
+		t.Errorf("For (%s, %s), unpacked (%s, %s)", manifestId, containerName, returnedManifestId, returnedContainerName)
+	}
+}
+
+func TestContainerManifestNaming(t *testing.T) {
+	verifyPackUnpack(t, "manifest1234", "container5678")
+	verifyPackUnpack(t, "manifest--", "container__")
+	verifyPackUnpack(t, "--manifest", "__container")
+	verifyPackUnpack(t, "m___anifest_", "container-_-")
+	verifyPackUnpack(t, "_m___anifest", "-_-container")
+}
+
+func TestContainerExists(t *testing.T) {
+	fakeDocker := FakeDockerClient{
+		err: nil,
+	}
+	kubelet := Kubelet{
+		DockerClient: &fakeDocker,
+	}
+	manifest := api.ContainerManifest{
+		Id: "qux",
+	}
+	container := api.Container{
+		Name: "foo",
+	}
+	fakeDocker.containerList = []docker.APIContainers{
+		docker.APIContainers{
+			Names: []string{"foo--qux--1234"},
+		},
+		docker.APIContainers{
+			Names: []string{"bar--qux--1234"},
+		},
+	}
+	fakeDocker.container = &docker.Container{
+		ID: "foobar",
+	}
+
+	exists, _, err := kubelet.ContainerExists(&manifest, &container)
+	verifyCalls(t, fakeDocker, []string{"list", "list", "inspect"})
+	if !exists {
+		t.Errorf("Failed to find container %#v", container)
+	}
+	if err != nil {
+		t.Errorf("Unexpected error: %#v", err)
+	}
+}
+
+func TestGetContainerID(t *testing.T) {
+	fakeDocker := FakeDockerClient{
+		err: nil,
+	}
+	kubelet := Kubelet{
+		DockerClient: &fakeDocker,
+	}
+	fakeDocker.containerList = []docker.APIContainers{
+		docker.APIContainers{
+			Names: []string{"foo"},
+			ID:    "1234",
+		},
+		docker.APIContainers{
+			Names: []string{"bar"},
+			ID:    "4567",
+		},
+	}
+
+	id, err := kubelet.GetContainerID("foo")
+	verifyStringEquals(t, id, "1234")
+	verifyNoError(t, err)
+	verifyCalls(t, fakeDocker, []string{"list"})
+	fakeDocker.clearCalls()
+
+	id, err = kubelet.GetContainerID("bar")
+	verifyStringEquals(t, id, "4567")
+	verifyNoError(t, err)
+	verifyCalls(t, fakeDocker, []string{"list"})
+	fakeDocker.clearCalls()
+
+	id, err = kubelet.GetContainerID("NotFound")
+	verifyError(t, err)
+	verifyCalls(t, fakeDocker, []string{"list"})
+}
+
+func TestGetContainerByName(t *testing.T) {
+	fakeDocker := FakeDockerClient{
+		err: nil,
+	}
+	kubelet := Kubelet{
+		DockerClient: &fakeDocker,
+	}
+	fakeDocker.containerList = []docker.APIContainers{
+		docker.APIContainers{
+			Names: []string{"foo"},
+		},
+		docker.APIContainers{
+			Names: []string{"bar"},
+		},
+	}
+	fakeDocker.container = &docker.Container{
+		ID: "foobar",
+	}
+
+	container, err := kubelet.GetContainerByName("foo")
+	verifyCalls(t, fakeDocker, []string{"list", "inspect"})
+	if container == nil {
+		t.Errorf("Unexpected nil container")
+	}
+	verifyStringEquals(t, container.ID, "foobar")
+	verifyNoError(t, err)
+}
+
+func TestListContainers(t *testing.T) {
+	fakeDocker := FakeDockerClient{
+		err: nil,
+	}
+	kubelet := Kubelet{
+		DockerClient: &fakeDocker,
+	}
+	fakeDocker.containerList = []docker.APIContainers{
+		docker.APIContainers{
+			Names: []string{"foo"},
+		},
+		docker.APIContainers{
+			Names: []string{"bar"},
+		},
+	}
+
+	containers, err := kubelet.ListContainers()
+	verifyStringArrayEquals(t, containers, []string{"foo", "bar"})
+	verifyNoError(t, err)
+	verifyCalls(t, fakeDocker, []string{"list"})
+}
+
+func TestKillContainerWithError(t *testing.T) {
+	fakeDocker := FakeDockerClient{
+		err: fmt.Errorf("Sample Error"),
+		containerList: []docker.APIContainers{
+			docker.APIContainers{
+				Names: []string{"foo"},
+			},
+			docker.APIContainers{
+				Names: []string{"bar"},
+			},
+		},
+	}
+	kubelet := Kubelet{
+		DockerClient: &fakeDocker,
+	}
+	err := kubelet.KillContainer("foo")
+	verifyError(t, err)
+	verifyCalls(t, fakeDocker, []string{"list"})
+}
+
+func TestKillContainer(t *testing.T) {
+	fakeDocker := FakeDockerClient{
+		err: nil,
+	}
+	kubelet := Kubelet{
+		DockerClient: &fakeDocker,
+	}
+	fakeDocker.containerList = []docker.APIContainers{
+		docker.APIContainers{
+			Names: []string{"foo"},
+		},
+		docker.APIContainers{
+			Names: []string{"bar"},
+		},
+	}
+	fakeDocker.container = &docker.Container{
+		ID: "foobar",
+	}
+
+	err := kubelet.KillContainer("foo")
+	verifyNoError(t, err)
+	verifyCalls(t, fakeDocker, []string{"list", "stop"})
+}
+
+func TestSyncHTTP(t *testing.T) {
+	containers := api.ContainerManifest{
+		Containers: []api.Container{
+			api.Container{
+				Name:  "foo",
+				Image: "dockerfile/foo",
+			},
+			api.Container{
+				Name:  "bar",
+				Image: "dockerfile/bar",
+			},
+		},
+	}
+	data, _ := json.Marshal(containers)
+	fakeHandler := util.FakeHandler{
+		StatusCode:   200,
+		ResponseBody: string(data),
+	}
+	testServer := httptest.NewServer(&fakeHandler)
+	kubelet := Kubelet{}
+
+	var containersOut api.ContainerManifest
+	data, err := kubelet.SyncHTTP(&http.Client{}, testServer.URL, &containersOut)
+	if err != nil {
+		t.Errorf("Unexpected error: %#v", err)
+	}
+	if len(containers.Containers) != len(containersOut.Containers) {
+		t.Errorf("Container sizes don't match.  Expected: %d Received %d, %#v", len(containers.Containers), len(containersOut.Containers), containersOut)
+	}
+	expectedData, _ := json.Marshal(containers)
+	actualData, _ := json.Marshal(containersOut)
+	if string(expectedData) != string(actualData) {
+		t.Errorf("Container data doesn't match.  Expected: %s Received %s", string(expectedData), string(actualData))
+	}
+}
+
+func TestResponseToContainersNil(t *testing.T) {
+	kubelet := Kubelet{}
+	list, err := kubelet.ResponseToManifests(&etcd.Response{Node: nil})
+	if len(list) != 0 {
+		t.Errorf("Unexpected non-zero list: %#v", list)
+	}
+	if err == nil {
+		t.Error("Unexpected non-error")
+	}
+}
+
+func TestResponseToManifests(t *testing.T) {
+	kubelet := Kubelet{}
+	list, err := kubelet.ResponseToManifests(&etcd.Response{
+		Node: &etcd.Node{
+			Value: util.MakeJSONString([]api.ContainerManifest{
+				api.ContainerManifest{Id: "foo"},
+				api.ContainerManifest{Id: "bar"},
+			}),
+		},
+	})
+	if len(list) != 2 || list[0].Id != "foo" || list[1].Id != "bar" {
+		t.Errorf("Unexpected list: %#v", list)
+	}
+	expectNoError(t, err)
+}
+
+type channelReader struct {
+	list [][]api.ContainerManifest
+	wg   sync.WaitGroup
+}
+
+func startReading(channel <-chan []api.ContainerManifest) *channelReader {
+	cr := &channelReader{}
+	cr.wg.Add(1)
+	go func() {
+		for {
+			containers, ok := <-channel
+			if !ok {
+				break
+			}
+			cr.list = append(cr.list, containers)
+		}
+		cr.wg.Done()
+	}()
+	return cr
+}
+
+func (cr *channelReader) GetList() [][]api.ContainerManifest {
+	cr.wg.Wait()
+	return cr.list
+}
+
+func TestGetKubeletStateFromEtcdNoData(t *testing.T) {
+	fakeClient := registry.MakeFakeEtcdClient(t)
+	kubelet := Kubelet{
+		Client: fakeClient,
+	}
+	channel := make(chan []api.ContainerManifest)
+	reader := startReading(channel)
+	fakeClient.Data["/registry/hosts/machine/kubelet"] = registry.EtcdResponseWithError{
+		R: &etcd.Response{},
+		E: nil,
+	}
+	err := kubelet.getKubeletStateFromEtcd("/registry/hosts/machine", channel)
+	if err == nil {
+		t.Error("Unexpected no err.")
+	}
+	close(channel)
+	list := reader.GetList()
+	if len(list) != 0 {
+		t.Errorf("Unexpected list: %#v", list)
+	}
+}
+
+func TestGetKubeletStateFromEtcd(t *testing.T) {
+	fakeClient := registry.MakeFakeEtcdClient(t)
+	kubelet := Kubelet{
+		Client: fakeClient,
+	}
+	channel := make(chan []api.ContainerManifest)
+	reader := startReading(channel)
+	fakeClient.Data["/registry/hosts/machine/kubelet"] = registry.EtcdResponseWithError{
+		R: &etcd.Response{
+			Node: &etcd.Node{
+				Value: util.MakeJSONString([]api.Container{}),
+			},
+		},
+		E: nil,
+	}
+	err := kubelet.getKubeletStateFromEtcd("/registry/hosts/machine", channel)
+	expectNoError(t, err)
+	close(channel)
+	list := reader.GetList()
+	if len(list) != 1 {
+		t.Errorf("Unexpected list: %#v", list)
+	}
+}
+
+func TestGetKubeletStateFromEtcdNotFound(t *testing.T) {
+	fakeClient := registry.MakeFakeEtcdClient(t)
+	kubelet := Kubelet{
+		Client: fakeClient,
+	}
+	channel := make(chan []api.ContainerManifest)
+	reader := startReading(channel)
+	fakeClient.Data["/registry/hosts/machine/kubelet"] = registry.EtcdResponseWithError{
+		R: &etcd.Response{},
+		E: &etcd.EtcdError{
+			ErrorCode: 100,
+		},
+	}
+	err := kubelet.getKubeletStateFromEtcd("/registry/hosts/machine", channel)
+	expectNoError(t, err)
+	close(channel)
+	list := reader.GetList()
+	if len(list) != 0 {
+		t.Errorf("Unexpected list: %#v", list)
+	}
+}
+
+func TestGetKubeletStateFromEtcdError(t *testing.T) {
+	fakeClient := registry.MakeFakeEtcdClient(t)
+	kubelet := Kubelet{
+		Client: fakeClient,
+	}
+	channel := make(chan []api.ContainerManifest)
+	reader := startReading(channel)
+	fakeClient.Data["/registry/hosts/machine/kubelet"] = registry.EtcdResponseWithError{
+		R: &etcd.Response{},
+		E: &etcd.EtcdError{
+			ErrorCode: 200, // non not found error
+		},
+	}
+	err := kubelet.getKubeletStateFromEtcd("/registry/hosts/machine", channel)
+	if err == nil {
+		t.Error("Unexpected non-error")
+	}
+	close(channel)
+	list := reader.GetList()
+	if len(list) != 0 {
+		t.Errorf("Unexpected list: %#v", list)
+	}
+}
+
+func TestSyncManifestsDoesNothing(t *testing.T) {
+	fakeDocker := FakeDockerClient{
+		err: nil,
+	}
+	fakeDocker.containerList = []docker.APIContainers{
+		docker.APIContainers{
+			// format is <container-id>--<manifest-id>
+			Names: []string{"bar--foo"},
+			ID:    "1234",
+		},
+	}
+	fakeDocker.container = &docker.Container{
+		ID: "1234",
+	}
+	kubelet := Kubelet{
+		DockerClient: &fakeDocker,
+	}
+	err := kubelet.SyncManifests([]api.ContainerManifest{
+		api.ContainerManifest{
+			Id: "foo",
+			Containers: []api.Container{
+				api.Container{Name: "bar"},
+			},
+		},
+	})
+	expectNoError(t, err)
+	if len(fakeDocker.called) != 4 ||
+		fakeDocker.called[0] != "list" ||
+		fakeDocker.called[1] != "list" ||
+		fakeDocker.called[2] != "inspect" ||
+		fakeDocker.called[3] != "list" {
+		t.Errorf("Unexpected call sequence: %#v", fakeDocker.called)
+	}
+}
+
+func TestSyncManifestsDeletes(t *testing.T) {
+	fakeDocker := FakeDockerClient{
+		err: nil,
+	}
+	fakeDocker.containerList = []docker.APIContainers{
+		docker.APIContainers{
+			Names: []string{"foo"},
+			ID:    "1234",
+		},
+	}
+	kubelet := Kubelet{
+		DockerClient: &fakeDocker,
+	}
+	err := kubelet.SyncManifests([]api.ContainerManifest{})
+	expectNoError(t, err)
+	if len(fakeDocker.called) != 3 ||
+		fakeDocker.called[0] != "list" ||
+		fakeDocker.called[1] != "list" ||
+		fakeDocker.called[2] != "stop" {
+		t.Errorf("Unexpected call sequence: %#v", fakeDocker.called)
+	}
+}

pkg/proxy/config/config.go

@@ -0,0 +1,320 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+// Config provides decoupling between various configuration sources (etcd, files,...) and
+// the pieces that actually care about them (loadbalancer, proxy). Config takes 1 or more
+// configuration sources and allows for incremental (add/remove) and full replace (set)
+// changes from each of the sources, then creates a union of the configuration and provides
+// a unified view for both service handlers as well as endpoint handlers. There is no attempt
+// to resolve conflicts of any sort. Basic idea is that each configuration source gets a channel
+// from the Config service and pushes updates to it via that channel. Config then keeps track of
+// incremental & replace changes and distributes them to listeners as appropriate.
+package config
+
+import (
+	"log"
+	"sync"
+	"time"
+
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/api"
+)
+
+type Operation int
+
+const (
+	SET Operation = iota
+	ADD
+	REMOVE
+)
+
+// Defines an operation sent on the channel. You can add or remove single services by
+// sending an array of size one and Op == ADD|REMOVE. For setting the state of the system
+// to a given state for this source configuration, set Services as desired and Op to SET,
+// which will reset the system state to that specified in this operation for this source
+// channel. To remove all services, set Services to empty array and Op to SET
+type ServiceUpdate struct {
+	Services []api.Service
+	Op       Operation
+}
+
+// Defines an operation sent on the channel. You can add or remove single endpoints by
+// sending an array of size one and Op == ADD|REMOVE. For setting the state of the system
+// to a given state for this source configuration, set Endpoints as desired and Op to SET,
+// which will reset the system state to that specified in this operation for this source
+// channel. To remove all endpoints, set Endpoints to empty array and Op to SET
+type EndpointsUpdate struct {
+	Endpoints []api.Endpoints
+	Op        Operation
+}
+
+type ServiceConfigHandler interface {
+	// Sent when a configuration has been changed by one of the sources. This is the
+	// union of all the configuration sources.
+	OnUpdate(services []api.Service)
+}
+
+type EndpointsConfigHandler interface {
+	// OnUpdate gets called when endpoints configuration is changed for a given
+	// service on any of the configuration sources. An example is when a new
+	// service comes up, or when containers come up or down for an existing service.
+	OnUpdate(endpoints []api.Endpoints)
+}
+
+type ServiceConfig struct {
+	// Configuration sources and their lock.
+	configSourceLock       sync.RWMutex
+	serviceConfigSources   map[string]chan ServiceUpdate
+	endpointsConfigSources map[string]chan EndpointsUpdate
+
+	// Handlers for changes to services and endpoints and their lock.
+	handlerLock      sync.RWMutex
+	serviceHandlers  []ServiceConfigHandler
+	endpointHandlers []EndpointsConfigHandler
+
+	// Last known configuration for union of the sources and the locks. Map goes
+	// from each source to array of services/endpoints that have been configured
+	// through that channel.
+	configLock     sync.RWMutex
+	serviceConfig  map[string]map[string]api.Service
+	endpointConfig map[string]map[string]api.Endpoints
+
+	// Channel that service configuration source listeners use to signal of new
+	// configurations.
+	// Value written is the source of the change.
+	serviceNotifyChannel chan string
+
+	// Channel that endpoint configuration source listeners use to signal of new
+	// configurations.
+	// Value written is the source of the change.
+	endpointsNotifyChannel chan string
+}
+
+func NewServiceConfig() ServiceConfig {
+	config := ServiceConfig{
+		serviceConfigSources:   make(map[string]chan ServiceUpdate),
+		endpointsConfigSources: make(map[string]chan EndpointsUpdate),
+		serviceHandlers:        make([]ServiceConfigHandler, 10),
+		endpointHandlers:       make([]EndpointsConfigHandler, 10),
+		serviceConfig:          make(map[string]map[string]api.Service),
+		endpointConfig:         make(map[string]map[string]api.Endpoints),
+		serviceNotifyChannel:   make(chan string),
+		endpointsNotifyChannel: make(chan string),
+	}
+	go config.Run()
+	return config
+}
+
+func (impl *ServiceConfig) Run() {
+	log.Printf("Starting the config Run loop")
+	for {
+		select {
+		case source := <-impl.serviceNotifyChannel:
+			log.Printf("Got new service configuration from source %s", source)
+			impl.NotifyServiceUpdate()
+		case source := <-impl.endpointsNotifyChannel:
+			log.Printf("Got new endpoint configuration from source %s", source)
+			impl.NotifyEndpointsUpdate()
+		case <-time.After(1 * time.Second):
+		}
+	}
+}
+
+func (impl *ServiceConfig) ServiceChannelListener(source string, listenChannel chan ServiceUpdate) {
+	// Represents the current services configuration for this channel.
+	serviceMap := make(map[string]api.Service)
+	for {
+		select {
+		case update := <-listenChannel:
+			switch update.Op {
+			case ADD:
+				log.Printf("Adding new service from source %s : %v", source, update.Services)
+				for _, value := range update.Services {
+					serviceMap[value.ID] = value
+				}
+			case REMOVE:
+				log.Printf("Removing a service %v", update)
+				for _, value := range update.Services {
+					delete(serviceMap, value.ID)
+				}
+			case SET:
+				log.Printf("Setting services %v", update)
+				// Clear the old map entries by just creating a new map
+				serviceMap = make(map[string]api.Service)
+				for _, value := range update.Services {
+					serviceMap[value.ID] = value
+				}
+			default:
+				log.Printf("Received invalid update type: %v", update)
+				continue
+			}
+			impl.configLock.Lock()
+			impl.serviceConfig[source] = serviceMap
+			impl.configLock.Unlock()
+			impl.serviceNotifyChannel <- source
+		}
+	}
+}
+
+func (impl *ServiceConfig) EndpointsChannelListener(source string, listenChannel chan EndpointsUpdate) {
+	endpointMap := make(map[string]api.Endpoints)
+	for {
+		select {
+		case update := <-listenChannel:
+			switch update.Op {
+			case ADD:
+				log.Printf("Adding a new endpoint %v", update)
+				for _, value := range update.Endpoints {
+					endpointMap[value.Name] = value
+				}
+			case REMOVE:
+				log.Printf("Removing an endpoint %v", update)
+				for _, value := range update.Endpoints {
+					delete(endpointMap, value.Name)
+				}
+
+			case SET:
+				log.Printf("Setting services %v", update)
+				// Clear the old map entries by just creating a new map
+				endpointMap = make(map[string]api.Endpoints)
+				for _, value := range update.Endpoints {
+					endpointMap[value.Name] = value
+				}
+			default:
+				log.Printf("Received invalid update type: %v", update)
+				continue
+			}
+			impl.configLock.Lock()
+			impl.endpointConfig[source] = endpointMap
+			impl.configLock.Unlock()
+			impl.endpointsNotifyChannel <- source
+		}
+
+	}
+}
+
+// GetServiceConfigurationChannel returns a channel where a configuration source
+// can send updates of new service configurations. Multiple calls with the same
+// source will return the same channel. This allows change and state based sources
+// to use the same channel. Difference source names however will be treated as a
+// union.
+func (impl *ServiceConfig) GetServiceConfigurationChannel(source string) chan ServiceUpdate {
+	if len(source) == 0 {
+		panic("GetServiceConfigurationChannel given an empty service name")
+	}
+	impl.configSourceLock.Lock()
+	defer impl.configSourceLock.Unlock()
+	channel, exists := impl.serviceConfigSources[source]
+	if exists {
+		return channel
+	}
+	newChannel := make(chan ServiceUpdate)
+	impl.serviceConfigSources[source] = newChannel
+	go impl.ServiceChannelListener(source, newChannel)
+	return newChannel
+}
+
+// GetEndpointConfigurationChannel returns a channel where a configuration source
+// can send updates of new endpoint configurations. Multiple calls with the same
+// source will return the same channel. This allows change and state based sources
+// to use the same channel. Difference source names however will be treated as a
+// union.
+func (impl *ServiceConfig) GetEndpointsConfigurationChannel(source string) chan EndpointsUpdate {
+	if len(source) == 0 {
+		panic("GetEndpointConfigurationChannel given an empty service name")
+	}
+	impl.configSourceLock.Lock()
+	defer impl.configSourceLock.Unlock()
+	channel, exists := impl.endpointsConfigSources[source]
+	if exists {
+		return channel
+	}
+	newChannel := make(chan EndpointsUpdate)
+	impl.endpointsConfigSources[source] = newChannel
+	go impl.EndpointsChannelListener(source, newChannel)
+	return newChannel
+}
+
+// Register ServiceConfigHandler to receive updates of changes to services.
+func (impl *ServiceConfig) RegisterServiceHandler(handler ServiceConfigHandler) {
+	impl.handlerLock.Lock()
+	defer impl.handlerLock.Unlock()
+	for i, h := range impl.serviceHandlers {
+		if h == nil {
+			impl.serviceHandlers[i] = handler
+			return
+		}
+	}
+	// TODO(vaikas): Grow the array here instead of panic.
+	// In practice we are expecting there to be 1 handler anyways,
+	// so not a big deal for now
+	panic("Only up to 10 service handlers supported for now")
+}
+
+// Register ServiceConfigHandler to receive updates of changes to services.
+func (impl *ServiceConfig) RegisterEndpointsHandler(handler EndpointsConfigHandler) {
+	impl.handlerLock.Lock()
+	defer impl.handlerLock.Unlock()
+	for i, h := range impl.endpointHandlers {
+		if h == nil {
+			impl.endpointHandlers[i] = handler
+			return
+		}
+	}
+	// TODO(vaikas): Grow the array here instead of panic.
+	// In practice we are expecting there to be 1 handler anyways,
+	// so not a big deal for now
+	panic("Only up to 10 endpoint handlers supported for now")
+}
+
+func (impl *ServiceConfig) NotifyServiceUpdate() {
+	services := make([]api.Service, 0)
+	impl.configLock.RLock()
+	for _, sourceServices := range impl.serviceConfig {
+		for _, value := range sourceServices {
+			services = append(services, value)
+		}
+	}
+	impl.configLock.RUnlock()
+	log.Printf("Unified configuration %+v", services)
+	impl.handlerLock.RLock()
+	handlers := impl.serviceHandlers
+	impl.handlerLock.RUnlock()
+	for _, handler := range handlers {
+		if handler != nil {
+			handler.OnUpdate(services)
+		}
+	}
+}
+
+func (impl *ServiceConfig) NotifyEndpointsUpdate() {
+	endpoints := make([]api.Endpoints, 0)
+	impl.configLock.RLock()
+	for _, sourceEndpoints := range impl.endpointConfig {
+		for _, value := range sourceEndpoints {
+			endpoints = append(endpoints, value)
+		}
+	}
+	impl.configLock.RUnlock()
+	log.Printf("Unified configuration %+v", endpoints)
+	impl.handlerLock.RLock()
+	handlers := impl.endpointHandlers
+	impl.handlerLock.RUnlock()
+	for _, handler := range handlers {
+		if handler != nil {
+			handler.OnUpdate(endpoints)
+		}
+	}
+}

pkg/proxy/config/config_test.go

@@ -0,0 +1,240 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package config
+
+import (
+	"reflect"
+	"testing"
+
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/api"
+)
+
+const TomcatPort int = 8080
+const TomcatName = "tomcat"
+
+var TomcatEndpoints = map[string]string{"c0": "1.1.1.1:18080", "c1": "2.2.2.2:18081"}
+
+const MysqlPort int = 3306
+const MysqlName = "mysql"
+
+var MysqlEndpoints = map[string]string{"c0": "1.1.1.1:13306", "c3": "2.2.2.2:13306"}
+
+type ServiceHandlerMock struct {
+	services []api.Service
+}
+
+func NewServiceHandlerMock() ServiceHandlerMock {
+	return ServiceHandlerMock{services: make([]api.Service, 0)}
+}
+
+func (impl ServiceHandlerMock) OnUpdate(services []api.Service) {
+	impl.services = services
+}
+
+func (impl ServiceHandlerMock) ValidateServices(t *testing.T, expectedServices []api.Service) {
+	if reflect.DeepEqual(impl.services, expectedServices) {
+		t.Errorf("Services don't match %+v expected: %+v", impl.services, expectedServices)
+	}
+}
+
+type EndpointsHandlerMock struct {
+	endpoints []api.Endpoints
+}
+
+func NewEndpointsHandlerMock() EndpointsHandlerMock {
+	return EndpointsHandlerMock{endpoints: make([]api.Endpoints, 0)}
+}
+
+func (impl EndpointsHandlerMock) OnUpdate(endpoints []api.Endpoints) {
+	impl.endpoints = endpoints
+}
+
+func (impl EndpointsHandlerMock) ValidateEndpoints(t *testing.T, expectedEndpoints []api.Endpoints) {
+	if reflect.DeepEqual(impl.endpoints, expectedEndpoints) {
+		t.Errorf("Endpoints don't match %+v", impl.endpoints, expectedEndpoints)
+	}
+}
+
+func CreateServiceUpdate(op Operation, services ...api.Service) ServiceUpdate {
+	ret := ServiceUpdate{Op: op}
+	ret.Services = make([]api.Service, len(services))
+	for i, value := range services {
+		ret.Services[i] = value
+	}
+	return ret
+}
+
+func CreateEndpointsUpdate(op Operation, endpoints ...api.Endpoints) EndpointsUpdate {
+	ret := EndpointsUpdate{Op: op}
+	ret.Endpoints = make([]api.Endpoints, len(endpoints))
+	for i, value := range endpoints {
+		ret.Endpoints[i] = value
+	}
+	return ret
+}
+
+func TestServiceConfigurationChannels(t *testing.T) {
+	config := NewServiceConfig()
+	channelOne := config.GetServiceConfigurationChannel("one")
+	if channelOne != config.GetServiceConfigurationChannel("one") {
+		t.Error("Didn't get the same service configuration channel back with the same name")
+	}
+	channelTwo := config.GetServiceConfigurationChannel("two")
+	if channelOne == channelTwo {
+		t.Error("Got back the same service configuration channel for different names")
+	}
+}
+
+func TestEndpointConfigurationChannels(t *testing.T) {
+	config := NewServiceConfig()
+	channelOne := config.GetEndpointsConfigurationChannel("one")
+	if channelOne != config.GetEndpointsConfigurationChannel("one") {
+		t.Error("Didn't get the same endpoint configuration channel back with the same name")
+	}
+	channelTwo := config.GetEndpointsConfigurationChannel("two")
+	if channelOne == channelTwo {
+		t.Error("Got back the same endpoint configuration channel for different names")
+	}
+}
+
+func TestNewServiceAddedAndNotified(t *testing.T) {
+	config := NewServiceConfig()
+	channel := config.GetServiceConfigurationChannel("one")
+	handler := NewServiceHandlerMock()
+	config.RegisterServiceHandler(&handler)
+	serviceUpdate := CreateServiceUpdate(ADD, api.Service{JSONBase: api.JSONBase{ID: "foo"}, Port: 10})
+	channel <- serviceUpdate
+	handler.ValidateServices(t, serviceUpdate.Services)
+
+}
+
+func TestServiceAddedRemovedSetAndNotified(t *testing.T) {
+	config := NewServiceConfig()
+	channel := config.GetServiceConfigurationChannel("one")
+	handler := NewServiceHandlerMock()
+	config.RegisterServiceHandler(&handler)
+	serviceUpdate := CreateServiceUpdate(ADD, api.Service{JSONBase: api.JSONBase{ID: "foo"}, Port: 10})
+	channel <- serviceUpdate
+	handler.ValidateServices(t, serviceUpdate.Services)
+
+	serviceUpdate2 := CreateServiceUpdate(ADD, api.Service{JSONBase: api.JSONBase{ID: "bar"}, Port: 20})
+	channel <- serviceUpdate2
+	services := []api.Service{serviceUpdate.Services[0], serviceUpdate2.Services[0]}
+	handler.ValidateServices(t, services)
+
+	serviceUpdate3 := CreateServiceUpdate(REMOVE, api.Service{JSONBase: api.JSONBase{ID: "foo"}})
+	channel <- serviceUpdate3
+	services = []api.Service{serviceUpdate2.Services[0]}
+	handler.ValidateServices(t, services)
+
+	serviceUpdate4 := CreateServiceUpdate(SET, api.Service{JSONBase: api.JSONBase{ID: "foobar"}, Port: 99})
+	channel <- serviceUpdate4
+	services = []api.Service{serviceUpdate4.Services[0]}
+	handler.ValidateServices(t, services)
+}
+
+func TestNewMultipleSourcesServicesAddedAndNotified(t *testing.T) {
+	config := NewServiceConfig()
+	channelOne := config.GetServiceConfigurationChannel("one")
+	channelTwo := config.GetServiceConfigurationChannel("two")
+	if channelOne == channelTwo {
+		t.Error("Same channel handed back for one and two")
+	}
+	handler := NewServiceHandlerMock()
+	config.RegisterServiceHandler(handler)
+	serviceUpdate1 := CreateServiceUpdate(ADD, api.Service{JSONBase: api.JSONBase{ID: "foo"}, Port: 10})
+	serviceUpdate2 := CreateServiceUpdate(ADD, api.Service{JSONBase: api.JSONBase{ID: "bar"}, Port: 20})
+	channelOne <- serviceUpdate1
+	channelTwo <- serviceUpdate2
+	services := []api.Service{serviceUpdate1.Services[0], serviceUpdate2.Services[0]}
+	handler.ValidateServices(t, services)
+}
+
+func TestNewMultipleSourcesServicesMultipleHandlersAddedAndNotified(t *testing.T) {
+	config := NewServiceConfig()
+	channelOne := config.GetServiceConfigurationChannel("one")
+	channelTwo := config.GetServiceConfigurationChannel("two")
+	handler := NewServiceHandlerMock()
+	handler2 := NewServiceHandlerMock()
+	config.RegisterServiceHandler(handler)
+	config.RegisterServiceHandler(handler2)
+	serviceUpdate1 := CreateServiceUpdate(ADD, api.Service{JSONBase: api.JSONBase{ID: "foo"}, Port: 10})
+	serviceUpdate2 := CreateServiceUpdate(ADD, api.Service{JSONBase: api.JSONBase{ID: "bar"}, Port: 20})
+	channelOne <- serviceUpdate1
+	channelTwo <- serviceUpdate2
+	services := []api.Service{serviceUpdate1.Services[0], serviceUpdate2.Services[0]}
+	handler.ValidateServices(t, services)
+	handler2.ValidateServices(t, services)
+}
+
+func TestNewMultipleSourcesEndpointsMultipleHandlersAddedAndNotified(t *testing.T) {
+	config := NewServiceConfig()
+	channelOne := config.GetEndpointsConfigurationChannel("one")
+	channelTwo := config.GetEndpointsConfigurationChannel("two")
+	handler := NewEndpointsHandlerMock()
+	handler2 := NewEndpointsHandlerMock()
+	config.RegisterEndpointsHandler(handler)
+	config.RegisterEndpointsHandler(handler2)
+	endpointsUpdate1 := CreateEndpointsUpdate(ADD, api.Endpoints{Name: "foo", Endpoints: []string{"endpoint1", "endpoint2"}})
+	endpointsUpdate2 := CreateEndpointsUpdate(ADD, api.Endpoints{Name: "bar", Endpoints: []string{"endpoint3", "endpoint4"}})
+	channelOne <- endpointsUpdate1
+	channelTwo <- endpointsUpdate2
+
+	endpoints := []api.Endpoints{endpointsUpdate1.Endpoints[0], endpointsUpdate2.Endpoints[0]}
+	handler.ValidateEndpoints(t, endpoints)
+	handler2.ValidateEndpoints(t, endpoints)
+}
+
+func TestNewMultipleSourcesEndpointsMultipleHandlersAddRemoveSetAndNotified(t *testing.T) {
+	config := NewServiceConfig()
+	channelOne := config.GetEndpointsConfigurationChannel("one")
+	channelTwo := config.GetEndpointsConfigurationChannel("two")
+	handler := NewEndpointsHandlerMock()
+	handler2 := NewEndpointsHandlerMock()
+	config.RegisterEndpointsHandler(handler)
+	config.RegisterEndpointsHandler(handler2)
+	endpointsUpdate1 := CreateEndpointsUpdate(ADD, api.Endpoints{Name: "foo", Endpoints: []string{"endpoint1", "endpoint2"}})
+	endpointsUpdate2 := CreateEndpointsUpdate(ADD, api.Endpoints{Name: "bar", Endpoints: []string{"endpoint3", "endpoint4"}})
+	channelOne <- endpointsUpdate1
+	channelTwo <- endpointsUpdate2
+
+	endpoints := []api.Endpoints{endpointsUpdate1.Endpoints[0], endpointsUpdate2.Endpoints[0]}
+	handler.ValidateEndpoints(t, endpoints)
+	handler2.ValidateEndpoints(t, endpoints)
+
+	// Add one more
+	endpointsUpdate3 := CreateEndpointsUpdate(ADD, api.Endpoints{Name: "foobar", Endpoints: []string{"endpoint5", "endpoint6"}})
+	channelTwo <- endpointsUpdate3
+	endpoints = []api.Endpoints{endpointsUpdate1.Endpoints[0], endpointsUpdate2.Endpoints[0], endpointsUpdate3.Endpoints[0]}
+	handler.ValidateEndpoints(t, endpoints)
+	handler2.ValidateEndpoints(t, endpoints)
+
+	// Update the "foo" service with new endpoints
+	endpointsUpdate1 = CreateEndpointsUpdate(ADD, api.Endpoints{Name: "foo", Endpoints: []string{"endpoint77"}})
+	channelOne <- endpointsUpdate1
+	endpoints = []api.Endpoints{endpointsUpdate1.Endpoints[0], endpointsUpdate2.Endpoints[0], endpointsUpdate3.Endpoints[0]}
+	handler.ValidateEndpoints(t, endpoints)
+	handler2.ValidateEndpoints(t, endpoints)
+
+	// Remove "bar" service
+	endpointsUpdate2 = CreateEndpointsUpdate(REMOVE, api.Endpoints{Name: "bar"})
+	channelTwo <- endpointsUpdate2
+
+	endpoints = []api.Endpoints{endpointsUpdate1.Endpoints[0], endpointsUpdate3.Endpoints[0]}
+	handler.ValidateEndpoints(t, endpoints)
+	handler2.ValidateEndpoints(t, endpoints)
+}

pkg/proxy/config/etcd.go

@@ -0,0 +1,227 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+// Watches etcd and gets the full configuration on preset intervals.
+// Expects the list of exposed services to live under:
+// registry/services
+// which in etcd is exposed like so:
+// http://<etcd server>/v2/keys/registry/services
+//
+// The port that proxy needs to listen in for each service is a value in:
+// registry/services/<service>
+//
+// The endpoints for each of the services found is a json string
+// representing that service at:
+// /registry/services/<service>/endpoint
+// and the format is:
+// '[ { "machine": <host>, "name": <name", "port": <port> },
+//    { "machine": <host2>, "name": <name2", "port": <port2> }
+//  ]',
+//
+package config
+
+import (
+	"encoding/json"
+	"fmt"
+	"log"
+	"strings"
+	"time"
+
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/api"
+	"github.com/coreos/go-etcd/etcd"
+)
+
+const RegistryRoot = "registry/services"
+
+type ConfigSourceEtcd struct {
+	client           *etcd.Client
+	serviceChannel   chan ServiceUpdate
+	endpointsChannel chan EndpointsUpdate
+}
+
+func NewConfigSourceEtcd(client *etcd.Client, serviceChannel chan ServiceUpdate, endpointsChannel chan EndpointsUpdate) ConfigSourceEtcd {
+	config := ConfigSourceEtcd{
+		client:           client,
+		serviceChannel:   serviceChannel,
+		endpointsChannel: endpointsChannel,
+	}
+	go config.Run()
+	return config
+}
+
+func (impl ConfigSourceEtcd) Run() {
+	// Initially, just wait for the etcd to come up before doing anything more complicated.
+	var services []api.Service
+	var endpoints []api.Endpoints
+	var err error
+	for {
+		services, endpoints, err = impl.GetServices()
+		if err == nil {
+			break
+		}
+		log.Printf("Failed to get any services: %v", err)
+		time.Sleep(2 * time.Second)
+	}
+
+	if len(services) > 0 {
+		serviceUpdate := ServiceUpdate{Op: SET, Services: services}
+		impl.serviceChannel <- serviceUpdate
+	}
+	if len(endpoints) > 0 {
+		endpointsUpdate := EndpointsUpdate{Op: SET, Endpoints: endpoints}
+		impl.endpointsChannel <- endpointsUpdate
+	}
+
+	// Ok, so we got something back from etcd. Let's set up a watch for new services, and
+	// their endpoints
+	go impl.WatchForChanges()
+
+	for {
+		services, endpoints, err = impl.GetServices()
+		if err != nil {
+			log.Printf("ConfigSourceEtcd: Failed to get services: %v", err)
+		} else {
+			if len(services) > 0 {
+				serviceUpdate := ServiceUpdate{Op: SET, Services: services}
+				impl.serviceChannel <- serviceUpdate
+			}
+			if len(endpoints) > 0 {
+				endpointsUpdate := EndpointsUpdate{Op: SET, Endpoints: endpoints}
+				impl.endpointsChannel <- endpointsUpdate
+			}
+		}
+		time.Sleep(30 * time.Second)
+	}
+}
+
+// Finds the list of services and their endpoints from etcd.
+// This operation is akin to a set a known good at regular intervals.
+func (impl ConfigSourceEtcd) GetServices() ([]api.Service, []api.Endpoints, error) {
+	response, err := impl.client.Get(RegistryRoot+"/specs", true, false)
+	if err != nil {
+		log.Printf("Failed to get the key %s: %v", RegistryRoot, err)
+		return make([]api.Service, 0), make([]api.Endpoints, 0), err
+	}
+	if response.Node.Dir == true {
+		retServices := make([]api.Service, len(response.Node.Nodes))
+		retEndpoints := make([]api.Endpoints, len(response.Node.Nodes))
+		// Ok, so we have directories, this list should be the list
+		// of services. Find the local port to listen on and remote endpoints
+		// and create a Service entry for it.
+		for i, node := range response.Node.Nodes {
+			var svc api.Service
+			err = json.Unmarshal([]byte(node.Value), &svc)
+			if err != nil {
+				log.Printf("Failed to load Service: %s (%#v)", node.Value, err)
+				continue
+			}
+			retServices[i] = svc
+			endpoints, err := impl.GetEndpoints(svc.ID)
+			if err != nil {
+				log.Printf("Couldn't get endpoints for %s : %v skipping", svc.ID, err)
+			}
+			log.Printf("Got service: %s on localport %d mapping to: %s", svc.ID, svc.Port, endpoints)
+			retEndpoints[i] = endpoints
+		}
+		return retServices, retEndpoints, err
+	}
+	return nil, nil, fmt.Errorf("did not get the root of the registry %s", RegistryRoot)
+}
+
+func (impl ConfigSourceEtcd) GetEndpoints(service string) (api.Endpoints, error) {
+	key := fmt.Sprintf(RegistryRoot + "/endpoints/" + service)
+	response, err := impl.client.Get(key, true, false)
+	if err != nil {
+		log.Printf("Failed to get the key: %s %v", key, err)
+		return api.Endpoints{}, err
+	}
+	// Parse all the endpoint specifications in this value.
+	return ParseEndpoints(response.Node.Value)
+}
+
+// EtcdResponseToServiceAndLocalport takes an etcd response and pulls it apart to find
+// service
+func EtcdResponseToService(response *etcd.Response) (*api.Service, error) {
+	if response.Node == nil {
+		return nil, fmt.Errorf("invalid response from etcd: %#v", response)
+	}
+	var svc api.Service
+	err := json.Unmarshal([]byte(response.Node.Value), &svc)
+	if err != nil {
+		return nil, err
+	}
+	return &svc, err
+}
+
+func ParseEndpoints(jsonString string) (api.Endpoints, error) {
+	var e api.Endpoints
+	err := json.Unmarshal([]byte(jsonString), &e)
+	return e, err
+}
+
+func (impl ConfigSourceEtcd) WatchForChanges() {
+	log.Print("Setting up a watch for new services")
+	watchChannel := make(chan *etcd.Response)
+	go impl.client.Watch("/registry/services/", 0, true, watchChannel, nil)
+	for {
+		watchResponse := <-watchChannel
+		impl.ProcessChange(watchResponse)
+	}
+}
+
+func (impl ConfigSourceEtcd) ProcessChange(response *etcd.Response) {
+	log.Printf("Processing a change in service configuration... %s", *response)
+
+	// If it's a new service being added (signified by a localport being added)
+	// then process it as such
+	if strings.Contains(response.Node.Key, "/endpoints/") {
+		impl.ProcessEndpointResponse(response)
+	} else if response.Action == "set" {
+		service, err := EtcdResponseToService(response)
+		if err != nil {
+			log.Printf("Failed to parse %s Port: %s", response, err)
+			return
+		}
+
+		log.Printf("New service added/updated: %#v", service)
+		serviceUpdate := ServiceUpdate{Op: ADD, Services: []api.Service{*service}}
+		impl.serviceChannel <- serviceUpdate
+		return
+	}
+	if response.Action == "delete" {
+		parts := strings.Split(response.Node.Key[1:], "/")
+		if len(parts) == 4 {
+			log.Printf("Deleting service: %s", parts[3])
+			serviceUpdate := ServiceUpdate{Op: REMOVE, Services: []api.Service{api.Service{JSONBase: api.JSONBase{ID: parts[3]}}}}
+			impl.serviceChannel <- serviceUpdate
+			return
+		} else {
+			log.Printf("Unknown service delete: %#v", parts)
+		}
+	}
+}
+
+func (impl ConfigSourceEtcd) ProcessEndpointResponse(response *etcd.Response) {
+	log.Printf("Processing a change in endpoint configuration... %s", *response)
+	var endpoints api.Endpoints
+	err := json.Unmarshal([]byte(response.Node.Value), &endpoints)
+	if err != nil {
+		log.Printf("Failed to parse service out of etcd key: %v : %+v", response.Node.Value, err)
+		return
+	}
+	endpointsUpdate := EndpointsUpdate{Op: ADD, Endpoints: []api.Endpoints{endpoints}}
+	impl.endpointsChannel <- endpointsUpdate
+}

pkg/proxy/config/etcd_test.go

@@ -0,0 +1,56 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package config
+
+import (
+	"encoding/json"
+	"reflect"
+	"testing"
+
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/api"
+)
+
+const TomcatContainerEtcdKey = "/registry/services/tomcat/endpoints/tomcat-3bd5af34"
+const TomcatService = "tomcat"
+const TomcatContainerId = "tomcat-3bd5af34"
+
+func ValidateJsonParsing(t *testing.T, jsonString string, expectedEndpoints api.Endpoints, expectError bool) {
+	endpoints, err := ParseEndpoints(jsonString)
+	if err == nil && expectError {
+		t.Errorf("ValidateJsonParsing did not get expected error when parsing %s", jsonString)
+	}
+	if err != nil && !expectError {
+		t.Errorf("ValidateJsonParsing got unexpected error %+v when parsing %s", err, jsonString)
+	}
+	if !reflect.DeepEqual(expectedEndpoints, endpoints) {
+		t.Errorf("Didn't get expected endpoints %+v got: %+v", expectedEndpoints, endpoints)
+	}
+}
+
+func TestParseJsonEndpoints(t *testing.T) {
+	ValidateJsonParsing(t, "", api.Endpoints{}, true)
+	endpoints := api.Endpoints{
+		Name:      "foo",
+		Endpoints: []string{"foo", "bar", "baz"},
+	}
+	data, err := json.Marshal(endpoints)
+	if err != nil {
+		t.Errorf("Unexpected error: %#v", err)
+	}
+	ValidateJsonParsing(t, string(data), endpoints, false)
+	//	ValidateJsonParsing(t, "[{\"port\":8000,\"name\":\"mysql\",\"machine\":\"foo\"},{\"port\":9000,\"name\":\"mysql\",\"machine\":\"bar\"}]", []string{"foo:8000", "bar:9000"}, false)
+}

pkg/proxy/config/file.go

@@ -0,0 +1,111 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+// Reads the configuration from the file. Example file for two services [nodejs & mysql]
+//{"Services": [
+//   {
+//      "Name":"nodejs",
+//      "Port":10000,
+//      "Endpoints":["10.240.180.168:8000", "10.240.254.199:8000", "10.240.62.150:8000"]
+//   },
+//   {
+//      "Name":"mysql",
+//      "Port":10001,
+//      "Endpoints":["10.240.180.168:9000", "10.240.254.199:9000", "10.240.62.150:9000"]
+//   }
+//]
+//}
+package config
+
+import (
+	"bytes"
+	"encoding/json"
+	"io/ioutil"
+	"log"
+	"reflect"
+	"time"
+
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/api"
+)
+
+// TODO: kill this struct.
+type ServiceJSON struct {
+	Name      string
+	Port      int
+	Endpoints []string
+}
+type ConfigFile struct {
+	Services []ServiceJSON
+}
+
+type ConfigSourceFile struct {
+	serviceChannel   chan ServiceUpdate
+	endpointsChannel chan EndpointsUpdate
+	filename         string
+}
+
+func NewConfigSourceFile(filename string, serviceChannel chan ServiceUpdate, endpointsChannel chan EndpointsUpdate) ConfigSourceFile {
+	config := ConfigSourceFile{
+		filename:         filename,
+		serviceChannel:   serviceChannel,
+		endpointsChannel: endpointsChannel,
+	}
+	go config.Run()
+	return config
+}
+
+func (impl ConfigSourceFile) Run() {
+	log.Printf("Watching file %s", impl.filename)
+	var lastData []byte
+	var lastServices []api.Service
+	var lastEndpoints []api.Endpoints
+
+	for {
+		data, err := ioutil.ReadFile(impl.filename)
+		if err != nil {
+			log.Printf("Couldn't read file: %s : %v", impl.filename, err)
+		} else {
+			var config ConfigFile
+			err = json.Unmarshal(data, &config)
+			if err != nil {
+				log.Printf("Couldn't unmarshal configuration from file : %s %v", data, err)
+			} else {
+				if !bytes.Equal(lastData, data) {
+					lastData = data
+					// Ok, we have a valid configuration, send to channel for
+					// rejiggering.
+					newServices := make([]api.Service, len(config.Services))
+					newEndpoints := make([]api.Endpoints, len(config.Services))
+					for i, service := range config.Services {
+						newServices[i] = api.Service{JSONBase: api.JSONBase{ID: service.Name}, Port: service.Port}
+						newEndpoints[i] = api.Endpoints{Name: service.Name, Endpoints: service.Endpoints}
+					}
+					if !reflect.DeepEqual(lastServices, newServices) {
+						serviceUpdate := ServiceUpdate{Op: SET, Services: newServices}
+						impl.serviceChannel <- serviceUpdate
+						lastServices = newServices
+					}
+					if !reflect.DeepEqual(lastEndpoints, newEndpoints) {
+						endpointsUpdate := EndpointsUpdate{Op: SET, Endpoints: newEndpoints}
+						impl.endpointsChannel <- endpointsUpdate
+						lastEndpoints = newEndpoints
+					}
+				}
+			}
+		}
+		time.Sleep(5 * time.Second)
+	}
+}

pkg/proxy/loadbalancer.go

@@ -0,0 +1,29 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+// Loadbalancer interface. Implementations use loadbalancer_<strategy> naming.
+
+package proxy
+
+import (
+	"net"
+)
+
+type LoadBalancer interface {
+	// LoadBalance takes an incoming request and figures out where to route it to.
+	// Determination is based on destination service (for example, 'mysql') as
+	// well as the source making the connection.
+	LoadBalance(service string, srcAddr net.Addr) (string, error)
+}

pkg/proxy/proxier.go

@@ -0,0 +1,117 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+// Simple proxy for tcp connections between a localhost:lport and services that provide
+// the actual implementations.
+
+package proxy
+
+import (
+	"fmt"
+	"io"
+	"log"
+	"net"
+	"time"
+
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/api"
+)
+
+type Proxier struct {
+	loadBalancer LoadBalancer
+	serviceMap   map[string]int
+}
+
+func NewProxier(loadBalancer LoadBalancer) *Proxier {
+	return &Proxier{loadBalancer: loadBalancer, serviceMap: make(map[string]int)}
+}
+
+func CopyBytes(in, out *net.TCPConn) {
+	log.Printf("Copying from %v <-> %v <-> %v <-> %v",
+		in.RemoteAddr(), in.LocalAddr(), out.LocalAddr(), out.RemoteAddr())
+	_, err := io.Copy(in, out)
+	if err != nil && err != io.EOF {
+		log.Printf("I/O error: %v", err)
+	}
+
+	in.CloseRead()
+	out.CloseWrite()
+}
+
+// Create a bidirectional byte shuffler. Copies bytes to/from each connection.
+func ProxyConnection(in, out *net.TCPConn) {
+	log.Printf("Creating proxy between %v <-> %v <-> %v <-> %v",
+		in.RemoteAddr(), in.LocalAddr(), out.LocalAddr(), out.RemoteAddr())
+	go CopyBytes(in, out)
+	go CopyBytes(out, in)
+}
+
+func (proxier Proxier) AcceptHandler(service string, listener net.Listener) {
+	for {
+		inConn, err := listener.Accept()
+		if err != nil {
+			log.Printf("Accept failed: %v", err)
+			continue
+		}
+		log.Printf("Accepted connection from: %v to %v", inConn.RemoteAddr(), inConn.LocalAddr())
+
+		// Figure out where this request should go.
+		endpoint, err := proxier.loadBalancer.LoadBalance(service, inConn.RemoteAddr())
+		if err != nil {
+			log.Printf("Couldn't find an endpoint for %s %v", service, err)
+			inConn.Close()
+			continue
+		}
+
+		log.Printf("Mapped service %s to endpoint %s", service, endpoint)
+		outConn, err := net.DialTimeout("tcp", endpoint, time.Duration(5)*time.Second)
+		// We basically need to take everything from inConn and send to outConn
+		// and anything coming from outConn needs to be sent to inConn.
+		if err != nil {
+			log.Printf("Dial failed: %v", err)
+			inConn.Close()
+			continue
+		}
+		go ProxyConnection(inConn.(*net.TCPConn), outConn.(*net.TCPConn))
+	}
+}
+
+// AddService starts listening for a new service on a given port.
+func (proxier Proxier) AddService(service string, port int) error {
+	// Make sure we can start listening on the port before saying all's well.
+	ln, err := net.Listen("tcp", fmt.Sprintf(":%d", port))
+	if err != nil {
+		return err
+	}
+	log.Printf("Listening for %s on %d", service, port)
+	// If that succeeds, start the accepting loop.
+	go proxier.AcceptHandler(service, ln)
+	return nil
+}
+
+func (proxier Proxier) OnUpdate(services []api.Service) {
+	log.Printf("Received update notice: %+v", services)
+	for _, service := range services {
+		port, exists := proxier.serviceMap[service.ID]
+		if !exists || port != service.Port {
+			log.Printf("Adding a new service %s on port %d", service.ID, service.Port)
+			err := proxier.AddService(service.ID, service.Port)
+			if err == nil {
+				proxier.serviceMap[service.ID] = service.Port
+			} else {
+				log.Printf("Failed to start listening for %s on %d", service.ID, service.Port)
+			}
+		}
+	}
+}

pkg/proxy/proxier_test.go

@@ -0,0 +1,73 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package proxy
+
+import (
+	"fmt"
+	"io"
+	"net"
+	"testing"
+
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/api"
+)
+
+// a simple echoServer that only accept one connection
+func echoServer(addr string) error {
+	l, err := net.Listen("tcp", addr)
+	if err != nil {
+		return fmt.Errorf("failed to start echo service: %v", err)
+	}
+	defer l.Close()
+	conn, err := l.Accept()
+	if err != nil {
+		return fmt.Errorf("failed to accept new conn to echo service: %v", err)
+	}
+	io.Copy(conn, conn)
+	conn.Close()
+	return nil
+}
+
+func TestProxy(t *testing.T) {
+	go func() {
+		if err := echoServer("127.0.0.1:2222"); err != nil {
+			t.Fatal(err)
+		}
+	}()
+
+	lb := NewLoadBalancerRR()
+	lb.OnUpdate([]api.Endpoints{{"echo", []string{"127.0.0.1:2222"}}})
+
+	p := NewProxier(lb)
+	if err := p.AddService("echo", 2223); err != nil {
+		t.Fatalf("error adding new service: %v", err)
+	}
+	conn, err := net.Dial("tcp", "127.0.0.1:2223")
+	if err != nil {
+		t.Fatalf("error connecting to proxy: %v", err)
+	}
+	magic := "aaaaa"
+	if _, err := conn.Write([]byte(magic)); err != nil {
+		t.Fatalf("error writing to proxy: %v", err)
+	}
+	buf := make([]byte, 5)
+	if _, err := conn.Read(buf); err != nil {
+		t.Fatalf("error reading from proxy: %v", err)
+	}
+	if string(buf) != magic {
+		t.Fatalf("bad echo from proxy: got: %q, expected %q", string(buf), magic)
+	}
+}

pkg/proxy/roundrobbin.go

@@ -0,0 +1,103 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+// RoundRobin Loadbalancer
+
+package proxy
+
+import (
+	"errors"
+	"log"
+	"net"
+	"reflect"
+	"strconv"
+	"strings"
+	"sync"
+
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/api"
+)
+
+type LoadBalancerRR struct {
+	lock         sync.RWMutex
+	endpointsMap map[string][]string
+	rrIndex      map[string]int
+}
+
+func NewLoadBalancerRR() *LoadBalancerRR {
+	return &LoadBalancerRR{endpointsMap: make(map[string][]string), rrIndex: make(map[string]int)}
+}
+
+func (impl LoadBalancerRR) LoadBalance(service string, srcAddr net.Addr) (string, error) {
+	impl.lock.RLock()
+	endpoints, exists := impl.endpointsMap[service]
+	index := impl.rrIndex[service]
+	impl.lock.RUnlock()
+	if exists == false {
+		return "", errors.New("no service entry for:" + service)
+	}
+	if len(endpoints) == 0 {
+		return "", errors.New("no endpoints for: " + service)
+	}
+	endpoint := endpoints[index]
+	impl.rrIndex[service] = (index + 1) % len(endpoints)
+	return endpoint, nil
+}
+
+func (impl LoadBalancerRR) IsValid(spec string) bool {
+	index := strings.Index(spec, ":")
+	if index == -1 {
+		return false
+	}
+	value, err := strconv.Atoi(spec[index+1:])
+	if err != nil {
+		return false
+	}
+	return value > 0
+}
+
+func (impl LoadBalancerRR) FilterValidEndpoints(endpoints []string) []string {
+	var result []string
+	for _, spec := range endpoints {
+		if impl.IsValid(spec) {
+			result = append(result, spec)
+		}
+	}
+	return result
+}
+
+func (impl LoadBalancerRR) OnUpdate(endpoints []api.Endpoints) {
+	tmp := make(map[string]bool)
+	impl.lock.Lock()
+	defer impl.lock.Unlock()
+	// First update / add all new endpoints for services.
+	for _, value := range endpoints {
+		existingEndpoints, exists := impl.endpointsMap[value.Name]
+		if !exists || !reflect.DeepEqual(value.Endpoints, existingEndpoints) {
+			log.Printf("LoadBalancerRR: Setting endpoints for %s to %+v", value.Name, value.Endpoints)
+			impl.endpointsMap[value.Name] = impl.FilterValidEndpoints(value.Endpoints)
+			// Start RR from the beginning if added or updated.
+			impl.rrIndex[value.Name] = 0
+		}
+		tmp[value.Name] = true
+	}
+	// Then remove any endpoints no longer relevant
+	for key, value := range impl.endpointsMap {
+		_, exists := tmp[key]
+		if !exists {
+			log.Printf("LoadBalancerRR: Removing endpoints for %s -> %+v", key, value)
+			delete(impl.endpointsMap, key)
+		}
+	}
+}

pkg/proxy/roundrobbin_test.go

@@ -0,0 +1,178 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+package proxy
+
+import (
+	"testing"
+
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/api"
+)
+
+func TestLoadBalanceValidateWorks(t *testing.T) {
+	loadBalancer := NewLoadBalancerRR()
+	if loadBalancer.IsValid("") {
+		t.Errorf("Didn't fail for empty string")
+	}
+	if loadBalancer.IsValid("foobar") {
+		t.Errorf("Didn't fail with no port")
+	}
+	if loadBalancer.IsValid("foobar:-1") {
+		t.Errorf("Didn't fail with a negative port")
+	}
+	if !loadBalancer.IsValid("foobar:8080") {
+		t.Errorf("Failed a valid config.")
+	}
+}
+
+func TestLoadBalanceFilterWorks(t *testing.T) {
+	loadBalancer := NewLoadBalancerRR()
+	endpoints := []string{"foobar:1", "foobar:2", "foobar:-1", "foobar:3", "foobar:-2"}
+	filtered := loadBalancer.FilterValidEndpoints(endpoints)
+
+	if len(filtered) != 3 {
+		t.Errorf("Failed to filter to the correct size")
+	}
+	if filtered[0] != "foobar:1" {
+		t.Errorf("Index zero is not foobar:1")
+	}
+	if filtered[1] != "foobar:2" {
+		t.Errorf("Index one is not foobar:2")
+	}
+	if filtered[2] != "foobar:3" {
+		t.Errorf("Index two is not foobar:3")
+	}
+}
+
+func TestLoadBalanceFailsWithNoEndpoints(t *testing.T) {
+	loadBalancer := NewLoadBalancerRR()
+	endpoints := make([]api.Endpoints, 0)
+	loadBalancer.OnUpdate(endpoints)
+	endpoint, err := loadBalancer.LoadBalance("foo", nil)
+	if err == nil {
+		t.Errorf("Didn't fail with non-existent service")
+	}
+	if len(endpoint) != 0 {
+		t.Errorf("Got an endpoint")
+	}
+}
+
+func expectEndpoint(t *testing.T, loadBalancer *LoadBalancerRR, service string, expected string) {
+	endpoint, err := loadBalancer.LoadBalance(service, nil)
+	if err != nil {
+		t.Errorf("Didn't find a service for %s, expected %s, failed with: %v", service, expected, err)
+	}
+	if endpoint != expected {
+		t.Errorf("Didn't get expected endpoint for service %s, expected %s, got: %s", service, expected, endpoint)
+	}
+}
+
+func TestLoadBalanceWorksWithSingleEndpoint(t *testing.T) {
+	loadBalancer := NewLoadBalancerRR()
+	endpoint, err := loadBalancer.LoadBalance("foo", nil)
+	if err == nil || len(endpoint) != 0 {
+		t.Errorf("Didn't fail with non-existent service")
+	}
+	endpoints := make([]api.Endpoints, 1)
+	endpoints[0] = api.Endpoints{Name: "foo", Endpoints: []string{"endpoint1:40"}}
+	loadBalancer.OnUpdate(endpoints)
+	expectEndpoint(t, loadBalancer, "foo", "endpoint1:40")
+	expectEndpoint(t, loadBalancer, "foo", "endpoint1:40")
+	expectEndpoint(t, loadBalancer, "foo", "endpoint1:40")
+	expectEndpoint(t, loadBalancer, "foo", "endpoint1:40")
+}
+
+func TestLoadBalanceWorksWithMultipleEndpoints(t *testing.T) {
+	loadBalancer := NewLoadBalancerRR()
+	endpoint, err := loadBalancer.LoadBalance("foo", nil)
+	if err == nil || len(endpoint) != 0 {
+		t.Errorf("Didn't fail with non-existent service")
+	}
+	endpoints := make([]api.Endpoints, 1)
+	endpoints[0] = api.Endpoints{Name: "foo", Endpoints: []string{"endpoint:1", "endpoint:2", "endpoint:3"}}
+	loadBalancer.OnUpdate(endpoints)
+	expectEndpoint(t, loadBalancer, "foo", "endpoint:1")
+	expectEndpoint(t, loadBalancer, "foo", "endpoint:2")
+	expectEndpoint(t, loadBalancer, "foo", "endpoint:3")
+	expectEndpoint(t, loadBalancer, "foo", "endpoint:1")
+}
+
+func TestLoadBalanceWorksWithMultipleEndpointsAndUpdates(t *testing.T) {
+	loadBalancer := NewLoadBalancerRR()
+	endpoint, err := loadBalancer.LoadBalance("foo", nil)
+	if err == nil || len(endpoint) != 0 {
+		t.Errorf("Didn't fail with non-existent service")
+	}
+	endpoints := make([]api.Endpoints, 1)
+	endpoints[0] = api.Endpoints{Name: "foo", Endpoints: []string{"endpoint:1", "endpoint:2", "endpoint:3"}}
+	loadBalancer.OnUpdate(endpoints)
+	expectEndpoint(t, loadBalancer, "foo", "endpoint:1")
+	expectEndpoint(t, loadBalancer, "foo", "endpoint:2")
+	expectEndpoint(t, loadBalancer, "foo", "endpoint:3")
+	expectEndpoint(t, loadBalancer, "foo", "endpoint:1")
+	expectEndpoint(t, loadBalancer, "foo", "endpoint:2")
+	// Then update the configuration with one fewer endpoints, make sure
+	// we start in the beginning again
+	endpoints[0] = api.Endpoints{Name: "foo", Endpoints: []string{"endpoint:8", "endpoint:9"}}
+	loadBalancer.OnUpdate(endpoints)
+	expectEndpoint(t, loadBalancer, "foo", "endpoint:8")
+	expectEndpoint(t, loadBalancer, "foo", "endpoint:9")
+	expectEndpoint(t, loadBalancer, "foo", "endpoint:8")
+	expectEndpoint(t, loadBalancer, "foo", "endpoint:9")
+	// Clear endpoints
+	endpoints[0] = api.Endpoints{Name: "foo", Endpoints: []string{}}
+	loadBalancer.OnUpdate(endpoints)
+
+	endpoint, err = loadBalancer.LoadBalance("foo", nil)
+	if err == nil || len(endpoint) != 0 {
+		t.Errorf("Didn't fail with non-existent service")
+	}
+}
+
+func TestLoadBalanceWorksWithServiceRemoval(t *testing.T) {
+	loadBalancer := NewLoadBalancerRR()
+	endpoint, err := loadBalancer.LoadBalance("foo", nil)
+	if err == nil || len(endpoint) != 0 {
+		t.Errorf("Didn't fail with non-existent service")
+	}
+	endpoints := make([]api.Endpoints, 2)
+	endpoints[0] = api.Endpoints{Name: "foo", Endpoints: []string{"endpoint:1", "endpoint:2", "endpoint:3"}}
+	endpoints[1] = api.Endpoints{Name: "bar", Endpoints: []string{"endpoint:4", "endpoint:5"}}
+	loadBalancer.OnUpdate(endpoints)
+	expectEndpoint(t, loadBalancer, "foo", "endpoint:1")
+	expectEndpoint(t, loadBalancer, "foo", "endpoint:2")
+	expectEndpoint(t, loadBalancer, "foo", "endpoint:3")
+	expectEndpoint(t, loadBalancer, "foo", "endpoint:1")
+	expectEndpoint(t, loadBalancer, "foo", "endpoint:2")
+
+	expectEndpoint(t, loadBalancer, "bar", "endpoint:4")
+	expectEndpoint(t, loadBalancer, "bar", "endpoint:5")
+	expectEndpoint(t, loadBalancer, "bar", "endpoint:4")
+	expectEndpoint(t, loadBalancer, "bar", "endpoint:5")
+	expectEndpoint(t, loadBalancer, "bar", "endpoint:4")
+
+	// Then update the configuration by removing foo
+	loadBalancer.OnUpdate(endpoints[1:])
+	endpoint, err = loadBalancer.LoadBalance("foo", nil)
+	if err == nil || len(endpoint) != 0 {
+		t.Errorf("Didn't fail with non-existent service")
+	}
+
+	// but bar is still there, and we continue RR from where we left off.
+	expectEndpoint(t, loadBalancer, "bar", "endpoint:5")
+	expectEndpoint(t, loadBalancer, "bar", "endpoint:4")
+	expectEndpoint(t, loadBalancer, "bar", "endpoint:5")
+	expectEndpoint(t, loadBalancer, "bar", "endpoint:4")
+}

pkg/registry/controller_registry.go

@@ -0,0 +1,68 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+package registry
+
+import (
+	"encoding/json"
+	"net/url"
+
+	. "github.com/GoogleCloudPlatform/kubernetes/pkg/api"
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/apiserver"
+)
+
+// Implementation of RESTStorage for the api server.
+type ControllerRegistryStorage struct {
+	registry ControllerRegistry
+}
+
+func MakeControllerRegistryStorage(registry ControllerRegistry) apiserver.RESTStorage {
+	return &ControllerRegistryStorage{
+		registry: registry,
+	}
+}
+
+func (storage *ControllerRegistryStorage) List(*url.URL) (interface{}, error) {
+	var result ReplicationControllerList
+	controllers, err := storage.registry.ListControllers()
+	if err == nil {
+		result = ReplicationControllerList{
+			Items: controllers,
+		}
+	}
+	return result, err
+}
+
+func (storage *ControllerRegistryStorage) Get(id string) (interface{}, error) {
+	return storage.registry.GetController(id)
+}
+
+func (storage *ControllerRegistryStorage) Delete(id string) error {
+	return storage.registry.DeleteController(id)
+}
+
+func (storage *ControllerRegistryStorage) Extract(body string) (interface{}, error) {
+	result := ReplicationController{}
+	err := json.Unmarshal([]byte(body), &result)
+	return result, err
+}
+
+func (storage *ControllerRegistryStorage) Create(controller interface{}) error {
+	return storage.registry.CreateController(controller.(ReplicationController))
+}
+
+func (storage *ControllerRegistryStorage) Update(controller interface{}) error {
+	return storage.registry.UpdateController(controller.(ReplicationController))
+}

pkg/registry/controller_registry_test.go

@@ -0,0 +1,187 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+package registry
+
+import (
+	"encoding/json"
+	"fmt"
+	"io/ioutil"
+	"reflect"
+	"testing"
+
+	. "github.com/GoogleCloudPlatform/kubernetes/pkg/api"
+)
+
+type MockControllerRegistry struct {
+	err         error
+	controllers []ReplicationController
+}
+
+func (registry *MockControllerRegistry) ListControllers() ([]ReplicationController, error) {
+	return registry.controllers, registry.err
+}
+
+func (registry *MockControllerRegistry) GetController(ID string) (*ReplicationController, error) {
+	return &ReplicationController{}, registry.err
+}
+
+func (registry *MockControllerRegistry) CreateController(controller ReplicationController) error {
+	return registry.err
+}
+
+func (registry *MockControllerRegistry) UpdateController(controller ReplicationController) error {
+	return registry.err
+}
+func (registry *MockControllerRegistry) DeleteController(ID string) error {
+	return registry.err
+}
+
+func TestListControllersError(t *testing.T) {
+	mockRegistry := MockControllerRegistry{
+		err: fmt.Errorf("Test Error"),
+	}
+	storage := ControllerRegistryStorage{
+		registry: &mockRegistry,
+	}
+	controllersObj, err := storage.List(nil)
+	controllers := controllersObj.(ReplicationControllerList)
+	if err != mockRegistry.err {
+		t.Errorf("Expected %#v, Got %#v", mockRegistry.err, err)
+	}
+	if len(controllers.Items) != 0 {
+		t.Errorf("Unexpected non-zero task list: %#v", controllers)
+	}
+}
+
+func TestListEmptyControllerList(t *testing.T) {
+	mockRegistry := MockControllerRegistry{}
+	storage := ControllerRegistryStorage{
+		registry: &mockRegistry,
+	}
+	controllers, err := storage.List(nil)
+	expectNoError(t, err)
+	if len(controllers.(ReplicationControllerList).Items) != 0 {
+		t.Errorf("Unexpected non-zero task list: %#v", controllers)
+	}
+}
+
+func TestListControllerList(t *testing.T) {
+	mockRegistry := MockControllerRegistry{
+		controllers: []ReplicationController{
+			ReplicationController{
+				JSONBase: JSONBase{
+					ID: "foo",
+				},
+			},
+			ReplicationController{
+				JSONBase: JSONBase{
+					ID: "bar",
+				},
+			},
+		},
+	}
+	storage := ControllerRegistryStorage{
+		registry: &mockRegistry,
+	}
+	controllersObj, err := storage.List(nil)
+	controllers := controllersObj.(ReplicationControllerList)
+	expectNoError(t, err)
+	if len(controllers.Items) != 2 {
+		t.Errorf("Unexpected controller list: %#v", controllers)
+	}
+	if controllers.Items[0].ID != "foo" {
+		t.Errorf("Unexpected controller: %#v", controllers.Items[0])
+	}
+	if controllers.Items[1].ID != "bar" {
+		t.Errorf("Unexpected controller: %#v", controllers.Items[1])
+	}
+}
+
+func TestExtractControllerJson(t *testing.T) {
+	mockRegistry := MockControllerRegistry{}
+	storage := ControllerRegistryStorage{
+		registry: &mockRegistry,
+	}
+	controller := ReplicationController{
+		JSONBase: JSONBase{
+			ID: "foo",
+		},
+	}
+	body, err := json.Marshal(controller)
+	expectNoError(t, err)
+	controllerOut, err := storage.Extract(string(body))
+	expectNoError(t, err)
+	jsonOut, err := json.Marshal(controllerOut)
+	expectNoError(t, err)
+	if string(body) != string(jsonOut) {
+		t.Errorf("Expected %#v, found %#v", controller, controllerOut)
+	}
+}
+
+func TestControllerParsing(t *testing.T) {
+	expectedController := ReplicationController{
+		JSONBase: JSONBase{
+			ID: "nginxController",
+		},
+		DesiredState: ReplicationControllerState{
+			Replicas: 2,
+			ReplicasInSet: map[string]string{
+				"name": "nginx",
+			},
+			TaskTemplate: TaskTemplate{
+				DesiredState: TaskState{
+					Manifest: ContainerManifest{
+						Containers: []Container{
+							Container{
+								Image: "dockerfile/nginx",
+								Ports: []Port{
+									Port{
+										ContainerPort: 80,
+										HostPort:      8080,
+									},
+								},
+							},
+						},
+					},
+				},
+				Labels: map[string]string{
+					"name": "nginx",
+				},
+			},
+		},
+		Labels: map[string]string{
+			"name": "nginx",
+		},
+	}
+	file, err := ioutil.TempFile("", "controller")
+	fileName := file.Name()
+	expectNoError(t, err)
+	data, err := json.Marshal(expectedController)
+	expectNoError(t, err)
+	_, err = file.Write(data)
+	expectNoError(t, err)
+	err = file.Close()
+	expectNoError(t, err)
+	data, err = ioutil.ReadFile(fileName)
+	expectNoError(t, err)
+	var controller ReplicationController
+	err = json.Unmarshal(data, &controller)
+	expectNoError(t, err)
+
+	if !reflect.DeepEqual(controller, expectedController) {
+		t.Errorf("Parsing failed: %s %#v %#v", string(data), controller, expectedController)
+	}
+}

pkg/registry/endpoints.go

@@ -0,0 +1,65 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+package registry
+
+import (
+	"fmt"
+	"log"
+
+	. "github.com/GoogleCloudPlatform/kubernetes/pkg/api"
+)
+
+func MakeEndpointController(serviceRegistry ServiceRegistry, taskRegistry TaskRegistry) *EndpointController {
+	return &EndpointController{
+		serviceRegistry: serviceRegistry,
+		taskRegistry:    taskRegistry,
+	}
+}
+
+type EndpointController struct {
+	serviceRegistry ServiceRegistry
+	taskRegistry    TaskRegistry
+}
+
+func (e *EndpointController) SyncServiceEndpoints() error {
+	services, err := e.serviceRegistry.ListServices()
+	if err != nil {
+		return err
+	}
+	var resultErr error
+	for _, service := range services.Items {
+		tasks, err := e.taskRegistry.ListTasks(&service.Labels)
+		if err != nil {
+			log.Printf("Error syncing service: %#v, skipping.", service)
+			resultErr = err
+			continue
+		}
+		endpoints := make([]string, len(tasks))
+		for ix, task := range tasks {
+			// TODO: Use port names in the service object, don't just use port #0
+			endpoints[ix] = fmt.Sprintf("%s:%d", task.CurrentState.Host, task.DesiredState.Manifest.Containers[0].Ports[0].HostPort)
+		}
+		err = e.serviceRegistry.UpdateEndpoints(Endpoints{
+			Name:      service.ID,
+			Endpoints: endpoints,
+		})
+		if err != nil {
+			log.Printf("Error updating endpoints: %#v", err)
+			continue
+		}
+	}
+	return resultErr
+}

pkg/registry/endpoints_test.go

@@ -0,0 +1,108 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+package registry
+
+import (
+	"fmt"
+	"testing"
+
+	. "github.com/GoogleCloudPlatform/kubernetes/pkg/api"
+)
+
+func TestSyncEndpointsEmpty(t *testing.T) {
+	serviceRegistry := MockServiceRegistry{}
+	taskRegistry := MockTaskRegistry{}
+
+	endpoints := MakeEndpointController(&serviceRegistry, &taskRegistry)
+	err := endpoints.SyncServiceEndpoints()
+	expectNoError(t, err)
+}
+
+func TestSyncEndpointsError(t *testing.T) {
+	serviceRegistry := MockServiceRegistry{
+		err: fmt.Errorf("Test Error"),
+	}
+	taskRegistry := MockTaskRegistry{}
+
+	endpoints := MakeEndpointController(&serviceRegistry, &taskRegistry)
+	err := endpoints.SyncServiceEndpoints()
+	if err != serviceRegistry.err {
+		t.Errorf("Errors don't match: %#v %#v", err, serviceRegistry.err)
+	}
+}
+
+func TestSyncEndpointsItems(t *testing.T) {
+	serviceRegistry := MockServiceRegistry{
+		list: ServiceList{
+			Items: []Service{
+				Service{
+					Labels: map[string]string{
+						"foo": "bar",
+					},
+				},
+			},
+		},
+	}
+	taskRegistry := MockTaskRegistry{
+		tasks: []Task{
+			Task{
+				DesiredState: TaskState{
+					Manifest: ContainerManifest{
+						Containers: []Container{
+							Container{
+								Ports: []Port{
+									Port{
+										HostPort: 8080,
+									},
+								},
+							},
+						},
+					},
+				},
+			},
+		},
+	}
+
+	endpoints := MakeEndpointController(&serviceRegistry, &taskRegistry)
+	err := endpoints.SyncServiceEndpoints()
+	expectNoError(t, err)
+	if len(serviceRegistry.endpoints.Endpoints) != 1 {
+		t.Errorf("Unexpected endpoints update: %#v", serviceRegistry.endpoints)
+	}
+}
+
+func TestSyncEndpointsTaskError(t *testing.T) {
+	serviceRegistry := MockServiceRegistry{
+		list: ServiceList{
+			Items: []Service{
+				Service{
+					Labels: map[string]string{
+						"foo": "bar",
+					},
+				},
+			},
+		},
+	}
+	taskRegistry := MockTaskRegistry{
+		err: fmt.Errorf("test error."),
+	}
+
+	endpoints := MakeEndpointController(&serviceRegistry, &taskRegistry)
+	err := endpoints.SyncServiceEndpoints()
+	if err == nil {
+		t.Error("Unexpected non-error")
+	}
+}

pkg/registry/etcd_registry.go

@@ -0,0 +1,392 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+package registry
+
+import (
+	"encoding/json"
+	"fmt"
+	"log"
+
+	"github.com/coreos/go-etcd/etcd"
+
+	. "github.com/GoogleCloudPlatform/kubernetes/pkg/api"
+)
+
+// TODO: Need to add a reconciler loop that makes sure that things in tasks are reflected into
+//       kubelet (and vice versa)
+
+// EtcdClient is an injectable interface for testing.
+type EtcdClient interface {
+	AddChild(key, data string, ttl uint64) (*etcd.Response, error)
+	Get(key string, sort, recursive bool) (*etcd.Response, error)
+	Set(key, value string, ttl uint64) (*etcd.Response, error)
+	Create(key, value string, ttl uint64) (*etcd.Response, error)
+	Delete(key string, recursive bool) (*etcd.Response, error)
+	// I'd like to use directional channels here (e.g. <-chan) but this interface mimics
+	// the etcd client interface which doesn't, and it doesn't seem worth it to wrap the api.
+	Watch(prefix string, waitIndex uint64, recursive bool, receiver chan *etcd.Response, stop chan bool) (*etcd.Response, error)
+}
+
+// EtcdRegistry is an implementation of both ControllerRegistry and TaskRegistry which is backed with etcd.
+type EtcdRegistry struct {
+	etcdClient      EtcdClient
+	machines        []string
+	manifestFactory ManifestFactory
+}
+
+// MakeEtcdRegistry creates an etcd registry.
+// 'client' is the connection to etcd
+// 'machines' is the list of machines
+// 'scheduler' is the scheduling algorithm to use.
+func MakeEtcdRegistry(client EtcdClient, machines []string) *EtcdRegistry {
+	registry := &EtcdRegistry{
+		etcdClient: client,
+		machines:   machines,
+	}
+	registry.manifestFactory = &BasicManifestFactory{
+		serviceRegistry: registry,
+	}
+	return registry
+}
+
+func makeTaskKey(machine, taskID string) string {
+	return "/registry/hosts/" + machine + "/tasks/" + taskID
+}
+
+func (registry *EtcdRegistry) ListTasks(query *map[string]string) ([]Task, error) {
+	tasks := []Task{}
+	for _, machine := range registry.machines {
+		machineTasks, err := registry.listTasksForMachine(machine)
+		if err != nil {
+			return tasks, err
+		}
+		for _, task := range machineTasks {
+			if LabelsMatch(task, query) {
+				tasks = append(tasks, task)
+			}
+		}
+	}
+	return tasks, nil
+}
+
+func (registry *EtcdRegistry) listEtcdNode(key string) ([]*etcd.Node, error) {
+	result, err := registry.etcdClient.Get(key, false, true)
+	if err != nil {
+		nodes := make([]*etcd.Node, 0)
+		if isEtcdNotFound(err) {
+			return nodes, nil
+		} else {
+			return nodes, err
+		}
+	}
+	return result.Node.Nodes, nil
+}
+
+func (registry *EtcdRegistry) listTasksForMachine(machine string) ([]Task, error) {
+	tasks := []Task{}
+	key := "/registry/hosts/" + machine + "/tasks"
+	nodes, err := registry.listEtcdNode(key)
+	for _, node := range nodes {
+		task := Task{}
+		err = json.Unmarshal([]byte(node.Value), &task)
+		if err != nil {
+			return tasks, err
+		}
+		task.CurrentState.Host = machine
+		tasks = append(tasks, task)
+	}
+	return tasks, err
+}
+
+func (registry *EtcdRegistry) GetTask(taskID string) (*Task, error) {
+	task, _, err := registry.findTask(taskID)
+	return &task, err
+}
+
+func makeContainerKey(machine string) string {
+	return "/registry/hosts/" + machine + "/kubelet"
+}
+
+func (registry *EtcdRegistry) loadManifests(machine string) ([]ContainerManifest, error) {
+	var manifests []ContainerManifest
+	response, err := registry.etcdClient.Get(makeContainerKey(machine), false, false)
+
+	if err != nil {
+		if isEtcdNotFound(err) {
+			err = nil
+			manifests = []ContainerManifest{}
+		}
+	} else {
+		err = json.Unmarshal([]byte(response.Node.Value), &manifests)
+	}
+	return manifests, err
+}
+
+func (registry *EtcdRegistry) updateManifests(machine string, manifests []ContainerManifest) error {
+	containerData, err := json.Marshal(manifests)
+	if err != nil {
+		return err
+	}
+	_, err = registry.etcdClient.Set(makeContainerKey(machine), string(containerData), 0)
+	return err
+}
+
+func (registry *EtcdRegistry) CreateTask(machineIn string, task Task) error {
+	taskOut, machine, err := registry.findTask(task.ID)
+	if err == nil {
+		return fmt.Errorf("A task named %s already exists on %s (%#v)", task.ID, machine, taskOut)
+	}
+	return registry.runTask(task, machineIn)
+}
+
+func (registry *EtcdRegistry) runTask(task Task, machine string) error {
+	manifests, err := registry.loadManifests(machine)
+	if err != nil {
+		return err
+	}
+
+	key := makeTaskKey(machine, task.ID)
+	data, err := json.Marshal(task)
+	if err != nil {
+		return err
+	}
+	_, err = registry.etcdClient.Create(key, string(data), 0)
+
+	manifest, err := registry.manifestFactory.MakeManifest(machine, task)
+	if err != nil {
+		return err
+	}
+	manifests = append(manifests, manifest)
+	return registry.updateManifests(machine, manifests)
+}
+
+func (registry *EtcdRegistry) UpdateTask(task Task) error {
+	return fmt.Errorf("Unimplemented!")
+}
+
+func (registry *EtcdRegistry) DeleteTask(taskID string) error {
+	_, machine, err := registry.findTask(taskID)
+	if err != nil {
+		return err
+	}
+	return registry.deleteTaskFromMachine(machine, taskID)
+}
+
+func (registry *EtcdRegistry) deleteTaskFromMachine(machine, taskID string) error {
+	manifests, err := registry.loadManifests(machine)
+	if err != nil {
+		return err
+	}
+	newManifests := make([]ContainerManifest, 0)
+	found := false
+	for _, manifest := range manifests {
+		if manifest.Id != taskID {
+			newManifests = append(newManifests, manifest)
+		} else {
+			found = true
+		}
+	}
+	if !found {
+		// This really shouldn't happen, it indicates something is broken, and likely
+		// there is a lost task somewhere.
+		// However it is "deleted" so log it and move on
+		log.Printf("Couldn't find: %s in %#v", taskID, manifests)
+	}
+	if err = registry.updateManifests(machine, newManifests); err != nil {
+		return err
+	}
+	key := makeTaskKey(machine, taskID)
+	_, err = registry.etcdClient.Delete(key, true)
+	return err
+}
+
+func (registry *EtcdRegistry) getTaskForMachine(machine, taskID string) (Task, error) {
+	key := makeTaskKey(machine, taskID)
+	result, err := registry.etcdClient.Get(key, false, false)
+	if err != nil {
+		if isEtcdNotFound(err) {
+			return Task{}, fmt.Errorf("Not found (%#v).", err)
+		} else {
+			return Task{}, err
+		}
+	}
+	if result.Node == nil || len(result.Node.Value) == 0 {
+		return Task{}, fmt.Errorf("no nodes field: %#v", result)
+	}
+	task := Task{}
+	err = json.Unmarshal([]byte(result.Node.Value), &task)
+	task.CurrentState.Host = machine
+	return task, err
+}
+
+func (registry *EtcdRegistry) findTask(taskID string) (Task, string, error) {
+	for _, machine := range registry.machines {
+		task, err := registry.getTaskForMachine(machine, taskID)
+		if err == nil {
+			return task, machine, nil
+		}
+	}
+	return Task{}, "", fmt.Errorf("Task not found %s", taskID)
+}
+
+func isEtcdNotFound(err error) bool {
+	if err == nil {
+		return false
+	}
+	switch err.(type) {
+	case *etcd.EtcdError:
+		etcdError := err.(*etcd.EtcdError)
+		if etcdError == nil {
+			return false
+		}
+		if etcdError.ErrorCode == 100 {
+			return true
+		}
+	}
+	return false
+}
+
+func (registry *EtcdRegistry) ListControllers() ([]ReplicationController, error) {
+	var controllers []ReplicationController
+	key := "/registry/controllers"
+	nodes, err := registry.listEtcdNode(key)
+	for _, node := range nodes {
+		var controller ReplicationController
+		err = json.Unmarshal([]byte(node.Value), &controller)
+		if err != nil {
+			return controllers, err
+		}
+		controllers = append(controllers, controller)
+	}
+	return controllers, nil
+}
+
+func makeControllerKey(id string) string {
+	return "/registry/controllers/" + id
+}
+
+func (registry *EtcdRegistry) GetController(controllerID string) (*ReplicationController, error) {
+	var controller ReplicationController
+	key := makeControllerKey(controllerID)
+	result, err := registry.etcdClient.Get(key, false, false)
+	if err != nil {
+		if isEtcdNotFound(err) {
+			return nil, fmt.Errorf("Controller %s not found", controllerID)
+		} else {
+			return nil, err
+		}
+	}
+	if result.Node == nil || len(result.Node.Value) == 0 {
+		return nil, fmt.Errorf("no nodes field: %#v", result)
+	}
+	err = json.Unmarshal([]byte(result.Node.Value), &controller)
+	return &controller, err
+}
+
+func (registry *EtcdRegistry) CreateController(controller ReplicationController) error {
+	// TODO : check for existence here and error.
+	return registry.UpdateController(controller)
+}
+
+func (registry *EtcdRegistry) UpdateController(controller ReplicationController) error {
+	controllerData, err := json.Marshal(controller)
+	if err != nil {
+		return err
+	}
+	key := makeControllerKey(controller.ID)
+	_, err = registry.etcdClient.Set(key, string(controllerData), 0)
+	return err
+}
+
+func (registry *EtcdRegistry) DeleteController(controllerID string) error {
+	key := makeControllerKey(controllerID)
+	_, err := registry.etcdClient.Delete(key, false)
+	return err
+}
+
+func makeServiceKey(name string) string {
+	return "/registry/services/specs/" + name
+}
+
+func (registry *EtcdRegistry) ListServices() (ServiceList, error) {
+	nodes, err := registry.listEtcdNode("/registry/services/specs")
+	if err != nil {
+		return ServiceList{}, err
+	}
+
+	var services []Service
+	for _, node := range nodes {
+		var svc Service
+		err := json.Unmarshal([]byte(node.Value), &svc)
+		if err != nil {
+			return ServiceList{}, err
+		}
+		services = append(services, svc)
+	}
+	return ServiceList{Items: services}, nil
+}
+
+func (registry *EtcdRegistry) CreateService(svc Service) error {
+	key := makeServiceKey(svc.ID)
+	data, err := json.Marshal(svc)
+	if err != nil {
+		return err
+	}
+	_, err = registry.etcdClient.Set(key, string(data), 0)
+	return err
+}
+
+func (registry *EtcdRegistry) GetService(name string) (*Service, error) {
+	key := makeServiceKey(name)
+	response, err := registry.etcdClient.Get(key, false, false)
+	if err != nil {
+		if isEtcdNotFound(err) {
+			return nil, fmt.Errorf("Service %s was not found.", name)
+		} else {
+			return nil, err
+		}
+	}
+	var svc Service
+	err = json.Unmarshal([]byte(response.Node.Value), &svc)
+	if err != nil {
+		return nil, err
+	}
+	return &svc, err
+}
+
+func (registry *EtcdRegistry) DeleteService(name string) error {
+	key := makeServiceKey(name)
+	_, err := registry.etcdClient.Delete(key, true)
+	if err != nil {
+		return err
+	}
+	key = "/registry/services/endpoints/" + name
+	_, err = registry.etcdClient.Delete(key, true)
+	return err
+}
+
+func (registry *EtcdRegistry) UpdateService(svc Service) error {
+	return registry.CreateService(svc)
+}
+
+func (registry *EtcdRegistry) UpdateEndpoints(e Endpoints) error {
+	data, err := json.Marshal(e)
+	if err != nil {
+		return err
+	}
+	_, err = registry.etcdClient.Set("/registry/services/endpoints/"+e.Name, string(data), 0)
+	return err
+}

pkg/registry/etcd_registry_test.go

@@ -0,0 +1,623 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+package registry
+
+import (
+	"encoding/json"
+	"reflect"
+	"testing"
+
+	. "github.com/GoogleCloudPlatform/kubernetes/pkg/api"
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/util"
+	"github.com/coreos/go-etcd/etcd"
+)
+
+func TestEtcdGetTask(t *testing.T) {
+	fakeClient := MakeFakeEtcdClient(t)
+	fakeClient.Set("/registry/hosts/machine/tasks/foo", util.MakeJSONString(Task{JSONBase: JSONBase{ID: "foo"}}), 0)
+	registry := MakeTestEtcdRegistry(fakeClient, []string{"machine"})
+	task, err := registry.GetTask("foo")
+	expectNoError(t, err)
+	if task.ID != "foo" {
+		t.Errorf("Unexpected task: %#v", task)
+	}
+}
+
+func TestEtcdGetTaskNotFound(t *testing.T) {
+	fakeClient := MakeFakeEtcdClient(t)
+	fakeClient.Data["/registry/hosts/machine/tasks/foo"] = EtcdResponseWithError{
+		R: &etcd.Response{
+			Node: nil,
+		},
+		E: &etcd.EtcdError{
+			ErrorCode: 100,
+		},
+	}
+	registry := MakeTestEtcdRegistry(fakeClient, []string{"machine"})
+	_, err := registry.GetTask("foo")
+	if err == nil {
+		t.Errorf("Unexpected non-error.")
+	}
+}
+
+func TestEtcdCreateTask(t *testing.T) {
+	fakeClient := MakeFakeEtcdClient(t)
+	fakeClient.Data["/registry/hosts/machine/tasks/foo"] = EtcdResponseWithError{
+		R: &etcd.Response{
+			Node: nil,
+		},
+		E: &etcd.EtcdError{ErrorCode: 100},
+	}
+	fakeClient.Set("/registry/hosts/machine/kubelet", util.MakeJSONString([]ContainerManifest{}), 0)
+	registry := MakeTestEtcdRegistry(fakeClient, []string{"machine"})
+	err := registry.CreateTask("machine", Task{
+		JSONBase: JSONBase{
+			ID: "foo",
+		},
+		DesiredState: TaskState{
+			Manifest: ContainerManifest{
+				Containers: []Container{
+					Container{
+						Name: "foo",
+					},
+				},
+			},
+		},
+	})
+	expectNoError(t, err)
+	resp, err := fakeClient.Get("/registry/hosts/machine/tasks/foo", false, false)
+	expectNoError(t, err)
+	var task Task
+	err = json.Unmarshal([]byte(resp.Node.Value), &task)
+	expectNoError(t, err)
+	if task.ID != "foo" {
+		t.Errorf("Unexpected task: %#v %s", task, resp.Node.Value)
+	}
+	var manifests []ContainerManifest
+	resp, err = fakeClient.Get("/registry/hosts/machine/kubelet", false, false)
+	expectNoError(t, err)
+	err = json.Unmarshal([]byte(resp.Node.Value), &manifests)
+	if len(manifests) != 1 || manifests[0].Id != "foo" {
+		t.Errorf("Unexpected manifest list: %#v", manifests)
+	}
+}
+
+func TestEtcdCreateTaskAlreadyExisting(t *testing.T) {
+	fakeClient := MakeFakeEtcdClient(t)
+	fakeClient.Data["/registry/hosts/machine/tasks/foo"] = EtcdResponseWithError{
+		R: &etcd.Response{
+			Node: &etcd.Node{
+				Value: util.MakeJSONString(Task{JSONBase: JSONBase{ID: "foo"}}),
+			},
+		},
+		E: nil,
+	}
+	registry := MakeTestEtcdRegistry(fakeClient, []string{"machine"})
+	err := registry.CreateTask("machine", Task{
+		JSONBase: JSONBase{
+			ID: "foo",
+		},
+	})
+	if err == nil {
+		t.Error("Unexpected non-error")
+	}
+}
+
+func TestEtcdCreateTaskWithContainersError(t *testing.T) {
+	fakeClient := MakeFakeEtcdClient(t)
+	fakeClient.Data["/registry/hosts/machine/tasks/foo"] = EtcdResponseWithError{
+		R: &etcd.Response{
+			Node: nil,
+		},
+		E: &etcd.EtcdError{ErrorCode: 100},
+	}
+	fakeClient.Data["/registry/hosts/machine/kubelet"] = EtcdResponseWithError{
+		R: &etcd.Response{
+			Node: nil,
+		},
+		E: &etcd.EtcdError{ErrorCode: 200},
+	}
+	registry := MakeTestEtcdRegistry(fakeClient, []string{"machine"})
+	err := registry.CreateTask("machine", Task{
+		JSONBase: JSONBase{
+			ID: "foo",
+		},
+	})
+	if err == nil {
+		t.Error("Unexpected non-error")
+	}
+	_, err = fakeClient.Get("/registry/hosts/machine/tasks/foo", false, false)
+	if err == nil {
+		t.Error("Unexpected non-error")
+	}
+	if err != nil && err.(*etcd.EtcdError).ErrorCode != 100 {
+		t.Errorf("Unexpected error: %#v", err)
+	}
+}
+
+func TestEtcdCreateTaskWithContainersNotFound(t *testing.T) {
+	fakeClient := MakeFakeEtcdClient(t)
+	fakeClient.Data["/registry/hosts/machine/tasks/foo"] = EtcdResponseWithError{
+		R: &etcd.Response{
+			Node: nil,
+		},
+		E: &etcd.EtcdError{ErrorCode: 100},
+	}
+	fakeClient.Data["/registry/hosts/machine/kubelet"] = EtcdResponseWithError{
+		R: &etcd.Response{
+			Node: nil,
+		},
+		E: &etcd.EtcdError{ErrorCode: 100},
+	}
+	registry := MakeTestEtcdRegistry(fakeClient, []string{"machine"})
+	err := registry.CreateTask("machine", Task{
+		JSONBase: JSONBase{
+			ID: "foo",
+		},
+		DesiredState: TaskState{
+			Manifest: ContainerManifest{
+				Id: "foo",
+				Containers: []Container{
+					Container{
+						Name: "foo",
+					},
+				},
+			},
+		},
+	})
+	expectNoError(t, err)
+	resp, err := fakeClient.Get("/registry/hosts/machine/tasks/foo", false, false)
+	expectNoError(t, err)
+	var task Task
+	err = json.Unmarshal([]byte(resp.Node.Value), &task)
+	expectNoError(t, err)
+	if task.ID != "foo" {
+		t.Errorf("Unexpected task: %#v %s", task, resp.Node.Value)
+	}
+	var manifests []ContainerManifest
+	resp, err = fakeClient.Get("/registry/hosts/machine/kubelet", false, false)
+	expectNoError(t, err)
+	err = json.Unmarshal([]byte(resp.Node.Value), &manifests)
+	if len(manifests) != 1 || manifests[0].Id != "foo" {
+		t.Errorf("Unexpected manifest list: %#v", manifests)
+	}
+}
+
+func TestEtcdCreateTaskWithExistingContainers(t *testing.T) {
+	fakeClient := MakeFakeEtcdClient(t)
+	fakeClient.Data["/registry/hosts/machine/tasks/foo"] = EtcdResponseWithError{
+		R: &etcd.Response{
+			Node: nil,
+		},
+		E: &etcd.EtcdError{ErrorCode: 100},
+	}
+	fakeClient.Set("/registry/hosts/machine/kubelet", util.MakeJSONString([]ContainerManifest{
+		ContainerManifest{
+			Id: "bar",
+		},
+	}), 0)
+	registry := MakeTestEtcdRegistry(fakeClient, []string{"machine"})
+	err := registry.CreateTask("machine", Task{
+		JSONBase: JSONBase{
+			ID: "foo",
+		},
+		DesiredState: TaskState{
+			Manifest: ContainerManifest{
+				Id: "foo",
+				Containers: []Container{
+					Container{
+						Name: "foo",
+					},
+				},
+			},
+		},
+	})
+	expectNoError(t, err)
+	resp, err := fakeClient.Get("/registry/hosts/machine/tasks/foo", false, false)
+	expectNoError(t, err)
+	var task Task
+	err = json.Unmarshal([]byte(resp.Node.Value), &task)
+	expectNoError(t, err)
+	if task.ID != "foo" {
+		t.Errorf("Unexpected task: %#v %s", task, resp.Node.Value)
+	}
+	var manifests []ContainerManifest
+	resp, err = fakeClient.Get("/registry/hosts/machine/kubelet", false, false)
+	expectNoError(t, err)
+	err = json.Unmarshal([]byte(resp.Node.Value), &manifests)
+	if len(manifests) != 2 || manifests[1].Id != "foo" {
+		t.Errorf("Unexpected manifest list: %#v", manifests)
+	}
+}
+
+func TestEtcdDeleteTask(t *testing.T) {
+	fakeClient := MakeFakeEtcdClient(t)
+	key := "/registry/hosts/machine/tasks/foo"
+	fakeClient.Set(key, util.MakeJSONString(Task{JSONBase: JSONBase{ID: "foo"}}), 0)
+	fakeClient.Set("/registry/hosts/machine/kubelet", util.MakeJSONString([]ContainerManifest{
+		ContainerManifest{
+			Id: "foo",
+		},
+	}), 0)
+	registry := MakeTestEtcdRegistry(fakeClient, []string{"machine"})
+	err := registry.DeleteTask("foo")
+	expectNoError(t, err)
+	if len(fakeClient.deletedKeys) != 1 {
+		t.Errorf("Expected 1 delete, found %#v", fakeClient.deletedKeys)
+	}
+	if fakeClient.deletedKeys[0] != key {
+		t.Errorf("Unexpected key: %s, expected %s", fakeClient.deletedKeys[0], key)
+	}
+	response, _ := fakeClient.Get("/registry/hosts/machine/kubelet", false, false)
+	if response.Node.Value != "[]" {
+		t.Errorf("Unexpected container set: %s, expected empty", response.Node.Value)
+	}
+}
+
+func TestEtcdDeleteTaskMultipleContainers(t *testing.T) {
+	fakeClient := MakeFakeEtcdClient(t)
+	key := "/registry/hosts/machine/tasks/foo"
+	fakeClient.Set(key, util.MakeJSONString(Task{JSONBase: JSONBase{ID: "foo"}}), 0)
+	fakeClient.Set("/registry/hosts/machine/kubelet", util.MakeJSONString([]ContainerManifest{
+		ContainerManifest{Id: "foo"},
+		ContainerManifest{Id: "bar"},
+	}), 0)
+	registry := MakeTestEtcdRegistry(fakeClient, []string{"machine"})
+	err := registry.DeleteTask("foo")
+	expectNoError(t, err)
+	if len(fakeClient.deletedKeys) != 1 {
+		t.Errorf("Expected 1 delete, found %#v", fakeClient.deletedKeys)
+	}
+	if fakeClient.deletedKeys[0] != key {
+		t.Errorf("Unexpected key: %s, expected %s", fakeClient.deletedKeys[0], key)
+	}
+	response, _ := fakeClient.Get("/registry/hosts/machine/kubelet", false, false)
+	var manifests []ContainerManifest
+	json.Unmarshal([]byte(response.Node.Value), &manifests)
+	if len(manifests) != 1 {
+		t.Errorf("Unexpected manifest set: %#v, expected empty", manifests)
+	}
+	if manifests[0].Id != "bar" {
+		t.Errorf("Deleted wrong manifest: %#v", manifests)
+	}
+}
+
+func TestEtcdEmptyListTasks(t *testing.T) {
+	fakeClient := MakeFakeEtcdClient(t)
+	key := "/registry/hosts/machine/tasks"
+	fakeClient.Data[key] = EtcdResponseWithError{
+		R: &etcd.Response{
+			Node: &etcd.Node{
+				Nodes: []*etcd.Node{},
+			},
+		},
+		E: nil,
+	}
+	registry := MakeTestEtcdRegistry(fakeClient, []string{"machine"})
+	tasks, err := registry.ListTasks(nil)
+	expectNoError(t, err)
+	if len(tasks) != 0 {
+		t.Errorf("Unexpected task list: %#v", tasks)
+	}
+}
+
+func TestEtcdListTasksNotFound(t *testing.T) {
+	fakeClient := MakeFakeEtcdClient(t)
+	key := "/registry/hosts/machine/tasks"
+	fakeClient.Data[key] = EtcdResponseWithError{
+		R: &etcd.Response{},
+		E: &etcd.EtcdError{ErrorCode: 100},
+	}
+	registry := MakeTestEtcdRegistry(fakeClient, []string{"machine"})
+	tasks, err := registry.ListTasks(nil)
+	expectNoError(t, err)
+	if len(tasks) != 0 {
+		t.Errorf("Unexpected task list: %#v", tasks)
+	}
+}
+
+func TestEtcdListTasks(t *testing.T) {
+	fakeClient := MakeFakeEtcdClient(t)
+	key := "/registry/hosts/machine/tasks"
+	fakeClient.Data[key] = EtcdResponseWithError{
+		R: &etcd.Response{
+			Node: &etcd.Node{
+				Nodes: []*etcd.Node{
+					&etcd.Node{
+						Value: util.MakeJSONString(Task{JSONBase: JSONBase{ID: "foo"}}),
+					},
+					&etcd.Node{
+						Value: util.MakeJSONString(Task{JSONBase: JSONBase{ID: "bar"}}),
+					},
+				},
+			},
+		},
+		E: nil,
+	}
+	registry := MakeTestEtcdRegistry(fakeClient, []string{"machine"})
+	tasks, err := registry.ListTasks(nil)
+	expectNoError(t, err)
+	if len(tasks) != 2 || tasks[0].ID != "foo" || tasks[1].ID != "bar" {
+		t.Errorf("Unexpected task list: %#v", tasks)
+	}
+}
+
+func TestEtcdListControllersNotFound(t *testing.T) {
+	fakeClient := MakeFakeEtcdClient(t)
+	key := "/registry/controllers"
+	fakeClient.Data[key] = EtcdResponseWithError{
+		R: &etcd.Response{},
+		E: &etcd.EtcdError{ErrorCode: 100},
+	}
+	registry := MakeTestEtcdRegistry(fakeClient, []string{"machine"})
+	controllers, err := registry.ListControllers()
+	expectNoError(t, err)
+	if len(controllers) != 0 {
+		t.Errorf("Unexpected controller list: %#v", controllers)
+	}
+}
+
+func TestEtcdListServicesNotFound(t *testing.T) {
+	fakeClient := MakeFakeEtcdClient(t)
+	key := "/registry/services/specs"
+	fakeClient.Data[key] = EtcdResponseWithError{
+		R: &etcd.Response{},
+		E: &etcd.EtcdError{ErrorCode: 100},
+	}
+	registry := MakeTestEtcdRegistry(fakeClient, []string{"machine"})
+	services, err := registry.ListServices()
+	expectNoError(t, err)
+	if len(services.Items) != 0 {
+		t.Errorf("Unexpected controller list: %#v", services)
+	}
+}
+
+func TestEtcdListControllers(t *testing.T) {
+	fakeClient := MakeFakeEtcdClient(t)
+	key := "/registry/controllers"
+	fakeClient.Data[key] = EtcdResponseWithError{
+		R: &etcd.Response{
+			Node: &etcd.Node{
+				Nodes: []*etcd.Node{
+					&etcd.Node{
+						Value: util.MakeJSONString(ReplicationController{JSONBase: JSONBase{ID: "foo"}}),
+					},
+					&etcd.Node{
+						Value: util.MakeJSONString(ReplicationController{JSONBase: JSONBase{ID: "bar"}}),
+					},
+				},
+			},
+		},
+		E: nil,
+	}
+	registry := MakeTestEtcdRegistry(fakeClient, []string{"machine"})
+	controllers, err := registry.ListControllers()
+	expectNoError(t, err)
+	if len(controllers) != 2 || controllers[0].ID != "foo" || controllers[1].ID != "bar" {
+		t.Errorf("Unexpected controller list: %#v", controllers)
+	}
+}
+
+func TestEtcdGetController(t *testing.T) {
+	fakeClient := MakeFakeEtcdClient(t)
+	fakeClient.Set("/registry/controllers/foo", util.MakeJSONString(ReplicationController{JSONBase: JSONBase{ID: "foo"}}), 0)
+	registry := MakeTestEtcdRegistry(fakeClient, []string{"machine"})
+	ctrl, err := registry.GetController("foo")
+	expectNoError(t, err)
+	if ctrl.ID != "foo" {
+		t.Errorf("Unexpected controller: %#v", ctrl)
+	}
+}
+
+func TestEtcdGetControllerNotFound(t *testing.T) {
+	fakeClient := MakeFakeEtcdClient(t)
+	fakeClient.Data["/registry/controllers/foo"] = EtcdResponseWithError{
+		R: &etcd.Response{
+			Node: nil,
+		},
+		E: &etcd.EtcdError{
+			ErrorCode: 100,
+		},
+	}
+	registry := MakeTestEtcdRegistry(fakeClient, []string{"machine"})
+	ctrl, err := registry.GetController("foo")
+	if ctrl != nil {
+		t.Errorf("Unexpected non-nil controller: %#v", ctrl)
+	}
+	if err == nil {
+		t.Error("Unexpected non-error.")
+	}
+}
+
+func TestEtcdDeleteController(t *testing.T) {
+	fakeClient := MakeFakeEtcdClient(t)
+	registry := MakeTestEtcdRegistry(fakeClient, []string{"machine"})
+	err := registry.DeleteController("foo")
+	expectNoError(t, err)
+	if len(fakeClient.deletedKeys) != 1 {
+		t.Errorf("Expected 1 delete, found %#v", fakeClient.deletedKeys)
+	}
+	key := "/registry/controllers/foo"
+	if fakeClient.deletedKeys[0] != key {
+		t.Errorf("Unexpected key: %s, expected %s", fakeClient.deletedKeys[0], key)
+	}
+}
+
+func TestEtcdCreateController(t *testing.T) {
+	fakeClient := MakeFakeEtcdClient(t)
+	registry := MakeTestEtcdRegistry(fakeClient, []string{"machine"})
+	err := registry.CreateController(ReplicationController{
+		JSONBase: JSONBase{
+			ID: "foo",
+		},
+	})
+	expectNoError(t, err)
+	resp, err := fakeClient.Get("/registry/controllers/foo", false, false)
+	expectNoError(t, err)
+	var ctrl ReplicationController
+	err = json.Unmarshal([]byte(resp.Node.Value), &ctrl)
+	expectNoError(t, err)
+	if ctrl.ID != "foo" {
+		t.Errorf("Unexpected task: %#v %s", ctrl, resp.Node.Value)
+	}
+}
+
+func TestEtcdUpdateController(t *testing.T) {
+	fakeClient := MakeFakeEtcdClient(t)
+	fakeClient.Set("/registry/controllers/foo", util.MakeJSONString(ReplicationController{JSONBase: JSONBase{ID: "foo"}}), 0)
+	registry := MakeTestEtcdRegistry(fakeClient, []string{"machine"})
+	err := registry.UpdateController(ReplicationController{
+		JSONBase: JSONBase{ID: "foo"},
+		DesiredState: ReplicationControllerState{
+			Replicas: 2,
+		},
+	})
+	expectNoError(t, err)
+	ctrl, err := registry.GetController("foo")
+	if ctrl.DesiredState.Replicas != 2 {
+		t.Errorf("Unexpected controller: %#v", ctrl)
+	}
+}
+
+func TestEtcdListServices(t *testing.T) {
+	fakeClient := MakeFakeEtcdClient(t)
+	key := "/registry/services/specs"
+	fakeClient.Data[key] = EtcdResponseWithError{
+		R: &etcd.Response{
+			Node: &etcd.Node{
+				Nodes: []*etcd.Node{
+					&etcd.Node{
+						Value: util.MakeJSONString(Service{JSONBase: JSONBase{ID: "foo"}}),
+					},
+					&etcd.Node{
+						Value: util.MakeJSONString(Service{JSONBase: JSONBase{ID: "bar"}}),
+					},
+				},
+			},
+		},
+		E: nil,
+	}
+	registry := MakeTestEtcdRegistry(fakeClient, []string{"machine"})
+	services, err := registry.ListServices()
+	expectNoError(t, err)
+	if len(services.Items) != 2 || services.Items[0].ID != "foo" || services.Items[1].ID != "bar" {
+		t.Errorf("Unexpected task list: %#v", services)
+	}
+}
+
+func TestEtcdCreateService(t *testing.T) {
+	fakeClient := MakeFakeEtcdClient(t)
+	fakeClient.Data["/registry/services/specs/foo"] = EtcdResponseWithError{
+		R: &etcd.Response{
+			Node: nil,
+		},
+		E: &etcd.EtcdError{ErrorCode: 100},
+	}
+	registry := MakeTestEtcdRegistry(fakeClient, []string{"machine"})
+	err := registry.CreateService(Service{
+		JSONBase: JSONBase{ID: "foo"},
+	})
+	expectNoError(t, err)
+	resp, err := fakeClient.Get("/registry/services/specs/foo", false, false)
+	expectNoError(t, err)
+	var service Service
+	err = json.Unmarshal([]byte(resp.Node.Value), &service)
+	expectNoError(t, err)
+	if service.ID != "foo" {
+		t.Errorf("Unexpected service: %#v %s", service, resp.Node.Value)
+	}
+}
+
+func TestEtcdGetService(t *testing.T) {
+	fakeClient := MakeFakeEtcdClient(t)
+	fakeClient.Set("/registry/services/specs/foo", util.MakeJSONString(Service{JSONBase: JSONBase{ID: "foo"}}), 0)
+	registry := MakeTestEtcdRegistry(fakeClient, []string{"machine"})
+	service, err := registry.GetService("foo")
+	expectNoError(t, err)
+	if service.ID != "foo" {
+		t.Errorf("Unexpected task: %#v", service)
+	}
+}
+
+func TestEtcdGetServiceNotFound(t *testing.T) {
+	fakeClient := MakeFakeEtcdClient(t)
+	fakeClient.Data["/registry/services/specs/foo"] = EtcdResponseWithError{
+		R: &etcd.Response{
+			Node: nil,
+		},
+		E: &etcd.EtcdError{
+			ErrorCode: 100,
+		},
+	}
+	registry := MakeTestEtcdRegistry(fakeClient, []string{"machine"})
+	_, err := registry.GetService("foo")
+	if err == nil {
+		t.Errorf("Unexpected non-error.")
+	}
+}
+
+func TestEtcdDeleteService(t *testing.T) {
+	fakeClient := MakeFakeEtcdClient(t)
+	registry := MakeTestEtcdRegistry(fakeClient, []string{"machine"})
+	err := registry.DeleteService("foo")
+	expectNoError(t, err)
+	if len(fakeClient.deletedKeys) != 2 {
+		t.Errorf("Expected 2 delete, found %#v", fakeClient.deletedKeys)
+	}
+	key := "/registry/services/specs/foo"
+	if fakeClient.deletedKeys[0] != key {
+		t.Errorf("Unexpected key: %s, expected %s", fakeClient.deletedKeys[0], key)
+	}
+	key = "/registry/services/endpoints/foo"
+	if fakeClient.deletedKeys[1] != key {
+		t.Errorf("Unexpected key: %s, expected %s", fakeClient.deletedKeys[1], key)
+	}
+}
+
+func TestEtcdUpdateService(t *testing.T) {
+	fakeClient := MakeFakeEtcdClient(t)
+	fakeClient.Set("/registry/services/specs/foo", util.MakeJSONString(Service{JSONBase: JSONBase{ID: "foo"}}), 0)
+	registry := MakeTestEtcdRegistry(fakeClient, []string{"machine"})
+	err := registry.UpdateService(Service{
+		JSONBase: JSONBase{ID: "foo"},
+		Labels: map[string]string{
+			"baz": "bar",
+		},
+	})
+	expectNoError(t, err)
+	svc, err := registry.GetService("foo")
+	if svc.Labels["baz"] != "bar" {
+		t.Errorf("Unexpected service: %#v", svc)
+	}
+}
+
+func TestEtcdUpdateEndpoints(t *testing.T) {
+	fakeClient := MakeFakeEtcdClient(t)
+	registry := MakeTestEtcdRegistry(fakeClient, []string{"machine"})
+	endpoints := Endpoints{
+		Name:      "foo",
+		Endpoints: []string{"baz", "bar"},
+	}
+	err := registry.UpdateEndpoints(endpoints)
+	expectNoError(t, err)
+	response, err := fakeClient.Get("/registry/services/endpoints/foo", false, false)
+	expectNoError(t, err)
+	var endpointsOut Endpoints
+	err = json.Unmarshal([]byte(response.Node.Value), &endpointsOut)
+	if !reflect.DeepEqual(endpoints, endpointsOut) {
+		t.Errorf("Unexpected endpoints: %#v, expected %#v", endpointsOut, endpoints)
+	}
+}

pkg/registry/fake_etcd_client.go

@@ -0,0 +1,86 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+package registry
+
+import (
+	"fmt"
+	"testing"
+
+	"github.com/coreos/go-etcd/etcd"
+)
+
+type EtcdResponseWithError struct {
+	R *etcd.Response
+	E error
+}
+
+type FakeEtcdClient struct {
+	Data        map[string]EtcdResponseWithError
+	deletedKeys []string
+	err         error
+	t           *testing.T
+}
+
+func MakeFakeEtcdClient(t *testing.T) *FakeEtcdClient {
+	return &FakeEtcdClient{
+		t:    t,
+		Data: map[string]EtcdResponseWithError{},
+	}
+}
+
+func (f *FakeEtcdClient) AddChild(key, data string, ttl uint64) (*etcd.Response, error) {
+	return f.Set(key, data, ttl)
+}
+
+func (f *FakeEtcdClient) Get(key string, sort, recursive bool) (*etcd.Response, error) {
+	result := f.Data[key]
+	if result.R == nil {
+		f.t.Errorf("Unexpected get for %s", key)
+		return &etcd.Response{}, &etcd.EtcdError{ErrorCode: 100}
+	}
+	return result.R, result.E
+}
+
+func (f *FakeEtcdClient) Set(key, value string, ttl uint64) (*etcd.Response, error) {
+	result := EtcdResponseWithError{
+		R: &etcd.Response{
+			Node: &etcd.Node{
+				Value: value,
+			},
+		},
+	}
+	f.Data[key] = result
+	return result.R, f.err
+}
+func (f *FakeEtcdClient) Create(key, value string, ttl uint64) (*etcd.Response, error) {
+	return f.Set(key, value, ttl)
+}
+func (f *FakeEtcdClient) Delete(key string, recursive bool) (*etcd.Response, error) {
+	f.deletedKeys = append(f.deletedKeys, key)
+	return &etcd.Response{}, f.err
+}
+
+func (f *FakeEtcdClient) Watch(prefix string, waitIndex uint64, recursive bool, receiver chan *etcd.Response, stop chan bool) (*etcd.Response, error) {
+	return nil, fmt.Errorf("Unimplemented")
+}
+
+func MakeTestEtcdRegistry(client EtcdClient, machines []string) *EtcdRegistry {
+	registry := MakeEtcdRegistry(client, machines)
+	registry.manifestFactory = &BasicManifestFactory{
+		serviceRegistry: &MockServiceRegistry{},
+	}
+	return registry
+}

pkg/registry/interfaces.go

@@ -0,0 +1,44 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+package registry
+
+import (
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/api"
+)
+
+// TaskRegistry is an interface implemented by things that know how to store Task objects
+type TaskRegistry interface {
+	// ListTasks obtains a list of tasks that match query.
+	// Query may be nil in which case all tasks are returned.
+	ListTasks(query *map[string]string) ([]api.Task, error)
+	// Get a specific task
+	GetTask(taskId string) (*api.Task, error)
+	// Create a task based on a specification, schedule it onto a specific machine.
+	CreateTask(machine string, task api.Task) error
+	// Update an existing task
+	UpdateTask(task api.Task) error
+	// Delete an existing task
+	DeleteTask(taskId string) error
+}
+
+// ControllerRegistry is an interface for things that know how to store Controllers
+type ControllerRegistry interface {
+	ListControllers() ([]api.ReplicationController, error)
+	GetController(controllerId string) (*api.ReplicationController, error)
+	CreateController(controller api.ReplicationController) error
+	UpdateController(controller api.ReplicationController) error
+	DeleteController(controllerId string) error
+}

pkg/registry/manifest_factory.go

@@ -0,0 +1,41 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+package registry
+
+import (
+	. "github.com/GoogleCloudPlatform/kubernetes/pkg/api"
+)
+
+type ManifestFactory interface {
+	// Make a container object for a given task, given the machine that the task is running on.
+	MakeManifest(machine string, task Task) (ContainerManifest, error)
+}
+
+type BasicManifestFactory struct {
+	serviceRegistry ServiceRegistry
+}
+
+func (b *BasicManifestFactory) MakeManifest(machine string, task Task) (ContainerManifest, error) {
+	envVars, err := GetServiceEnvironmentVariables(b.serviceRegistry, machine)
+	if err != nil {
+		return ContainerManifest{}, err
+	}
+	for ix, container := range task.DesiredState.Manifest.Containers {
+		task.DesiredState.Manifest.Id = task.ID
+		task.DesiredState.Manifest.Containers[ix].Env = append(container.Env, envVars...)
+	}
+	return task.DesiredState.Manifest, nil
+}

pkg/registry/manifest_factory_test.go

@@ -0,0 +1,133 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+package registry
+
+import (
+	"testing"
+
+	. "github.com/GoogleCloudPlatform/kubernetes/pkg/api"
+)
+
+func TestMakeManifestNoServices(t *testing.T) {
+	registry := MockServiceRegistry{}
+	factory := &BasicManifestFactory{
+		serviceRegistry: &registry,
+	}
+
+	manifest, err := factory.MakeManifest("machine", Task{
+		JSONBase: JSONBase{ID: "foobar"},
+		DesiredState: TaskState{
+			Manifest: ContainerManifest{
+				Containers: []Container{
+					Container{
+						Name: "foo",
+					},
+				},
+			},
+		},
+	})
+	expectNoError(t, err)
+	container := manifest.Containers[0]
+	if len(container.Env) != 1 ||
+		container.Env[0].Name != "SERVICE_HOST" ||
+		container.Env[0].Value != "machine" {
+		t.Errorf("Expected one env vars, got: %#v", manifest)
+	}
+	if manifest.Id != "foobar" {
+		t.Errorf("Failed to assign id to manifest: %#v")
+	}
+}
+
+func TestMakeManifestServices(t *testing.T) {
+	registry := MockServiceRegistry{
+		list: ServiceList{
+			Items: []Service{
+				Service{
+					JSONBase: JSONBase{ID: "test"},
+					Port:     8080,
+				},
+			},
+		},
+	}
+	factory := &BasicManifestFactory{
+		serviceRegistry: &registry,
+	}
+
+	manifest, err := factory.MakeManifest("machine", Task{
+		DesiredState: TaskState{
+			Manifest: ContainerManifest{
+				Containers: []Container{
+					Container{
+						Name: "foo",
+					},
+				},
+			},
+		},
+	})
+	expectNoError(t, err)
+	container := manifest.Containers[0]
+	if len(container.Env) != 2 ||
+		container.Env[0].Name != "TEST_SERVICE_PORT" ||
+		container.Env[0].Value != "8080" ||
+		container.Env[1].Name != "SERVICE_HOST" ||
+		container.Env[1].Value != "machine" {
+		t.Errorf("Expected 2 env vars, got: %#v", manifest)
+	}
+}
+
+func TestMakeManifestServicesExistingEnvVar(t *testing.T) {
+	registry := MockServiceRegistry{
+		list: ServiceList{
+			Items: []Service{
+				Service{
+					JSONBase: JSONBase{ID: "test"},
+					Port:     8080,
+				},
+			},
+		},
+	}
+	factory := &BasicManifestFactory{
+		serviceRegistry: &registry,
+	}
+
+	manifest, err := factory.MakeManifest("machine", Task{
+		DesiredState: TaskState{
+			Manifest: ContainerManifest{
+				Containers: []Container{
+					Container{
+						Env: []EnvVar{
+							EnvVar{
+								Name:  "foo",
+								Value: "bar",
+							},
+						},
+					},
+				},
+			},
+		},
+	})
+	expectNoError(t, err)
+	container := manifest.Containers[0]
+	if len(container.Env) != 3 ||
+		container.Env[0].Name != "foo" ||
+		container.Env[0].Value != "bar" ||
+		container.Env[1].Name != "TEST_SERVICE_PORT" ||
+		container.Env[1].Value != "8080" ||
+		container.Env[2].Name != "SERVICE_HOST" ||
+		container.Env[2].Value != "machine" {
+		t.Errorf("Expected no env vars, got: %#v", manifest)
+	}
+}

pkg/registry/memory_registry.go

@@ -0,0 +1,137 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+package registry
+
+import (
+	. "github.com/GoogleCloudPlatform/kubernetes/pkg/api"
+)
+
+// An implementation of TaskRegistry and ControllerRegistry that is backed by memory
+// Mainly used for testing.
+type MemoryRegistry struct {
+	taskData       map[string]Task
+	controllerData map[string]ReplicationController
+	serviceData    map[string]Service
+}
+
+func MakeMemoryRegistry() *MemoryRegistry {
+	return &MemoryRegistry{
+		taskData:       map[string]Task{},
+		controllerData: map[string]ReplicationController{},
+		serviceData:    map[string]Service{},
+	}
+}
+
+func (registry *MemoryRegistry) ListTasks(labelQuery *map[string]string) ([]Task, error) {
+	result := []Task{}
+	for _, value := range registry.taskData {
+		if LabelsMatch(value, labelQuery) {
+			result = append(result, value)
+		}
+	}
+	return result, nil
+}
+
+func (registry *MemoryRegistry) GetTask(taskID string) (*Task, error) {
+	task, found := registry.taskData[taskID]
+	if found {
+		return &task, nil
+	} else {
+		return nil, nil
+	}
+}
+
+func (registry *MemoryRegistry) CreateTask(machine string, task Task) error {
+	registry.taskData[task.ID] = task
+	return nil
+}
+
+func (registry *MemoryRegistry) DeleteTask(taskID string) error {
+	delete(registry.taskData, taskID)
+	return nil
+}
+
+func (registry *MemoryRegistry) UpdateTask(task Task) error {
+	registry.taskData[task.ID] = task
+	return nil
+}
+
+func (registry *MemoryRegistry) ListControllers() ([]ReplicationController, error) {
+	result := []ReplicationController{}
+	for _, value := range registry.controllerData {
+		result = append(result, value)
+	}
+	return result, nil
+}
+
+func (registry *MemoryRegistry) GetController(controllerID string) (*ReplicationController, error) {
+	controller, found := registry.controllerData[controllerID]
+	if found {
+		return &controller, nil
+	} else {
+		return nil, nil
+	}
+}
+
+func (registry *MemoryRegistry) CreateController(controller ReplicationController) error {
+	registry.controllerData[controller.ID] = controller
+	return nil
+}
+
+func (registry *MemoryRegistry) DeleteController(controllerId string) error {
+	delete(registry.controllerData, controllerId)
+	return nil
+}
+
+func (registry *MemoryRegistry) UpdateController(controller ReplicationController) error {
+	registry.controllerData[controller.ID] = controller
+	return nil
+}
+
+func (registry *MemoryRegistry) ListServices() (ServiceList, error) {
+	var list []Service
+	for _, value := range registry.serviceData {
+		list = append(list, value)
+	}
+	return ServiceList{Items: list}, nil
+}
+
+func (registry *MemoryRegistry) CreateService(svc Service) error {
+	registry.serviceData[svc.ID] = svc
+	return nil
+}
+
+func (registry *MemoryRegistry) GetService(name string) (*Service, error) {
+	svc, found := registry.serviceData[name]
+	if found {
+		return &svc, nil
+	} else {
+		return nil, nil
+	}
+}
+
+func (registry *MemoryRegistry) DeleteService(name string) error {
+	delete(registry.serviceData, name)
+	return nil
+}
+
+func (registry *MemoryRegistry) UpdateService(svc Service) error {
+	return registry.CreateService(svc)
+}
+
+func (registry *MemoryRegistry) UpdateEndpoints(e Endpoints) error {
+	return nil
+}

pkg/registry/memory_registry_test.go

@@ -0,0 +1,146 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+package registry
+
+import (
+	"testing"
+
+	. "github.com/GoogleCloudPlatform/kubernetes/pkg/api"
+)
+
+func TestListTasksEmpty(t *testing.T) {
+	registry := MakeMemoryRegistry()
+	tasks, err := registry.ListTasks(nil)
+	expectNoError(t, err)
+	if len(tasks) != 0 {
+		t.Errorf("Unexpected task list: %#v", tasks)
+	}
+}
+
+func TestMemoryListTasks(t *testing.T) {
+	registry := MakeMemoryRegistry()
+	registry.CreateTask("machine", Task{JSONBase: JSONBase{ID: "foo"}})
+	tasks, err := registry.ListTasks(nil)
+	expectNoError(t, err)
+	if len(tasks) != 1 || tasks[0].ID != "foo" {
+		t.Errorf("Unexpected task list: %#v", tasks)
+	}
+}
+
+func TestMemorySetGetTasks(t *testing.T) {
+	registry := MakeMemoryRegistry()
+	expectedTask := Task{JSONBase: JSONBase{ID: "foo"}}
+	registry.CreateTask("machine", expectedTask)
+	task, err := registry.GetTask("foo")
+	expectNoError(t, err)
+	if expectedTask.ID != task.ID {
+		t.Errorf("Unexpected task, expected %#v, actual %#v", expectedTask, task)
+	}
+}
+
+func TestMemorySetUpdateGetTasks(t *testing.T) {
+	registry := MakeMemoryRegistry()
+	oldTask := Task{JSONBase: JSONBase{ID: "foo"}}
+	expectedTask := Task{
+		JSONBase: JSONBase{
+			ID: "foo",
+		},
+		DesiredState: TaskState{
+			Host: "foo.com",
+		},
+	}
+	registry.CreateTask("machine", oldTask)
+	registry.UpdateTask(expectedTask)
+	task, err := registry.GetTask("foo")
+	expectNoError(t, err)
+	if expectedTask.ID != task.ID || task.DesiredState.Host != expectedTask.DesiredState.Host {
+		t.Errorf("Unexpected task, expected %#v, actual %#v", expectedTask, task)
+	}
+}
+
+func TestMemorySetDeleteGetTasks(t *testing.T) {
+	registry := MakeMemoryRegistry()
+	expectedTask := Task{JSONBase: JSONBase{ID: "foo"}}
+	registry.CreateTask("machine", expectedTask)
+	registry.DeleteTask("foo")
+	task, err := registry.GetTask("foo")
+	expectNoError(t, err)
+	if task != nil {
+		t.Errorf("Unexpected task: %#v", task)
+	}
+}
+
+func TestListControllersEmpty(t *testing.T) {
+	registry := MakeMemoryRegistry()
+	tasks, err := registry.ListControllers()
+	expectNoError(t, err)
+	if len(tasks) != 0 {
+		t.Errorf("Unexpected task list: %#v", tasks)
+	}
+}
+
+func TestMemoryListControllers(t *testing.T) {
+	registry := MakeMemoryRegistry()
+	registry.CreateController(ReplicationController{JSONBase: JSONBase{ID: "foo"}})
+	tasks, err := registry.ListControllers()
+	expectNoError(t, err)
+	if len(tasks) != 1 || tasks[0].ID != "foo" {
+		t.Errorf("Unexpected task list: %#v", tasks)
+	}
+}
+
+func TestMemorySetGetControllers(t *testing.T) {
+	registry := MakeMemoryRegistry()
+	expectedController := ReplicationController{JSONBase: JSONBase{ID: "foo"}}
+	registry.CreateController(expectedController)
+	task, err := registry.GetController("foo")
+	expectNoError(t, err)
+	if expectedController.ID != task.ID {
+		t.Errorf("Unexpected task, expected %#v, actual %#v", expectedController, task)
+	}
+}
+
+func TestMemorySetUpdateGetControllers(t *testing.T) {
+	registry := MakeMemoryRegistry()
+	oldController := ReplicationController{JSONBase: JSONBase{ID: "foo"}}
+	expectedController := ReplicationController{
+		JSONBase: JSONBase{
+			ID: "foo",
+		},
+		DesiredState: ReplicationControllerState{
+			Replicas: 2,
+		},
+	}
+	registry.CreateController(oldController)
+	registry.UpdateController(expectedController)
+	task, err := registry.GetController("foo")
+	expectNoError(t, err)
+	if expectedController.ID != task.ID || task.DesiredState.Replicas != expectedController.DesiredState.Replicas {
+		t.Errorf("Unexpected task, expected %#v, actual %#v", expectedController, task)
+	}
+}
+
+func TestMemorySetDeleteGetControllers(t *testing.T) {
+	registry := MakeMemoryRegistry()
+	expectedController := ReplicationController{JSONBase: JSONBase{ID: "foo"}}
+	registry.CreateController(expectedController)
+	registry.DeleteController("foo")
+	task, err := registry.GetController("foo")
+	expectNoError(t, err)
+	if task != nil {
+		t.Errorf("Unexpected task: %#v", task)
+	}
+}

pkg/registry/mock_service_registry.go

@@ -0,0 +1,51 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+package registry
+
+import (
+	. "github.com/GoogleCloudPlatform/kubernetes/pkg/api"
+)
+
+type MockServiceRegistry struct {
+	list      ServiceList
+	err       error
+	endpoints Endpoints
+}
+
+func (m *MockServiceRegistry) ListServices() (ServiceList, error) {
+	return m.list, m.err
+}
+
+func (m *MockServiceRegistry) CreateService(svc Service) error {
+	return m.err
+}
+
+func (m *MockServiceRegistry) GetService(name string) (*Service, error) {
+	return nil, m.err
+}
+
+func (m *MockServiceRegistry) DeleteService(name string) error {
+	return m.err
+}
+
+func (m *MockServiceRegistry) UpdateService(svc Service) error {
+	return m.err
+}
+
+func (m *MockServiceRegistry) UpdateEndpoints(e Endpoints) error {
+	m.endpoints = e
+	return m.err
+}

pkg/registry/replication_controller.go

@@ -0,0 +1,186 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+package registry
+
+import (
+	"encoding/json"
+	"fmt"
+	"log"
+	"math/rand"
+	"strings"
+	"sync"
+	"time"
+
+	. "github.com/GoogleCloudPlatform/kubernetes/pkg/api"
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/client"
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/util"
+	"github.com/coreos/go-etcd/etcd"
+)
+
+// ReplicationManager is responsible for synchronizing ReplicationController objects stored in etcd
+// with actual running tasks.
+// TODO: Remove the etcd dependency and re-factor in terms of a generic watch interface
+type ReplicationManager struct {
+	etcdClient  *etcd.Client
+	kubeClient  client.ClientInterface
+	taskControl TaskControlInterface
+	updateLock  sync.Mutex
+}
+
+// An interface that knows how to add or delete tasks
+// created as an interface to allow testing.
+type TaskControlInterface interface {
+	createReplica(controllerSpec ReplicationController)
+	deleteTask(taskID string) error
+}
+
+type RealTaskControl struct {
+	kubeClient client.ClientInterface
+}
+
+func (r RealTaskControl) createReplica(controllerSpec ReplicationController) {
+	labels := controllerSpec.DesiredState.TaskTemplate.Labels
+	if labels != nil {
+		labels["replicationController"] = controllerSpec.ID
+	}
+	task := Task{
+		JSONBase: JSONBase{
+			ID: fmt.Sprintf("%x", rand.Int()),
+		},
+		DesiredState: controllerSpec.DesiredState.TaskTemplate.DesiredState,
+		Labels:       controllerSpec.DesiredState.TaskTemplate.Labels,
+	}
+	_, err := r.kubeClient.CreateTask(task)
+	if err != nil {
+		log.Printf("%#v\n", err)
+	}
+}
+
+func (r RealTaskControl) deleteTask(taskID string) error {
+	return r.kubeClient.DeleteTask(taskID)
+}
+
+func MakeReplicationManager(etcdClient *etcd.Client, kubeClient client.ClientInterface) *ReplicationManager {
+	return &ReplicationManager{
+		kubeClient: kubeClient,
+		etcdClient: etcdClient,
+		taskControl: RealTaskControl{
+			kubeClient: kubeClient,
+		},
+	}
+}
+
+func (rm *ReplicationManager) WatchControllers() {
+	watchChannel := make(chan *etcd.Response)
+	go util.Forever(func() { rm.etcdClient.Watch("/registry/controllers", 0, true, watchChannel, nil) }, 0)
+	for {
+		watchResponse := <-watchChannel
+		if watchResponse == nil {
+			time.Sleep(time.Second * 10)
+			continue
+		}
+		log.Printf("Got watch: %#v", watchResponse)
+		controller, err := rm.handleWatchResponse(watchResponse)
+		if err != nil {
+			log.Printf("Error handling data: %#v, %#v", err, watchResponse)
+			continue
+		}
+		rm.syncReplicationController(*controller)
+	}
+}
+
+func (rm *ReplicationManager) handleWatchResponse(response *etcd.Response) (*ReplicationController, error) {
+	if response.Action == "set" {
+		if response.Node != nil {
+			var controllerSpec ReplicationController
+			err := json.Unmarshal([]byte(response.Node.Value), &controllerSpec)
+			if err != nil {
+				return nil, err
+			}
+			return &controllerSpec, nil
+		} else {
+			return nil, fmt.Errorf("Response node is null %#v", response)
+		}
+	}
+	return nil, nil
+}
+
+func (rm *ReplicationManager) filterActiveTasks(tasks []Task) []Task {
+	var result []Task
+	for _, value := range tasks {
+		if strings.Index(value.CurrentState.Status, "Exit") == -1 {
+			result = append(result, value)
+		}
+	}
+	return result
+}
+
+func (rm *ReplicationManager) syncReplicationController(controllerSpec ReplicationController) error {
+	rm.updateLock.Lock()
+	taskList, err := rm.kubeClient.ListTasks(controllerSpec.DesiredState.ReplicasInSet)
+	if err != nil {
+		return err
+	}
+	filteredList := rm.filterActiveTasks(taskList.Items)
+	diff := len(filteredList) - controllerSpec.DesiredState.Replicas
+	log.Printf("%#v", filteredList)
+	if diff < 0 {
+		diff *= -1
+		log.Printf("Too few replicas, creating %d\n", diff)
+		for i := 0; i < diff; i++ {
+			rm.taskControl.createReplica(controllerSpec)
+		}
+	} else if diff > 0 {
+		log.Print("Too many replicas, deleting")
+		for i := 0; i < diff; i++ {
+			rm.taskControl.deleteTask(filteredList[i].ID)
+		}
+	}
+	rm.updateLock.Unlock()
+	return nil
+}
+
+func (rm *ReplicationManager) Synchronize() {
+	for {
+		response, err := rm.etcdClient.Get("/registry/controllers", false, false)
+		if err != nil {
+			log.Printf("Synchronization error %#v", err)
+		}
+		// TODO(bburns): There is a race here, if we get a version of the controllers, and then it is
+		// updated, its possible that the watch will pick up the change first, and then we will execute
+		// using the old version of the controller.
+		// Probably the correct thing to do is to use the version number in etcd to detect when
+		// we are stale.
+		// Punting on this for now, but this could lead to some nasty bugs, so we should really fix it
+		// sooner rather than later.
+		if response != nil && response.Node != nil && response.Node.Nodes != nil {
+			for _, value := range response.Node.Nodes {
+				var controllerSpec ReplicationController
+				err := json.Unmarshal([]byte(value.Value), &controllerSpec)
+				if err != nil {
+					log.Printf("Unexpected error: %#v", err)
+					continue
+				}
+				log.Printf("Synchronizing %s\n", controllerSpec.ID)
+				err = rm.syncReplicationController(controllerSpec)
+				if err != nil {
+					log.Printf("Error synchronizing: %#v", err)
+				}
+			}
+		}
+		time.Sleep(10 * time.Second)
+	}
+}

pkg/registry/replication_controller_test.go

@@ -0,0 +1,311 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+package registry
+
+import (
+	"encoding/json"
+	"fmt"
+	"net/http/httptest"
+	"reflect"
+	"testing"
+
+	. "github.com/GoogleCloudPlatform/kubernetes/pkg/api"
+	. "github.com/GoogleCloudPlatform/kubernetes/pkg/client"
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/util"
+	"github.com/coreos/go-etcd/etcd"
+)
+
+// TODO: Move this to a common place, it's needed in multiple tests.
+var apiPath = "/api/v1beta1"
+
+func makeUrl(suffix string) string {
+	return apiPath + suffix
+}
+
+type FakeTaskControl struct {
+	controllerSpec []ReplicationController
+	deleteTaskID   []string
+}
+
+func (f *FakeTaskControl) createReplica(spec ReplicationController) {
+	f.controllerSpec = append(f.controllerSpec, spec)
+}
+
+func (f *FakeTaskControl) deleteTask(taskID string) error {
+	f.deleteTaskID = append(f.deleteTaskID, taskID)
+	return nil
+}
+
+func makeReplicationController(replicas int) ReplicationController {
+	return ReplicationController{
+		DesiredState: ReplicationControllerState{
+			Replicas: replicas,
+			TaskTemplate: TaskTemplate{
+				DesiredState: TaskState{
+					Manifest: ContainerManifest{
+						Containers: []Container{
+							Container{
+								Image: "foo/bar",
+							},
+						},
+					},
+				},
+				Labels: map[string]string{
+					"name": "foo",
+					"type": "production",
+				},
+			},
+		},
+	}
+}
+
+func makeTaskList(count int) TaskList {
+	tasks := []Task{}
+	for i := 0; i < count; i++ {
+		tasks = append(tasks, Task{
+			JSONBase: JSONBase{
+				ID: fmt.Sprintf("task%d", i),
+			},
+		})
+	}
+	return TaskList{
+		Items: tasks,
+	}
+}
+
+func validateSyncReplication(t *testing.T, fakeTaskControl *FakeTaskControl, expectedCreates, expectedDeletes int) {
+	if len(fakeTaskControl.controllerSpec) != expectedCreates {
+		t.Errorf("Unexpected number of creates.  Expected %d, saw %d\n", expectedCreates, len(fakeTaskControl.controllerSpec))
+	}
+	if len(fakeTaskControl.deleteTaskID) != expectedDeletes {
+		t.Errorf("Unexpected number of deletes.  Expected %d, saw %d\n", expectedDeletes, len(fakeTaskControl.deleteTaskID))
+	}
+}
+
+func TestSyncReplicationControllerDoesNothing(t *testing.T) {
+	body, _ := json.Marshal(makeTaskList(2))
+	fakeHandler := util.FakeHandler{
+		StatusCode:   200,
+		ResponseBody: string(body),
+	}
+	testServer := httptest.NewTLSServer(&fakeHandler)
+	client := Client{
+		Host: testServer.URL,
+	}
+
+	fakeTaskControl := FakeTaskControl{}
+
+	manager := MakeReplicationManager(nil, &client)
+	manager.taskControl = &fakeTaskControl
+
+	controllerSpec := makeReplicationController(2)
+
+	manager.syncReplicationController(controllerSpec)
+	validateSyncReplication(t, &fakeTaskControl, 0, 0)
+}
+
+func TestSyncReplicationControllerDeletes(t *testing.T) {
+	body, _ := json.Marshal(makeTaskList(2))
+	fakeHandler := util.FakeHandler{
+		StatusCode:   200,
+		ResponseBody: string(body),
+	}
+	testServer := httptest.NewTLSServer(&fakeHandler)
+	client := Client{
+		Host: testServer.URL,
+	}
+
+	fakeTaskControl := FakeTaskControl{}
+
+	manager := MakeReplicationManager(nil, &client)
+	manager.taskControl = &fakeTaskControl
+
+	controllerSpec := makeReplicationController(1)
+
+	manager.syncReplicationController(controllerSpec)
+	validateSyncReplication(t, &fakeTaskControl, 0, 1)
+}
+
+func TestSyncReplicationControllerCreates(t *testing.T) {
+	body := "{ \"items\": [] }"
+	fakeHandler := util.FakeHandler{
+		StatusCode:   200,
+		ResponseBody: string(body),
+	}
+	testServer := httptest.NewTLSServer(&fakeHandler)
+	client := Client{
+		Host: testServer.URL,
+	}
+
+	fakeTaskControl := FakeTaskControl{}
+
+	manager := MakeReplicationManager(nil, &client)
+	manager.taskControl = &fakeTaskControl
+
+	controllerSpec := makeReplicationController(2)
+
+	manager.syncReplicationController(controllerSpec)
+	validateSyncReplication(t, &fakeTaskControl, 2, 0)
+}
+
+func TestCreateReplica(t *testing.T) {
+	body := "{}"
+	fakeHandler := util.FakeHandler{
+		StatusCode:   200,
+		ResponseBody: string(body),
+	}
+	testServer := httptest.NewTLSServer(&fakeHandler)
+	client := Client{
+		Host: testServer.URL,
+	}
+
+	taskControl := RealTaskControl{
+		kubeClient: client,
+	}
+
+	controllerSpec := ReplicationController{
+		DesiredState: ReplicationControllerState{
+			TaskTemplate: TaskTemplate{
+				DesiredState: TaskState{
+					Manifest: ContainerManifest{
+						Containers: []Container{
+							Container{
+								Image: "foo/bar",
+							},
+						},
+					},
+				},
+				Labels: map[string]string{
+					"name": "foo",
+					"type": "production",
+				},
+			},
+		},
+	}
+
+	taskControl.createReplica(controllerSpec)
+
+	//expectedTask := Task{
+	//	Labels:       controllerSpec.DesiredState.TaskTemplate.Labels,
+	//	DesiredState: controllerSpec.DesiredState.TaskTemplate.DesiredState,
+	//}
+	// TODO: fix this so that it validates the body.
+	fakeHandler.ValidateRequest(t, makeUrl("/tasks"), "POST", nil)
+}
+
+func TestHandleWatchResponseNotSet(t *testing.T) {
+	body, _ := json.Marshal(makeTaskList(2))
+	fakeHandler := util.FakeHandler{
+		StatusCode:   200,
+		ResponseBody: string(body),
+	}
+	testServer := httptest.NewTLSServer(&fakeHandler)
+	client := Client{
+		Host: testServer.URL,
+	}
+
+	fakeTaskControl := FakeTaskControl{}
+
+	manager := MakeReplicationManager(nil, &client)
+	manager.taskControl = &fakeTaskControl
+	_, err := manager.handleWatchResponse(&etcd.Response{
+		Action: "delete",
+	})
+	expectNoError(t, err)
+}
+
+func TestHandleWatchResponseNoNode(t *testing.T) {
+	body, _ := json.Marshal(makeTaskList(2))
+	fakeHandler := util.FakeHandler{
+		StatusCode:   200,
+		ResponseBody: string(body),
+	}
+	testServer := httptest.NewTLSServer(&fakeHandler)
+	client := Client{
+		Host: testServer.URL,
+	}
+
+	fakeTaskControl := FakeTaskControl{}
+
+	manager := MakeReplicationManager(nil, &client)
+	manager.taskControl = &fakeTaskControl
+	_, err := manager.handleWatchResponse(&etcd.Response{
+		Action: "set",
+	})
+	if err == nil {
+		t.Error("Unexpected non-error")
+	}
+}
+
+func TestHandleWatchResponseBadData(t *testing.T) {
+	body, _ := json.Marshal(makeTaskList(2))
+	fakeHandler := util.FakeHandler{
+		StatusCode:   200,
+		ResponseBody: string(body),
+	}
+	testServer := httptest.NewTLSServer(&fakeHandler)
+	client := Client{
+		Host: testServer.URL,
+	}
+
+	fakeTaskControl := FakeTaskControl{}
+
+	manager := MakeReplicationManager(nil, &client)
+	manager.taskControl = &fakeTaskControl
+	_, err := manager.handleWatchResponse(&etcd.Response{
+		Action: "set",
+		Node: &etcd.Node{
+			Value: "foobar",
+		},
+	})
+	if err == nil {
+		t.Error("Unexpected non-error")
+	}
+}
+
+func TestHandleWatchResponse(t *testing.T) {
+	body, _ := json.Marshal(makeTaskList(2))
+	fakeHandler := util.FakeHandler{
+		StatusCode:   200,
+		ResponseBody: string(body),
+	}
+	testServer := httptest.NewTLSServer(&fakeHandler)
+	client := Client{
+		Host: testServer.URL,
+	}
+
+	fakeTaskControl := FakeTaskControl{}
+
+	manager := MakeReplicationManager(nil, &client)
+	manager.taskControl = &fakeTaskControl
+
+	controller := makeReplicationController(2)
+
+	data, err := json.Marshal(controller)
+	expectNoError(t, err)
+	controllerOut, err := manager.handleWatchResponse(&etcd.Response{
+		Action: "set",
+		Node: &etcd.Node{
+			Value: string(data),
+		},
+	})
+	if err != nil {
+		t.Errorf("Unexpected error: %#v", err)
+	}
+	if !reflect.DeepEqual(controller, *controllerOut) {
+		t.Errorf("Unexpected mismatch.  Expected %#v, Saw: %#v", controller, controllerOut)
+	}
+}

pkg/registry/scheduler.go

@@ -0,0 +1,115 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+package registry
+
+import (
+	"fmt"
+	"math/rand"
+
+	. "github.com/GoogleCloudPlatform/kubernetes/pkg/api"
+)
+
+// Scheduler is an interface implemented by things that know how to schedule tasks onto machines.
+type Scheduler interface {
+	Schedule(Task) (string, error)
+}
+
+// RandomScheduler choses machines uniformly at random.
+type RandomScheduler struct {
+	machines []string
+	random   rand.Rand
+}
+
+func MakeRandomScheduler(machines []string, random rand.Rand) Scheduler {
+	return &RandomScheduler{
+		machines: machines,
+		random:   random,
+	}
+}
+
+func (s *RandomScheduler) Schedule(task Task) (string, error) {
+	return s.machines[s.random.Int()%len(s.machines)], nil
+}
+
+// RoundRobinScheduler chooses machines in order.
+type RoundRobinScheduler struct {
+	machines     []string
+	currentIndex int
+}
+
+func MakeRoundRobinScheduler(machines []string) Scheduler {
+	return &RoundRobinScheduler{
+		machines:     machines,
+		currentIndex: 0,
+	}
+}
+
+func (s *RoundRobinScheduler) Schedule(task Task) (string, error) {
+	result := s.machines[s.currentIndex]
+	s.currentIndex = (s.currentIndex + 1) % len(s.machines)
+	return result, nil
+}
+
+type FirstFitScheduler struct {
+	machines []string
+	registry TaskRegistry
+}
+
+func MakeFirstFitScheduler(machines []string, registry TaskRegistry) Scheduler {
+	return &FirstFitScheduler{
+		machines: machines,
+		registry: registry,
+	}
+}
+
+func (s *FirstFitScheduler) containsPort(task Task, port Port) bool {
+	for _, container := range task.DesiredState.Manifest.Containers {
+		for _, taskPort := range container.Ports {
+			if taskPort.HostPort == port.HostPort {
+				return true
+			}
+		}
+	}
+	return false
+}
+
+func (s *FirstFitScheduler) Schedule(task Task) (string, error) {
+	machineToTasks := map[string][]Task{}
+	tasks, err := s.registry.ListTasks(nil)
+	if err != nil {
+		return "", err
+	}
+	for _, scheduledTask := range tasks {
+		host := scheduledTask.CurrentState.Host
+		machineToTasks[host] = append(machineToTasks[host], scheduledTask)
+	}
+	for _, machine := range s.machines {
+		taskFits := true
+		for _, scheduledTask := range machineToTasks[machine] {
+			for _, container := range task.DesiredState.Manifest.Containers {
+				for _, port := range container.Ports {
+					if s.containsPort(scheduledTask, port) {
+						taskFits = false
+					}
+				}
+			}
+		}
+		if taskFits {
+			return machine, nil
+		}
+	}
+	return "", fmt.Errorf("Failed to find fit for %#v", task)
+}

pkg/registry/scheduler_test.go

@@ -0,0 +1,110 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+package registry
+
+import (
+	"math/rand"
+	"testing"
+
+	. "github.com/GoogleCloudPlatform/kubernetes/pkg/api"
+)
+
+func expectSchedule(scheduler Scheduler, task Task, expected string, t *testing.T) {
+	actual, err := scheduler.Schedule(task)
+	expectNoError(t, err)
+	if actual != expected {
+		t.Errorf("Unexpected scheduling value: %d, expected %d", actual, expected)
+	}
+}
+
+func TestRoundRobinScheduler(t *testing.T) {
+	scheduler := MakeRoundRobinScheduler([]string{"m1", "m2", "m3", "m4"})
+	expectSchedule(scheduler, Task{}, "m1", t)
+	expectSchedule(scheduler, Task{}, "m2", t)
+	expectSchedule(scheduler, Task{}, "m3", t)
+	expectSchedule(scheduler, Task{}, "m4", t)
+}
+
+func TestRandomScheduler(t *testing.T) {
+	random := rand.New(rand.NewSource(0))
+	scheduler := MakeRandomScheduler([]string{"m1", "m2", "m3", "m4"}, *random)
+	_, err := scheduler.Schedule(Task{})
+	expectNoError(t, err)
+}
+
+func TestFirstFitSchedulerNothingScheduled(t *testing.T) {
+	mockRegistry := MockTaskRegistry{}
+	scheduler := MakeFirstFitScheduler([]string{"m1", "m2", "m3"}, &mockRegistry)
+	expectSchedule(scheduler, Task{}, "m1", t)
+}
+
+func makeTask(host string, hostPorts ...int) Task {
+	networkPorts := []Port{}
+	for _, port := range hostPorts {
+		networkPorts = append(networkPorts, Port{HostPort: port})
+	}
+	return Task{
+		CurrentState: TaskState{
+			Host: host,
+		},
+		DesiredState: TaskState{
+			Manifest: ContainerManifest{
+				Containers: []Container{
+					Container{
+						Ports: networkPorts,
+					},
+				},
+			},
+		},
+	}
+}
+
+func TestFirstFitSchedulerFirstScheduled(t *testing.T) {
+	mockRegistry := MockTaskRegistry{
+		tasks: []Task{
+			makeTask("m1", 8080),
+		},
+	}
+	scheduler := MakeFirstFitScheduler([]string{"m1", "m2", "m3"}, &mockRegistry)
+	expectSchedule(scheduler, makeTask("", 8080), "m2", t)
+}
+
+func TestFirstFitSchedulerFirstScheduledComplicated(t *testing.T) {
+	mockRegistry := MockTaskRegistry{
+		tasks: []Task{
+			makeTask("m1", 80, 8080),
+			makeTask("m2", 8081, 8082, 8083),
+			makeTask("m3", 80, 443, 8085),
+		},
+	}
+	scheduler := MakeFirstFitScheduler([]string{"m1", "m2", "m3"}, &mockRegistry)
+	expectSchedule(scheduler, makeTask("", 8080, 8081), "m3", t)
+}
+
+func TestFirstFitSchedulerFirstScheduledImpossible(t *testing.T) {
+	mockRegistry := MockTaskRegistry{
+		tasks: []Task{
+			makeTask("m1", 8080),
+			makeTask("m2", 8081),
+			makeTask("m3", 8080),
+		},
+	}
+	scheduler := MakeFirstFitScheduler([]string{"m1", "m2", "m3"}, &mockRegistry)
+	_, err := scheduler.Schedule(makeTask("", 8080, 8081))
+	if err == nil {
+		t.Error("Unexpected non-error.")
+	}
+}

pkg/registry/service_registry.go

@@ -0,0 +1,86 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+package registry
+
+import (
+	"encoding/json"
+	"net/url"
+	"strconv"
+	"strings"
+
+	. "github.com/GoogleCloudPlatform/kubernetes/pkg/api"
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/apiserver"
+)
+
+type ServiceRegistry interface {
+	ListServices() (ServiceList, error)
+	CreateService(svc Service) error
+	GetService(name string) (*Service, error)
+	DeleteService(name string) error
+	UpdateService(svc Service) error
+	UpdateEndpoints(e Endpoints) error
+}
+
+type ServiceRegistryStorage struct {
+	registry ServiceRegistry
+}
+
+func MakeServiceRegistryStorage(registry ServiceRegistry) apiserver.RESTStorage {
+	return &ServiceRegistryStorage{registry: registry}
+}
+
+// GetServiceEnvironmentVariables populates a list of environment variables that are use
+// in the container environment to get access to services.
+func GetServiceEnvironmentVariables(registry ServiceRegistry, machine string) ([]EnvVar, error) {
+	var result []EnvVar
+	services, err := registry.ListServices()
+	if err != nil {
+		return result, err
+	}
+	for _, service := range services.Items {
+		name := strings.ToUpper(service.ID) + "_SERVICE_PORT"
+		value := strconv.Itoa(service.Port)
+		result = append(result, EnvVar{Name: name, Value: value})
+	}
+	result = append(result, EnvVar{Name: "SERVICE_HOST", Value: machine})
+	return result, nil
+}
+
+func (sr *ServiceRegistryStorage) List(*url.URL) (interface{}, error) {
+	return sr.registry.ListServices()
+}
+
+func (sr *ServiceRegistryStorage) Get(id string) (interface{}, error) {
+	return sr.registry.GetService(id)
+}
+
+func (sr *ServiceRegistryStorage) Delete(id string) error {
+	return sr.registry.DeleteService(id)
+}
+
+func (sr *ServiceRegistryStorage) Extract(body string) (interface{}, error) {
+	var svc Service
+	err := json.Unmarshal([]byte(body), &svc)
+	return svc, err
+}
+
+func (sr *ServiceRegistryStorage) Create(obj interface{}) error {
+	return sr.registry.CreateService(obj.(Service))
+}
+
+func (sr *ServiceRegistryStorage) Update(obj interface{}) error {
+	return sr.registry.UpdateService(obj.(Service))
+}

pkg/registry/task_registry.go

@@ -0,0 +1,119 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+package registry
+
+import (
+	"encoding/json"
+	"fmt"
+	"net/url"
+
+	. "github.com/GoogleCloudPlatform/kubernetes/pkg/api"
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/apiserver"
+	"github.com/GoogleCloudPlatform/kubernetes/pkg/client"
+)
+
+// TaskRegistryStorage implements the RESTStorage interface in terms of a TaskRegistry
+type TaskRegistryStorage struct {
+	registry      TaskRegistry
+	containerInfo client.ContainerInfo
+	scheduler     Scheduler
+}
+
+func MakeTaskRegistryStorage(registry TaskRegistry, containerInfo client.ContainerInfo, scheduler Scheduler) apiserver.RESTStorage {
+	return &TaskRegistryStorage{
+		registry:      registry,
+		containerInfo: containerInfo,
+		scheduler:     scheduler,
+	}
+}
+
+// LabelMatch tests to see if a Task's labels map contains 'key' mapping to 'value'
+func LabelMatch(task Task, queryKey, queryValue string) bool {
+	for key, value := range task.Labels {
+		if queryKey == key && queryValue == value {
+			return true
+		}
+	}
+	return false
+}
+
+// LabelMatch tests to see if a Task's labels map contains all key/value pairs in 'labelQuery'
+func LabelsMatch(task Task, labelQuery *map[string]string) bool {
+	if labelQuery == nil {
+		return true
+	}
+	for key, value := range *labelQuery {
+		if !LabelMatch(task, key, value) {
+			return false
+		}
+	}
+	return true
+}
+
+func (storage *TaskRegistryStorage) List(url *url.URL) (interface{}, error) {
+	var result TaskList
+	var query *map[string]string
+	if url != nil {
+		queryMap := client.DecodeLabelQuery(url.Query().Get("labels"))
+		query = &queryMap
+	}
+	tasks, err := storage.registry.ListTasks(query)
+	if err == nil {
+		result = TaskList{
+			Items: tasks,
+		}
+	}
+	return result, err
+}
+
+func (storage *TaskRegistryStorage) Get(id string) (interface{}, error) {
+	task, err := storage.registry.GetTask(id)
+	if err != nil {
+		return task, err
+	}
+	info, err := storage.containerInfo.GetContainerInfo(task.CurrentState.Host, id)
+	if err != nil {
+		return task, err
+	}
+	task.CurrentState.Info = info
+	return task, err
+}
+
+func (storage *TaskRegistryStorage) Delete(id string) error {
+	return storage.registry.DeleteTask(id)
+}
+
+func (storage *TaskRegistryStorage) Extract(body string) (interface{}, error) {
+	task := Task{}
+	err := json.Unmarshal([]byte(body), &task)
+	return task, err
+}
+
+func (storage *TaskRegistryStorage) Create(task interface{}) error {
+	taskObj := task.(Task)
+	if len(taskObj.ID) == 0 {
+		return fmt.Errorf("ID is unspecified: %#v", task)
+	}
+	machine, err := storage.scheduler.Schedule(taskObj)
+	if err != nil {
+		return err
+	}
+	return storage.registry.CreateTask(machine, taskObj)
+}
+
+func (storage *TaskRegistryStorage) Update(task interface{}) error {
+	return storage.registry.UpdateTask(task.(Task))
+}

pkg/registry/task_registry_test.go

@@ -0,0 +1,204 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+package registry
+
+import (
+	"encoding/json"
+	"fmt"
+	"testing"
+
+	. "github.com/GoogleCloudPlatform/kubernetes/pkg/api"
+)
+
+type MockTaskRegistry struct {
+	err   error
+	tasks []Task
+}
+
+func expectNoError(t *testing.T, err error) {
+	if err != nil {
+		t.Errorf("Unexpected error: %#v", err)
+	}
+}
+
+func (registry *MockTaskRegistry) ListTasks(*map[string]string) ([]Task, error) {
+	return registry.tasks, registry.err
+}
+
+func (registry *MockTaskRegistry) GetTask(taskId string) (*Task, error) {
+	return &Task{}, registry.err
+}
+
+func (registry *MockTaskRegistry) CreateTask(machine string, task Task) error {
+	return registry.err
+}
+
+func (registry *MockTaskRegistry) UpdateTask(task Task) error {
+	return registry.err
+}
+func (registry *MockTaskRegistry) DeleteTask(taskId string) error {
+	return registry.err
+}
+
+func TestListTasksError(t *testing.T) {
+	mockRegistry := MockTaskRegistry{
+		err: fmt.Errorf("Test Error"),
+	}
+	storage := TaskRegistryStorage{
+		registry: &mockRegistry,
+	}
+	tasks, err := storage.List(nil)
+	if err != mockRegistry.err {
+		t.Errorf("Expected %#v, Got %#v", mockRegistry.err, err)
+	}
+	if len(tasks.(TaskList).Items) != 0 {
+		t.Errorf("Unexpected non-zero task list: %#v", tasks)
+	}
+}
+
+func TestListEmptyTaskList(t *testing.T) {
+	mockRegistry := MockTaskRegistry{}
+	storage := TaskRegistryStorage{
+		registry: &mockRegistry,
+	}
+	tasks, err := storage.List(nil)
+	expectNoError(t, err)
+	if len(tasks.(TaskList).Items) != 0 {
+		t.Errorf("Unexpected non-zero task list: %#v", tasks)
+	}
+}
+
+func TestListTaskList(t *testing.T) {
+	mockRegistry := MockTaskRegistry{
+		tasks: []Task{
+			Task{
+				JSONBase: JSONBase{
+					ID: "foo",
+				},
+			},
+			Task{
+				JSONBase: JSONBase{
+					ID: "bar",
+				},
+			},
+		},
+	}
+	storage := TaskRegistryStorage{
+		registry: &mockRegistry,
+	}
+	tasksObj, err := storage.List(nil)
+	tasks := tasksObj.(TaskList)
+	expectNoError(t, err)
+	if len(tasks.Items) != 2 {
+		t.Errorf("Unexpected task list: %#v", tasks)
+	}
+	if tasks.Items[0].ID != "foo" {
+		t.Errorf("Unexpected task: %#v", tasks.Items[0])
+	}
+	if tasks.Items[1].ID != "bar" {
+		t.Errorf("Unexpected task: %#v", tasks.Items[1])
+	}
+}
+
+func TestExtractJson(t *testing.T) {
+	mockRegistry := MockTaskRegistry{}
+	storage := TaskRegistryStorage{
+		registry: &mockRegistry,
+	}
+	task := Task{
+		JSONBase: JSONBase{
+			ID: "foo",
+		},
+	}
+	body, err := json.Marshal(task)
+	expectNoError(t, err)
+	taskOut, err := storage.Extract(string(body))
+	expectNoError(t, err)
+	jsonOut, err := json.Marshal(taskOut)
+	expectNoError(t, err)
+	if string(body) != string(jsonOut) {
+		t.Errorf("Expected %#v, found %#v", task, taskOut)
+	}
+}
+
+func expectLabelMatch(t *testing.T, task Task, key, value string) {
+	if !LabelMatch(task, key, value) {
+		t.Errorf("Unexpected match failure: %#v %s %s", task, key, value)
+	}
+}
+
+func expectNoLabelMatch(t *testing.T, task Task, key, value string) {
+	if LabelMatch(task, key, value) {
+		t.Errorf("Unexpected match success: %#v %s %s", task, key, value)
+	}
+}
+
+func expectLabelsMatch(t *testing.T, task Task, query *map[string]string) {
+	if !LabelsMatch(task, query) {
+		t.Errorf("Unexpected match failure: %#v %#v", task, *query)
+	}
+}
+
+func expectNoLabelsMatch(t *testing.T, task Task, query *map[string]string) {
+	if LabelsMatch(task, query) {
+		t.Errorf("Unexpected match success: %#v %#v", task, *query)
+	}
+}
+
+func TestLabelMatch(t *testing.T) {
+	task := Task{
+		Labels: map[string]string{
+			"foo": "bar",
+			"baz": "blah",
+		},
+	}
+	expectLabelMatch(t, task, "foo", "bar")
+	expectLabelMatch(t, task, "baz", "blah")
+	expectNoLabelMatch(t, task, "foo", "blah")
+	expectNoLabelMatch(t, task, "baz", "bar")
+}
+
+func TestLabelsMatch(t *testing.T) {
+	task := Task{
+		Labels: map[string]string{
+			"foo": "bar",
+			"baz": "blah",
+		},
+	}
+	expectLabelsMatch(t, task, &map[string]string{})
+	expectLabelsMatch(t, task, &map[string]string{
+		"foo": "bar",
+	})
+	expectLabelsMatch(t, task, &map[string]string{
+		"baz": "blah",
+	})
+	expectLabelsMatch(t, task, &map[string]string{
+		"foo": "bar",
+		"baz": "blah",
+	})
+	expectNoLabelsMatch(t, task, &map[string]string{
+		"foo": "blah",
+	})
+	expectNoLabelsMatch(t, task, &map[string]string{
+		"baz": "bar",
+	})
+	expectNoLabelsMatch(t, task, &map[string]string{
+		"foo":    "bar",
+		"foobar": "bar",
+		"baz":    "blah",
+	})
+
+}

pkg/util/fake_handler.go

@@ -0,0 +1,56 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+package util
+
+import (
+	"io/ioutil"
+	"log"
+	"net/http"
+	"testing"
+)
+
+// FakeHandler is to assist in testing HTTP requests.
+type FakeHandler struct {
+	RequestReceived *http.Request
+	StatusCode      int
+	ResponseBody    string
+}
+
+func (f *FakeHandler) ServeHTTP(response http.ResponseWriter, request *http.Request) {
+	f.RequestReceived = request
+	response.WriteHeader(f.StatusCode)
+	response.Write([]byte(f.ResponseBody))
+
+	bodyReceived, err := ioutil.ReadAll(request.Body)
+	if err != nil {
+		log.Printf("Received read error: %#v", err)
+	}
+	f.ResponseBody = string(bodyReceived)
+}
+
+func (f FakeHandler) ValidateRequest(t *testing.T, expectedPath, expectedMethod string, body *string) {
+	if f.RequestReceived.URL.Path != expectedPath {
+		t.Errorf("Unexpected request path: %s", f.RequestReceived.URL.Path)
+	}
+	if f.RequestReceived.Method != expectedMethod {
+		t.Errorf("Unexpected method: %s", f.RequestReceived.Method)
+	}
+	if body != nil {
+		if *body != f.ResponseBody {
+			t.Errorf("Received body:\n%s\n Doesn't match expected body:\n%s", f.ResponseBody, *body)
+		}
+	}
+}

pkg/util/stringlist.go

@@ -0,0 +1,37 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+package util
+
+import (
+	"fmt"
+	"strings"
+)
+
+type StringList []string
+
+func (sl *StringList) String() string {
+	return fmt.Sprint(*sl)
+}
+
+func (sl *StringList) Set(value string) error {
+	for _, s := range strings.Split(value, ",") {
+		if len(s) == 0 {
+			return fmt.Errorf("value should not be an empty string")
+		}
+		*sl = append(*sl, s)
+	}
+	return nil
+}

pkg/util/stringlist_test.go

@@ -0,0 +1,41 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+package util
+
+import (
+	"reflect"
+	"testing"
+)
+
+func TestStringListSet(t *testing.T) {
+	var sl StringList
+	sl.Set("foo,bar")
+	sl.Set("hop")
+	expected := []string{"foo", "bar", "hop"}
+	if reflect.DeepEqual(expected, []string(sl)) == false {
+		t.Errorf("expected: %v, got: %v:", expected, sl)
+	}
+}
+
+func TestStringListSetErr(t *testing.T) {
+	var sl StringList
+	if err := sl.Set(""); err == nil {
+		t.Errorf("expected error for empty string")
+	}
+	if err := sl.Set(","); err == nil {
+		t.Errorf("expected error for list of empty strings")
+	}
+}

pkg/util/util.go

@@ -0,0 +1,47 @@
+/*
+Copyright 2014 Google Inc. All rights reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+package util
+
+import (
+	"encoding/json"
+	"log"
+	"time"
+)
+
+// Simply catches a crash and logs an error. Meant to be called via defer.
+func HandleCrash() {
+	r := recover()
+	if r != nil {
+		log.Printf("Recovered from panic: %#v", r)
+	}
+}
+
+// Loops forever running f every d.  Catches any panics, and keeps going.
+func Forever(f func(), period time.Duration) {
+	for {
+		func() {
+			defer HandleCrash()
+			f()
+		}()
+		time.Sleep(period)
+	}
+}
+
+// Returns o marshalled as a JSON string, ignoring any errors.
+func MakeJSONString(o interface{}) string {
+	data, _ := json.Marshal(o)
+	return string(data)
+}

src/release/config.sh

@@ -0,0 +1,83 @@
+# Copyright 2014 Google Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# A set of defaults for Kubernetes releases
+
+PROJECT=$(gcloud config list project | tail -n 1 | cut -f 3 -d ' ')
+
+if which md5 > /dev/null; then
+  HASH=$(md5 -q -s $PROJECT)
+else
+  HASH=$(echo -n "$PROJECT" | md5sum)
+fi
+HASH=${HASH:0:5}
+RELEASE_BUCKET=${RELEASE_BUCKET-gs://kubernetes-releases-$HASH/}
+RELEASE_PREFIX=${RELEASE_PREFIX-devel/$USER/}
+RELEASE_NAME=${RELEASE_NAME-r$(date -u +%Y%m%d-%H%M%S)}
+
+# This is a 'soft link' to the release in question.  It is a single line file to
+# the full GS path for a release.
+RELEASE_TAG=${RELEASE_TAG-testing}
+
+RELEASE_TAR_FILE=master-release.tgz
+
+RELEASE_FULL_PATH=$RELEASE_BUCKET$RELEASE_PREFIX$RELEASE_NAME
+RELEASE_FULL_TAG_PATH=$RELEASE_BUCKET$RELEASE_PREFIX$RELEASE_TAG
+
+# Takes a release path ($1 if passed, otherwise $RELEASE_FULL_TAG_PATH) and
+# computes the normalized release path. Results are stored in
+# $RELEASE_NORMALIZED.  Returns 0 if a valid release can be found.
+function normalize_release() {
+  RELEASE_NORMALIZED=${1-$RELEASE_FULL_TAG_PATH}
+
+  # First test to see if there is a valid release at this path.
+  if gsutil -q stat $RELEASE_NORMALIZED/$RELEASE_TAR_FILE; then
+    return 0
+  fi
+
+  # Check if this is a simple file.  If so, read it and use the result as the
+  # new RELEASE_NORMALIZED.
+  if gsutil -q stat $RELEASE_NORMALIZED; then
+    RELEASE_NORMALIZED=$(gsutil -q cat $RELEASE_NORMALIZED)
+    normalize_release $RELEASE_NORMALIZED
+    return
+  fi
+  return 1
+}
+
+# Sets a tag ($1) to a release ($2)
+function set_tag() {
+  echo $2 | gsutil -q cp - $1
+
+  gsutil -q setmeta -h "Cache-Control:private, max-age=0, no-transform" $1
+  make_public_readable $1
+}
+
+# Makes a GCS object ($1) publicly readable
+function make_public_readable() {
+  # Ideally we'd run the command below.  But this is currently broken in the
+  # newest version of gsutil.  Instead, download the ACL and edit the json
+  # quickly.
+
+  # gsutil -q acl ch -g AllUsers:R $1
+
+  TMPFILE=$(mktemp -t release 2>/dev/null || mktemp -t release.XXXX)
+
+  gsutil -q acl get $1 \
+    | python $(dirname $0)/make-public-gcs-acl.py \
+    > $TMPFILE
+    gsutil -q acl set $TMPFILE $RELEASE_FULL_PATH/$x
+
+  rm $TMPFILE
+}

src/release/launch-kubernetes-base.sh

@@ -0,0 +1,48 @@
+# Copyright 2014 Google Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Prerequisites
+# TODO (bburns): Perhaps install cloud SDK automagically if we can't find it?
+
+# Exit on any error
+set -e
+
+echo "Auto installer for launching Kubernetes"
+echo "Release: $RELEASE_PREFIX$RELEASE_NAME"
+
+# Make sure that prerequisites are installed.
+for x in gcloud gsutil; do
+  if [ "$(which $x)" == "" ]; then
+    echo "Can't find $x in PATH, please fix and retry."
+    exit 1
+  fi
+done
+
+# TODO(jbeda): Provide a way to install this in to someplace beyond a temp dir
+# so that users have access to local tools.
+TMPDIR=$(mktemp -d /tmp/installer.kubernetes.XXXXXX)
+
+cd $TMPDIR
+
+echo "Downloading support files"
+gsutil cp $RELEASE_FULL_PATH/launch-kubernetes.tgz .
+
+tar xzf launch-kubernetes.tgz
+
+./src/scripts/kube-up.sh $RELEASE_FULL_PATH
+
+cd /
+
+# clean up
+# rm -rf $TMPDIR

src/release/make-public-gcs-acl.py

@@ -0,0 +1,12 @@
+# This is a quick script that adds AllUsers as READER to a JSON file
+# representing an ACL on a GCS object.  This is a quick workaround for a bug in
+# gsutil.
+import json
+import sys
+
+acl = json.load(sys.stdin)
+acl.append({
+    "entity": "allUsers",
+    "role": "READER"
+  })
+json.dump(acl, sys.stdout)

src/release/master-release-install.sh

@@ -0,0 +1,46 @@
+#!/bin/bash
+
+# Copyright 2014 Google Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# This file is meant to run on the master.  It takes the release in the current
+# directory and installs everything that needs to be installed.  It will then
+# also kick off a saltstack config pass
+
+RELEASE_BASE=$(dirname $0)/../..
+
+echo "Installing release files"
+
+# Put all of the salt stuff under /srv
+mkdir -p /srv
+cp -R --preserve=mode $RELEASE_BASE/src/saltbase/* /srv
+
+# Copy various go source code into the right places in the salt directory
+# hieararchy so it can be downloaded/built on all the nodes.
+mkdir -p /srv/salt/apiserver/go
+cp -R --preserve=mode $RELEASE_BASE/src/go/* /srv/salt/apiserver/go
+
+mkdir -p /srv/salt/kube-proxy/go
+cp -R --preserve=mode $RELEASE_BASE/src/go/* /srv/salt/kube-proxy/go
+
+mkdir -p /srv/salt/controller-manager/go
+cp -R --preserve=mode $RELEASE_BASE/src/go/* /srv/salt/controller-manager/go
+
+mkdir -p /srv/salt/kubelet/go
+cp -R --preserve=mode $RELEASE_BASE/src/go/* /srv/salt/kubelet/go
+
+mkdir -p /srv/salt/third-party/go
+cp -R --preserve=mode $RELEASE_BASE/third_party/go/* /srv/salt/third-party/go
+
+

src/release/release.sh

@@ -0,0 +1,96 @@
+#!/bin/bash
+
+# Copyright 2014 Google Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# This script will build and release Kubernetes.
+#
+# The main parameters to this script come from the config.sh file.  This is set
+# up by default for development releases.  Feel free to edit it or override some
+# of the variables there.
+
+# exit on any error
+set -e
+
+source $(dirname $0)/config.sh
+
+cd $(dirname $0)/../..
+
+# First build the release tar.  This gets copied on to the master and installed
+# from there.  It includes the go source for the necessary servers along with
+# the salt configs.
+rm -rf release/*
+
+MASTER_RELEASE_DIR=release/master-release
+mkdir -p $MASTER_RELEASE_DIR/bin
+mkdir -p $MASTER_RELEASE_DIR/src/scripts
+mkdir -p $MASTER_RELEASE_DIR/third_party/go
+
+echo "Building release tree"
+cp src/release/master-release-install.sh $MASTER_RELEASE_DIR/src/scripts/master-release-install.sh
+cp -r src/saltbase $MASTER_RELEASE_DIR/src/saltbase
+cp -r third_party $MASTER_RELEASE_DIR/third_party/go/src
+
+function find_go_files() {
+  find * -not \( \
+      \( \
+        -wholename 'third_party' \
+        -o -wholename 'release' \
+      \) -prune \
+    \) -name '*.go'
+}
+for f in $(find_go_files); do
+  mkdir -p $MASTER_RELEASE_DIR/src/go/$(dirname ${f})
+  cp ${f} ${MASTER_RELEASE_DIR}/src/go/${f}
+done
+
+echo "Packaging release"
+tar cz -C release -f release/master-release.tgz master-release
+
+echo "Building launch script"
+# Create the local install script.  These are the tools to install the local
+# tools and launch a new cluster.
+LOCAL_RELEASE_DIR=release/local-release
+mkdir -p $LOCAL_RELEASE_DIR/src
+
+cp -r src/templates $LOCAL_RELEASE_DIR/src/templates
+cp -r src/scripts $LOCAL_RELEASE_DIR/src/scripts
+
+tar cz -C $LOCAL_RELEASE_DIR -f release/launch-kubernetes.tgz .
+
+echo "#!/bin/bash" >> release/launch-kubernetes.sh
+echo "RELEASE_TAG=$RELEASE_TAG" >> release/launch-kubernetes.sh
+echo "RELEASE_PREFIX=$RELEASE_PREFIX" >> release/launch-kubernetes.sh
+echo "RELEASE_NAME=$RELEASE_NAME" >> release/launch-kubernetes.sh
+echo "RELEASE_FULL_PATH=$RELEASE_FULL_PATH" >> release/launch-kubernetes.sh
+cat src/release/launch-kubernetes-base.sh >> release/launch-kubernetes.sh
+chmod a+x release/launch-kubernetes.sh
+
+# Now copy everything up to the release structure on GS
+echo "Uploading to Google Storage"
+if ! gsutil ls $RELEASE_BUCKET > /dev/null; then
+  echo "Creating $RELEASE_BUCKET"
+  gsutil mb $RELEASE_BUCKET
+fi
+for x in master-release.tgz launch-kubernetes.tgz launch-kubernetes.sh; do
+  gsutil -q cp release/$x $RELEASE_FULL_PATH/$x
+
+  make_public_readable $RELEASE_FULL_PATH/$x
+done
+set_tag $RELEASE_FULL_TAG_PATH $RELEASE_FULL_PATH
+
+echo "Release pushed ($RELEASE_PREFIX$RELEASE_NAME).  Launch with:"
+echo
+echo "  curl -s -L ${RELEASE_FULL_PATH/gs:\/\//http://storage.googleapis.com/}/launch-kubernetes.sh | bash"
+echo

src/saltbase/pillar/mine.sls

@@ -0,0 +1,4 @@
+# Allow everyone to see cached values of who sits at what IP
+mine_functions:
+  network.ip_addrs: [eth0]
+  grains.items: []

src/saltbase/pillar/top.sls

@@ -0,0 +1,3 @@
+base:
+  '*':
+    - mine

src/saltbase/reactor/start.sls

@@ -0,0 +1,5 @@
+
+# This runs highstate on the target node
+highstate_run:
+  cmd.state.highstate:
+    - tgt: {{ data['id'] }}

src/saltbase/salt/_states/container_bridge.py

@@ -0,0 +1,163 @@
+# Copyright 2014 Google Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import re
+
+import salt.exceptions
+import salt.utils.ipaddr as ipaddr
+
+def ensure(name, cidr, mtu=1460):
+    '''
+    Ensure that a bridge (named <name>) is configured for contianers.
+
+    Under the covers we will make sure that
+      - The bridge exists
+      - The MTU is set
+      - The correct network is added to the bridge
+      - iptables is set up for MASQUARADE for egress
+
+    cidr:
+        The cidr range in the form of 10.244.x.0/24
+    mtu:
+        The MTU to set on the interface
+    '''
+    ret = {'name': name, 'changes': {}, 'result': False, 'comment': ''}
+
+    iptables_rule = {
+        'table': 'nat',
+        'chain': 'POSTROUTING',
+        'rule': '-o eth0 -j MASQUERADE \! -d 10.0.0.0/8'
+    }
+
+    def bridge_exists(name):
+        'Determine if a bridge exists already.'
+        out = __salt__['cmd.run_stdout']('brctl show {0}'.format(name))
+        for line in out.splitlines():
+            # get rid of first line
+            if line.startswith('bridge name'):
+                continue
+            # get rid of ^\n's
+            vals = line.split()
+            if not vals:
+                continue
+            if len(vals) > 1:
+                return True
+        return False
+
+    def get_ip_addr_details(name):
+        'For the given interface, get address details.'
+        out = __salt__['cmd.run']('ip addr show dev {0}'.format(name))
+        ret = { 'networks': [] }
+        for line in out.splitlines():
+            match = re.match(
+                r'^\d*:\s+([\w.\-]+)(?:@)?([\w.\-]+)?:\s+<(.+)>.*mtu (\d+)',
+                line)
+            if match:
+                iface, parent, attrs, mtu = match.groups()
+                if 'UP' in attrs.split(','):
+                    ret['up'] = True
+                else:
+                    ret['up'] = False
+                if parent:
+                    ret['parent'] = parent
+                ret['mtu'] = int(mtu)
+                continue
+            cols = line.split()
+            if len(cols) > 2 and cols[0] == 'inet':
+                ret['networks'].append(cols[1])
+        return ret
+
+
+    def get_current_state():
+        'Helper that returns a dict of current bridge state.'
+        ret = {}
+        ret['name'] = name
+        ret['exists'] = bridge_exists(name)
+        if ret['exists']:
+            ret['details'] = get_ip_addr_details(name)
+        else:
+            ret['details'] = {}
+        # This module function is strange and returns True if the rule exists.
+        # If not, it returns a string with the error from the call to iptables.
+        ret['iptables_rule_exists'] = \
+          __salt__['iptables.check'](**iptables_rule) == True
+        return ret
+
+    # This is a little hacky.  I should probably import a real library for this
+    # but this'll work for now.
+    try:
+        cidr_network = ipaddr.IPv4Network(cidr, strict=True)
+    except Exception:
+        raise salt.exceptions.SaltInvocationError(
+            'Invalid CIDR \'{0}\''.format(cidr))
+
+    desired_network = '{0}/{1}'.format(
+        str(ipaddr.IPv4Address(cidr_network._ip + 1)),
+        str(cidr_network.prefixlen))
+
+    current_state = get_current_state()
+
+    if (current_state['exists']
+        and current_state['details']['mtu'] == mtu
+        and desired_network in current_state['details']['networks']
+        and current_state['details']['up']
+        and current_state['iptables_rule_exists']):
+        ret['result'] = True
+        ret['comment'] = 'System already in the correct state'
+        return ret
+
+    # The state of the system does need to be changed. Check if we're running
+    # in ``test=true`` mode.
+    if __opts__['test'] == True:
+        ret['comment'] = 'The state of "{0}" will be changed.'.format(name)
+        ret['changes'] = {
+            'old': current_state,
+            'new': 'Create and configure bridge'
+        }
+
+        # Return ``None`` when running with ``test=true``.
+        ret['result'] = None
+
+        return ret
+
+    # Finally, make the actual change and return the result.
+    if not current_state['exists']:
+        __salt__['cmd.run']('brctl addbr {0}'.format(name))
+    new_state = get_current_state()
+    if new_state['details']['mtu'] != mtu:
+        __salt__['cmd.run'](
+            'ip link set dev {0} mtu {1}'.format(name, str(mtu)))
+    new_state = get_current_state()
+    if desired_network not in new_state['details']['networks']:
+        __salt__['cmd.run'](
+            'ip addr add {0} dev {1}'.format(desired_network, name))
+    new_state = get_current_state()
+    if not new_state['details']['up']:
+        __salt__['cmd.run'](
+            'ip link set dev {0} up'.format(name))
+    new_state = get_current_state()
+    if not new_state['iptables_rule_exists']:
+        __salt__['iptables.append'](**iptables_rule)
+    new_state = get_current_state()
+
+    ret['comment'] = 'The state of "{0}" was changed!'.format(name)
+
+    ret['changes'] = {
+        'old': current_state,
+        'new': new_state,
+    }
+
+    ret['result'] = True
+
+    return ret

src/saltbase/salt/apiserver/default

@@ -0,0 +1,5 @@
+{%- set ips = salt['mine.get']('roles:kubernetes-master', 'network.ip_addrs', 'grain').values() %}
+DAEMON_ARGS="$DAEMON_ARGS -etcd_servers=http://{{ ips[0][0] }}:4001"
+
+MACHINES="{{ ','.join(salt['mine.get']('roles:kubernetes-pool', 'network.ip_addrs', expr_form='grain').keys()) }}"
+DAEMON_ARGS="$DAEMON_ARGS --machines $MACHINES"

src/saltbase/salt/apiserver/init.sls

@@ -0,0 +1,81 @@
+{% set root = '/var/src/apiserver' %}
+{% set package = 'github.com/GoogleCloudPlatform/kubernetes' %}
+{% set package_dir = root + '/src/' + package %}
+
+{{ package_dir }}:
+  file.recurse:
+    - source: salt://apiserver/go
+    - user: root
+    - group: staff
+    - dir_mode: 775
+    - file_mode: 664
+    - makedirs: True
+    - recurse:
+      - user
+      - group
+      - mode
+
+apiserver-third-party-go:
+  file.recurse:
+    - name: {{ root }}/src
+    - source: salt://third-party/go/src
+    - user: root
+    - group: staff
+    - dir_mode: 775
+    - file_mode: 664
+    - makedirs: True
+    - recurse:
+      - user
+      - group
+      - mode
+
+/etc/default/apiserver:
+  file.managed:
+    - source: salt://apiserver/default
+    - template: jinja
+    - user: root
+    - group: root
+    - mode: 644
+
+apiserver-build:
+  cmd.wait:
+    - cwd: {{ root }}
+    - names:
+      - go build {{ package }}/cmd/apiserver
+    - env:
+      - PATH: {{ grains['path'] }}:/usr/local/bin
+      - GOPATH: {{ root }}
+    - watch:
+      - file: {{ package_dir }}
+
+/usr/local/bin/apiserver:
+  file.symlink:
+    - target: {{ root }}/apiserver
+    - watch:
+      - cmd: apiserver-build
+
+/etc/init.d/apiserver:
+  file.managed:
+    - source: salt://apiserver/initd
+    - user: root
+    - group: root
+    - mode: 755
+
+apiserver:
+  group.present:
+    - system: True
+  user.present:
+    - system: True
+    - gid_from_name: True
+    - shell: /sbin/nologin
+    - home: /var/apiserver
+    - require:
+      - group: apiserver
+  service.running:
+    - enable: True
+    - watch:
+      - cmd: apiserver-build
+      - file: /etc/default/apiserver
+      - file: /usr/local/bin/apiserver
+      - file: /etc/init.d/apiserver
+

src/saltbase/salt/apiserver/initd

@@ -0,0 +1,119 @@
+#!/bin/bash
+#
+### BEGIN INIT INFO
+# Provides:   apiserver
+# Required-Start:    $local_fs $network $syslog
+# Required-Stop:
+# Default-Start:     2 3 4 5
+# Default-Stop:      0 1 6
+# Short-Description: The Kubernetes API server
+# Description:
+#   The Kubernetes API server maintains docker state against a state file.
+### END INIT INFO
+
+
+# PATH should only include /usr/* if it runs after the mountnfs.sh script
+PATH=/sbin:/usr/sbin:/bin:/usr/bin
+DESC="The Kubernetes API server"
+NAME=apiserver
+DAEMON=/usr/local/bin/apiserver
+DAEMON_ARGS=""
+DAEMON_LOG_FILE=/var/log/$NAME.log
+PIDFILE=/var/run/$NAME.pid
+SCRIPTNAME=/etc/init.d/$NAME
+DAEMON_USER=apiserver
+
+# Exit if the package is not installed
+[ -x "$DAEMON" ] || exit 0
+
+# Read configuration variable file if it is present
+[ -r /etc/default/$NAME ] && . /etc/default/$NAME
+
+# Define LSB log_* functions.
+# Depend on lsb-base (>= 3.2-14) to ensure that this file is present
+# and status_of_proc is working.
+. /lib/lsb/init-functions
+
+#
+# Function that starts the daemon/service
+#
+do_start()
+{
+        # Return
+        #   0 if daemon has been started
+        #   1 if daemon was already running
+        #   2 if daemon could not be started
+        start-stop-daemon --start --quiet --background --no-close \
+                --make-pidfile --pidfile $PIDFILE \
+                --exec $DAEMON -c $DAEMON_USER --test > /dev/null \
+                || return 1
+        start-stop-daemon --start --quiet --background --no-close \
+                --make-pidfile --pidfile $PIDFILE \
+                --exec $DAEMON -c $DAEMON_USER -- \
+                $DAEMON_ARGS >> $DAEMON_LOG_FILE 2>&1 \
+                || return 2
+}
+
+#
+# Function that stops the daemon/service
+#
+do_stop()
+{
+        # Return
+        #   0 if daemon has been stopped
+        #   1 if daemon was already stopped
+        #   2 if daemon could not be stopped
+        #   other if a failure occurred
+        start-stop-daemon --stop --quiet --retry=TERM/30/KILL/5 --pidfile $PIDFILE --name $NAME
+        RETVAL="$?"
+        [ "$RETVAL" = 2 ] && return 2
+        # Many daemons don't delete their pidfiles when they exit.
+        rm -f $PIDFILE
+        return "$RETVAL"
+}
+
+
+case "$1" in
+  start)
+        log_daemon_msg "Starting $DESC" "$NAME"
+        do_start
+        case "$?" in
+                0|1) log_end_msg 0 || exit 0 ;;
+                2) verblog_end_msg 1 || exit 1 ;;
+        esac
+        ;;
+  stop)
+        log_daemon_msg "Stopping $DESC" "$NAME"
+        do_stop
+        case "$?" in
+                0|1) log_end_msg 0 ;;
+                2) exit 1 ;;
+        esac
+        ;;
+  status)
+        status_of_proc -p $PIDFILE "$DAEMON" "$NAME" && exit 0 || exit $?
+        ;;
+
+  restart|force-reload)
+        log_daemon_msg "Restarting $DESC" "$NAME"
+        do_stop
+        case "$?" in
+          0|1)
+                do_start
+                case "$?" in
+                        0) log_end_msg 0 ;;
+                        1) log_end_msg 1 ;; # Old process is still running
+                        *) log_end_msg 1 ;; # Failed to start
+                esac
+                ;;
+          *)
+                # Failed to stop
+                log_end_msg 1
+                ;;
+        esac
+        ;;
+  *)
+        echo "Usage: $SCRIPTNAME {start|stop|status|restart|force-reload}" >&2
+        exit 3
+        ;;
+esac

src/saltbase/salt/base.sls

@@ -0,0 +1,6 @@
+
+pkg-core:
+  pkg.latest:
+    - names:
+      - apt-transport-https
+      - python-apt

src/saltbase/salt/controller-manager/default

@@ -0,0 +1,2 @@
+{%- set ips = salt['mine.get']('roles:kubernetes-master', 'network.ip_addrs', 'grain').values() %}
+DAEMON_ARGS="$DAEMON_ARGS -etcd_servers=http://{{ ips[0][0] }}:4001"

src/saltbase/salt/controller-manager/init.sls

@@ -0,0 +1,81 @@
+{% set root = '/var/src/controller-manager' %}
+{% set package = 'github.com/GoogleCloudPlatform/kubernetes' %}
+{% set package_dir = root + '/src/' + package %}
+
+{{ package_dir }}:
+  file.recurse:
+    - source: salt://controller-manager/go
+    - user: root
+    - group: staff
+    - dir_mode: 775
+    - file_mode: 664
+    - makedirs: True
+    - recurse:
+      - user
+      - group
+      - mode
+
+controller-manager-third-party-go:
+  file.recurse:
+    - name: {{ root }}/src
+    - source: salt://third-party/go/src
+    - user: root
+    - group: staff
+    - dir_mode: 775
+    - file_mode: 664
+    - makedirs: True
+    - recurse:
+      - user
+      - group
+      - mode
+
+/etc/default/controller-manager:
+  file.managed:
+    - source: salt://controller-manager/default
+    - template: jinja
+    - user: root
+    - group: root
+    - mode: 644
+
+controller-manager-build:
+  cmd.wait:
+    - cwd: {{ root }}
+    - names:
+      - go build {{ package }}/cmd/controller-manager
+    - env:
+      - PATH: {{ grains['path'] }}:/usr/local/bin
+      - GOPATH: {{ root }}
+    - watch:
+      - file: {{ package_dir }}
+
+/usr/local/bin/controller-manager:
+  file.symlink:
+    - target: {{ root }}/controller-manager
+    - watch:
+      - cmd: controller-manager-build
+
+/etc/init.d/controller-manager:
+  file.managed:
+    - source: salt://controller-manager/initd
+    - user: root
+    - group: root
+    - mode: 755
+
+controller-manager:
+  group.present:
+    - system: True
+  user.present:
+    - system: True
+    - gid_from_name: True
+    - shell: /sbin/nologin
+    - home: /var/controller-manager
+    - require:
+      - group: controller-manager
+  service.running:
+    - enable: True
+    - watch:
+      - cmd: controller-manager-build
+      - file: /usr/local/bin/controller-manager
+      - file: /etc/init.d/controller-manager
+      - file: /etc/default/controller-manager
+

src/saltbase/salt/controller-manager/initd

@@ -0,0 +1,120 @@
+#!/bin/bash
+#
+### BEGIN INIT INFO
+# Provides:    controller-manager
+# Required-Start:    $local_fs $network $syslog
+# Required-Stop:
+# Default-Start:     2 3 4 5
+# Default-Stop:      0 1 6
+# Short-Description: The Kubernetes controller manager
+# Description:
+#   The Kubernetes controller manager is responsible for monitoring replication
+#   controllers, and creating corresponding tasks to achieve the desired state.
+### END INIT INFO
+
+
+# PATH should only include /usr/* if it runs after the mountnfs.sh script
+PATH=/sbin:/usr/sbin:/bin:/usr/bin
+DESC="The Kubernetes container manager"
+NAME=controller-manager
+DAEMON=/usr/local/bin/controller-manager
+DAEMON_ARGS=" --master=127.0.0.1:8080"
+DAEMON_LOG_FILE=/var/log/$NAME.log
+PIDFILE=/var/run/$NAME.pid
+SCRIPTNAME=/etc/init.d/$NAME
+DAEMON_USER=controller-manager
+
+# Exit if the package is not installed
+[ -x "$DAEMON" ] || exit 0
+
+# Read configuration variable file if it is present
+[ -r /etc/default/$NAME ] && . /etc/default/$NAME
+
+# Define LSB log_* functions.
+# Depend on lsb-base (>= 3.2-14) to ensure that this file is present
+# and status_of_proc is working.
+. /lib/lsb/init-functions
+
+#
+# Function that starts the daemon/service
+#
+do_start()
+{
+        # Return
+        #   0 if daemon has been started
+        #   1 if daemon was already running
+        #   2 if daemon could not be started
+        start-stop-daemon --start --quiet --background --no-close \
+                --make-pidfile --pidfile $PIDFILE \
+                --exec $DAEMON -c $DAEMON_USER --test > /dev/null \
+                || return 1
+        start-stop-daemon --start --quiet --background --no-close \
+                --make-pidfile --pidfile $PIDFILE \
+                --exec $DAEMON -c $DAEMON_USER -- \
+                $DAEMON_ARGS >> $DAEMON_LOG_FILE 2>&1 \
+                || return 2
+}
+
+#
+# Function that stops the daemon/service
+#
+do_stop()
+{
+        # Return
+        #   0 if daemon has been stopped
+        #   1 if daemon was already stopped
+        #   2 if daemon could not be stopped
+        #   other if a failure occurred
+        start-stop-daemon --stop --quiet --retry=TERM/30/KILL/5 --pidfile $PIDFILE --exec $DAEMON
+        RETVAL="$?"
+        [ "$RETVAL" = 2 ] && return 2
+        # Many daemons don't delete their pidfiles when they exit.
+        rm -f $PIDFILE
+        return "$RETVAL"
+}
+
+
+case "$1" in
+  start)
+        log_daemon_msg "Starting $DESC" "$NAME"
+        do_start
+        case "$?" in
+                0|1) log_end_msg 0 || exit 0 ;;
+                2) verblog_end_msg 1 || exit 1 ;;
+        esac
+        ;;
+  stop)
+        log_daemon_msg "Stopping $DESC" "$NAME"
+        do_stop
+        case "$?" in
+                0|1) log_end_msg 0 ;;
+                2) exit 1 ;;
+        esac
+        ;;
+  status)
+        status_of_proc -p $PIDFILE "$DAEMON" "$NAME" && exit 0 || exit $?
+        ;;
+
+  restart|force-reload)
+        log_daemon_msg "Restarting $DESC" "$NAME"
+        do_stop
+        case "$?" in
+          0|1)
+                do_start
+                case "$?" in
+                        0) log_end_msg 0 ;;
+                        1) log_end_msg 1 ;; # Old process is still running
+                        *) log_end_msg 1 ;; # Failed to start
+                esac
+                ;;
+          *)
+                # Failed to stop
+                log_end_msg 1
+                ;;
+        esac
+        ;;
+  *)
+        echo "Usage: $SCRIPTNAME {start|stop|status|restart|force-reload}" >&2
+        exit 3
+        ;;
+esac

src/saltbase/salt/docker/docker-defaults

@@ -0,0 +1 @@
+DOCKER_OPTS="--bridge cbr0 --iptables=false"

src/saltbase/salt/docker/init.sls

@@ -0,0 +1,53 @@
+docker-repo:
+  pkgrepo.managed:
+    - humanname: Docker Repo
+    - name: deb https://get.docker.io/ubuntu docker main
+    - key_url: https://get.docker.io/gpg
+    - require:
+      - pkg: pkg-core
+
+# The default GCE images have ip_forwarding explicitly set to 0.
+# Here we take care of commenting that out.
+/etc/sysctl.d/11-gce-network-security.conf:
+  file.replace:
+    - pattern: '^net.ipv4.ip_forward=0'
+    - repl: '# net.ipv4.ip_forward=0'
+
+net.ipv4.ip_forward:
+  sysctl.present:
+    - value: 1
+
+bridge-utils:
+  pkg.latest
+
+cbr0:
+  container_bridge.ensure:
+    - cidr: {{ grains['cbr-cidr'] }}
+    - mtu: 1460
+
+/etc/default/docker:
+  file.managed:
+    - source: salt://docker/docker-defaults
+    - template: jinja
+    - user: root
+    - group: root
+    - mode: 644
+    - makedirs: true
+
+lxc-docker:
+  pkg.latest
+
+# There is a race here, I think.  As the package is installed, it will start
+# docker.  If it doesn't write its pid file fast enough then this next stanza
+# will try to ensure that docker is running.  That might start another copy of
+# docker causing the thing to get wedged.
+#
+# See docker issue https://github.com/dotcloud/docker/issues/6184
+
+# docker:
+#   service.running:
+#     - enable: True
+#     - require:
+#       - pkg: lxc-docker
+#     - watch:
+#       - file: /etc/default/docker

src/saltbase/salt/etcd/etcd.conf

@@ -0,0 +1,4 @@
+bind_addr = "0.0.0.0"
+peer_bind_addr = "0.0.0.0"
+data_dir = "/var/etcd"
+max_retry_attempts = 60

src/saltbase/salt/etcd/init.sls

@@ -0,0 +1,63 @@
+etcd-install:
+  git.latest:
+    - target: /var/src/etcd
+    - name: git://github.com/coreos/etcd
+  cmd.wait:
+    - cwd: /var/src/etcd
+    - names:
+      - ./build
+    - env:
+      - PATH: {{ grains['path'] }}:/usr/local/bin
+    - watch:
+      - git: etcd-install
+  file.symlink:
+    - name: /usr/local/bin/etcd
+    - target: /var/src/etcd/bin/etcd
+    - watch:
+      - cmd: etcd-install
+
+etcd:
+  group.present:
+    - system: True
+  user.present:
+    - system: True
+    - gid_from_name: True
+    - shell: /sbin/nologin
+    - home: /var/etcd
+    - require:
+      - group: etcd
+
+/etc/etcd:
+  file.directory:
+    - user: root
+    - group: root
+    - dir_mode: 755
+
+/etc/etcd/etcd.conf:
+  file.managed:
+    - source: salt://etcd/etcd.conf
+    - user: root
+    - group: root
+    - mode: 644
+
+/var/etcd:
+  file.directory:
+    - user: etcd
+    - group: etcd
+    - dir_mode: 700
+
+/etc/init.d/etcd:
+  file.managed:
+    - source: salt://etcd/initd
+    - user: root
+    - group: root
+    - mode: 755
+
+etcd-service:
+  service.running:
+    - name: etcd
+    - enable: True
+    - watch:
+      - file: /etc/etcd/etcd.conf
+      - cmd: etcd-install
+

src/saltbase/salt/etcd/initd

@@ -0,0 +1,118 @@
+#!/bin/bash
+#
+### BEGIN INIT INFO
+# Provides:    etcd
+# Required-Start:    $local_fs $network $syslog
+# Required-Stop:
+# Default-Start:     2 3 4 5
+# Default-Stop:      0 1 6
+# Short-Description: The etcd key-value share configuration service.
+# Description:       This launches and controls the etcd daemon.
+### END INIT INFO
+
+
+# PATH should only include /usr/* if it runs after the mountnfs.sh script
+PATH=/sbin:/usr/sbin:/bin:/usr/bin
+DESC="The etcd key-value share configuration service"
+NAME=etcd
+DAEMON=/usr/local/bin/$NAME
+DAEMON_ARGS="-peer-addr $HOSTNAME:7001 -name $HOSTNAME"
+DAEMON_LOG_FILE=/var/log/$NAME.log
+PIDFILE=/var/run/$NAME.pid
+SCRIPTNAME=/etc/init.d/$NAME
+DAEMON_USER=etcd
+
+# Exit if the package is not installed
+[ -x "$DAEMON" ] || exit 0
+
+# Read configuration variable file if it is present
+[ -r /etc/default/$NAME ] && . /etc/default/$NAME
+
+# Define LSB log_* functions.
+# Depend on lsb-base (>= 3.2-14) to ensure that this file is present
+# and status_of_proc is working.
+. /lib/lsb/init-functions
+
+#
+# Function that starts the daemon/service
+#
+do_start()
+{
+        # Return
+        #   0 if daemon has been started
+        #   1 if daemon was already running
+        #   2 if daemon could not be started
+        start-stop-daemon --start --quiet --background --no-close \
+                --make-pidfile --pidfile $PIDFILE \
+                --exec $DAEMON -c $DAEMON_USER --test > /dev/null \
+                || return 1
+        start-stop-daemon --start --quiet --background --no-close \
+                --make-pidfile --pidfile $PIDFILE \
+                --exec $DAEMON -c $DAEMON_USER -- \
+                $DAEMON_ARGS >> $DAEMON_LOG_FILE 2>&1 \
+                || return 2
+}
+
+#
+# Function that stops the daemon/service
+#
+do_stop()
+{
+        # Return
+        #   0 if daemon has been stopped
+        #   1 if daemon was already stopped
+        #   2 if daemon could not be stopped
+        #   other if a failure occurred
+        start-stop-daemon --stop --quiet --retry=TERM/30/KILL/5 --pidfile $PIDFILE --name $NAME
+        RETVAL="$?"
+        [ "$RETVAL" = 2 ] && return 2
+        # Many daemons don't delete their pidfiles when they exit.
+        rm -f $PIDFILE
+        return "$RETVAL"
+}
+
+
+case "$1" in
+  start)
+        log_daemon_msg "Starting $DESC" "$NAME"
+        do_start
+        case "$?" in
+                0|1) log_end_msg 0 || exit 0 ;;
+                2) verblog_end_msg 1 || exit 1 ;;
+        esac
+        ;;
+  stop)
+        log_daemon_msg "Stopping $DESC" "$NAME"
+        do_stop
+        case "$?" in
+                0|1) log_end_msg 0 ;;
+                2) exit 1 ;;
+        esac
+        ;;
+  status)
+        status_of_proc "$DAEMON" "$NAME" && exit 0 || exit $?
+        ;;
+
+  restart|force-reload)
+        log_daemon_msg "Restarting $DESC" "$NAME"
+        do_stop
+        case "$?" in
+          0|1)
+                do_start
+                case "$?" in
+                        0) log_end_msg 0 ;;
+                        1) log_end_msg 1 ;; # Old process is still running
+                        *) log_end_msg 1 ;; # Failed to start
+                esac
+                ;;
+          *)
+                # Failed to stop
+                log_end_msg 1
+                ;;
+        esac
+        ;;
+  *)
+        echo "Usage: $SCRIPTNAME {start|stop|status|restart|force-reload}" >&2
+        exit 3
+        ;;
+esac

src/saltbase/salt/golang.sls

@@ -0,0 +1,24 @@
+{% set go_version = '1.2' %}
+{% set go_arch = 'linux-amd64' %}
+{% set go_archive = 'go%s.%s.tar.gz' | format(go_version, go_arch) %}
+{% set go_url = 'https://go.googlecode.com/files/' + go_archive %}
+{% set go_hash = 'md5=68901bbf8a04e71e0b30aa19c3946b21' %}
+
+
+get-golang:
+  file.managed:
+    - name: /var/cache/{{ go_archive }}
+    - source: {{ go_url }}
+    - source_hash: {{ go_hash }}
+  cmd.wait:
+    - cwd: /usr/local
+    - name: tar xzf /var/cache/{{ go_archive }}
+    - watch:
+      - file: get-golang
+
+install-golang:
+  file.symlink:
+    - name: /usr/local/bin/go
+    - target: /usr/local/go/bin/go
+    - watch:
+      - cmd: get-golang

src/saltbase/salt/kube-proxy/default

@@ -0,0 +1,2 @@
+{%- set ips = salt['mine.get']('roles:kubernetes-master', 'network.ip_addrs', 'grain').values() %}
+DAEMON_ARGS="$DAEMON_ARGS --etcd_servers=http://{{ ips[0][0] }}:4001"

src/saltbase/salt/kube-proxy/init.sls

@@ -0,0 +1,79 @@
+{% set root = '/var/src/kube-proxy' %}
+{% set package = 'github.com/GoogleCloudPlatform/kubernetes' %}
+{% set package_dir = root + '/src/' + package %}
+
+{{ package_dir }}:
+  file.recurse:
+    - source: salt://kube-proxy/go
+    - user: root
+    - group: staff
+    - dir_mode: 775
+    - file_mode: 664
+    - makedirs: True
+    - recurse:
+      - user
+      - group
+      - mode
+
+third-party-go:
+  file.recurse:
+    - name: {{ root }}/src
+    - source: salt://third-party/go/src
+    - user: root
+    - group: staff
+    - dir_mode: 775
+    - file_mode: 664
+    - makedirs: True
+    - recurse:
+      - user
+      - group
+      - mode
+
+kube-proxy-build:
+  cmd.wait:
+    - cwd: {{ root }}
+    - names:
+      - go build {{ package }}/cmd/proxy
+    - env:
+      - PATH: {{ grains['path'] }}:/usr/local/bin
+      - GOPATH: {{ root }}
+    - watch:
+      - file: {{ package_dir }}
+
+/usr/local/bin/kube-proxy:
+  file.symlink:
+    - target: {{ root }}/proxy
+    - watch:
+      - cmd: kube-proxy-build
+
+/etc/init.d/kube-proxy:
+  file.managed:
+    - source: salt://kube-proxy/initd
+    - user: root
+    - group: root
+    - mode: 755
+
+/etc/default/kube-proxy:
+  file.managed:
+    - source: salt://kube-proxy/default
+    - template: jinja
+    - user: root
+    - group: root
+    - mode: 644
+
+kube-proxy:
+  group.present:
+    - system: True
+  user.present:
+    - system: True
+    - gid_from_name: True
+    - shell: /sbin/nologin
+    - home: /var/kube-proxy
+    - require:
+      - group: kube-proxy
+  service.running:
+    - enable: True
+    - watch:
+      - cmd: kube-proxy-build
+      - file: /etc/default/kube-proxy
+      - file: /etc/init.d/kube-proxy

src/saltbase/salt/kube-proxy/initd

@@ -0,0 +1,120 @@
+#!/bin/bash
+#
+### BEGIN INIT INFO
+# Provides:    kube-proxy
+# Required-Start:    $local_fs $network $syslog
+# Required-Stop:
+# Default-Start:     2 3 4 5
+# Default-Stop:      0 1 6
+# Short-Description: The Kubernetes network proxy
+# Description:
+#   The Kubernetes network proxy enables network redirection and
+#   loadbalancing for dynamically placed containers.
+### END INIT INFO
+
+
+# PATH should only include /usr/* if it runs after the mountnfs.sh script
+PATH=/sbin:/usr/sbin:/bin:/usr/bin
+DESC="The Kubernetes network proxy"
+NAME=kube-proxy
+DAEMON=/usr/local/bin/kube-proxy
+DAEMON_ARGS=""
+DAEMON_LOG_FILE=/var/log/$NAME.log
+PIDFILE=/var/run/$NAME.pid
+SCRIPTNAME=/etc/init.d/$NAME
+DAEMON_USER=kube-proxy
+
+# Exit if the package is not installed
+[ -x "$DAEMON" ] || exit 0
+
+# Read configuration variable file if it is present
+[ -r /etc/default/$NAME ] && . /etc/default/$NAME
+
+# Define LSB log_* functions.
+# Depend on lsb-base (>= 3.2-14) to ensure that this file is present
+# and status_of_proc is working.
+. /lib/lsb/init-functions
+
+#
+# Function that starts the daemon/service
+#
+do_start()
+{
+        # Return
+        #   0 if daemon has been started
+        #   1 if daemon was already running
+        #   2 if daemon could not be started
+        start-stop-daemon --start --quiet --background --no-close \
+                --make-pidfile --pidfile $PIDFILE \
+                --exec $DAEMON -c $DAEMON_USER --test > /dev/null \
+                || return 1
+        start-stop-daemon --start --quiet --background --no-close \
+                --make-pidfile --pidfile $PIDFILE \
+                --exec $DAEMON -c $DAEMON_USER -- \
+                $DAEMON_ARGS >> $DAEMON_LOG_FILE 2>&1 \
+                || return 2
+}
+
+#
+# Function that stops the daemon/service
+#
+do_stop()
+{
+        # Return
+        #   0 if daemon has been stopped
+        #   1 if daemon was already stopped
+        #   2 if daemon could not be stopped
+        #   other if a failure occurred
+        start-stop-daemon --stop --quiet --retry=TERM/30/KILL/5 --pidfile $PIDFILE --name $NAME
+        RETVAL="$?"
+        [ "$RETVAL" = 2 ] && return 2
+        # Many daemons don't delete their pidfiles when they exit.
+        rm -f $PIDFILE
+        return "$RETVAL"
+}
+
+
+case "$1" in
+  start)
+        log_daemon_msg "Starting $DESC" "$NAME"
+        do_start
+        case "$?" in
+                0|1) log_end_msg 0 || exit 0 ;;
+                2) verblog_end_msg 1 || exit 1 ;;
+        esac
+        ;;
+  stop)
+        log_daemon_msg "Stopping $DESC" "$NAME"
+        do_stop
+        case "$?" in
+                0|1) log_end_msg 0 ;;
+                2) exit 1 ;;
+        esac
+        ;;
+  status)
+        status_of_proc -p $PIDFILE "$DAEMON" "$NAME" && exit 0 || exit $?
+        ;;
+
+  restart|force-reload)
+        log_daemon_msg "Restarting $DESC" "$NAME"
+        do_stop
+        case "$?" in
+          0|1)
+                do_start
+                case "$?" in
+                        0) log_end_msg 0 ;;
+                        1) log_end_msg 1 ;; # Old process is still running
+                        *) log_end_msg 1 ;; # Failed to start
+                esac
+                ;;
+          *)
+                # Failed to stop
+                log_end_msg 1
+                ;;
+        esac
+        ;;
+  *)
+        echo "Usage: $SCRIPTNAME {start|stop|status|restart|force-reload}" >&2
+        exit 3
+        ;;
+esac