Merge pull request #1523 from rneugeba/demo-cloud

Update demo files/config/docs

projects/demo/README.md

@@ -0,0 +1,37 @@
+This directory contains files used in moby demos.
+
+# Prerequisites
+
+Most of the scripts/files assume you are on a Mac.
+
+- Recent Docker for Mac installed (edge, nightly, master channel)
+- For the GCP portion: `brew install google-cloud-sdk`
+- For `etcd`: `brew install etcd`
+- Infrakit: Clone [infrakit](https://github.com/docker/infrakit) and
+  the [GCP plugin](https://github.com/docker/infrakit.gcp) for
+  infrakit.  For each, `make build-in-container` and then copy the
+  contents of `./build` somewhere in your path.
+
+# GCP Setup
+
+You probably want to change the project/zone
+```
+export CLOUDSDK_CORE_PROJECT=docker4x
+export CLOUDSDK_COMPUTE_ZONE=europe-west1-d
+gcloud auth login
+gcloud auth application-default login
+```
+
+You may also want to create ssh-keys and upload them. See the [Generating a new SSH key-pair section](https://cloud.google.com/compute/docs/instances/connecting-to-instance)
+
+One time configuration of the network:
+```
+gcloud compute networks create rneugeba-demo --mode auto
+gcloud compute networks subnets list
+# get IP subnet for rneugeba-demo
+gcloud compute firewall-rules create rneugeba-demo-internal --network \
+    rneugeba-demo --allow tcp,udp,icmp --source-ranges 10.128.0.0/9
+```
+The firewall setup means that all our projects networks can talk to the demo network.
+
+

projects/demo/etcd/README.md

@@ -6,7 +6,7 @@ Docker for Mac container to bootstrap the cluster. For a cloud based demo, we'd
 The moby `etcd` package is build with [build-pkg.sh](./build-pkg.sh). It take the official `etcd` container and adds a [script](./etcd.sh) to start `etcd`.
 
 
-## Simple single node cluster
+## Simple single node cluster (OUTDATED)
 
 - Edit `./dfm-setup.sh` and set `NUMPEERS` to `1`
 - Start the etcd bootstrap container in on window:
@@ -21,7 +21,7 @@ moby build etcd
 moby run etcd
 ```
 
-## InfraKit cluster setup
+## InfraKit cluster setup (OUTDATED)
 
 This should create a HyperKit based, InfraKit managed `etcd` cluster with 5 `etcd` instances.
 
@@ -34,10 +34,10 @@ rm -rf ~/.infrakit
 ```
 - Start the infrakit plugins, each in it's own window from the root of the infrakit source tree:
 ```
-./build/infrakit-group-default
+infrakit-group-default
 ```
 ```
-./build/infrakit-flavor-vanilla
+infrakit-flavor-vanilla
 ```
 - Start the hyperkit instance plugin from this directory:
 ```
@@ -53,7 +53,7 @@ rm -rf ~/.infrakit
 
 - Commit the infrakit config:
 ```
-~/src/docker/infrakit/build/infrakit group commit infrakit.json
+infrakit group commit infrakit.json
 ```
 
 To check if everything is fine, note down the IP address from one of
@@ -61,3 +61,26 @@ the nodes and then:
 ```
 docker run --rm -t quay.io/coreos/etcd:v3.1.5 etcdctl --endpoints http://192.168.65.24:2379 member list
 ```
+
+## Infrakit GCP setup
+
+Note: This setup is somewhat specific to our GCP setup (IP addresses
+and account info) and needs to be adjusted to your setting. The
+configuration is documented in the top-level README.md.
+
+Build the image and upload it:
+```
+moby build etcd
+```
+
+Start the infrakit components in separate windows:
+```
+infrakit-group-default
+infrakit-flavor-vanilla
+infrakit-instance-gcp
+```
+
+Commit the configuration:
+```
+infrakit group commit infrakit-gce.json
+```

projects/demo/etcd/etcd.sh

@@ -4,12 +4,19 @@
 set -x
 set -v
 
-# Needs to come from metadata
+# --initial-cluster argument should come from meta data
-UUID=6c007a14875d53d9bf0ef5a6fc0257c817f0fb83
-DISCOVER_URL=http://192.168.65.2:2381/v2/keys/discovery/${UUID}
 
+# Wait till we have an IP address
+IP=""
+while [ -z "$IP" ]; do
     IP=$(ifconfig eth0 2>/dev/null|awk '/inet addr:/ {print $2}'|sed 's/addr://')
-NAME=$(hostname)
+    sleep 1
+done
+
+# Name is infra+last octet of IP address
+NUM=$(echo ${IP} | cut -d . -f 4)
+PREFIX=$(echo ${IP} | cut -d . -f 1,2,3)
+NAME=infra${NUM}
 
 /usr/local/bin/etcd \
     --name ${NAME} \
@@ -19,4 +26,6 @@ NAME=$(hostname)
     --listen-peer-urls http://${IP}:2380 \
     --listen-client-urls http://${IP}:2379,http://127.0.0.1:2379 \
     --advertise-client-urls http://${IP}:2379 \
-    --discovery ${DISCOVER_URL}
+    --initial-cluster-token etcd-cluster-1 \
+    --initial-cluster infra200=http://${PREFIX}.200:2380,infra201=http://${PREFIX}.201:2380,infra202=http://${PREFIX}.202:2380 \
+    --initial-cluster-state new

projects/demo/etcd/etcd.yml

@@ -40,3 +40,7 @@ daemon:
     net: host
 outputs:
   - format: kernel+initrd
+  - format: gce
+    project: docker4x
+    bucket: rolf
+    replace: true

projects/demo/etcd/infrakit-gce.json

@@ -0,0 +1,36 @@
+{
+    "ID": "etcd",
+    "Properties": {
+        "Allocation": {
+            "LogicalIDs": [
+                "10.132.0.200",
+                "10.132.0.201",
+                "10.132.0.202"
+            ]
+        },
+        "Instance": {
+            "Plugin": "instance-gcp",
+            "Properties": {
+                "NamePrefix": "etcdtest",
+                "Description": "ETCD test cluster",
+                "Network": "rneugeba-demo",
+                "Connect" : true,
+                "MachineType": "n1-standard-1",
+                "DiskSizeMb": 60,
+                "DiskImage": "https://www.googleapis.com/compute/v1/projects/docker4x/global/images/etcd",
+                "DiskType": "pd-standard",
+                "Scopes": [ ]
+            }
+        },
+        "Flavor": {
+            "Plugin": "flavor-vanilla",
+            "Properties": {
+                "Init": [ ],
+                "Tags": {
+                    "tier": "etcd-cluster",
+                    "project": "infrakit"
+                }
+            }
+        }
+    }
+}

projects/demo/etcd/infrakit.json

@@ -1,13 +1,11 @@
 {
-    "ID": "cattle",
+    "ID": "etcd",
     "Properties": {
         "Allocation": {
             "LogicalIDs": [
                 "192.168.65.200",
                 "192.168.65.201",
-                "192.168.65.202",
+                "192.168.65.202"
-                "192.168.65.203",
-                "192.168.65.204"
             ]
         },
         "Instance": {
@@ -16,7 +14,7 @@
                 "Moby": "etcd",
                 "Disk" : 0,
                 "CPUs" : 1,
-                "Memory" : 512,
+                "Memory" : 512
             }
         },
         "Flavor": {