Commit Graph

2815 Commits

Author SHA1 Message Date
Justin Cormack
dc1818147c Merge pull request #1522 from dave-tucker/gce-gcp
Use GCP everywhere for consistency
2017-04-06 17:27:15 +01:00
Rolf Neugebauer
2bd75a621d demo: Switch etcd bootstrap from discovery service to static IPs
This makes the configuration simpler but requires us to be able
to set IP addresses on instances.

This also, for simplicity, reduces the number of nodes to 3.

The script does not make assumption about specific IP addresses,
but does assume that the nodes have IP addresses such as:
a.b.c.200, a.b.c.201, and a.b.c.202.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-06 17:22:56 +01:00
Dave Tucker
1b485eff4c moby: Add gcp platform to usage in moby run
Signed-off-by: Dave Tucker <dt@docker.com>
2017-04-06 17:10:04 +01:00
Dave Tucker
df1c66dd04 moby: Replace references to GCE with GCP
Signed-off-by: Dave Tucker <dt@docker.com>
2017-04-06 17:00:53 +01:00
Dave Tucker
bf7ab9f04c tools: rename mkimage-gce to mkimage-gcp
Also removes `mobyplatform` from the cmdline as this is no longer used

Signed-off-by: Dave Tucker <dt@docker.com>
2017-04-06 16:59:57 +01:00
Thomas Gazagnaire
bb536803be miragesdk: use the latest image dhcp-client image
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:25 +02:00
Thomas Gazagnaire
f44e2ffbcb miragesdk: cleaner errors
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:25 +02:00
Thomas Gazagnaire
fd447ee082 miragesdk: really allow to configure the DB path with the CLI
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:24 +02:00
Thomas Gazagnaire
4437c4a23b miragesdk: add a config.json file for the calf
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:24 +02:00
Thomas Gazagnaire
f5306b23ef miragesdk: fix the fork/exec init code and add a test
Make all the low-level init code synchronous to avoid weird blocks on `close`.
Also move the net and ctl file descriptor at the beginning of the fd space for
the calf.

The SDK also allow to spamn multiple exec calves, which will all have the same
fd map:

- 0: stdin  = /dev/null
- 1: stdout = pipe to parent stdout
- 2: stderr = pipe to parent stderr
- 3: net    = socketpair to parent "net" pipe
- 4: ctl    = socketpair to parent "ctl" pipe

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:24 +02:00
Thomas Gazagnaire
3cec2b1f5e miragesdk: refactor the SDK
Expose a non-unix dependent flow-like API, so it is easier to test/use in a
unikernel.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:24 +02:00
Thomas Gazagnaire
a07952d4e6 miragesdk: shell out to ifconfig and ip to set the IP and routes
This forces us to bind mount /lib but will be replaced by calling the proper
bindings later on.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:24 +02:00
Thomas Gazagnaire
0938ae44f6 miragesdk: fail early on short read/write returning 0
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:24 +02:00
Thomas Gazagnaire
25d3e42204 miragesdk: update init image
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:24 +02:00
Justin Cormack
485452d9bd Merge pull request #1460 from justincormack/automount
Add a formatting container and persistent disk support for docker
2017-04-06 16:08:20 +01:00
Justin Cormack
3675eb17a5 Merge pull request #1518 from justincormack/dhcpcd-cleanup
Clean up dhcpcd container
2017-04-06 15:43:56 +01:00
Justin Cormack
008def2ed4 Merge pull request #1516 from justincormack/more-oci
Add more OCI features
2017-04-06 15:42:56 +01:00
Justin Cormack
0a030dc219 Clean up dhcpcd container
It is not necessary to bring up `eth0`, the program does it fine.

This means we can remove shell script, clean up build.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-06 15:32:22 +01:00
Justin Cormack
40cc6d8110 Merge pull request #1517 from samoht/make-qemu
Really fix `make qemu`
2017-04-06 15:14:22 +01:00
Thomas Gazagnaire
7f0761c5f1 Really fix make qemu
Previously, the cmdline file was never sent in the build context which
caused the script to pick the default options.

Also, when mulitple options are specified (not the default) the right
escaping needs to be used.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 16:05:35 +02:00
Justin Cormack
3244898919 Add more OCI features
- masked paths
- readonly paths
- allow attaching to existing namespaces, eg if bind mounted by a system container

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-06 15:05:08 +01:00
Justin Cormack
cf7b952995 Add persistent drive support to Docker container
This works and runs containers now, if you eg `runc exec` into it.
Needs a few tweaks for rlimits, but will pull and run containers.

Will integrate better with ssh/dev containers to make more usable.

For a simple test use
```
./bin/moby build examples/docker.yml
./bin/moby run hyperkit -disk-size 100 docker

```

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-06 14:25:42 +01:00
Justin Cormack
8bcb2c9e0e Add a formatting container
This is based on the code we used for Docker Editions, and
will format an external drive, to be used for example for
`/var` for Docker image persistence.

It does not `mount` the drive yet, as splitting format and mount
gives better modularity.

Example yaml fragment:
```
  - name: format
    image: "mobylinux/format:097d4f22b20f976b1f89d8f0b8a5d074d35b856c"
    binds:
     - /dev:/dev
    capabilities:
     - CAP_SYS_ADMIN
     - CAP_MKNOD
```

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-06 13:19:46 +01:00
Justin Cormack
3223897232 Merge pull request #1514 from rneugeba/cli-up
Improvements to the CLI
2017-04-06 13:01:58 +01:00
Justin Cormack
4a1b8a3e32 Merge pull request #1515 from samoht/make-qemu
Fix `make qemu`
2017-04-06 12:30:19 +01:00
Rolf Neugebauer
3e53aab4ce cli: Add a 'version' version
Pass version and git commit hash from the Makefile
into main.go. Add a 'version' subcommand to print
the information.

While at it also tweak the help output to only print the
command name and not the entire path.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-06 12:29:52 +01:00
Thomas Gazagnaire
8f64191dee Fix make qemu
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 13:20:33 +02:00
Justin Cormack
6917f6d23a Merge pull request #1488 from t-koulouris/master
[WIP] Support launching containers in outer-kernel mode for moby-okernel demo
2017-04-06 11:48:06 +01:00
Rolf Neugebauer
fdee4a099c demo: Fix JSON file
This was corrupted broken by a previous commit.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-06 11:32:26 +01:00
Rolf Neugebauer
f76f2b6654 tools: Add --ldflags to go-compile
This flags allows passing additional ldflags to the build.
It is primarily there to pass -C arguments to specify values
of variables at compile time.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-06 10:55:18 +01:00
Rolf Neugebauer
d33892e9f8 cli: Fix "build" when the basename contains a "."
Something like "moby-4.10.yml" did not work when invoked
like "moby build moby-4.10".

While at it, also allow .yaml as an extension.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-06 10:55:18 +01:00
Justin Cormack
d740624db6 Merge pull request #1511 from justincormack/oci-add
Add more OCI options
2017-04-06 10:23:47 +01:00
Justin Cormack
f129645c23 Merge pull request #1505 from ijc25/use-exec-to-avoid-stale-sh-processes
Use exec in a couple of places to avoid needless lingering /bin/sh pr…
2017-04-06 10:12:54 +01:00
Justin Cormack
7b1e0be404 Merge pull request #1513 from justincormack/del-extra-dtakit
remove unnecessary .datakitci.json
2017-04-06 10:09:54 +01:00
Ian Campbell
2b54e18f9f Drop unnecessary use of start-stop-daemon with containerd.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-04-06 09:34:38 +01:00
Ian Campbell
8270bdfe33 Use exec in a couple of places to avoid needless lingering /bin/sh processes
```
$ apk -U add procps
$ ps xf
    1 ?        Ss     0:05 /sbin/init
  357 ?        Ss     0:00 /bin/sh /etc/init.d/containerd
  359 ?        Sl     0:00  \_ /usr/bin/containerd
  360 ?        Ss     0:00 /bin/sh /etc/init.d/containers
  432 ?        Sl     0:00  \_ /usr/bin/runc run --bundle /containers/daemon/swa
  466 ?        Ssl    0:00  |   \_ /usr/bin/swarmd --containerd-addr=/run/contai
  427 ?        Sl     0:00  \_ /usr/bin/runc run --bundle /containers/daemon/dhc
  457 ?        Ss     0:00  |   \_ bin/sh /usr/local/bin/start_dhcpcd.sh
  474 ?        S      0:00  |       \_ /sbin/dhcpcd --nobackground
  429 ?        Sl     0:00  \_ /usr/bin/runc run --bundle /containers/daemon/rng
  576 ?        Ss     0:00      \_ /bin/tini /usr/sbin/rngd -f
  580 ?        S      0:00          \_ /usr/sbin/rngd -f
```

becomes

```
$ ps xf
    1 ?        Ss     0:06 /sbin/init
  358 ?        Ss     0:00 /bin/sh /etc/init.d/containers
  426 ?        Sl     0:00  \_ /usr/bin/runc run --bundle /containers/daemon/dhc
  458 ?        Ss     0:00  |   \_ /sbin/dhcpcd --nobackground
  431 ?        Sl     0:00  \_ /usr/bin/runc run --bundle /containers/daemon/swa
  460 ?        Ssl    0:00  |   \_ /usr/bin/swarmd --containerd-addr=/run/contai
  428 ?        Sl     0:00  \_ /usr/bin/runc run --bundle /containers/daemon/rng
  574 ?        Ss     0:00      \_ /bin/tini /usr/sbin/rngd -f
  578 ?        S      0:00          \_ /usr/sbin/rngd -f
  356 ?        Ssl    0:00 /usr/bin/containerd

```

Specifically these are gone:
  357 ?        Ss     0:00 /bin/sh /etc/init.d/containerd
  457 ?        Ss     0:00  |   \_ bin/sh /usr/local/bin/start_dhcpcd.sh

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-04-06 09:34:38 +01:00
Justin Cormack
106d769d0e remove unnecessary .datakitci.json
Confusing to have it, we are not using variants now.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-05 23:57:17 +01:00
Rolf Neugebauer
fa7c85e235 build: Improve dependency for CLI/infrakit build
Use vendor.conf as a proxy for ./vendor contents

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-05 23:10:08 +01:00
Justin Cormack
ea75bd96b6 Merge pull request #1510 from riyazdf/sec-arch-feedback
Update security arch docs with feedback
2017-04-05 20:50:31 +01:00
Justin Cormack
316fe5da79 Add more OCI options
- Sysctl
- CgroupsPath
- RootfsPropagation

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-05 20:46:16 +01:00
Rolf Neugebauer
df993d8b0d build: Fix copy&paste error for Infrakit dependencies
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-05 19:52:31 +01:00
Riyaz Faizullabhoy
5fe0233dad Update security arch docs with feedback
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-04-05 11:29:07 -07:00
Rolf Neugebauer
466f59189a Merge pull request #1508 from rneugeba/meta
Allow passing LogicalIDs into hyperkit instances
2017-04-05 18:11:12 +01:00
Rolf Neugebauer
df5c858d38 demo: Use IP addresses as Logical IDs in etcd infrakit config
Instead of specifying the number of instances, provide a list
of IP addresses for instances. These are passed to the instance
plugin as LogicalID.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-05 18:01:52 +01:00
Rolf Neugebauer
ea3bfccf5e infrakit: Fix LogicalID handling in HyperKit plugin
InfraKit may pass an optional LogicalID into an instance. It expects
this LogicalID to be returned via DescribeInstances(). If they don't
match, it assumes something is wrong.

Here, we use the LogicalID passed in (or construct one based on
the internal ID) and stash it in the state directory. It is retrieved
in DescribeInstances().

While at it, also improve logging.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-05 18:01:52 +01:00
Riyaz Faizullabhoy
a6257844d1 Merge pull request #1506 from justincormack/gcp-indent
Fix indentation in examples/gcp.yml
2017-04-05 09:05:05 -07:00
Justin Cormack
c21996fc0b Fix indentation in examples/gcp.yml
Yaml is fussy...

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-05 16:24:03 +01:00
Justin Cormack
4516d360ed Merge pull request #1492 from dave-tucker/gcp
Add gcp backend for moby run
2017-04-05 15:17:01 +01:00
Justin Cormack
50c1bd9df3 Merge pull request #1500 from justincormack/mount-overrides
Allow overriding the default mount options
2017-04-05 15:13:06 +01:00
Justin Cormack
3bffae8fe7 Allow overriding the default mount options
This refactors the mount handling, without changing any defaults.

Any specification of a mount destination will override the default,
so if you want to make `sysfs` read only you can add

```
mounts:
  - type: sysfs
    options: ["ro"]
```

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-05 15:00:13 +01:00