Commit Graph

249 Commits

Author SHA1 Message Date
Rolf Neugebauer
17e4aa3e58 demo: Increase memory of etcd VMs to 1G
512MB is too small to install etcd via apk in the RAM disk.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-07 10:52:40 +01:00
Justin Cormack
f4127faec3 Merge pull request #1526 from samoht/runc-calf
miragesdk: start the calf using runc
2017-04-07 10:42:58 +01:00
Ilya Dmitrichenko
961acc4c21 Working Kubernetes master
Signed-off-by: Ilya Dmitrichenko <errordeveloper@gmail.com>
2017-04-07 10:09:10 +01:00
Thomas Gazagnaire
a60ac17233 miragesdk: start the calf using runc
`nested runc` unfortunately needs a lot of caps/privileged. The removal of `readonly: true` is also a bit unfortunate.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 19:39:21 +02:00
Thomas Gazagnaire
05089f9249 Merge pull request #1469 from samoht/system-handlers
mirage-sdk: add system handlers
2017-04-06 19:28:36 +02:00
Justin Cormack
0511e13bd4 Merge pull request #1523 from rneugeba/demo-cloud
Update demo files/config/docs
2017-04-06 18:10:27 +01:00
Ilya Dmitrichenko
e68e42c0fa WIP Kubernetes
Signed-off-by: Ilya Dmitrichenko <errordeveloper@gmail.com>
2017-04-06 17:57:48 +01:00
Rolf Neugebauer
dca2b4d223 demo: Add files/instructions for a GCP based etcd cluster
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-06 17:44:19 +01:00
Rolf Neugebauer
2bd75a621d demo: Switch etcd bootstrap from discovery service to static IPs
This makes the configuration simpler but requires us to be able
to set IP addresses on instances.

This also, for simplicity, reduces the number of nodes to 3.

The script does not make assumption about specific IP addresses,
but does assume that the nodes have IP addresses such as:
a.b.c.200, a.b.c.201, and a.b.c.202.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-06 17:22:56 +01:00
Thomas Gazagnaire
bb536803be miragesdk: use the latest image dhcp-client image
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:25 +02:00
Thomas Gazagnaire
f44e2ffbcb miragesdk: cleaner errors
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:25 +02:00
Thomas Gazagnaire
fd447ee082 miragesdk: really allow to configure the DB path with the CLI
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:24 +02:00
Thomas Gazagnaire
4437c4a23b miragesdk: add a config.json file for the calf
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:24 +02:00
Thomas Gazagnaire
f5306b23ef miragesdk: fix the fork/exec init code and add a test
Make all the low-level init code synchronous to avoid weird blocks on `close`.
Also move the net and ctl file descriptor at the beginning of the fd space for
the calf.

The SDK also allow to spamn multiple exec calves, which will all have the same
fd map:

- 0: stdin  = /dev/null
- 1: stdout = pipe to parent stdout
- 2: stderr = pipe to parent stderr
- 3: net    = socketpair to parent "net" pipe
- 4: ctl    = socketpair to parent "ctl" pipe

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:24 +02:00
Thomas Gazagnaire
3cec2b1f5e miragesdk: refactor the SDK
Expose a non-unix dependent flow-like API, so it is easier to test/use in a
unikernel.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:24 +02:00
Thomas Gazagnaire
a07952d4e6 miragesdk: shell out to ifconfig and ip to set the IP and routes
This forces us to bind mount /lib but will be replaced by calling the proper
bindings later on.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:24 +02:00
Thomas Gazagnaire
0938ae44f6 miragesdk: fail early on short read/write returning 0
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:24 +02:00
Thomas Gazagnaire
25d3e42204 miragesdk: update init image
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:24 +02:00
Justin Cormack
0a030dc219 Clean up dhcpcd container
It is not necessary to bring up `eth0`, the program does it fine.

This means we can remove shell script, clean up build.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-06 15:32:22 +01:00
Justin Cormack
6917f6d23a Merge pull request #1488 from t-koulouris/master
[WIP] Support launching containers in outer-kernel mode for moby-okernel demo
2017-04-06 11:48:06 +01:00
Rolf Neugebauer
fdee4a099c demo: Fix JSON file
This was corrupted broken by a previous commit.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-06 11:32:26 +01:00
Ian Campbell
2b54e18f9f Drop unnecessary use of start-stop-daemon with containerd.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-04-06 09:34:38 +01:00
Ian Campbell
8270bdfe33 Use exec in a couple of places to avoid needless lingering /bin/sh processes
```
$ apk -U add procps
$ ps xf
    1 ?        Ss     0:05 /sbin/init
  357 ?        Ss     0:00 /bin/sh /etc/init.d/containerd
  359 ?        Sl     0:00  \_ /usr/bin/containerd
  360 ?        Ss     0:00 /bin/sh /etc/init.d/containers
  432 ?        Sl     0:00  \_ /usr/bin/runc run --bundle /containers/daemon/swa
  466 ?        Ssl    0:00  |   \_ /usr/bin/swarmd --containerd-addr=/run/contai
  427 ?        Sl     0:00  \_ /usr/bin/runc run --bundle /containers/daemon/dhc
  457 ?        Ss     0:00  |   \_ bin/sh /usr/local/bin/start_dhcpcd.sh
  474 ?        S      0:00  |       \_ /sbin/dhcpcd --nobackground
  429 ?        Sl     0:00  \_ /usr/bin/runc run --bundle /containers/daemon/rng
  576 ?        Ss     0:00      \_ /bin/tini /usr/sbin/rngd -f
  580 ?        S      0:00          \_ /usr/sbin/rngd -f
```

becomes

```
$ ps xf
    1 ?        Ss     0:06 /sbin/init
  358 ?        Ss     0:00 /bin/sh /etc/init.d/containers
  426 ?        Sl     0:00  \_ /usr/bin/runc run --bundle /containers/daemon/dhc
  458 ?        Ss     0:00  |   \_ /sbin/dhcpcd --nobackground
  431 ?        Sl     0:00  \_ /usr/bin/runc run --bundle /containers/daemon/swa
  460 ?        Ssl    0:00  |   \_ /usr/bin/swarmd --containerd-addr=/run/contai
  428 ?        Sl     0:00  \_ /usr/bin/runc run --bundle /containers/daemon/rng
  574 ?        Ss     0:00      \_ /bin/tini /usr/sbin/rngd -f
  578 ?        S      0:00          \_ /usr/sbin/rngd -f
  356 ?        Ssl    0:00 /usr/bin/containerd

```

Specifically these are gone:
  357 ?        Ss     0:00 /bin/sh /etc/init.d/containerd
  457 ?        Ss     0:00  |   \_ bin/sh /usr/local/bin/start_dhcpcd.sh

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-04-06 09:34:38 +01:00
Rolf Neugebauer
df5c858d38 demo: Use IP addresses as Logical IDs in etcd infrakit config
Instead of specifying the number of instances, provide a list
of IP addresses for instances. These are passed to the instance
plugin as LogicalID.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-05 18:01:52 +01:00
Rolf Neugebauer
dd58bd75eb Merge pull request #1495 from rneugeba/demo-up
Update demo project
2017-04-05 11:12:28 +01:00
Rolf Neugebauer
a131ee823b demo: Update YAML files to include DHCP
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-05 11:08:48 +01:00
Rolf Neugebauer
1cb8dfc222 demo: Remove infrakit example.
This is superseded by the etcd infrakit setup now in place.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-05 11:07:39 +01:00
Ian Campbell
f01d77930c swarmd: Switch to DHCPD container
Follows on from #1316, copies stanza from moby.yml but also picks up update
from #1493.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-04-05 10:39:04 +01:00
Justin Cormack
a97db7b406 Merge pull request #1487 from rneugeba/ik
Update InfraKit
2017-04-04 17:22:44 +01:00
Theo Koulouris
f6c85a9980 build file fixes for okernel
Signed-off-by: Theo Koulouris <theo.koulouris@hpe.com>
2017-04-04 16:44:26 +01:00
Rolf Neugebauer
411a8b2f94 demo: Update etcd README to new InfraKit version
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-04 16:27:15 +01:00
Ian Campbell
8859a7c0a6 swarmd: use standard init
In #1485 I was still using a local mobylinux/init containing #1436, even though
I had included the necessary files in the swarmd container.

Switch to the current standard init package and drop the unnecessary bind.

Also `git add .gitignore` which I forgot last time too.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-04-04 16:23:25 +01:00
Justin Cormack
08e62e0ebd Merge pull request #1485 from ijc25/swarmd
Initial swarmkit package
2017-04-04 16:09:34 +01:00
Ian Campbell
6a0bd7d035 Initial swarmkit package
This is based on https://github.com/docker/swarmkit/pull/1965 which adds a
direct containerd executor to swarmkit. It is very much a work in progress.

With a suitable moby image (such as projects/swarmd/swarmd.yml) something like
this should work:

runc exec swarmd swarmctl service create --image docker.io/library/nginx:alpine --name nginx
runc exec swarmd swarmctl service ls

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-04-04 16:00:03 +01:00
Rolf Neugebauer
fc84079db3 demo: Initial etcd cluster setup with InfraKit
This just sets up the initial cluster via bootstrap.

It does *not* manage state changes correctly afterwards. If one
node crashes (get's killed) it InfraKit will start a new node,
but the new node does not join the cluster (and the old node
is not removed, either).

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-04 15:37:41 +01:00
Rolf Neugebauer
4b3422f76c demo: WIP for a local etcd setup
- Script to set up a DfM etcd for bootstrapping a cluser
- Custom/local etcd package for moby
  derived from the official image with script to start etcd
- YAML file to create a etcd moby image
- README with current instructions

This has a bunch of stuff, including the discovery URL hard coded.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-04 13:46:19 +01:00
Rolf Neugebauer
218ec6d3d5 demo: Dummy infrakit demo using nginx
This will change to something more distributed...

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-04 13:45:32 +01:00
Rolf Neugebauer
5122afffb1 demo: Add simplest YAML for introducing moby
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-04 13:45:03 +01:00
Justin Cormack
01fba0f3f5 Merge pull request #1479 from justincormack/command
Revert Command->Args but remove from yaml where not needed
2017-04-04 10:49:32 +01:00
Justin Cormack
57c75741e9 Revert Command->Args but remove from yaml where not needed
In the riddler change I changed "command" in the yaml to "args"
but did not change the files. In fact we basically used the
default command everywhere so this did not actually break.

Remove the unnecessary "command" lines to simplify yaml.

Revert the command to args change for now as I think I prefer
command, but its easier to switch now. Need to think if the
entrypoint/command distinction matters before finalizing.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-04 09:51:17 +01:00
Thomas Gazagnaire
2dbcf99030 miragesdk: do not force the creation of pipes when linking with the SDK
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-04 10:06:59 +02:00
Thomas Gazagnaire
4944652d37 miragesdk: improve dev README
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-04 10:05:22 +02:00
Justin Cormack
065af9707c Replace riddler with code that constructs config.json directly
Generated largely from the specified config; small parts taken from `docker image inspect`,
such as the command line.

Renamed some of the yaml keys to match the OCI spec rather than Docker Compose as
we decided they are more readable, no more underscores.

Add some extra functionality
- tmpfs specification
- fully general mount specification
- no new privileges can be specified now

For nostalgic reasons, using engine-api to talk to the docker cli as
we only need an old API version, and it is nice and easy to vendor...

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-03 23:28:55 +01:00
Thomas Gazagnaire
76b33742ac Merge pull request #1448 from samoht/i-got-an-ip
miragesdk: put the DHCP lease info into the host filesystem
2017-04-02 18:32:51 +02:00
Rolf Neugebauer
63c89507e7 docs: Move ebpf.md to projects
This was missed when moving the ebpf stuff into a project.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-02 13:09:29 +01:00
Thomas Gazagnaire
e3939e03c0 miragesdk: fix the build after the switch from mirage tool to jbuilder
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-02 13:17:26 +02:00
Thomas Gazagnaire
e976a6c4aa miragesdk: ship Git the init image (to debug)
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-02 13:17:26 +02:00
Thomas Gazagnaire
1495dabccb miragesdk: allow the command run by the privileged container to be overwritten
Use a file (that we can drop easily into the rootfs) instead of passing the full
command on the CLI (as config.json needs to be edited otherwise).

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-02 13:17:25 +02:00
Thomas Gazagnaire
8d3cea1980 miragesdk: do not use inotify in the privileged container
The inotify bindings that we are using is a bit sensitive to init
conditions, and it seems to not like being run inside in a container.

See https://github.com/samoht/irmin-watcher/issues/10

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-02 13:17:25 +02:00
Thomas Gazagnaire
dfb078825b miragesdk: simplify the build of the calf binary
For now, use jbuilder to build the calf as well, this gives us a bit
more control than the mirage tool. We will switch back to the mirage
tool later on if we want to use more fancy backends (such as KVM).

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-02 13:17:25 +02:00