Commit Graph

67 Commits

Author SHA1 Message Date
Thomas Gazagnaire
1e83a9f9b4 miragesdk: update roadmap
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-12 17:47:42 +02:00
Thomas Gazagnaire
238879f2d4 miragesdk: add temporary auto-gen files
The capnp compiler is not yet available in alpine.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-12 16:29:49 +02:00
Thomas Gazagnaire
a0546bba88 miragesdk: use cap-n-proto instead of custom binary protocol for calf/priv API
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-12 16:29:49 +02:00
Thomas Gazagnaire
0a18bf3a00 miragesdk: update to latest base init
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-12 15:56:29 +02:00
Thomas Gazagnaire
dc4ff8accd miragesdk: update DHCP client example to use latest images
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-12 12:09:30 +02:00
Thomas Gazagnaire
2b0009ee06 miragesdk: be a bit less verbose in debug mode
It's not very interesting to see the forwarded stdout/stderr messages as
they will end-up being printed anyway.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-12 12:08:32 +02:00
Thomas Gazagnaire
b18d50b5e4 miragesdk: rebase init pkg to catch up with upstream changes
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-12 12:06:50 +02:00
Thomas Gazagnaire
396ffabc41 miragesdk: tune down some debug code
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-12 11:04:33 +02:00
Thomas Gazagnaire
df9833ed68 miragesdk: add missing Sdk.Net files
Probably my inconscient which refused to show the world these hacks.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-12 11:04:16 +02:00
Thomas Gazagnaire
abe96b0447 miragesdk: use the interface's MAC address instead of using a random one
The priv container populate the `/mac` key on startup, that the calf can
then read.

Also add more fine-grained control over read/write delete capabilities attached
to the routes, e.g. the calf can read /mac but not write to it.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-12 11:04:16 +02:00
Thomas Gazagnaire
0d7e584e32 miragesdk: remove dead-code and code repetition
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-12 11:04:16 +02:00
Thomas Gazagnaire
95d362ab7e miragesdk: clean error handling for Sdk.Ctl.Client
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-12 11:04:16 +02:00
Thomas Gazagnaire
703657a54f miragesdk: update Dockerfiles
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-12 11:04:16 +02:00
Justin Cormack
eb22d6909f system → onboot daemon → services
As suggested by @shykes these are clearer

- onboot for things that are run at boot time to completion
- services for persistent services

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-08 16:56:47 +01:00
Justin Cormack
d9faecdee9 Make init accept a list of images not just a single one.
fix #1527

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-07 14:25:28 +01:00
Thomas Gazagnaire
81debfcd74 miragesdk: update README with moby build/run instructions
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-07 12:13:06 +02:00
Thomas Gazagnaire
914d27bed9 miragesdk: remove CAP_SYS_PTRACE
Since https://github.com/opencontainers/runc/pull/774 we don't need this anymore.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-07 12:10:35 +02:00
Thomas Gazagnaire
d289de6416 miragesdk: update to latest runc
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-07 12:10:35 +02:00
Thomas Gazagnaire
a60ac17233 miragesdk: start the calf using runc
`nested runc` unfortunately needs a lot of caps/privileged. The removal of `readonly: true` is also a bit unfortunate.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 19:39:21 +02:00
Thomas Gazagnaire
bb536803be miragesdk: use the latest image dhcp-client image
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:25 +02:00
Thomas Gazagnaire
f44e2ffbcb miragesdk: cleaner errors
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:25 +02:00
Thomas Gazagnaire
fd447ee082 miragesdk: really allow to configure the DB path with the CLI
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:24 +02:00
Thomas Gazagnaire
4437c4a23b miragesdk: add a config.json file for the calf
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:24 +02:00
Thomas Gazagnaire
f5306b23ef miragesdk: fix the fork/exec init code and add a test
Make all the low-level init code synchronous to avoid weird blocks on `close`.
Also move the net and ctl file descriptor at the beginning of the fd space for
the calf.

The SDK also allow to spamn multiple exec calves, which will all have the same
fd map:

- 0: stdin  = /dev/null
- 1: stdout = pipe to parent stdout
- 2: stderr = pipe to parent stderr
- 3: net    = socketpair to parent "net" pipe
- 4: ctl    = socketpair to parent "ctl" pipe

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:24 +02:00
Thomas Gazagnaire
3cec2b1f5e miragesdk: refactor the SDK
Expose a non-unix dependent flow-like API, so it is easier to test/use in a
unikernel.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:24 +02:00
Thomas Gazagnaire
a07952d4e6 miragesdk: shell out to ifconfig and ip to set the IP and routes
This forces us to bind mount /lib but will be replaced by calling the proper
bindings later on.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:24 +02:00
Thomas Gazagnaire
0938ae44f6 miragesdk: fail early on short read/write returning 0
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:24 +02:00
Thomas Gazagnaire
25d3e42204 miragesdk: update init image
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:24 +02:00
Justin Cormack
01fba0f3f5 Merge pull request #1479 from justincormack/command
Revert Command->Args but remove from yaml where not needed
2017-04-04 10:49:32 +01:00
Justin Cormack
57c75741e9 Revert Command->Args but remove from yaml where not needed
In the riddler change I changed "command" in the yaml to "args"
but did not change the files. In fact we basically used the
default command everywhere so this did not actually break.

Remove the unnecessary "command" lines to simplify yaml.

Revert the command to args change for now as I think I prefer
command, but its easier to switch now. Need to think if the
entrypoint/command distinction matters before finalizing.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-04 09:51:17 +01:00
Thomas Gazagnaire
2dbcf99030 miragesdk: do not force the creation of pipes when linking with the SDK
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-04 10:06:59 +02:00
Thomas Gazagnaire
4944652d37 miragesdk: improve dev README
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-04 10:05:22 +02:00
Justin Cormack
065af9707c Replace riddler with code that constructs config.json directly
Generated largely from the specified config; small parts taken from `docker image inspect`,
such as the command line.

Renamed some of the yaml keys to match the OCI spec rather than Docker Compose as
we decided they are more readable, no more underscores.

Add some extra functionality
- tmpfs specification
- fully general mount specification
- no new privileges can be specified now

For nostalgic reasons, using engine-api to talk to the docker cli as
we only need an old API version, and it is nice and easy to vendor...

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-03 23:28:55 +01:00
Thomas Gazagnaire
e3939e03c0 miragesdk: fix the build after the switch from mirage tool to jbuilder
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-02 13:17:26 +02:00
Thomas Gazagnaire
e976a6c4aa miragesdk: ship Git the init image (to debug)
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-02 13:17:26 +02:00
Thomas Gazagnaire
1495dabccb miragesdk: allow the command run by the privileged container to be overwritten
Use a file (that we can drop easily into the rootfs) instead of passing the full
command on the CLI (as config.json needs to be edited otherwise).

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-02 13:17:25 +02:00
Thomas Gazagnaire
8d3cea1980 miragesdk: do not use inotify in the privileged container
The inotify bindings that we are using is a bit sensitive to init
conditions, and it seems to not like being run inside in a container.

See https://github.com/samoht/irmin-watcher/issues/10

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-02 13:17:25 +02:00
Thomas Gazagnaire
dfb078825b miragesdk: simplify the build of the calf binary
For now, use jbuilder to build the calf as well, this gives us a bit
more control than the mirage tool. We will switch back to the mirage
tool later on if we want to use more fancy backends (such as KVM).

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-02 13:17:25 +02:00
Thomas Gazagnaire
6500becfea miragesdk: do not die brutally when the calf terminates
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 19:57:36 +02:00
Thomas Gazagnaire
0f0b721b19 miragesdk: better computation of the image tag
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 19:57:03 +02:00
Thomas Gazagnaire
5223c08d05 miragesdk: fix the compilation of the calf
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 19:22:41 +02:00
Thomas Gazagnaire
725a6e8070 miragesdk: update the dhcp-client example with latest image
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 18:36:30 +02:00
Thomas Gazagnaire
d4db6c65a1 miragesdk: fix compilation of the dhcp-client image
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 18:35:52 +02:00
Thomas Gazagnaire
faa9e1811d miragesdk: implement a simple binary RPCs for the control plane
The protocol allows the client to send concurrent RPCs to the server.
The server replies by keeping the client ID, and the client keeps a
dispatch table of queries to route the retries. By doing things like
that, the server has a strong control over resource allocation, so
a bad client cannot exhaust all the server memory.

Also add some simple tests for the serialization + wire protocol which
are all passing. Proper concurrency/resource exhaustion usage will be
added later.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 17:22:16 +02:00
Thomas Gazagnaire
a61171e87e miragesdk: add tests for control plane messages
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 17:22:16 +02:00
Thomas Gazagnaire
c582aafe45 miragesdk: add tests for stdout/stderr pipes
And the tests pass!

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 13:40:57 +02:00
Thomas Gazagnaire
2b48442fee miragesdk: update .gitignore
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 13:40:57 +02:00
Thomas Gazagnaire
9db898ceaf miragesdk: enable all warnings when compiling the projects
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 13:40:57 +02:00
Thomas Gazagnaire
b5a3d4b2aa miragesdk: rework the control plane protocol
Previously, the control plane was using HTTP client/server, that various people
found way too complex to run in a privileged container (for very good reasons).

So switching to a simpler binary protocol, using c-like structures. Will
probably switch to an other serialization protocol later (eg. protobuf
or cap-n-proto).

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 13:40:57 +02:00
Thomas Gazagnaire
df71c0f299 miragesdk: add mimimal tests for the SDK
Very minimal so far, but the plan is to add much more of them.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-29 15:41:23 +02:00