Commit Graph

451 Commits

Author SHA1 Message Date
Thomas Gazagnaire
914d27bed9 miragesdk: remove CAP_SYS_PTRACE
Since https://github.com/opencontainers/runc/pull/774 we don't need this anymore.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-07 12:10:35 +02:00
Thomas Gazagnaire
d289de6416 miragesdk: update to latest runc
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-07 12:10:35 +02:00
Thomas Gazagnaire
a60ac17233 miragesdk: start the calf using runc
`nested runc` unfortunately needs a lot of caps/privileged. The removal of `readonly: true` is also a bit unfortunate.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 19:39:21 +02:00
Thomas Gazagnaire
bb536803be miragesdk: use the latest image dhcp-client image
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:25 +02:00
Thomas Gazagnaire
f44e2ffbcb miragesdk: cleaner errors
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:25 +02:00
Thomas Gazagnaire
fd447ee082 miragesdk: really allow to configure the DB path with the CLI
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:24 +02:00
Thomas Gazagnaire
4437c4a23b miragesdk: add a config.json file for the calf
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:24 +02:00
Thomas Gazagnaire
f5306b23ef miragesdk: fix the fork/exec init code and add a test
Make all the low-level init code synchronous to avoid weird blocks on `close`.
Also move the net and ctl file descriptor at the beginning of the fd space for
the calf.

The SDK also allow to spamn multiple exec calves, which will all have the same
fd map:

- 0: stdin  = /dev/null
- 1: stdout = pipe to parent stdout
- 2: stderr = pipe to parent stderr
- 3: net    = socketpair to parent "net" pipe
- 4: ctl    = socketpair to parent "ctl" pipe

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:24 +02:00
Thomas Gazagnaire
3cec2b1f5e miragesdk: refactor the SDK
Expose a non-unix dependent flow-like API, so it is easier to test/use in a
unikernel.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:24 +02:00
Thomas Gazagnaire
a07952d4e6 miragesdk: shell out to ifconfig and ip to set the IP and routes
This forces us to bind mount /lib but will be replaced by calling the proper
bindings later on.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:24 +02:00
Thomas Gazagnaire
0938ae44f6 miragesdk: fail early on short read/write returning 0
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:24 +02:00
Thomas Gazagnaire
25d3e42204 miragesdk: update init image
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:24 +02:00
Justin Cormack
01fba0f3f5 Merge pull request #1479 from justincormack/command
Revert Command->Args but remove from yaml where not needed
2017-04-04 10:49:32 +01:00
Justin Cormack
57c75741e9 Revert Command->Args but remove from yaml where not needed
In the riddler change I changed "command" in the yaml to "args"
but did not change the files. In fact we basically used the
default command everywhere so this did not actually break.

Remove the unnecessary "command" lines to simplify yaml.

Revert the command to args change for now as I think I prefer
command, but its easier to switch now. Need to think if the
entrypoint/command distinction matters before finalizing.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-04 09:51:17 +01:00
Thomas Gazagnaire
2dbcf99030 miragesdk: do not force the creation of pipes when linking with the SDK
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-04 10:06:59 +02:00
Thomas Gazagnaire
4944652d37 miragesdk: improve dev README
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-04 10:05:22 +02:00
Justin Cormack
065af9707c Replace riddler with code that constructs config.json directly
Generated largely from the specified config; small parts taken from `docker image inspect`,
such as the command line.

Renamed some of the yaml keys to match the OCI spec rather than Docker Compose as
we decided they are more readable, no more underscores.

Add some extra functionality
- tmpfs specification
- fully general mount specification
- no new privileges can be specified now

For nostalgic reasons, using engine-api to talk to the docker cli as
we only need an old API version, and it is nice and easy to vendor...

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-03 23:28:55 +01:00
Thomas Gazagnaire
e3939e03c0 miragesdk: fix the build after the switch from mirage tool to jbuilder
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-02 13:17:26 +02:00
Thomas Gazagnaire
e976a6c4aa miragesdk: ship Git the init image (to debug)
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-02 13:17:26 +02:00
Thomas Gazagnaire
1495dabccb miragesdk: allow the command run by the privileged container to be overwritten
Use a file (that we can drop easily into the rootfs) instead of passing the full
command on the CLI (as config.json needs to be edited otherwise).

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-02 13:17:25 +02:00
Thomas Gazagnaire
8d3cea1980 miragesdk: do not use inotify in the privileged container
The inotify bindings that we are using is a bit sensitive to init
conditions, and it seems to not like being run inside in a container.

See https://github.com/samoht/irmin-watcher/issues/10

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-02 13:17:25 +02:00
Thomas Gazagnaire
dfb078825b miragesdk: simplify the build of the calf binary
For now, use jbuilder to build the calf as well, this gives us a bit
more control than the mirage tool. We will switch back to the mirage
tool later on if we want to use more fancy backends (such as KVM).

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-02 13:17:25 +02:00
Thomas Gazagnaire
6500becfea miragesdk: do not die brutally when the calf terminates
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 19:57:36 +02:00
Thomas Gazagnaire
0f0b721b19 miragesdk: better computation of the image tag
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 19:57:03 +02:00
Thomas Gazagnaire
5223c08d05 miragesdk: fix the compilation of the calf
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 19:22:41 +02:00
Thomas Gazagnaire
725a6e8070 miragesdk: update the dhcp-client example with latest image
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 18:36:30 +02:00
Thomas Gazagnaire
d4db6c65a1 miragesdk: fix compilation of the dhcp-client image
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 18:35:52 +02:00
Thomas Gazagnaire
faa9e1811d miragesdk: implement a simple binary RPCs for the control plane
The protocol allows the client to send concurrent RPCs to the server.
The server replies by keeping the client ID, and the client keeps a
dispatch table of queries to route the retries. By doing things like
that, the server has a strong control over resource allocation, so
a bad client cannot exhaust all the server memory.

Also add some simple tests for the serialization + wire protocol which
are all passing. Proper concurrency/resource exhaustion usage will be
added later.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 17:22:16 +02:00
Thomas Gazagnaire
a61171e87e miragesdk: add tests for control plane messages
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 17:22:16 +02:00
Thomas Gazagnaire
c582aafe45 miragesdk: add tests for stdout/stderr pipes
And the tests pass!

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 13:40:57 +02:00
Thomas Gazagnaire
2b48442fee miragesdk: update .gitignore
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 13:40:57 +02:00
Thomas Gazagnaire
9db898ceaf miragesdk: enable all warnings when compiling the projects
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 13:40:57 +02:00
Thomas Gazagnaire
b5a3d4b2aa miragesdk: rework the control plane protocol
Previously, the control plane was using HTTP client/server, that various people
found way too complex to run in a privileged container (for very good reasons).

So switching to a simpler binary protocol, using c-like structures. Will
probably switch to an other serialization protocol later (eg. protobuf
or cap-n-proto).

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 13:40:57 +02:00
Thomas Gazagnaire
df71c0f299 miragesdk: add mimimal tests for the SDK
Very minimal so far, but the plan is to add much more of them.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-29 15:41:23 +02:00
Thomas Gazagnaire
c06454273b miragesdk: add a dev-clean target
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-29 14:28:19 +02:00
Thomas Gazagnaire
ec988eb93b miragesdk: fix dev target
Paths have changed since 47b9f08b16

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-29 14:27:38 +02:00
Thomas Gazagnaire
7d08325c7a miragesdk: move files around
The new hiearchy is:
- pkg/{init,mirage-compile}: additional Moby packages
- src/sdk -> the begining of the MirageOS SDK for Moby
- src/dhcp-client -> the code for the MirageOS dhcp-client service

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-29 13:39:20 +02:00
Thomas Gazagnaire
d008e859c6 miragesdk: first cut of a MirageSDK
Today the SDK only contains helper code to create secure Moby services based on
MirageOS. Today the SDK only defines the architecture and the communication
pipes between the privileged service and the calf; the proper communication
API will be specified after we have a few more use-cases.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-29 12:34:48 +02:00
Thomas Gazagnaire
13d110e2c7 miragesdk: minor cleanups
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-29 12:34:41 +02:00
Anil Madhavapeddy
5c9d5315e5 miragesdk: more tweaks to README markdown
Signed-off-by: Anil Madhavapeddy <anil@docker.com>
2017-03-28 22:07:15 +01:00
Anil Madhavapeddy
ab72dbb0e4 miragesdk: add a README explaining the background of the project
Also link it to the roadmap and why-dhcp docs, and include review
comments from @yomimono

Signed-off-by: Anil Madhavapeddy <anil@docker.com>
2017-03-28 19:13:17 +01:00
Thomas Gazagnaire
56085a3e6c miragesdk: re-org source code
Split the bits which can be re-used in other services (e.g. init dance
and the server-side of the control path). `main.ml` now only contains what
is specific to the DHCP logic (+ the /caf directory).

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-28 14:42:02 +02:00
Thomas Gazagnaire
7fa21377b5 miragesdk: update TODO list
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-28 11:11:17 +02:00
Thomas Gazagnaire
1bee082c6c miragesdk: use a custom BPF filter to allow the calf to get a DHCP lease
Plus a few more minor improvements:

- compile with jbuilder.
- start working on the control path.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-28 11:09:55 +02:00
Thomas Gazagnaire
b0f758a20d miragesdk: add strace in the init image
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-28 11:09:50 +02:00
Thomas Gazagnaire
18635ed2e1 miragesdk: remove dhccpd in the init container
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-28 11:09:50 +02:00
Thomas Gazagnaire
ebdf371421 miragesdk: fork pkg/init to experiment with removal of dhcpcd
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-28 11:09:50 +02:00
Mindy Preston
46e650ede4 more documentation on why & what we're doing with dhcp
Signed-off-by: Mindy Preston <mindy.preston@docker.com>
2017-03-27 11:55:51 -05:00
Rolf Neugebauer
e9fbe43b34 Rename .yaml to .yml
docker-compose and other utilities use the .yml extension.
For consistency rename all .yaml to .yml

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-27 09:53:26 +02:00
Thomas Gazagnaire
ad357c21b8 mirageSDK: rename README.md into roadmap.md
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-20 15:00:47 +01:00
Thomas Gazagnaire
30be4647ad Restructure the mirage/dhcp container into the new project structure
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-17 17:43:20 +01:00