Commit Graph

59 Commits

Author SHA1 Message Date
Thomas Gazagnaire
14a5b64dcc Update to latest capnp & capnp-rpc
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-09-04 16:21:53 +02:00
Thomas Gazagnaire
f5716ce985 Split the DHCP client into three components communicating via named pipes
3 components:

- network: read eht0 and proxy only DHCP traffic
- engine: read DHCP traffic, handle DHCP client state machine, and call the
  host actuator to change the host config when a lease is obtained
  host system configuration.
- actuator: perform the acutall net syscalls, read and write host configuration
  files, etc

These three components can either be linked together in a single binary
(see src/dhcp-client/main.ml) or can be used as 3 binaries communicating
over cap-n-proto.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-07-17 17:46:17 +02:00
Thomas Gazagnaire
5525937c10 sdk: add fdd exec to map socketpair shares to local fds
To enable this:

```
$ fdd init &
$ fdd share /tmp/foo

$ fdd exec -m /tmp/foo:1:2 -- echo hello world!

$ fdd exec -m /tmp/foo:0 -- cat
hello world!
```

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-06-30 16:11:41 +02:00
Thomas Gazagnaire
b4feb71f78 sdk: add a yml example on how to use fdd to create container channels
Lots of boilerplate for now on, will work on upstreaming that in the tool
properly if needed later.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-06-30 16:11:41 +02:00
Thomas Gazagnaire
875b877047 sdk: add Dockerfile root pivot runs + static linking
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-06-29 18:17:02 +02:00
Thomas Gazagnaire
56229e486b sdk: add a file-descriptor sharing server
```
$ fdd init &
$ fdd share /tmp/foo # serve a fresh socketpair on that path
$ fdd test /tmp/foo  # read the socketpair and test that it works
```

Instead of `fdd test` (which is only useful for testing), users are expected to
connect to the unix domain socket and call `recvmsg(2)`. They will get one side
of the socketpair. Two different processes can do this and they will be able to
talk to each other.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-06-29 17:53:49 +02:00
Rolf Neugebauer
43ac05e4dd Merge pull request #2072 from samoht/sdk-upstream-cleanup
Sdk upstream cleanup
2017-06-19 10:01:14 -07:00
Thomas Gazagnaire
be7a54b5c7 sdk: update Docker to pin dev versions of upstream libraries
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-06-19 17:33:28 +02:00
Thomas Gazagnaire
681781651f sdk: add empty mli files to executable to catch up more warnings
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-06-19 17:33:28 +02:00
Thomas Gazagnaire
2ef504405b sdk: remove some dead-code
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-06-19 17:33:28 +02:00
Thomas Gazagnaire
852468bc99 sdk: use upstream libraires for SDK helpers
- IO has been upstreamed in mirage-flow-lwt
- Init.Flow.Fd has been upstreamed in mirage-flow-unix
- Init.Flow.Rawlink has been upstreamed in mirage-flow-rawlink
- Remove some dead-code in unikernel.ml

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-06-19 17:33:28 +02:00
Thomas Gazagnaire
821a9f2e06 sdk: add missing files
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-06-19 16:57:49 +02:00
Thomas Gazagnaire
e76d6c09a4 sdk: fix dhcp's Dockerfile
The rootfs were containing way too much binaries and runc command where not
started in the correct directory.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-06-14 16:13:40 +01:00
Thomas Gazagnaire
6fb3820925 sdk: push images to miragesdk org on hub
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-06-14 12:09:02 +01:00
Thomas Gazagnaire
03cd4d6fd3 sdk: replace custom transport protocol by Capnproto
Initial patch by @talex5

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-06-14 11:43:27 +01:00
Mindy Preston
1ab32f9ca7 use dhcp_client_lwt instead of dhcp_client_mirage
Use the `with-cdhcpc` branch of charrua-client, which exposes `Dhcp_client_lwt`.  Dhcp_client_lwt exposes similar functions to `Dhcp_client_mirage`, but does not impose the structure of a Mirage_types_lwt.ipv4_config on the returned object, rather returning the full lease; the engine can then expose whatever information from the lease it finds to be pertinent.

Signed-off-by: Mindy Preston <mindy.preston@docker.com>
2017-06-09 13:57:07 -05:00
Mindy Preston
bcb0c52fff nudge some build updates around
build from (now-released) canpnproto 0.6.0; use the latest jbuilder beta.

Signed-off-by: Mindy Preston <mindy.preston@docker.com>
2017-06-09 13:38:55 -05:00
Thomas Gazagnaire
c4c44253cb sdk: the tests need logs.fmt
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-06-06 13:42:45 +02:00
Thomas Gazagnaire
c0d999dbab sdk: use mtime 1.0
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-06-06 13:42:42 +02:00
Thomas Gazagnaire
94583013aa miragesdk: simplify the build by using multi-stage Dockerfile
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-14 17:05:36 +02:00
Thomas Gazagnaire
238879f2d4 miragesdk: add temporary auto-gen files
The capnp compiler is not yet available in alpine.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-12 16:29:49 +02:00
Thomas Gazagnaire
a0546bba88 miragesdk: use cap-n-proto instead of custom binary protocol for calf/priv API
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-12 16:29:49 +02:00
Thomas Gazagnaire
2b0009ee06 miragesdk: be a bit less verbose in debug mode
It's not very interesting to see the forwarded stdout/stderr messages as
they will end-up being printed anyway.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-12 12:08:32 +02:00
Thomas Gazagnaire
396ffabc41 miragesdk: tune down some debug code
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-12 11:04:33 +02:00
Thomas Gazagnaire
df9833ed68 miragesdk: add missing Sdk.Net files
Probably my inconscient which refused to show the world these hacks.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-12 11:04:16 +02:00
Thomas Gazagnaire
abe96b0447 miragesdk: use the interface's MAC address instead of using a random one
The priv container populate the `/mac` key on startup, that the calf can
then read.

Also add more fine-grained control over read/write delete capabilities attached
to the routes, e.g. the calf can read /mac but not write to it.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-12 11:04:16 +02:00
Thomas Gazagnaire
0d7e584e32 miragesdk: remove dead-code and code repetition
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-12 11:04:16 +02:00
Thomas Gazagnaire
95d362ab7e miragesdk: clean error handling for Sdk.Ctl.Client
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-12 11:04:16 +02:00
Thomas Gazagnaire
703657a54f miragesdk: update Dockerfiles
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-12 11:04:16 +02:00
Thomas Gazagnaire
a60ac17233 miragesdk: start the calf using runc
`nested runc` unfortunately needs a lot of caps/privileged. The removal of `readonly: true` is also a bit unfortunate.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 19:39:21 +02:00
Thomas Gazagnaire
bb536803be miragesdk: use the latest image dhcp-client image
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:25 +02:00
Thomas Gazagnaire
f44e2ffbcb miragesdk: cleaner errors
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:25 +02:00
Thomas Gazagnaire
fd447ee082 miragesdk: really allow to configure the DB path with the CLI
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:24 +02:00
Thomas Gazagnaire
4437c4a23b miragesdk: add a config.json file for the calf
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:24 +02:00
Thomas Gazagnaire
f5306b23ef miragesdk: fix the fork/exec init code and add a test
Make all the low-level init code synchronous to avoid weird blocks on `close`.
Also move the net and ctl file descriptor at the beginning of the fd space for
the calf.

The SDK also allow to spamn multiple exec calves, which will all have the same
fd map:

- 0: stdin  = /dev/null
- 1: stdout = pipe to parent stdout
- 2: stderr = pipe to parent stderr
- 3: net    = socketpair to parent "net" pipe
- 4: ctl    = socketpair to parent "ctl" pipe

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:24 +02:00
Thomas Gazagnaire
3cec2b1f5e miragesdk: refactor the SDK
Expose a non-unix dependent flow-like API, so it is easier to test/use in a
unikernel.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:24 +02:00
Thomas Gazagnaire
a07952d4e6 miragesdk: shell out to ifconfig and ip to set the IP and routes
This forces us to bind mount /lib but will be replaced by calling the proper
bindings later on.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:24 +02:00
Thomas Gazagnaire
0938ae44f6 miragesdk: fail early on short read/write returning 0
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-06 17:44:24 +02:00
Thomas Gazagnaire
2dbcf99030 miragesdk: do not force the creation of pipes when linking with the SDK
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-04 10:06:59 +02:00
Thomas Gazagnaire
4944652d37 miragesdk: improve dev README
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-04 10:05:22 +02:00
Thomas Gazagnaire
e3939e03c0 miragesdk: fix the build after the switch from mirage tool to jbuilder
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-02 13:17:26 +02:00
Thomas Gazagnaire
1495dabccb miragesdk: allow the command run by the privileged container to be overwritten
Use a file (that we can drop easily into the rootfs) instead of passing the full
command on the CLI (as config.json needs to be edited otherwise).

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-02 13:17:25 +02:00
Thomas Gazagnaire
8d3cea1980 miragesdk: do not use inotify in the privileged container
The inotify bindings that we are using is a bit sensitive to init
conditions, and it seems to not like being run inside in a container.

See https://github.com/samoht/irmin-watcher/issues/10

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-02 13:17:25 +02:00
Thomas Gazagnaire
dfb078825b miragesdk: simplify the build of the calf binary
For now, use jbuilder to build the calf as well, this gives us a bit
more control than the mirage tool. We will switch back to the mirage
tool later on if we want to use more fancy backends (such as KVM).

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-02 13:17:25 +02:00
Thomas Gazagnaire
6500becfea miragesdk: do not die brutally when the calf terminates
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 19:57:36 +02:00
Thomas Gazagnaire
0f0b721b19 miragesdk: better computation of the image tag
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 19:57:03 +02:00
Thomas Gazagnaire
5223c08d05 miragesdk: fix the compilation of the calf
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 19:22:41 +02:00
Thomas Gazagnaire
d4db6c65a1 miragesdk: fix compilation of the dhcp-client image
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 18:35:52 +02:00
Thomas Gazagnaire
faa9e1811d miragesdk: implement a simple binary RPCs for the control plane
The protocol allows the client to send concurrent RPCs to the server.
The server replies by keeping the client ID, and the client keeps a
dispatch table of queries to route the retries. By doing things like
that, the server has a strong control over resource allocation, so
a bad client cannot exhaust all the server memory.

Also add some simple tests for the serialization + wire protocol which
are all passing. Proper concurrency/resource exhaustion usage will be
added later.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 17:22:16 +02:00
Thomas Gazagnaire
a61171e87e miragesdk: add tests for control plane messages
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 17:22:16 +02:00