Done as follows:
find -name build.yml | xargs dirname | while read d ; do
t=$(linuxkit pkg show-tag $d)
./scripts/update-component-sha.sh --image ${t%:*} ${t#*:}
done
git commit -s test pkg tools blueprints examples projects/kubernetes projects/swarmd docs linuxkit.yml Makefile src
This explicitly excludes projects/* which I did not know whether to update.
Then:
git reset --hard
for i in init runc containerd ca-certificates sysctl dhcpcd getty rngd ; do
o=$(git grep -h "\(image:\|-\) *linuxkit/$i:[0-9a-f]\{40\}" origin/master:linuxkit.yml | awk '// { print $2 }')
n=$(linuxkit pkg show-tag pkg/$i)
./scripts/update-component-sha.sh "$o" "$n"
done
git commit --amend projects
This updates any projects which were using components with the same hash as the
top-level linuxkit.yml.
Signed-off-by: Ian Campbell <ijc@docker.com>
In particular also fix the wireguard test whose kernel
tag hasn't been updated for quite some time...
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
3 components:
- network: read eht0 and proxy only DHCP traffic
- engine: read DHCP traffic, handle DHCP client state machine, and call the
host actuator to change the host config when a lease is obtained
host system configuration.
- actuator: perform the acutall net syscalls, read and write host configuration
files, etc
These three components can either be linked together in a single binary
(see src/dhcp-client/main.ml) or can be used as 3 binaries communicating
over cap-n-proto.
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
pull in newer containerd v1.0.0-alpha0 via updated alpine base, update runc to
429a5387123625040bacfbb60d96b1cbd02293ab which is vendored by that version of
containerd (and also update alpine base for runc)
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
These are not needed, but we are inconsistent. Been waiting for a
quiet moment to fix this since I noticed while doing a presentation...
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Lots of boilerplate for now on, will work on upstreaming that in the tool
properly if needed later.
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
```
$ fdd init &
$ fdd share /tmp/foo # serve a fresh socketpair on that path
$ fdd test /tmp/foo # read the socketpair and test that it works
```
Instead of `fdd test` (which is only useful for testing), users are expected to
connect to the unix domain socket and call `recvmsg(2)`. They will get one side
of the socketpair. Two different processes can do this and they will be able to
talk to each other.
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
- Use the new style kernel tags with the full kernel version
- Update packages with new alpine base and new/simplified Makefiles.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
- IO has been upstreamed in mirage-flow-lwt
- Init.Flow.Fd has been upstreamed in mirage-flow-unix
- Init.Flow.Rawlink has been upstreamed in mirage-flow-rawlink
- Remove some dead-code in unikernel.ml
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
This includes https://github.com/containerd/containerd/pull/994 and hence
requires updating the various instances of `/etc/containerd/config.toml`.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
Currently it supports only `service start <SERVICE>`, but it could grow e.g.
`stop`, `exec` etc in the future (although you can still use `ctr` for those).
In order to be able to use go-compile.sh the containerd build needs to move
from /root/go to /go as the GOPATH.
The vendoring situation is not ideal, but since this tool wants to be an exact
match for the containerd it seems tollerable to reuse its vendoring.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
This doesn't exist with newer ctr or in systems where service containers are
not started using the ctr tool. All it contains today are the stdio FIFOs,
which are not in general useful to access after container creation.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
The rootfs were containing way too much binaries and runc command where not
started in the correct directory.
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
Use the `with-cdhcpc` branch of charrua-client, which exposes `Dhcp_client_lwt`. Dhcp_client_lwt exposes similar functions to `Dhcp_client_mirage`, but does not impose the structure of a Mirage_types_lwt.ipv4_config on the returned object, rather returning the full lease; the engine can then expose whatever information from the lease it finds to be pertinent.
Signed-off-by: Mindy Preston <mindy.preston@docker.com>
The intention with the yaml fragment is that it specifies the
set of processes that form the daemon, with minimal privileges
for each component and each running inside a separate container.
In addition to the normal container capabilities, there is also
a new field which lets a startup process establish an RPC channel,
based on a Capnp specification. This allows for extremely
unprivileged components to be started, such as the `dhcp-engine`
in this example which can only communicate with the outside world
via the `dhcp-network` (to transmit) or `dhcp-actuator` (to alter
the state of the local Linux distribution).
This is a first cut at the yaml interface and the capnp, with the
intention to refine it as we combine it with the rest of the existing
prototype (which currently doesnt have an RPC layer). Expect
more changes...
Signed-off-by: Anil Madhavapeddy <anil@docker.com>
- Update to packages using the Alpine 3.6 base image
- Remove config for packages which now supply it
- Update/add trust section
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
The latest version of the `moby` tool now requires that the output formats
be specified in the CLI not in the yaml file.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Not sure when this arrived but it was stopping anything running.
Appears not to be in the older test `init` containers.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
It's not very interesting to see the forwarded stdout/stderr messages as
they will end-up being printed anyway.
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
The priv container populate the `/mac` key on startup, that the calf can
then read.
Also add more fine-grained control over read/write delete capabilities attached
to the routes, e.g. the calf can read /mac but not write to it.
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
As suggested by @shykes these are clearer
- onboot for things that are run at boot time to completion
- services for persistent services
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
`nested runc` unfortunately needs a lot of caps/privileged. The removal of `readonly: true` is also a bit unfortunate.
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
Make all the low-level init code synchronous to avoid weird blocks on `close`.
Also move the net and ctl file descriptor at the beginning of the fd space for
the calf.
The SDK also allow to spamn multiple exec calves, which will all have the same
fd map:
- 0: stdin = /dev/null
- 1: stdout = pipe to parent stdout
- 2: stderr = pipe to parent stderr
- 3: net = socketpair to parent "net" pipe
- 4: ctl = socketpair to parent "ctl" pipe
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
This forces us to bind mount /lib but will be replaced by calling the proper
bindings later on.
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
In the riddler change I changed "command" in the yaml to "args"
but did not change the files. In fact we basically used the
default command everywhere so this did not actually break.
Remove the unnecessary "command" lines to simplify yaml.
Revert the command to args change for now as I think I prefer
command, but its easier to switch now. Need to think if the
entrypoint/command distinction matters before finalizing.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Generated largely from the specified config; small parts taken from `docker image inspect`,
such as the command line.
Renamed some of the yaml keys to match the OCI spec rather than Docker Compose as
we decided they are more readable, no more underscores.
Add some extra functionality
- tmpfs specification
- fully general mount specification
- no new privileges can be specified now
For nostalgic reasons, using engine-api to talk to the docker cli as
we only need an old API version, and it is nice and easy to vendor...
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Use a file (that we can drop easily into the rootfs) instead of passing the full
command on the CLI (as config.json needs to be edited otherwise).
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
The inotify bindings that we are using is a bit sensitive to init
conditions, and it seems to not like being run inside in a container.
See https://github.com/samoht/irmin-watcher/issues/10
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
For now, use jbuilder to build the calf as well, this gives us a bit
more control than the mirage tool. We will switch back to the mirage
tool later on if we want to use more fancy backends (such as KVM).
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
The protocol allows the client to send concurrent RPCs to the server.
The server replies by keeping the client ID, and the client keeps a
dispatch table of queries to route the retries. By doing things like
that, the server has a strong control over resource allocation, so
a bad client cannot exhaust all the server memory.
Also add some simple tests for the serialization + wire protocol which
are all passing. Proper concurrency/resource exhaustion usage will be
added later.
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
Previously, the control plane was using HTTP client/server, that various people
found way too complex to run in a privileged container (for very good reasons).
So switching to a simpler binary protocol, using c-like structures. Will
probably switch to an other serialization protocol later (eg. protobuf
or cap-n-proto).
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
The new hiearchy is:
- pkg/{init,mirage-compile}: additional Moby packages
- src/sdk -> the begining of the MirageOS SDK for Moby
- src/dhcp-client -> the code for the MirageOS dhcp-client service
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
Today the SDK only contains helper code to create secure Moby services based on
MirageOS. Today the SDK only defines the architecture and the communication
pipes between the privileged service and the calf; the proper communication
API will be specified after we have a few more use-cases.
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
Split the bits which can be re-used in other services (e.g. init dance
and the server-side of the control path). `main.ml` now only contains what
is specific to the DHCP logic (+ the /caf directory).
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
Plus a few more minor improvements:
- compile with jbuilder.
- start working on the control path.
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
docker-compose and other utilities use the .yml extension.
For consistency rename all .yaml to .yml
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>