Currently they will be on a read only partition so broken;
previously this would have been a non persistent read write partition
in an initramfs but this no longer works.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Because Kubernetes is 1.5GB, ISO makes sense as the files do not
take up memory, so you can boot a 1GB machine rather than a 4GB one.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
It is not in any wa=y a required container, and now that arm64
and other architecture machines are widely available we should
start to deprecate it, as it has many issues, eg requires patches
to qemu for Go support, will mislabel images etc.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
These were removed as unused in 8acecf1b62 but with the update to 1.7.2 they
are now pulled in (again?) by the default system.
Signed-off-by: Ian Campbell <ijc@docker.com>
PR #2314 turned /var into a tmpfs (possibly overmounted by a persistent disk)
and made /var/run into a symlink to /run. Adjust various containers and bind
mount settings to allow for this change. In particular ensuring that everything
can find the correct shared /var/run/docker.sock, which due to the symlink is
now actually at /run.
Signed-off-by: Ian Campbell <ijc@docker.com>
After #2289 we need to bind /etc/resolv.conf into the docker and kubelet
containers on both node and master.
Also since #2289 the metadata container requires /etc/resolv.conf to be
populated on the host, which means running DHCP earlier in oneshot onboot mode,
do so.
Signed-off-by: Ian Campbell <ijc@docker.com>
These contribute ~140M to the common image cache but do not appear to be used
by either the base system nor the sock-shop demo. They can/will still be pulled
on demands as necessary.
Signed-off-by: Ian Campbell <ijc@docker.com>
Bumps kubernetes and associated tools and images to v1.6.7 (from v1.6.1).
Updates weave from v1.9.4 to v2.0.1
Updates cni from a snapshot to v0.5.2. Note that the download location has
changed and the tarball no longer includes the `bin` subdirectory, so adjust
build to compensate.
Signed-off-by: Ian Campbell <ijc@docker.com>
Much smaller than the CentOS based one.
Note that ijc25/alpine-ssh has entrypoint==ssh.
Drop Compression=yes, this is used for local ssh so no point compressing (just uses CPU).
Signed-off-by: Ian Campbell <ijc@docker.com>
Remove `-publish` (which is currently Linux/QEMU specific) and replace with a
generic $KUBE_RUN_ARGS envvar. Usage:
KUBE_RUN_ARGS="-publish 2222:22" ./boot.sh
KUBE_PORT_BASE is thus obsolete and removed.
Signed-off-by: Ian Campbell <ijc@docker.com>
This commits an initial version of the Memorizer tracing tool. It collects and
outputs detailed data on the objects (traced from kmalloc/kmem_cache_alloc) and
accesses, tracking the context of each event with respect to thread ID, program
counter, and for allocations name of process.
Signed-off-by: Nathan Dautenhahn <ndd@cis.upenn.edu>
It's slightly embarrassing that this old snapshot was kept around here
rotting for so long, but thankfully something is finally being done
about it.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Apart from adding the recursive target itself this required:
- Unescaping the @ in the image names, this was confusing `make` into always
rebuilding and wasn't necessary (I had previously thought I had seen oddities
due to these being interpreted by the `patsubst`, but I think that was just the
colons.
- Making the recursive rules silent (prepending an @), those command lines are
not especially enlightening and they obscure the output in the show-tags case.
With this the output is like:
$ make --no-print-directory -C image-cache/ show-tags
linuxkitprojects/kubernetes-image-cache-common:94a0715c6b3604e909bc0da74260dc7f1142d90d-dirty
linuxkitprojects/kubernetes-image-cache-control-plane:94a0715c6b3604e909bc0da74260dc7f1142d90d-dirty
Signed-off-by: Ian Campbell <ijc@docker.com>
The total size of the images in the common and control-plane cache is 251M and
528M respectively.
This changes drops the size of the cache images from 353M to 274M and from 630M
to 530M, reducing the overhead from ~100M to ~20M.
The initrd images shrink from 273M to 246M and from 416M to 363M (the initrd's
are compressed).
Signed-off-by: Ian Campbell <ijc@docker.com>
This updates the build of the two image caches to use the `pkg/package.mk`
infrastructure, albeit in a slightly (ok, very) atypical way.
In order to share the bulk of the build code (including the `Dockerfile` and
the `Makefile` machinery to download the images) we arrange for the necessary
bits to be copied at build time into distinct subdirectories and for the
`pkg/package.mk` to be aware of this possibility.
Since pkg/package.mk is only set up to build a single package we use a single
`image-cache/Makefile` to drive the whole process and recurse into
`Makefile.pkg` to build individual packages.
One particular subtlety is that the package hash is based on the `image-cache`
directory (which is in `git`) rather than the generated subdirectories (which
are not in `git`). Since all the generators (and their inputs) are in the
`image-cache` directory this is what we want. This means that the two images
are given the same tag, but this is deliberate and desirable.
The generated directories are completely temporary to avoid picking up stale
versions of images when versions are updated. Images are hardlinked into place.
The images are moved to the linuxkitprojects org. Using a dev tag for now, will
update once everything is in place.
Also use "tag" rather than "build" where appropriate in the Makefile.
There is no point in the .dockerignore now, but add a .gitignore.
Signed-off-by: Ian Campbell <ijc@docker.com>
It is pretty close to our docker package, if we adjust the command
that is run to avoid the actual dind startup script. We can't use
the normal docker image as it does not have mkfs and so on.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
3 components:
- network: read eht0 and proxy only DHCP traffic
- engine: read DHCP traffic, handle DHCP client state machine, and call the
host actuator to change the host config when a lease is obtained
host system configuration.
- actuator: perform the acutall net syscalls, read and write host configuration
files, etc
These three components can either be linked together in a single binary
(see src/dhcp-client/main.ml) or can be used as 3 binaries communicating
over cap-n-proto.
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
Still a flat/unstructured config space, but at least uses the mounting
machinery.
`boot.sh` continues to just work without modification.
Signed-off-by: Ian Campbell <ijc@docker.com>
These were originally generated by some box builder runes and then taken
wholesale here. Format them to be more readable.
Signed-off-by: Ian Campbell <ijc@docker.com>
It doesn't support it. This makes "make cache-images" work. Previously it would
fail with various:
Error: remote trust data does not exist for gcr.io/google_containers/pause-amd64: gcr.io does not have trust data for gcr.io/google_containers/pause-amd64
Signed-off-by: Ian Campbell <ijc@docker.com>
This is a pretty straight port of the previous box stuff, without much attempt
to clean things up.
Image label is a placeholder, will update once a batch of changes are complete.
Signed-off-by: Ian Campbell <ijc@docker.com>
Apart from the /var/lib mount itself the custom package:
- Made host /etc/cni and /opt/cni rshared. This has been handled by init make /
rshared since 3c326bebdf ("Make / rshared").
- Make /var/lib/kubeadm after mount. For now handle this with a dedicated start
of day container instead.
Signed-off-by: Ian Campbell <ijc@docker.com>
Port base is configurable (via $KUBE_PORT_BASE envvar). Master uses this and
nodes use subsequent ports.
Check that the node number is numeric so we can add them to things, but avoid
worker node 0 since the port will clash with master.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
pull in newer containerd v1.0.0-alpha0 via updated alpine base, update runc to
429a5387123625040bacfbb60d96b1cbd02293ab which is vendored by that version of
containerd (and also update alpine base for runc)
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
These are not needed, but we are inconsistent. Been waiting for a
quiet moment to fix this since I noticed while doing a presentation...
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Firstly add option to disable content trust, for the use of e.g. projects which
are pushing to the linuxkitprojects org (which has no trust setup) rather than
the main linuxkit org.
Secondly, when trust _is_ enabled then enable it globally, in particular it is
now active for the `docker build` and hence containers referenced in
Dockerfiles via "FROM" will be checked.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
This version is more up to date and works with the current containerd packages.
swarmd needs /tmp to share /tmp/containerd with containerd.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
Lots of boilerplate for now on, will work on upstreaming that in the tool
properly if needed later.
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
```
$ fdd init &
$ fdd share /tmp/foo # serve a fresh socketpair on that path
$ fdd test /tmp/foo # read the socketpair and test that it works
```
Instead of `fdd test` (which is only useful for testing), users are expected to
connect to the unix domain socket and call `recvmsg(2)`. They will get one side
of the socketpair. Two different processes can do this and they will be able to
talk to each other.
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
- Use the new style kernel tags with the full kernel version
- Update packages with new alpine base and new/simplified Makefiles.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
- IO has been upstreamed in mirage-flow-lwt
- Init.Flow.Fd has been upstreamed in mirage-flow-unix
- Init.Flow.Rawlink has been upstreamed in mirage-flow-rawlink
- Remove some dead-code in unikernel.ml
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
This includes https://github.com/containerd/containerd/pull/994 and hence
requires updating the various instances of `/etc/containerd/config.toml`.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
Currently it supports only `service start <SERVICE>`, but it could grow e.g.
`stop`, `exec` etc in the future (although you can still use `ctr` for those).
In order to be able to use go-compile.sh the containerd build needs to move
from /root/go to /go as the GOPATH.
The vendoring situation is not ideal, but since this tool wants to be an exact
match for the containerd it seems tollerable to reuse its vendoring.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
This doesn't exist with newer ctr or in systems where service containers are
not started using the ctr tool. All it contains today are the stdio FIFOs,
which are not in general useful to access after container creation.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
The new init adds the usermode helper which is needed with
the soon to be pushed new 4.11 kernel update.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
Instead of figuring out which config files to use inside of makeconfig.sh,
let's figure that out in the Dockerfile and pass them into the script.
Signed-off-by: Tycho Andersen <tycho@docker.com>
Instead of having a special case sed script, we can just put this in the
.debug config file, and have a special case when it's being checked.
Signed-off-by: Tycho Andersen <tycho@docker.com>
Let's use the kernel machine architecture for this value.
Also remove a broken check. The "arch" binary on OSX outputs different
stuff than on linux. Since we don't need this check anyway (the variable
is mostly to demonstrate how cross platform stuff would work, not to
actually do it yet), let's just remove the check.
Signed-off-by: Tycho Andersen <tycho@docker.com>
The rootfs were containing way too much binaries and runc command where not
started in the correct directory.
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
This way something comes up when you click on the project on
github, rather than having to hunt for something to explain the
project.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Use the `with-cdhcpc` branch of charrua-client, which exposes `Dhcp_client_lwt`. Dhcp_client_lwt exposes similar functions to `Dhcp_client_mirage`, but does not impose the structure of a Mirage_types_lwt.ipv4_config on the returned object, rather returning the full lease; the engine can then expose whatever information from the lease it finds to be pertinent.
Signed-off-by: Mindy Preston <mindy.preston@docker.com>
0eb21735ae accidentally broke some package
builds by switching linuxkit/alpine to linuxkit/containerd. Let's revert
the ones that shouldn't be there.
Closes#1991
Signed-off-by: Tycho Andersen <tycho@docker.com>
Note that this is not the latest (which was 95efd45db073 at time of writing)
but the next commit 6428b4bad0c2 merges "Port ctr to use client package" breaks
the use of `ctr run --runtime-config` (by removing that option).
This contains https://github.com/containerd/containerd/pull/954 which was
causing some services to fail to start.
All previous uses of 15541037b9 are updated to
5749f2e9e6.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
The intention with the yaml fragment is that it specifies the
set of processes that form the daemon, with minimal privileges
for each component and each running inside a separate container.
In addition to the normal container capabilities, there is also
a new field which lets a startup process establish an RPC channel,
based on a Capnp specification. This allows for extremely
unprivileged components to be started, such as the `dhcp-engine`
in this example which can only communicate with the outside world
via the `dhcp-network` (to transmit) or `dhcp-actuator` (to alter
the state of the local Linux distribution).
This is a first cut at the yaml interface and the capnp, with the
intention to refine it as we combine it with the rest of the existing
prototype (which currently doesnt have an RPC layer). Expect
more changes...
Signed-off-by: Anil Madhavapeddy <anil@docker.com>
This is much more functional (includes networking).
Requires switching to an alpine base because CNI networking backends (such as
weave) can expect iptables binaries to be present, or may want to shell out to
scripts etc.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
Uses the alpine base container. Networking is not disabled because we still
clone at build time.
No swarmd.yml update because the commit referenced here no longer exists in the
upstream repo after a force push. This will change in the next commit.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
The existing set was randomly rather than carefully chosen, lets just be honest
and use "all" until the proper set can be determined.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
Update everything including the current
linuxkit/containerd:b1766e4c4c09f63ac4925a6e4612852a93f7e73b to the new
linuxkit/containerd:deaf5bf838bf7f131c2287ecff3ed9835b0497e2.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
We don't actually build it here, we just use the alpine package, which is
already included in the linuxkit/alpine base.
Can be instantiated with a service stanza such as:
- name: qemu-ga
image: "linuxkit/qemu-ga:e5fbcf55926f6e1a96b3e49a392e547e8be1022c"
binds:
- /dev/vport0p1:/dev/vport0p1
The only functionality I've tested is to report IP addresses to the host (e.g.
via `virsh domifaddr`).
The image is configured to use `/dev/vport0p1` by default. If your VM image is
using a different port number then you can either incorporate the rename into
the bind ("/dev/vport1p1:/dev/vport0p1") or override the command with:
command: ["/usr/bin/qemu-ga", "-p", "/dev/vport1p1"]
The mdev tool appears to not create the symlinks which udev would provide
`/dev/virtio-ports/org.qemu.guest_agent.0` hence the need to hardcode the
potentially unstable `/dev/vportNp1`, `N` seems to depend on the number and
order of virtio devices in use. I don't know if it is possible to get mdev to
create these links. For reference the udev rule is:
SUBSYSTEM=="virtio-ports", KERNEL=="vport*", ATTR{name}=="?*", SYMLINK+="virtio-ports/$attr{name}"
See https://wiki.libvirt.org/page/Qemu_guest_agent for more info on the guest
agent.
A suitable stanza is added to `projects/swarmd/swarmd.yml` since that is where
I am using it.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
We were pointing to the old mobylinux docker hub repo. Let's update the
kernel build to be the new style one.
Note that I didn't bump the kernel version or update the patches at all. We
should do this soon, but for the purposes of our probational channel PoC,
I'm leaving wireguard at the old version for now.
Signed-off-by: Tycho Andersen <tycho@docker.com>
-wireguard is redundant, and with a standardized name, we can merge patches
"automatically", for our probational channel.
Signed-off-by: Tycho Andersen <tycho@docker.com>
Rather than re-build the whole init, let's just include the wireguard tools
in a tools repo.
This also moves *most* of the stuff to new linuxkit infrastructure, instead
of the legacy mobylinux. And checks an item off the TODO list.
Signed-off-by: Tycho Andersen <tycho@docker.com>
- Update to packages using the Alpine 3.6 base image
- Remove config for packages which now supply it
- Update/add trust section
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
The latest version of the `moby` tool now requires that the output formats
be specified in the CLI not in the yaml file.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
The kernel configs themselves are stored as diffs of what we want vs. each
version's defconfig.
Thus, things like e.g. CONFIG_DEVKMEM drop out after it was made
non-default. The implication of this is (I hope) that as upstream adopts
security features, our delta can shrink (or more realistically, only
include the next-next gen features).
Signed-off-by: Tycho Andersen <tycho@docker.com>
This is the script I used with [1] to generate the config diffs and
separate out the arch specific bits. Included mostly just so people can
play around with it if they want to generate their own diffs.
[1]: https://github.com/ulfalizer/Kconfiglib
Signed-off-by: Tycho Andersen <tycho@docker.com>
In particular, let's start with a defconfig and edit it, rather than try to
generate the config entirely from our own diff.
Signed-off-by: Tycho Andersen <tycho@docker.com>
Works around https://github.com/moby/moby/issues/33176 and fixes#1807.
Updated al users of linuxkit/runc:2649198589ef0020d99f613adaeda45ce0093a38 to
this new build.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
Since containers are spawned by containerd (which is in the host PID namespace)
and not in the swarmd container's namespace.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
Note these are only the ones using the `containerd` based `init` as
the `runc` ones are still using an old one.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
...and add straw man implementations of kernel_config.base and
kernel_config.x86 as examples.
First, splitting the build: to avoid duplication, we split the build into
three parts: a "source" stage, a "config" stage, and a "build" stage. The
"source" stage allows us to use a cached image, so we don't have to
re-download the kernel source every time. The "config" step applies our
patches and generates (and checks) the kernel config. I've left this as a
separate step for now so that we can build just an image with a config in
it, without having to ^C the build. However there's no real reason it needs
to be a separate step, assuming that this kernel config design is
acceptable. The third step is the actual kernel build.
Then there is kernel config management: the bulk of it occurs in
makeconfig.sh, with the idea being that we can specify base, arch, and
version specific config options as necessary.
The config files themselves are lists of options (both positive and
negative). We include the negative options, because we want to explicitly
turn off things that are on in the default config (e.g. CONFIG_USELIB), and
it seems cleaner to do things this way then to have some sort of negative
options list.
The options files are sorted with the default behavior of the "sort"
command, which ignores comment lines, meaning that negative options and
positive options are inline with each other. I don't have a strong opinion
on whether or not to group all negative options, or whether this default
behavior makes sense, so I just left it.
Finally, obviously the .base and .x86 files are incomplete. I mostly
selected a few options with interesting dependencies or special issues
(CONFIG_PANIC_ON_OOPS) with how we manage things, so as to demo how
everything would work. It's not really clear to me that there's a good way
to generate e.g. kernel_config.base, without a lot of painstaking work
(which I'm happy to do if we agree this is a good approach).
Signed-off-by: Tycho Andersen <tycho@docker.com>
This is just a direct import of the current kernel/ directory, with a
slight splitting up of the dockerfiles to build a kernel-source and kernel
image.
Signed-off-by: Tycho Andersen <tycho@docker.com>
Drop `hyperkit` from the `linuxkit run` invocation, thus causing the linuxkit
tool to pick the platform's default backend (which is qemu on my Linux system,
which works better than hyperkit in this environment).
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
Not sure when this arrived but it was stopping anything running.
Appears not to be in the older test `init` containers.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
We were using Debian but Alpine more consistent. Use nested build.
Currently extract the hash in a nasty way but this can be fixed later
when we switch over hashing method.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
This was missed when things were renamed.
The intention with this code was (apparently) to provide a (pseudo)unique
hostname in the case where something more specific was not provided (e.g. by
DHCP). Make this a little clearer by using '(none)' rather than 'linuxkit' as
the default, in the normal case this will be overwritten by something more
specific and if it isn't we will change it to something somewhat unique derived
from the MAC address (as before). nb: '(none)' is already used by Debian so I
think it is a safe choice as the sentinel value.
The use of both CONFIG_DEFAULT_HOSTNAME and the explicit /etc/hostname from
mkimage.sh is likely to be redundant in some cases, but neither seems to
completely cover all cases so keep both.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
With redis-os and README documentation moved elsewhere,
the demo directory only contains the etcd demo setup.
Rename it.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
We had serveral files with instructions, in particular for
networking, for macOS/Docker for Mac. Let's have just one place.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
The Kubernetes images have been migrated to Alpine Linux which
does not include bash by default.
Signed-off-by: Matt Bajor <matt@notevenremotelydorky.com>
Adds a logging daemon that collects logs in a ring buffer in a runc container.
The tools logwrite and logread can be used to read/write logs. The logging
daemon can be sent open file descriptors that will be read and included
in the logs.
Modifies init to start the daemon and use logwrite to capture logs from runc.
Signed-off-by: Magnus Skjegstad <magnus@skjegstad.com>
This commit adds the script qemu.sh that will be used in a
docker container (created with Dockerfile).
This script will crate qemu instance to lauch a Clear Container
base OS with a kernel generated with moby.
Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
For the time being I've decided to exploit shared mounts to make
`/opt/cni` and `/etc/cni` work as expected. We need these directories
to appear writable on the host, and allow Weave Net pod to bind-mount
out them in order to install plugin binaries, and allow for vanilla
CNI plugins to be also accessible to kubelet.
Signed-off-by: Ilya Dmitrichenko <errordeveloper@gmail.com>
Previously only `/var` was `rshared` but some people need to share
mounts in `/opt` etc so let us make everything `rshared` for now.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
The sha1 tag should be sufficient to uniquely identify the image
and the sha256 versions are just very long...
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
It's not very interesting to see the forwarded stdout/stderr messages as
they will end-up being printed anyway.
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
The priv container populate the `/mac` key on startup, that the calf can
then read.
Also add more fine-grained control over read/write delete capabilities attached
to the routes, e.g. the calf can read /mac but not write to it.
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
This is needed for cloud environments that want to get their metadata in
the onboot phase over the network.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
The infrakit plugin is not specific to Moby and should be able
to boot other Linux systems as long as a kernel image and
initial RAM disk are supplied. Reflect this in the property
passed to the plugin.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
Instead of mounting a new filesystem, revert to doing a `rw` bind.
However do not make `/` `rshared`, just `/var` as that is where we expect
filesystems to be mounted for persistence. Also only make the actual
container rootfs writeable, not the whole directory.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Based on kernel_config{,.debug} from commit
724561bf69
Enable Landlock and userland sandbox example:
* CONFIG_SECURITY_LANDLOCK=y
* CONFIG_SAMPLES=y
Signed-off-by: Mickaël Salaün <mic@digikod.net>
A few YAML files include the binfmt container, where it's not really
needed. Remove it to make the samples simpler.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
etcd works better with a persistent storage. So configure a
disk and add the formatting container to the image.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
As suggested by @shykes these are clearer
- onboot for things that are run at boot time to completion
- services for persistent services
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Also tweak the shell script a little and give the local and GCP
infrakit group different names.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
`nested runc` unfortunately needs a lot of caps/privileged. The removal of `readonly: true` is also a bit unfortunate.
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
This makes the configuration simpler but requires us to be able
to set IP addresses on instances.
This also, for simplicity, reduces the number of nodes to 3.
The script does not make assumption about specific IP addresses,
but does assume that the nodes have IP addresses such as:
a.b.c.200, a.b.c.201, and a.b.c.202.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
Make all the low-level init code synchronous to avoid weird blocks on `close`.
Also move the net and ctl file descriptor at the beginning of the fd space for
the calf.
The SDK also allow to spamn multiple exec calves, which will all have the same
fd map:
- 0: stdin = /dev/null
- 1: stdout = pipe to parent stdout
- 2: stderr = pipe to parent stderr
- 3: net = socketpair to parent "net" pipe
- 4: ctl = socketpair to parent "ctl" pipe
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
This forces us to bind mount /lib but will be replaced by calling the proper
bindings later on.
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
It is not necessary to bring up `eth0`, the program does it fine.
This means we can remove shell script, clean up build.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Instead of specifying the number of instances, provide a list
of IP addresses for instances. These are passed to the instance
plugin as LogicalID.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
In #1485 I was still using a local mobylinux/init containing #1436, even though
I had included the necessary files in the swarmd container.
Switch to the current standard init package and drop the unnecessary bind.
Also `git add .gitignore` which I forgot last time too.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
This is based on https://github.com/docker/swarmkit/pull/1965 which adds a
direct containerd executor to swarmkit. It is very much a work in progress.
With a suitable moby image (such as projects/swarmd/swarmd.yml) something like
this should work:
runc exec swarmd swarmctl service create --image docker.io/library/nginx:alpine --name nginx
runc exec swarmd swarmctl service ls
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
This just sets up the initial cluster via bootstrap.
It does *not* manage state changes correctly afterwards. If one
node crashes (get's killed) it InfraKit will start a new node,
but the new node does not join the cluster (and the old node
is not removed, either).
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
- Script to set up a DfM etcd for bootstrapping a cluser
- Custom/local etcd package for moby
derived from the official image with script to start etcd
- YAML file to create a etcd moby image
- README with current instructions
This has a bunch of stuff, including the discovery URL hard coded.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
In the riddler change I changed "command" in the yaml to "args"
but did not change the files. In fact we basically used the
default command everywhere so this did not actually break.
Remove the unnecessary "command" lines to simplify yaml.
Revert the command to args change for now as I think I prefer
command, but its easier to switch now. Need to think if the
entrypoint/command distinction matters before finalizing.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Generated largely from the specified config; small parts taken from `docker image inspect`,
such as the command line.
Renamed some of the yaml keys to match the OCI spec rather than Docker Compose as
we decided they are more readable, no more underscores.
Add some extra functionality
- tmpfs specification
- fully general mount specification
- no new privileges can be specified now
For nostalgic reasons, using engine-api to talk to the docker cli as
we only need an old API version, and it is nice and easy to vendor...
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Use a file (that we can drop easily into the rootfs) instead of passing the full
command on the CLI (as config.json needs to be edited otherwise).
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
The inotify bindings that we are using is a bit sensitive to init
conditions, and it seems to not like being run inside in a container.
See https://github.com/samoht/irmin-watcher/issues/10
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
For now, use jbuilder to build the calf as well, this gives us a bit
more control than the mirage tool. We will switch back to the mirage
tool later on if we want to use more fancy backends (such as KVM).
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
The protocol allows the client to send concurrent RPCs to the server.
The server replies by keeping the client ID, and the client keeps a
dispatch table of queries to route the retries. By doing things like
that, the server has a strong control over resource allocation, so
a bad client cannot exhaust all the server memory.
Also add some simple tests for the serialization + wire protocol which
are all passing. Proper concurrency/resource exhaustion usage will be
added later.
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
Previously, the control plane was using HTTP client/server, that various people
found way too complex to run in a privileged container (for very good reasons).
So switching to a simpler binary protocol, using c-like structures. Will
probably switch to an other serialization protocol later (eg. protobuf
or cap-n-proto).
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
The ebpf packages were somewhat neglected during the restructuring of the
the repository and currently do not build. They were also a little awkward
to use. So move them to ./projects for now until it matures.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
The new hiearchy is:
- pkg/{init,mirage-compile}: additional Moby packages
- src/sdk -> the begining of the MirageOS SDK for Moby
- src/dhcp-client -> the code for the MirageOS dhcp-client service
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
Today the SDK only contains helper code to create secure Moby services based on
MirageOS. Today the SDK only defines the architecture and the communication
pipes between the privileged service and the calf; the proper communication
API will be specified after we have a few more use-cases.
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
