We were pointing to the old mobylinux docker hub repo. Let's update the
kernel build to be the new style one.
Note that I didn't bump the kernel version or update the patches at all. We
should do this soon, but for the purposes of our probational channel PoC,
I'm leaving wireguard at the old version for now.
Signed-off-by: Tycho Andersen <tycho@docker.com>
-wireguard is redundant, and with a standardized name, we can merge patches
"automatically", for our probational channel.
Signed-off-by: Tycho Andersen <tycho@docker.com>
Rather than re-build the whole init, let's just include the wireguard tools
in a tools repo.
This also moves *most* of the stuff to new linuxkit infrastructure, instead
of the legacy mobylinux. And checks an item off the TODO list.
Signed-off-by: Tycho Andersen <tycho@docker.com>
- Update to packages using the Alpine 3.6 base image
- Remove config for packages which now supply it
- Update/add trust section
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
The latest version of the `moby` tool now requires that the output formats
be specified in the CLI not in the yaml file.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
The kernel configs themselves are stored as diffs of what we want vs. each
version's defconfig.
Thus, things like e.g. CONFIG_DEVKMEM drop out after it was made
non-default. The implication of this is (I hope) that as upstream adopts
security features, our delta can shrink (or more realistically, only
include the next-next gen features).
Signed-off-by: Tycho Andersen <tycho@docker.com>
This is the script I used with [1] to generate the config diffs and
separate out the arch specific bits. Included mostly just so people can
play around with it if they want to generate their own diffs.
[1]: https://github.com/ulfalizer/Kconfiglib
Signed-off-by: Tycho Andersen <tycho@docker.com>
In particular, let's start with a defconfig and edit it, rather than try to
generate the config entirely from our own diff.
Signed-off-by: Tycho Andersen <tycho@docker.com>
Works around https://github.com/moby/moby/issues/33176 and fixes#1807.
Updated al users of linuxkit/runc:2649198589ef0020d99f613adaeda45ce0093a38 to
this new build.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
Since containers are spawned by containerd (which is in the host PID namespace)
and not in the swarmd container's namespace.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
Note these are only the ones using the `containerd` based `init` as
the `runc` ones are still using an old one.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
...and add straw man implementations of kernel_config.base and
kernel_config.x86 as examples.
First, splitting the build: to avoid duplication, we split the build into
three parts: a "source" stage, a "config" stage, and a "build" stage. The
"source" stage allows us to use a cached image, so we don't have to
re-download the kernel source every time. The "config" step applies our
patches and generates (and checks) the kernel config. I've left this as a
separate step for now so that we can build just an image with a config in
it, without having to ^C the build. However there's no real reason it needs
to be a separate step, assuming that this kernel config design is
acceptable. The third step is the actual kernel build.
Then there is kernel config management: the bulk of it occurs in
makeconfig.sh, with the idea being that we can specify base, arch, and
version specific config options as necessary.
The config files themselves are lists of options (both positive and
negative). We include the negative options, because we want to explicitly
turn off things that are on in the default config (e.g. CONFIG_USELIB), and
it seems cleaner to do things this way then to have some sort of negative
options list.
The options files are sorted with the default behavior of the "sort"
command, which ignores comment lines, meaning that negative options and
positive options are inline with each other. I don't have a strong opinion
on whether or not to group all negative options, or whether this default
behavior makes sense, so I just left it.
Finally, obviously the .base and .x86 files are incomplete. I mostly
selected a few options with interesting dependencies or special issues
(CONFIG_PANIC_ON_OOPS) with how we manage things, so as to demo how
everything would work. It's not really clear to me that there's a good way
to generate e.g. kernel_config.base, without a lot of painstaking work
(which I'm happy to do if we agree this is a good approach).
Signed-off-by: Tycho Andersen <tycho@docker.com>
This is just a direct import of the current kernel/ directory, with a
slight splitting up of the dockerfiles to build a kernel-source and kernel
image.
Signed-off-by: Tycho Andersen <tycho@docker.com>
Drop `hyperkit` from the `linuxkit run` invocation, thus causing the linuxkit
tool to pick the platform's default backend (which is qemu on my Linux system,
which works better than hyperkit in this environment).
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
Not sure when this arrived but it was stopping anything running.
Appears not to be in the older test `init` containers.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
We were using Debian but Alpine more consistent. Use nested build.
Currently extract the hash in a nasty way but this can be fixed later
when we switch over hashing method.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
This was missed when things were renamed.
The intention with this code was (apparently) to provide a (pseudo)unique
hostname in the case where something more specific was not provided (e.g. by
DHCP). Make this a little clearer by using '(none)' rather than 'linuxkit' as
the default, in the normal case this will be overwritten by something more
specific and if it isn't we will change it to something somewhat unique derived
from the MAC address (as before). nb: '(none)' is already used by Debian so I
think it is a safe choice as the sentinel value.
The use of both CONFIG_DEFAULT_HOSTNAME and the explicit /etc/hostname from
mkimage.sh is likely to be redundant in some cases, but neither seems to
completely cover all cases so keep both.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
With redis-os and README documentation moved elsewhere,
the demo directory only contains the etcd demo setup.
Rename it.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
We had serveral files with instructions, in particular for
networking, for macOS/Docker for Mac. Let's have just one place.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
The Kubernetes images have been migrated to Alpine Linux which
does not include bash by default.
Signed-off-by: Matt Bajor <matt@notevenremotelydorky.com>
Adds a logging daemon that collects logs in a ring buffer in a runc container.
The tools logwrite and logread can be used to read/write logs. The logging
daemon can be sent open file descriptors that will be read and included
in the logs.
Modifies init to start the daemon and use logwrite to capture logs from runc.
Signed-off-by: Magnus Skjegstad <magnus@skjegstad.com>
This commit adds the script qemu.sh that will be used in a
docker container (created with Dockerfile).
This script will crate qemu instance to lauch a Clear Container
base OS with a kernel generated with moby.
Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>
For the time being I've decided to exploit shared mounts to make
`/opt/cni` and `/etc/cni` work as expected. We need these directories
to appear writable on the host, and allow Weave Net pod to bind-mount
out them in order to install plugin binaries, and allow for vanilla
CNI plugins to be also accessible to kubelet.
Signed-off-by: Ilya Dmitrichenko <errordeveloper@gmail.com>
Previously only `/var` was `rshared` but some people need to share
mounts in `/opt` etc so let us make everything `rshared` for now.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
The sha1 tag should be sufficient to uniquely identify the image
and the sha256 versions are just very long...
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
It's not very interesting to see the forwarded stdout/stderr messages as
they will end-up being printed anyway.
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
The priv container populate the `/mac` key on startup, that the calf can
then read.
Also add more fine-grained control over read/write delete capabilities attached
to the routes, e.g. the calf can read /mac but not write to it.
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
This is needed for cloud environments that want to get their metadata in
the onboot phase over the network.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
The infrakit plugin is not specific to Moby and should be able
to boot other Linux systems as long as a kernel image and
initial RAM disk are supplied. Reflect this in the property
passed to the plugin.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
Instead of mounting a new filesystem, revert to doing a `rw` bind.
However do not make `/` `rshared`, just `/var` as that is where we expect
filesystems to be mounted for persistence. Also only make the actual
container rootfs writeable, not the whole directory.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Based on kernel_config{,.debug} from commit
724561bf69
Enable Landlock and userland sandbox example:
* CONFIG_SECURITY_LANDLOCK=y
* CONFIG_SAMPLES=y
Signed-off-by: Mickaël Salaün <mic@digikod.net>
A few YAML files include the binfmt container, where it's not really
needed. Remove it to make the samples simpler.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
etcd works better with a persistent storage. So configure a
disk and add the formatting container to the image.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
As suggested by @shykes these are clearer
- onboot for things that are run at boot time to completion
- services for persistent services
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Also tweak the shell script a little and give the local and GCP
infrakit group different names.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
`nested runc` unfortunately needs a lot of caps/privileged. The removal of `readonly: true` is also a bit unfortunate.
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
This makes the configuration simpler but requires us to be able
to set IP addresses on instances.
This also, for simplicity, reduces the number of nodes to 3.
The script does not make assumption about specific IP addresses,
but does assume that the nodes have IP addresses such as:
a.b.c.200, a.b.c.201, and a.b.c.202.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
Make all the low-level init code synchronous to avoid weird blocks on `close`.
Also move the net and ctl file descriptor at the beginning of the fd space for
the calf.
The SDK also allow to spamn multiple exec calves, which will all have the same
fd map:
- 0: stdin = /dev/null
- 1: stdout = pipe to parent stdout
- 2: stderr = pipe to parent stderr
- 3: net = socketpair to parent "net" pipe
- 4: ctl = socketpair to parent "ctl" pipe
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
This forces us to bind mount /lib but will be replaced by calling the proper
bindings later on.
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
It is not necessary to bring up `eth0`, the program does it fine.
This means we can remove shell script, clean up build.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Instead of specifying the number of instances, provide a list
of IP addresses for instances. These are passed to the instance
plugin as LogicalID.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
In #1485 I was still using a local mobylinux/init containing #1436, even though
I had included the necessary files in the swarmd container.
Switch to the current standard init package and drop the unnecessary bind.
Also `git add .gitignore` which I forgot last time too.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
This is based on https://github.com/docker/swarmkit/pull/1965 which adds a
direct containerd executor to swarmkit. It is very much a work in progress.
With a suitable moby image (such as projects/swarmd/swarmd.yml) something like
this should work:
runc exec swarmd swarmctl service create --image docker.io/library/nginx:alpine --name nginx
runc exec swarmd swarmctl service ls
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
This just sets up the initial cluster via bootstrap.
It does *not* manage state changes correctly afterwards. If one
node crashes (get's killed) it InfraKit will start a new node,
but the new node does not join the cluster (and the old node
is not removed, either).
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
- Script to set up a DfM etcd for bootstrapping a cluser
- Custom/local etcd package for moby
derived from the official image with script to start etcd
- YAML file to create a etcd moby image
- README with current instructions
This has a bunch of stuff, including the discovery URL hard coded.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
In the riddler change I changed "command" in the yaml to "args"
but did not change the files. In fact we basically used the
default command everywhere so this did not actually break.
Remove the unnecessary "command" lines to simplify yaml.
Revert the command to args change for now as I think I prefer
command, but its easier to switch now. Need to think if the
entrypoint/command distinction matters before finalizing.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Generated largely from the specified config; small parts taken from `docker image inspect`,
such as the command line.
Renamed some of the yaml keys to match the OCI spec rather than Docker Compose as
we decided they are more readable, no more underscores.
Add some extra functionality
- tmpfs specification
- fully general mount specification
- no new privileges can be specified now
For nostalgic reasons, using engine-api to talk to the docker cli as
we only need an old API version, and it is nice and easy to vendor...
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Use a file (that we can drop easily into the rootfs) instead of passing the full
command on the CLI (as config.json needs to be edited otherwise).
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
The inotify bindings that we are using is a bit sensitive to init
conditions, and it seems to not like being run inside in a container.
See https://github.com/samoht/irmin-watcher/issues/10
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
For now, use jbuilder to build the calf as well, this gives us a bit
more control than the mirage tool. We will switch back to the mirage
tool later on if we want to use more fancy backends (such as KVM).
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
The protocol allows the client to send concurrent RPCs to the server.
The server replies by keeping the client ID, and the client keeps a
dispatch table of queries to route the retries. By doing things like
that, the server has a strong control over resource allocation, so
a bad client cannot exhaust all the server memory.
Also add some simple tests for the serialization + wire protocol which
are all passing. Proper concurrency/resource exhaustion usage will be
added later.
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
Previously, the control plane was using HTTP client/server, that various people
found way too complex to run in a privileged container (for very good reasons).
So switching to a simpler binary protocol, using c-like structures. Will
probably switch to an other serialization protocol later (eg. protobuf
or cap-n-proto).
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
The ebpf packages were somewhat neglected during the restructuring of the
the repository and currently do not build. They were also a little awkward
to use. So move them to ./projects for now until it matures.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
The new hiearchy is:
- pkg/{init,mirage-compile}: additional Moby packages
- src/sdk -> the begining of the MirageOS SDK for Moby
- src/dhcp-client -> the code for the MirageOS dhcp-client service
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
Today the SDK only contains helper code to create secure Moby services based on
MirageOS. Today the SDK only defines the architecture and the communication
pipes between the privileged service and the calf; the proper communication
API will be specified after we have a few more use-cases.
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
The aim of the split kernel is to introduce a level of intra-kernel
protection into the kernel so that, amongst other things, it can
offer lifetime guarantees over kernel code and data integrity.
These patches only wire in the kernel build from a 4.11-rc3 snapshot.
The userspace tools will follow shortly. Instructions came via
https://github.com/linux-okernel/linux-okernel (linux-okernel branch)
and via @edwards-n and @t-koulouris.
The build can be done via `cd projects/okernel && make`.
Signed-off-by: Anil Madhavapeddy <anil@docker.com>
Split the bits which can be re-used in other services (e.g. init dance
and the server-side of the control path). `main.ml` now only contains what
is specific to the DHCP logic (+ the /caf directory).
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
Plus a few more minor improvements:
- compile with jbuilder.
- start working on the control path.
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
docker-compose and other utilities use the .yml extension.
For consistency rename all .yaml to .yml
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
