Commit Graph

976 Commits

Author SHA1 Message Date
Rolf Neugebauer
df5c858d38 demo: Use IP addresses as Logical IDs in etcd infrakit config
Instead of specifying the number of instances, provide a list
of IP addresses for instances. These are passed to the instance
plugin as LogicalID.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-05 18:01:52 +01:00
Rolf Neugebauer
dd58bd75eb Merge pull request #1495 from rneugeba/demo-up
Update demo project
2017-04-05 11:12:28 +01:00
Rolf Neugebauer
a131ee823b demo: Update YAML files to include DHCP
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-05 11:08:48 +01:00
Rolf Neugebauer
1cb8dfc222 demo: Remove infrakit example.
This is superseded by the etcd infrakit setup now in place.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-05 11:07:39 +01:00
Ian Campbell
f01d77930c swarmd: Switch to DHCPD container
Follows on from #1316, copies stanza from moby.yml but also picks up update
from #1493.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-04-05 10:39:04 +01:00
Justin Cormack
a97db7b406 Merge pull request #1487 from rneugeba/ik
Update InfraKit
2017-04-04 17:22:44 +01:00
Theo Koulouris
f6c85a9980 build file fixes for okernel
Signed-off-by: Theo Koulouris <theo.koulouris@hpe.com>
2017-04-04 16:44:26 +01:00
Rolf Neugebauer
411a8b2f94 demo: Update etcd README to new InfraKit version
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-04 16:27:15 +01:00
Ian Campbell
8859a7c0a6 swarmd: use standard init
In #1485 I was still using a local mobylinux/init containing #1436, even though
I had included the necessary files in the swarmd container.

Switch to the current standard init package and drop the unnecessary bind.

Also `git add .gitignore` which I forgot last time too.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-04-04 16:23:25 +01:00
Justin Cormack
08e62e0ebd Merge pull request #1485 from ijc25/swarmd
Initial swarmkit package
2017-04-04 16:09:34 +01:00
Ian Campbell
6a0bd7d035 Initial swarmkit package
This is based on https://github.com/docker/swarmkit/pull/1965 which adds a
direct containerd executor to swarmkit. It is very much a work in progress.

With a suitable moby image (such as projects/swarmd/swarmd.yml) something like
this should work:

runc exec swarmd swarmctl service create --image docker.io/library/nginx:alpine --name nginx
runc exec swarmd swarmctl service ls

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-04-04 16:00:03 +01:00
Rolf Neugebauer
fc84079db3 demo: Initial etcd cluster setup with InfraKit
This just sets up the initial cluster via bootstrap.

It does *not* manage state changes correctly afterwards. If one
node crashes (get's killed) it InfraKit will start a new node,
but the new node does not join the cluster (and the old node
is not removed, either).

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-04 15:37:41 +01:00
Rolf Neugebauer
4b3422f76c demo: WIP for a local etcd setup
- Script to set up a DfM etcd for bootstrapping a cluser
- Custom/local etcd package for moby
  derived from the official image with script to start etcd
- YAML file to create a etcd moby image
- README with current instructions

This has a bunch of stuff, including the discovery URL hard coded.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-04 13:46:19 +01:00
Rolf Neugebauer
218ec6d3d5 demo: Dummy infrakit demo using nginx
This will change to something more distributed...

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-04 13:45:32 +01:00
Rolf Neugebauer
5122afffb1 demo: Add simplest YAML for introducing moby
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-04 13:45:03 +01:00
Justin Cormack
01fba0f3f5 Merge pull request #1479 from justincormack/command
Revert Command->Args but remove from yaml where not needed
2017-04-04 10:49:32 +01:00
Justin Cormack
57c75741e9 Revert Command->Args but remove from yaml where not needed
In the riddler change I changed "command" in the yaml to "args"
but did not change the files. In fact we basically used the
default command everywhere so this did not actually break.

Remove the unnecessary "command" lines to simplify yaml.

Revert the command to args change for now as I think I prefer
command, but its easier to switch now. Need to think if the
entrypoint/command distinction matters before finalizing.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-04 09:51:17 +01:00
Thomas Gazagnaire
2dbcf99030 miragesdk: do not force the creation of pipes when linking with the SDK
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-04 10:06:59 +02:00
Thomas Gazagnaire
4944652d37 miragesdk: improve dev README
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-04 10:05:22 +02:00
Justin Cormack
065af9707c Replace riddler with code that constructs config.json directly
Generated largely from the specified config; small parts taken from `docker image inspect`,
such as the command line.

Renamed some of the yaml keys to match the OCI spec rather than Docker Compose as
we decided they are more readable, no more underscores.

Add some extra functionality
- tmpfs specification
- fully general mount specification
- no new privileges can be specified now

For nostalgic reasons, using engine-api to talk to the docker cli as
we only need an old API version, and it is nice and easy to vendor...

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-03 23:28:55 +01:00
Thomas Gazagnaire
76b33742ac Merge pull request #1448 from samoht/i-got-an-ip
miragesdk: put the DHCP lease info into the host filesystem
2017-04-02 18:32:51 +02:00
Rolf Neugebauer
63c89507e7 docs: Move ebpf.md to projects
This was missed when moving the ebpf stuff into a project.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-02 13:09:29 +01:00
Thomas Gazagnaire
e3939e03c0 miragesdk: fix the build after the switch from mirage tool to jbuilder
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-02 13:17:26 +02:00
Thomas Gazagnaire
e976a6c4aa miragesdk: ship Git the init image (to debug)
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-02 13:17:26 +02:00
Thomas Gazagnaire
1495dabccb miragesdk: allow the command run by the privileged container to be overwritten
Use a file (that we can drop easily into the rootfs) instead of passing the full
command on the CLI (as config.json needs to be edited otherwise).

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-02 13:17:25 +02:00
Thomas Gazagnaire
8d3cea1980 miragesdk: do not use inotify in the privileged container
The inotify bindings that we are using is a bit sensitive to init
conditions, and it seems to not like being run inside in a container.

See https://github.com/samoht/irmin-watcher/issues/10

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-02 13:17:25 +02:00
Thomas Gazagnaire
dfb078825b miragesdk: simplify the build of the calf binary
For now, use jbuilder to build the calf as well, this gives us a bit
more control than the mirage tool. We will switch back to the mirage
tool later on if we want to use more fancy backends (such as KVM).

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-02 13:17:25 +02:00
Thomas Gazagnaire
6500becfea miragesdk: do not die brutally when the calf terminates
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 19:57:36 +02:00
Thomas Gazagnaire
0f0b721b19 miragesdk: better computation of the image tag
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 19:57:03 +02:00
Thomas Gazagnaire
5223c08d05 miragesdk: fix the compilation of the calf
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 19:22:41 +02:00
Thomas Gazagnaire
725a6e8070 miragesdk: update the dhcp-client example with latest image
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 18:36:30 +02:00
Thomas Gazagnaire
d4db6c65a1 miragesdk: fix compilation of the dhcp-client image
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 18:35:52 +02:00
Thomas Gazagnaire
faa9e1811d miragesdk: implement a simple binary RPCs for the control plane
The protocol allows the client to send concurrent RPCs to the server.
The server replies by keeping the client ID, and the client keeps a
dispatch table of queries to route the retries. By doing things like
that, the server has a strong control over resource allocation, so
a bad client cannot exhaust all the server memory.

Also add some simple tests for the serialization + wire protocol which
are all passing. Proper concurrency/resource exhaustion usage will be
added later.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 17:22:16 +02:00
Thomas Gazagnaire
a61171e87e miragesdk: add tests for control plane messages
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 17:22:16 +02:00
Thomas Gazagnaire
3520ee928b Merge pull request #1413 from samoht/tests
Add tests for the MirageSDK and start the new control plane protocol
2017-03-30 13:48:13 +02:00
Thomas Gazagnaire
c582aafe45 miragesdk: add tests for stdout/stderr pipes
And the tests pass!

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 13:40:57 +02:00
Thomas Gazagnaire
2b48442fee miragesdk: update .gitignore
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 13:40:57 +02:00
Thomas Gazagnaire
9db898ceaf miragesdk: enable all warnings when compiling the projects
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 13:40:57 +02:00
Thomas Gazagnaire
b5a3d4b2aa miragesdk: rework the control plane protocol
Previously, the control plane was using HTTP client/server, that various people
found way too complex to run in a privileged container (for very good reasons).

So switching to a simpler binary protocol, using c-like structures. Will
probably switch to an other serialization protocol later (eg. protobuf
or cap-n-proto).

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 13:40:57 +02:00
Rolf Neugebauer
5592b58549 projects: Move non-working ebpf bits under projects
The ebpf packages were somewhat neglected during the restructuring of the
the repository and currently do not build. They were also a little awkward
to use. So move them to ./projects for now until it matures.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-29 17:36:25 +01:00
Rolf Neugebauer
2c49714ef9 projects: Move AWS bits to projects until they are usable again
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-29 16:28:21 +01:00
Thomas Gazagnaire
df71c0f299 miragesdk: add mimimal tests for the SDK
Very minimal so far, but the plan is to add much more of them.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-29 15:41:23 +02:00
Thomas Gazagnaire
c06454273b miragesdk: add a dev-clean target
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-29 14:28:19 +02:00
Thomas Gazagnaire
ec988eb93b miragesdk: fix dev target
Paths have changed since 47b9f08b16

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-29 14:27:38 +02:00
Thomas Gazagnaire
7d08325c7a miragesdk: move files around
The new hiearchy is:
- pkg/{init,mirage-compile}: additional Moby packages
- src/sdk -> the begining of the MirageOS SDK for Moby
- src/dhcp-client -> the code for the MirageOS dhcp-client service

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-29 13:39:20 +02:00
Thomas Gazagnaire
d008e859c6 miragesdk: first cut of a MirageSDK
Today the SDK only contains helper code to create secure Moby services based on
MirageOS. Today the SDK only defines the architecture and the communication
pipes between the privileged service and the calf; the proper communication
API will be specified after we have a few more use-cases.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-29 12:34:48 +02:00
Thomas Gazagnaire
13d110e2c7 miragesdk: minor cleanups
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-29 12:34:41 +02:00
Anil Madhavapeddy
df33c8a4d3 Merge pull request #1407 from avsm/miragesdk-readme
miragesdk: more tweaks to README markdown
2017-03-28 22:20:18 +01:00
Justin Cormack
e85b432f19 Merge pull request #1373 from avsm/okernel
kernel: add build config for the experimental "split kernel"
2017-03-28 23:11:12 +02:00
Anil Madhavapeddy
5c9d5315e5 miragesdk: more tweaks to README markdown
Signed-off-by: Anil Madhavapeddy <anil@docker.com>
2017-03-28 22:07:15 +01:00
Anil Madhavapeddy
3de5c415ef projects: remove double word typo in README.md
Signed-off-by: Anil Madhavapeddy <anil@docker.com>
2017-03-28 19:13:17 +01:00
Anil Madhavapeddy
ab72dbb0e4 miragesdk: add a README explaining the background of the project
Also link it to the roadmap and why-dhcp docs, and include review
comments from @yomimono

Signed-off-by: Anil Madhavapeddy <anil@docker.com>
2017-03-28 19:13:17 +01:00
Anil Madhavapeddy
c3d4049180 okernel: Make README more Markdown friendly
Signed-off-by: Anil Madhavapeddy <anil@docker.com>
2017-03-28 14:32:06 +01:00
Anil Madhavapeddy
bb49cab9c2 kernel: add build config for the experimental "split kernel"
The aim of the split kernel is to introduce a level of intra-kernel
protection into the kernel so that, amongst other things, it can
offer lifetime guarantees over kernel code and data integrity.

These patches only wire in the kernel build from a 4.11-rc3 snapshot.
The userspace tools will follow shortly.  Instructions came via
https://github.com/linux-okernel/linux-okernel (linux-okernel branch)
and via @edwards-n and @t-koulouris.

The build can be done via `cd projects/okernel && make`.

Signed-off-by: Anil Madhavapeddy <anil@docker.com>
2017-03-28 14:32:06 +01:00
Thomas Gazagnaire
56085a3e6c miragesdk: re-org source code
Split the bits which can be re-used in other services (e.g. init dance
and the server-side of the control path). `main.ml` now only contains what
is specific to the DHCP logic (+ the /caf directory).

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-28 14:42:02 +02:00
Thomas Gazagnaire
7fa21377b5 miragesdk: update TODO list
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-28 11:11:17 +02:00
Thomas Gazagnaire
1bee082c6c miragesdk: use a custom BPF filter to allow the calf to get a DHCP lease
Plus a few more minor improvements:

- compile with jbuilder.
- start working on the control path.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-28 11:09:55 +02:00
Thomas Gazagnaire
b0f758a20d miragesdk: add strace in the init image
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-28 11:09:50 +02:00
Thomas Gazagnaire
18635ed2e1 miragesdk: remove dhccpd in the init container
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-28 11:09:50 +02:00
Thomas Gazagnaire
ebdf371421 miragesdk: fork pkg/init to experiment with removal of dhcpcd
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-28 11:09:50 +02:00
Mindy Preston
46e650ede4 more documentation on why & what we're doing with dhcp
Signed-off-by: Mindy Preston <mindy.preston@docker.com>
2017-03-27 11:55:51 -05:00
Rolf Neugebauer
e9fbe43b34 Rename .yaml to .yml
docker-compose and other utilities use the .yml extension.
For consistency rename all .yaml to .yml

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-27 09:53:26 +02:00
Riyaz Faizullabhoy
60816eaa48 wireguard: more details in roadmap
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-03-22 14:18:42 -07:00
Justin Cormack
7c4768bb54 Merge pull request #1353 from tych0/selinux
projects: add selinux project
2017-03-21 18:43:18 +00:00
Tycho Andersen
7c873da17b selinux: include libselinux-utils as well
This adds things like getenforce and setenforce.

Signed-off-by: Tycho Andersen <tycho@docker.com>
2017-03-21 12:11:00 -06:00
Tycho Andersen
ae423e428b selinux: add readme
Signed-off-by: Tycho Andersen <tycho@docker.com>
2017-03-21 12:08:14 -06:00
Tycho Andersen
a8bab0bddb projects: add selinux project
Signed-off-by: Tycho Andersen <tycho@docker.com>
2017-03-21 11:42:11 -06:00
Riyaz Faizullabhoy
cc7d80be13 wireguard: include Dockerfile and kernel debug config for build
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-03-21 09:59:40 -07:00
Thomas Gazagnaire
ad357c21b8 mirageSDK: rename README.md into roadmap.md
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-20 15:00:47 +01:00
Justin Cormack
569652fd36 Initial overview of the okernel project
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-19 14:22:53 +00:00
Justin Cormack
9854394dda fix typo security->secure
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-19 13:50:21 +00:00
Justin Cormack
ecea479443 Add Summary page for projects
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-19 11:56:42 +00:00
Justin Cormack
f4bf27e044 Merge pull request #1322 from samoht/mirage
[RFC] Very early draft of using a MirageOS unikernel as a system container
2017-03-17 21:56:56 +00:00
Thomas Gazagnaire
30be4647ad Restructure the mirage/dhcp container into the new project structure
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-17 17:43:20 +01:00
Riyaz Faizullabhoy
4caa2d24b4 project: KSPP
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-03-17 16:39:46 +00:00
Riyaz Faizullabhoy
8b8a544ce8 Keep WireGuard configuration local to projects subdir
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-03-17 15:49:22 +00:00