Commit Graph

2734 Commits

Author SHA1 Message Date
Justin Cormack
faad84431e Add a doc for topics to discuss at the Summit at Dockercon
(as per containerd)

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-02 12:43:52 +01:00
Thomas Gazagnaire
e3939e03c0 miragesdk: fix the build after the switch from mirage tool to jbuilder
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-02 13:17:26 +02:00
Thomas Gazagnaire
e976a6c4aa miragesdk: ship Git the init image (to debug)
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-02 13:17:26 +02:00
Thomas Gazagnaire
1495dabccb miragesdk: allow the command run by the privileged container to be overwritten
Use a file (that we can drop easily into the rootfs) instead of passing the full
command on the CLI (as config.json needs to be edited otherwise).

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-02 13:17:25 +02:00
Thomas Gazagnaire
8d3cea1980 miragesdk: do not use inotify in the privileged container
The inotify bindings that we are using is a bit sensitive to init
conditions, and it seems to not like being run inside in a container.

See https://github.com/samoht/irmin-watcher/issues/10

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-02 13:17:25 +02:00
Thomas Gazagnaire
dfb078825b miragesdk: simplify the build of the calf binary
For now, use jbuilder to build the calf as well, this gives us a bit
more control than the mirage tool. We will switch back to the mirage
tool later on if we want to use more fancy backends (such as KVM).

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-02 13:17:25 +02:00
Justin Cormack
8513d3a995 Merge pull request #1445 from tych0/security-non-events
docs: add a security-events.md
2017-04-02 12:17:20 +01:00
Tycho Andersen
12624b60bf add some headings, fix some spelling mistakes
Signed-off-by: Tycho Andersen <tycho@docker.com>
2017-04-01 11:32:41 -06:00
Justin Cormack
4fab33d34b Merge pull request #1452 from rneugeba/efiiso
Fix EFI ISO generation
2017-04-01 18:03:56 +02:00
Rolf Neugebauer
616b073c13 cli: Use fixed mkimage-iso-efi image
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-01 16:25:02 +01:00
Rolf Neugebauer
be16cf67b4 tools: Fix make-efi script
mkfs.vfat prints output (even without the -v option) which ends up
in front of the ISO, which is cat'ed to stdout. This made the generated
ISO unbootable. Redirect stdout of mkfs.vfat to /dev/null.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-01 16:22:35 +01:00
Rolf Neugebauer
b5e07659bb Merge pull request #1449 from rneugeba/hk-cli-fix
cli: Add option to specify hyperkit to use
2017-03-31 23:05:02 +01:00
Riyaz Faizullabhoy
98a2bc7f1d Merge pull request #1446 from amirmc/patch-1
Fix links in weekly report
2017-03-31 11:50:20 -07:00
Rolf Neugebauer
db7b8a3b71 cli: Add option to specify hyperkit to use
Some users seem to have Docker for Mac/hyperkit in a non-standard
path. Allow them to specify the path to the hyperkit executable.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-31 19:21:02 +01:00
Rolf Neugebauer
91c66d5633 Merge pull request #1444 from ijc25/kernel-rtl8139
kernel Enable support for RTL8139 NIC
2017-03-31 17:24:06 +01:00
Amir Chaudhry
64eadee5d6 Fix links in weekly report
Signed-off-by: Amir Chaudhry <amir@docker.com>
2017-03-31 16:50:36 +01:00
Tycho Andersen
ad7ddba0dc docs: add a security-events.md
In the same vein as [1], let's start talking about security events. I
suppose we want to talk about security events as well as non-events,
though, to give a little discussion about post moretem. But we can rename
this to security-non-events if we want.

[1]: https://github.com/docker/docker.github.io/blob/master/engine/security/non-events.md

Signed-off-by: Tycho Andersen <tycho@docker.com>
2017-03-31 09:40:38 -06:00
Ian Campbell
80b3f45973 kernel Enable support for RTL8139 NIC
This is the default NIC provided by virt-install, I think it is also pretty
common on other virtualisation platforms since both the drivers and the
emulation are pretty widespread (IIRC Xen HVM guests used to get this by
default, and may still do).

Personally I'd probably try and remember to switch to virtio (or even e1000) in
preference, but that's one more thing to do.

Bump the image number.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-03-31 15:54:27 +01:00
Ian Campbell
5e293f8479 kernel: Rerun make oldconfig on default kernel config
... and accept the defaults. Doing so enables some hw monitoring on Intel
(which enables some I2C thing) and explicitly disables a few Mellanox options.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-03-31 15:54:22 +01:00
Rolf Neugebauer
c77d932321 Merge pull request #1438 from rneugeba/moby
Support multiple "run" backends for the CLI
2017-03-31 13:30:51 +01:00
Rolf Neugebauer
d731eabb71 Merge pull request #1432 from rneugeba/patchup
Update kernels to 4.4.58/4.9.19/4.10.7 plus add VMBus patches
2017-03-31 13:30:18 +01:00
Thomas Gazagnaire
07d55713a7 Merge pull request #1434 from samoht/ctl
miragesdk: implement a simple binary RPCs for the control plane
2017-03-31 12:16:45 +02:00
Riyaz Faizullabhoy
52fa33d6d1 first pass at security docs
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-03-30 17:22:58 -07:00
Rolf Neugebauer
4423d3e5de cli: Add support for multiple backends for "moby run"
- Move HyperKit code into a separate file. It should be compilable
  on all supported OSes now.
- Add a (optional) subcommand to "moby run" to select a backend
  i.e., "moby run hyperkit [options] [prefix]"
- On macOS the default is "hyperkit" so that:
  "moby run [options] [prefix]"
  just works
- Add enough command line parsing to make it easy to add new
  backends to the run command
Update help messages.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-30 18:59:07 +01:00
Thomas Gazagnaire
6500becfea miragesdk: do not die brutally when the calf terminates
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 19:57:36 +02:00
Thomas Gazagnaire
0f0b721b19 miragesdk: better computation of the image tag
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 19:57:03 +02:00
Thomas Gazagnaire
5223c08d05 miragesdk: fix the compilation of the calf
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 19:22:41 +02:00
Rolf Neugebauer
671aec28a1 Merge pull request #1435 from ijc25/init-binary-dependencies
pkg/init: ruinc and containerd binaries depend on Makefile
2017-03-30 17:48:28 +01:00
Thomas Gazagnaire
725a6e8070 miragesdk: update the dhcp-client example with latest image
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 18:36:30 +02:00
Thomas Gazagnaire
d4db6c65a1 miragesdk: fix compilation of the dhcp-client image
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 18:35:52 +02:00
Ian Campbell
31e0a848e1 pkg/init: ruinc and containerd binaries depend on Makefile
Since Makefile contains the hashes we need a dependency otherwise changing the
hashes does not necessarily result in the binaries getting updated.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-03-30 17:20:59 +01:00
Rolf Neugebauer
024b1d6d98 vendor: Update hyperkit go bindings
The updated bindings primarily allow compilation on non-darwin
operating systems.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-30 17:00:07 +01:00
Thomas Gazagnaire
faa9e1811d miragesdk: implement a simple binary RPCs for the control plane
The protocol allows the client to send concurrent RPCs to the server.
The server replies by keeping the client ID, and the client keeps a
dispatch table of queries to route the retries. By doing things like
that, the server has a strong control over resource allocation, so
a bad client cannot exhaust all the server memory.

Also add some simple tests for the serialization + wire protocol which
are all passing. Proper concurrency/resource exhaustion usage will be
added later.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 17:22:16 +02:00
Thomas Gazagnaire
a61171e87e miragesdk: add tests for control plane messages
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 17:22:16 +02:00
Rolf Neugebauer
13e3d88bdd cli: Move "run" flag processing into the run implmentation
While at it also fix up the HyperKit run help message.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-30 16:00:06 +01:00
Rolf Neugebauer
bc5e4c8a14 cli: Move "build" flag processing into build.go
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-30 15:59:52 +01:00
Justin Cormack
c7826ef8c2 Merge pull request #1415 from tych0/fake-hardlink-support
init: fake supporting hard links slightly differently
2017-03-30 14:05:02 +02:00
Thomas Gazagnaire
3520ee928b Merge pull request #1413 from samoht/tests
Add tests for the MirageSDK and start the new control plane protocol
2017-03-30 13:48:13 +02:00
Thomas Gazagnaire
c582aafe45 miragesdk: add tests for stdout/stderr pipes
And the tests pass!

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 13:40:57 +02:00
Thomas Gazagnaire
2b48442fee miragesdk: update .gitignore
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 13:40:57 +02:00
Thomas Gazagnaire
9db898ceaf miragesdk: enable all warnings when compiling the projects
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 13:40:57 +02:00
Thomas Gazagnaire
b5a3d4b2aa miragesdk: rework the control plane protocol
Previously, the control plane was using HTTP client/server, that various people
found way too complex to run in a privileged container (for very good reasons).

So switching to a simpler binary protocol, using c-like structures. Will
probably switch to an other serialization protocol later (eg. protobuf
or cap-n-proto).

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 13:40:57 +02:00
Rolf Neugebauer
87c4092b26 kernel: Add patches to fix VMBus channel ID re-use
On some hv_sock workloads which quickly open/close many connections
occasionally, channel IDs would get re-used while still having work
pending. This can cause a kernel crash on a NULL pointer exception.

The three patches added to the 4.9.x and 4.10.x kernels fixes
these bugs. The patches are being prepared to be upstreamed, but for
now we cherry-picked them from the developers tree.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-30 10:54:05 +01:00
Rolf Neugebauer
eaef1ca8fb docs: Update kernel patches doc
It's useful for cherry-picked patches to know which tree
they got patched from. Include a "Origin:" line.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-30 10:46:49 +01:00
Rolf Neugebauer
730e11757c kernel: Update to 4.4.58/4.9.19/4.10.7
The 4.9.19/4.10.7 kernels include the fix for the VMBus
memory leak, so we don't need to carry these patches anymore.

The patches against 4.9.x/4.10.x now also all have added a
"Origin" line pointing to the git tree the patches were cherry
picked from.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-30 10:29:50 +01:00
Tycho Andersen
3c5040b0f6 init: fake supporting hard links slightly differently
Instead, make a hard link a symlink. This isn't much better, but it allows
some cases (e.g. installing GCC on moby via alpine) to work.

Signed-off-by: Tycho Andersen <tycho@docker.com>
2017-03-29 22:15:11 -06:00
Justin Cormack
317e48b9dd Merge pull request #1412 from rneugeba/ebpf
projects: Move non-working ebpf bits under projects
2017-03-29 19:21:59 +02:00
Rolf Neugebauer
5592b58549 projects: Move non-working ebpf bits under projects
The ebpf packages were somewhat neglected during the restructuring of the
the repository and currently do not build. They were also a little awkward
to use. So move them to ./projects for now until it matures.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-29 17:36:25 +01:00
Rolf Neugebauer
b4a3e2e0da Merge pull request #1411 from rneugeba/cleanup
more repository cleanup
2017-03-29 16:52:32 +01:00
Justin Cormack
8d934f6914 Merge pull request #1410 from rneugeba/aws
projects: Move AWS bits to projects until they are usable again
2017-03-29 17:40:33 +02:00