Commit Graph

2523 Commits

Author SHA1 Message Date
Rolf Neugebauer
4ebcd6ff61 Merge pull request #1432 from rneugeba/patchup
Update kernels to 4.4.58/4.9.19/4.10.7 plus add VMBus patches
2017-03-31 13:30:18 +01:00
Thomas Gazagnaire
1bdcaadd24 Merge pull request #1434 from samoht/ctl
miragesdk: implement a simple binary RPCs for the control plane
2017-03-31 12:16:45 +02:00
Thomas Gazagnaire
b3ecefe72a miragesdk: do not die brutally when the calf terminates
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 19:57:36 +02:00
Thomas Gazagnaire
bf2f7dafab miragesdk: better computation of the image tag
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 19:57:03 +02:00
Thomas Gazagnaire
279b157ec7 miragesdk: fix the compilation of the calf
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 19:22:41 +02:00
Rolf Neugebauer
927d2610b0 Merge pull request #1435 from ijc25/init-binary-dependencies
pkg/init: ruinc and containerd binaries depend on Makefile
2017-03-30 17:48:28 +01:00
Thomas Gazagnaire
94b932d67f miragesdk: update the dhcp-client example with latest image
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 18:36:30 +02:00
Thomas Gazagnaire
bc6e0a50b0 miragesdk: fix compilation of the dhcp-client image
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 18:35:52 +02:00
Ian Campbell
cce041f686 pkg/init: ruinc and containerd binaries depend on Makefile
Since Makefile contains the hashes we need a dependency otherwise changing the
hashes does not necessarily result in the binaries getting updated.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-03-30 17:20:59 +01:00
Thomas Gazagnaire
fe2accb8c8 miragesdk: implement a simple binary RPCs for the control plane
The protocol allows the client to send concurrent RPCs to the server.
The server replies by keeping the client ID, and the client keeps a
dispatch table of queries to route the retries. By doing things like
that, the server has a strong control over resource allocation, so
a bad client cannot exhaust all the server memory.

Also add some simple tests for the serialization + wire protocol which
are all passing. Proper concurrency/resource exhaustion usage will be
added later.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 17:22:16 +02:00
Thomas Gazagnaire
6488ba59ff miragesdk: add tests for control plane messages
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 17:22:16 +02:00
Justin Cormack
e95f0f1e0b Merge pull request #1415 from tych0/fake-hardlink-support
init: fake supporting hard links slightly differently
2017-03-30 14:05:02 +02:00
Thomas Gazagnaire
5622f43b3c Merge pull request #1413 from samoht/tests
Add tests for the MirageSDK and start the new control plane protocol
2017-03-30 13:48:13 +02:00
Thomas Gazagnaire
469472b48d miragesdk: add tests for stdout/stderr pipes
And the tests pass!

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 13:40:57 +02:00
Thomas Gazagnaire
60a6936e53 miragesdk: update .gitignore
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 13:40:57 +02:00
Thomas Gazagnaire
5dd3d36954 miragesdk: enable all warnings when compiling the projects
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 13:40:57 +02:00
Thomas Gazagnaire
fb2d6b6252 miragesdk: rework the control plane protocol
Previously, the control plane was using HTTP client/server, that various people
found way too complex to run in a privileged container (for very good reasons).

So switching to a simpler binary protocol, using c-like structures. Will
probably switch to an other serialization protocol later (eg. protobuf
or cap-n-proto).

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-30 13:40:57 +02:00
Rolf Neugebauer
73de69d7b4 kernel: Add patches to fix VMBus channel ID re-use
On some hv_sock workloads which quickly open/close many connections
occasionally, channel IDs would get re-used while still having work
pending. This can cause a kernel crash on a NULL pointer exception.

The three patches added to the 4.9.x and 4.10.x kernels fixes
these bugs. The patches are being prepared to be upstreamed, but for
now we cherry-picked them from the developers tree.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-30 10:54:05 +01:00
Rolf Neugebauer
5fea53bb09 docs: Update kernel patches doc
It's useful for cherry-picked patches to know which tree
they got patched from. Include a "Origin:" line.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-30 10:46:49 +01:00
Rolf Neugebauer
e9026919cf kernel: Update to 4.4.58/4.9.19/4.10.7
The 4.9.19/4.10.7 kernels include the fix for the VMBus
memory leak, so we don't need to carry these patches anymore.

The patches against 4.9.x/4.10.x now also all have added a
"Origin" line pointing to the git tree the patches were cherry
picked from.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-30 10:29:50 +01:00
Tycho Andersen
01a1caa3a7 init: fake supporting hard links slightly differently
Instead, make a hard link a symlink. This isn't much better, but it allows
some cases (e.g. installing GCC on moby via alpine) to work.

Signed-off-by: Tycho Andersen <tycho@docker.com>
2017-03-29 22:15:11 -06:00
Justin Cormack
94b54c0ec8 Merge pull request #1412 from rneugeba/ebpf
projects: Move non-working ebpf bits under projects
2017-03-29 19:21:59 +02:00
Rolf Neugebauer
209f317fd1 projects: Move non-working ebpf bits under projects
The ebpf packages were somewhat neglected during the restructuring of the
the repository and currently do not build. They were also a little awkward
to use. So move them to ./projects for now until it matures.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-29 17:36:25 +01:00
Rolf Neugebauer
ce90e6290c Merge pull request #1411 from rneugeba/cleanup
more repository cleanup
2017-03-29 16:52:32 +01:00
Justin Cormack
fd8b217efc Merge pull request #1410 from rneugeba/aws
projects: Move AWS bits to projects until they are usable again
2017-03-29 17:40:33 +02:00
Rolf Neugebauer
534fd2940f base: Remove mksh
It does not seem to be used by anything here anymore.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-29 16:38:57 +01:00
Rolf Neugebauer
a319b1b1ef base: Move guestfs to tools.
Guestfs is not really a base package and is only used
byt other images in the ./tools directory. Move it there.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-29 16:37:59 +01:00
Rolf Neugebauer
ee536158a2 projects: Move AWS bits to projects until they are usable again
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-29 16:28:21 +01:00
Justin Cormack
c1780a92e1 Merge pull request #1404 from ijc25/enable-printk-time
Enable CONFIG_PRINTK_TIME
2017-03-29 17:05:44 +02:00
Justin Cormack
9b2d560515 Merge pull request #1409 from rneugeba/build
Add logging to "moby build"
2017-03-29 16:50:13 +02:00
Thomas Gazagnaire
2ae5b63d02 miragesdk: add mimimal tests for the SDK
Very minimal so far, but the plan is to add much more of them.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-29 15:41:23 +02:00
Rolf Neugebauer
1e25622cac moby: Create custom formatter for Info() log events
This formatter strips the prefix from Info() events to
make the default output of "moby build" more readable.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-29 13:43:27 +01:00
Thomas Gazagnaire
94c437cf87 miragesdk: add a dev-clean target
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-29 14:28:19 +02:00
Thomas Gazagnaire
5b3f80a104 miragesdk: fix dev target
Paths have changed since 47b9f08b16

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-29 14:27:38 +02:00
Justin Cormack
7d89d68b61 Merge pull request #1408 from samoht/reorg
mirageSDK: re-organise the file hierarchy
2017-03-29 14:05:19 +02:00
Thomas Gazagnaire
47b9f08b16 miragesdk: move files around
The new hiearchy is:
- pkg/{init,mirage-compile}: additional Moby packages
- src/sdk -> the begining of the MirageOS SDK for Moby
- src/dhcp-client -> the code for the MirageOS dhcp-client service

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-29 13:39:20 +02:00
Thomas Gazagnaire
2497528665 miragesdk: first cut of a MirageSDK
Today the SDK only contains helper code to create secure Moby services based on
MirageOS. Today the SDK only defines the architecture and the communication
pipes between the privileged service and the calf; the proper communication
API will be specified after we have a few more use-cases.

Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-29 12:34:48 +02:00
Thomas Gazagnaire
e18e7ebeb8 miragesdk: minor cleanups
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-03-29 12:34:41 +02:00
Rolf Neugebauer
1367bd2cde moby: Add informational output to build
This adds log.Info() to the main steps of the "moby build"
process. By default the Info() output is shown to the user
so it provides some idea of progress and what is happening.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-29 10:53:18 +01:00
Rolf Neugebauer
ddb1b410a6 moby: Add debug logging to "moby build" code
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-29 10:25:28 +01:00
Rolf Neugebauer
da3dc16642 moby: Add verbose and quiet flags to moby commandline
These set the log level to Debug and Error. The default
log level is set to Info.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-29 10:25:28 +01:00
Rolf Neugebauer
616b69559d moby: Switch to use logrus instead of default logger
logrus offers better control over log-levels.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-29 10:25:28 +01:00
Anil Madhavapeddy
16b09454a5 Merge pull request #1407 from avsm/miragesdk-readme
miragesdk: more tweaks to README markdown
2017-03-28 22:20:18 +01:00
Justin Cormack
38327f2cb2 Merge pull request #1373 from avsm/okernel
kernel: add build config for the experimental "split kernel"
2017-03-28 23:11:12 +02:00
Justin Cormack
5c1b1e6eb6 Merge pull request #1397 from justincormack/more-docs
Architecture overview
2017-03-28 23:10:12 +02:00
Anil Madhavapeddy
738683af14 miragesdk: more tweaks to README markdown
Signed-off-by: Anil Madhavapeddy <anil@docker.com>
2017-03-28 22:07:15 +01:00
Anil Madhavapeddy
6b017c5524 Merge pull request #1406 from avsm/miragesdk-readme
Add projects/miragesdk README
2017-03-28 22:05:27 +01:00
Anil Madhavapeddy
b198765bc7 projects: remove double word typo in README.md
Signed-off-by: Anil Madhavapeddy <anil@docker.com>
2017-03-28 19:13:17 +01:00
Anil Madhavapeddy
018d82f70a miragesdk: add a README explaining the background of the project
Also link it to the roadmap and why-dhcp docs, and include review
comments from @yomimono

Signed-off-by: Anil Madhavapeddy <anil@docker.com>
2017-03-28 19:13:17 +01:00
Ian Campbell
656a598559 Enable CONFIG_PRINTK_TIME
This adds a timestamp to the start of the kernel command line. Like this (from
a random system I have lying around, line truncated by me):
[    0.000000] tsc: Detected 2665.038 MHz processor
[    0.000021] Calibrating delay loop (skipped), value calculated using tim...
[    0.000023] pid_max: default: 32768 minimum: 301
[    0.000041] ACPI: Core revision 20160831
[    0.003782] ACPI: 2 ACPI AML tables successfully acquired and loaded

This would be handy in relation to #1403.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-03-28 14:54:13 +01:00