Commit Graph

4010 Commits

Author SHA1 Message Date
Riyaz Faizullabhoy
ec7f1df1b4 Merge pull request #2112 from djs55/trim-after-delete2
RFC: add "trim after image delete" to the Docker for Mac blueprint
2017-06-27 14:12:00 -07:00
Riyaz Faizullabhoy
fe5b683d8f Merge pull request #2114 from riyazdf/sig-agenda-memorizer
sig-security: agenda for 2017-07-05
2017-06-27 13:28:57 -07:00
Rolf Neugebauer
b4d50ec405 Merge pull request #2113 from rn/pkg
Improve package and kernel builds
2017-06-27 21:19:07 +01:00
David Scott
4a6db6eb51 blueprints: add trim-after-delete to the docker-for-mac
This starts the trim-after-delete daemon which calls
`fstrim /var/lib/docker` after image deletes are observed.

Signed-off-by: David Scott <dave.scott@docker.com>
2017-06-27 20:32:57 +01:00
Rolf Neugebauer
093dae22d5 Update YAML files
- Use the new style kernel tags with the full kernel version
- Update packages with new alpine base and new/simplified Makefiles.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-06-27 19:56:08 +01:00
Riyaz Faizullabhoy
ce335f71b9 sig-security: agenda for 2017-07-05
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-06-27 11:52:04 -07:00
Rolf Neugebauer
2fb65e8f1e doc: Remove explicit hashes from documentation in ./pkg
People typically update hashes in YAML files with
'git grep ... | sed ...' this will change the README.md files
in the ./pkg directory and thus change the git tree hash.

Remove the use of explicit hashes from those files to avoid
the obvious chicken and egg problem.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-06-27 18:01:48 +01:00
David Scott
38787b1ddd Add pkg/trim-after-delete
This adds a helper which monitors the Docker event stream and, when it
receives an image delete event, queues a call to `fstrim /var/lib/docker`
to trigger a space reclamation. Previously we would rely on a cron job
running every 15 minutes.

Signed-off-by: David Scott <dave.scott@docker.com>
2017-06-27 17:55:42 +01:00
Rolf Neugebauer
56d94318d9 pkg: Use curl instead of ADD to fetch rng-tools
There seems to be a change in docker 17.06-rc5 on how remote
URLs and tar archives work with ADD. Use curl instead.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-06-27 17:43:58 +01:00
Rolf Neugebauer
cab27698f4 pkg: Update all packages to the new alpine base
In a subsequent commit, all YAML files will be updated with
new package hashes since all packages needed rebuild due to
build system changes in commit adae27b8d1 ("Simplify
Makefiles for Packages"). So, we might as well bring all
packages up to the latest alpine base package.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-06-27 17:43:57 +01:00
Rolf Neugebauer
73e5ea8f01 tools: Update the Alpine base image to the latest versions
A subsequent commit will update all packages, so we might as well
update all packages to the latest base.

The new hash is: linuxkit/alpine:c608d404c1cb36cce0c7d9303e30b52c9d81ccf0

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-06-27 17:43:57 +01:00
Rolf Neugebauer
40ea10065c kernel: Tweak kernel build
- Combine 'sign' and 'push' targets like it is done for
  package builds.
- Append '-dirty' to the tag if the repository is dirty.
- Don't push to hub if the repository is dirty.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-06-27 17:43:57 +01:00
Rolf Neugebauer
da24542d78 pkg: Improve package build
- Only build if the hash has changed. Previosuly
  we'd unconditionally build and not push of the tag existed.
- Add '.m' to the hash if the repository is dirty.
  This allows developers to build locally without committing
  but makes it clear that the hash is from a modified repository.
- Don't push to hub if the repository is dirty
- If the current commit has a tag, also push this to hub.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-06-27 17:43:57 +01:00
Justin Cormack
e6515cec4b Merge pull request #2104 from ijc/test-ctr-over-getty
test: ensure ctr works in getty container
2017-06-27 16:48:17 +01:00
Justin Cormack
6fb1ab285e Merge pull request #2110 from ijc/qemu-kvm-control
qemu: Add -enable-kvm to force (attempted) use (or not) of KVM
2017-06-27 14:47:48 +01:00
Ian Campbell
3809391545 test: ensure ctr works in getty container
This is based on examples/getty.yml modified to drop console=tty0 from the
command line since we will be capturing/logging only the serial so want
everything to go there.

Also updates the getty example to pickup the latest containerd and
ca-certificates as used in the top level linuxkit.yml.

Fix a typo in the containerd test too.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-06-27 12:29:29 +01:00
Ian Campbell
7151f1af52 qemu: Add -enable-kvm to force (attempted) use (or not) of KVM
The default is set by probing for /dev/kvm. Use of KVM can be forcibly disabled
with `-enable-kvm=false`. Note that passing `-enable-kvm=true` when `/dev/kvm` is
not present still boots but complains:

    Could not access KVM kernel module: No such file or directory
    failed to initialize KVM: No such file or directory
    Back to tcg accelerator.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-06-27 12:20:25 +01:00
Rolf Neugebauer
dc8d881d2c Merge pull request #2111 from rn/kup
kernel: Update to 4.11.7/4.9.34/4.4.74
2017-06-27 12:14:18 +01:00
Rolf Neugebauer
c12eafeeb2 kernel: Update to 4.11.7/4.9.34/4.4.74
In particular this contains 1be7107fbe18eed3e319 ("mm: larger stack
guard gap, between vmas") which is a fix for CVE-2017-1000364.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-06-27 10:39:23 +01:00
Rolf Neugebauer
1809b8d497 Merge pull request #2108 from avsm/week-25-dev-report
reports: add week 25 dev report
2017-06-27 10:09:18 +01:00
Riyaz Faizullabhoy
ac80cd2574 Merge pull request #2101 from furious-luke/master
Add a metadata provider for Vultr
2017-06-26 14:59:21 -07:00
Riyaz Faizullabhoy
d536abf68f Merge pull request #2107 from justincormack/get-started-moby
Slighly clearer getting started docs
2017-06-26 14:51:34 -07:00
Anil Madhavapeddy
3d45046bbe reports: add week 25 dev report
Signed-off-by: Anil Madhavapeddy <anil@docker.com>
2017-06-26 22:42:29 +01:00
Riyaz Faizullabhoy
206b6add4f Merge pull request #2105 from justincormack/authorsup
Update Authors
2017-06-26 10:29:32 -07:00
Justin Cormack
d573e37fec Slighly clearer getting started docs
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-06-26 17:40:34 +01:00
Justin Cormack
50785b4be7 Merge pull request #2103 from justincormack/make-cleaner
Makefile cleanups
2017-06-26 16:41:30 +01:00
Justin Cormack
014a08b15c Update Authors
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-06-26 13:54:04 +01:00
Justin Cormack
3f5c236de7 Add a target for the cross build tests and make parallel
Trying to speed up build a bit.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-06-26 13:39:47 +01:00
Justin Cormack
fb4c168af7 Merge pull request #2102 from ijc/getty-and-ssh-improvements
Getty and ssh improvements
2017-06-26 13:38:59 +01:00
Justin Cormack
50c3c38a18 Clean up Makefile
- clean up incorrect lines such as `tar xf tmp_linuxkit_bin.tar > $@`
- split out targets to make tarballs and to untar

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-06-26 13:29:37 +01:00
Ian Campbell
b6071df200 Update getty and sshd sha's in *.yml to current.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-06-26 11:52:41 +01:00
Ian Campbell
bbbbf3cb5d sshd: sync bind mounts with getty container
This adds all the binds from pkg/getty/Dockerfile to pkg/sshd/Dockerfile and
retains the existing /root/.ssh and /etc/resolv.conf binds.

With this `dist` and `ctr` function in a ssh session.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-06-26 11:52:41 +01:00
Ian Campbell
2864f467b9 getty,sshd: Include ca-certificates.
Otherwise:

    dist pull docker.io/library/redis:alpine
    dist: failed to do request: Head https://registry-1.docker.io/v2/library/redis/manifests/alpine: x509: failed to load system roots and no roots provided

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-06-26 11:52:41 +01:00
Ian Campbell
34fbe42615 getty,sshd: Mount host /tmp into containers.
ctr (which runs in the getty or ssh container) relies on sharing files in /tmp
with containerd (which runs in the host mount namespace). Specifically it
currently uses paths under /tmp/containerd for the stdio FIFOs of containers, resulting in:

    # ctr run -t docker.io/library/redis:alpine test
    ctr: rpc error: code = Unknown desc = runtime create failed: runc create failed: container with id exists: test

Currently it is not possible to specify a non-existent source directory for a
bind mount, so we cannot easily bind just /tmp/containerd. Sharing all of /tmp
doesn't sound like a terrible idea anyway.

Defering updating the sha in *.yml until after some further changes to these packages.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-06-26 11:52:41 +01:00
Ian Campbell
ea79748830 pkg/containerd: Add /etc/localtime set to UTC
containerd/ctr includes this in the default set of bind mounts for a container.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-06-26 11:52:30 +01:00
Ian Campbell
e880052d35 alpine: Add tzdata to mirror.
Moved qemu-img to keep list fully alphabetical.

Rebuild pushed as linuxkit/alpine:c96f56ef989ccb31fd115d25c1b90f05b992271d

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-06-26 11:51:27 +01:00
Luke Hodkinson
3dc23b96ac Add a metadata provider for Vultr
Vultr uses a very similar approach to AWS, including using the
same IP address for serving metadata. In fact, it seems
as though if AWS appears first in the list of providers, that
provider mistakenly believes to be running on AWS (hence the
insertion of `NewVultr` in between GCP and AWS. I don't believe
AWS servers will accidentally try to use the Vultr provider,
as it seems that the `/v1/` endpoint doesn't exist on AWS.

Signed-off-by: Luke Hodkinson <furious.luke@gmail.com>
2017-06-24 07:43:07 +10:00
Justin Cormack
8be67983ff Merge pull request #2100 from justincormack/no-efi-test
Disable Qemu EFI ISO test for now
2017-06-23 11:50:35 -07:00
Justin Cormack
c2bd284ccb Disable Qemu EFI ISO test for now
See #1872 this is very flaky and without a controlled known
working environment this is not helpful.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-06-23 09:46:45 -07:00
Justin Cormack
f077c7c7f7 Merge pull request #2095 from rn/kern-up
Kernel and kernel config update
2017-06-23 09:45:05 -07:00
Rolf Neugebauer
8d4dfb25e6 test: Load modules before testing for filesystems
The previous commit moved some filesystems to kernel modules.
modprobe them before checking if they are enabled.

Also update the YAML files for the tests.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-06-22 23:31:39 +01:00
Rolf Neugebauer
492b6c3fdb kernel: Move some less commonly used features to modules
Module loading on hotplug and boot seems to work now, so
move some less commonly used kernel features and drivers
out of the kernel into drivers. Specifically:
- Devices: All non-virtual network device drivers
- Networking: GRE, GENEVE, PPP, non-essential IPv6 protos,
  L2TP, MPLS_GSO, bonding, IPSec (XFRM), openvswitch,
  queueing/schedulers
- FS: SUNRPC, NFS, NFSD, LOCKD, NTFS
- Misc: ATA over Ethernet

Remove Nozomi serial driver. It doesn't seem to be used
on any of our platforms.

The config files were also run through 'make defconfig &&
make oldconfig' to update any missing options.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-06-22 19:32:13 +01:00
Rolf Neugebauer
4d8f8956b3 kernel: Update to 4.11.6/4.9.33/4.4.73
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-06-22 19:32:13 +01:00
Rolf Neugebauer
974bc56886 kernel: Remove elfutils-dev package
It clashes with libelf-dev but libelf-dev is sufficient
to compile the kernel. This also allows us to remove the
'|| true' from the 'apk add', catching errors with the
tools installation.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-06-22 19:32:12 +01:00
Riyaz Faizullabhoy
125469a4c5 Merge pull request #2093 from MagnusS/add-dfm-build-test
tests: add build test for Docker for Mac blueprint
2017-06-22 09:24:35 -07:00
Justin Cormack
6cfc0ad7b2 Merge pull request #2094 from samoht/typo
Fix typo in report
2017-06-22 09:20:57 -07:00
Thomas Gazagnaire
e1ab0e75a9 Replace Solar5 by Solo5
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-06-22 17:26:59 +02:00
Thomas Gazagnaire
83d615e2d1 Remove spaces in report
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-06-22 17:26:39 +02:00
Magnus Skjegstad
7bbc39374e tests: add build test for Docker for Mac blueprint
Adds a new test group "linuxkit.build.blueprints" and a test for
building blueprints/docker-for-mac.yml.

Signed-off-by: Magnus Skjegstad <magnus@skjegstad.com>
2017-06-22 16:30:31 +02:00
Rolf Neugebauer
d65bb86c37 Merge pull request #2080 from dave-tucker/pkg-mk
Simplify Makefiles for Packages
2017-06-22 13:35:43 +01:00