Commit Graph

258 Commits

Author SHA1 Message Date
Avi Deitcher
8de1d9bf2c Mount /var/log to getty and ssh again
Signed-off-by: Avi Deitcher <avi@deitcher.net>
2017-07-21 19:11:22 +03:00
Jason A. Donenfeld
ec7dadcddd wireguard: add to getty and sshd
People really want to play around with this, so adding them here makes
it possible. Just as iproute2 is part of these, so should
wireguard-tools.

Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2017-07-20 23:50:13 +02:00
Ian Campbell
c48acd5d9d sshd+getty: Add apk to these containers
Since these are the user login containers, having the ability to add packages
is useful (e.g. I quite often find I want strace).

Doing this requires that we not share `/var` with the login containers since we
want the apk database therein. Previously it was thought that the containers
might need some parts of `/var` for `ctr` to work (e.g. `/var/lib/containerd`)
but this is not the case now (if it ever was) based on my testing.

Fixes #2206.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-07-20 15:08:27 +01:00
Justin Cormack
5545f3085a Merge pull request #2258 from ijc/containerd-v1.0.0-alpha1
Containerd v1.0.0 alpha1
2017-07-20 15:07:24 +01:00
Ian Campbell
b3db1a887c Update to containerd v1.0.0-alpha1
This is actually containerd#1141 rebased onto v1.0.0-alpha1.

The `dist` command has been integreated into `ctr` and so is removed, including
from the getty and sshd bind mounts and the test which uses it is updated..

There is no change to the version of runc vendored by containerd, so this is
unchanged.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-07-20 14:10:51 +01:00
Rolf Neugebauer
c1eacd07b7 Merge pull request #2256 from djs55/secrets
metadata: avoid printing large chunks of configuration json to the console
2017-07-20 13:17:50 +01:00
David Scott
5bf6526380 metadata: avoid printing large chunks of configuration json to the console
If the configuration .json has contents like:

{
  "etc": {
    "ssl": {
      "certs": {
        "ca-certificates.crt": {
          "perm": "0644",
          "content": "large amount of certificate text"
        }
      }
    }
  },
...
}

then we print a warning because the node "ssl" has no "perm".
Previously the warning would include the contents of "ssl", which
would be large (and in theory could include secret information).

This patch modifies the warning print to only print the key and
not the value.

Without this patch, I see on the console:

2017/07/20 10:03:04 CDROM: Probe succeeded
2017/07/20 10:03:04 No permission provided ssl:map[certs:map[ca-certificates.crt:map[perm:0644 content:large amount of certificate text]]]
 - 000-metadata

With this patch, I see on the console:

2017/07/20 09:54:18 CDROM: Probe succeeded
2017/07/20 09:54:18 No permission provided ssl
 - 000-metadata

Signed-off-by: David Scott <dave.scott@docker.com>
2017-07-20 11:10:59 +01:00
Rolf Neugebauer
c28a472ba2 pkg/init-lcow: Use main opengcs repository again
My patches have been merged.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-07-20 11:02:05 +01:00
Rolf Neugebauer
35e274c7d8 Merge pull request #2245 from rn/lcow-init
Additional LCOW fixes
2017-07-19 15:27:01 +01:00
Rolf Neugebauer
7eb81ba38c pkg/init-lcow: Include fix for rootfs permissions
Update the git commit to point at a branch with the joint
networking and rootfs permission fixes at:
https://github.com/rn/opengcs/tree/net+rootfs

This is includes the patches from PRs:
https://github.com/Microsoft/opengcs/pull/63
https://github.com/Microsoft/opengcs/pull/70

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-07-19 12:20:52 +01:00
Rolf Neugebauer
9ba7fee7e6 pkg/init-lcow: Use our own init
The sample init script from opengcs is quite basic (and doesn't
set up several mounts and symlinks). Use the relevant portion
from rc.init from the LinuxKit init package instead.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-07-19 10:40:49 +01:00
Ian Campbell
56a8d2468a kubernetes: modernise image cache builds
This updates the build of the two image caches to use the `pkg/package.mk`
infrastructure, albeit in a slightly (ok, very) atypical way.

In order to share the bulk of the build code (including the `Dockerfile` and
the `Makefile` machinery to download the images) we arrange for the necessary
bits to be copied at build time into distinct subdirectories and for the
`pkg/package.mk` to be aware of this possibility.

Since pkg/package.mk is only set up to build a single package we use a single
`image-cache/Makefile` to drive the whole process and recurse into
`Makefile.pkg` to build individual packages.

One particular subtlety is that the package hash is based on the `image-cache`
directory (which is in `git`) rather than the generated subdirectories (which
are not in `git`). Since all the generators (and their inputs) are in the
`image-cache` directory this is what we want. This means that the two images
are given the same tag, but this is deliberate and desirable.

The generated directories are completely temporary to avoid picking up stale
versions of images when versions are updated. Images are hardlinked into place.

The images are moved to the linuxkitprojects org. Using a dev tag for now, will
update once everything is in place.

Also use "tag" rather than "build" where appropriate in the Makefile.

There is no point in the .dockerignore now, but add a .gitignore.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-07-19 09:51:32 +01:00
Rolf Neugebauer
c073b0ebd6 Merge pull request #2242 from rn/lcow-net
Fix networking in LCOW
2017-07-18 20:02:47 +01:00
Avi Deitcher
0f725c366e Merge pull request #2241 from deitch/fix-getty-background
flag in init and background only if not in init
2017-07-18 21:39:53 +03:00
Justin Cormack
09e9357499 Merge pull request #2235 from justincormack/use-library-docker
Use the upstream dind package to run docker
2017-07-18 17:27:45 +01:00
Rolf Neugebauer
4ff7916e10 pkg/init-lcow: Temporarily use a branch of my dev tree
The branch has fixes to the networking setup (see
https://github.com/Microsoft/opengcs/issues/56 and
https://github.com/Microsoft/opengcs/pull/63)

Also remove runc from the package as the hard-coded runc
path has been removed (see
https://github.com/Microsoft/opengcs/pull/58)

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-07-18 15:44:20 +01:00
Avi Deitcher
af0331d382 flag in init and background only if not in init
Signed-off-by: Avi Deitcher <avi@deitcher.net>
2017-07-18 16:37:05 +03:00
Justin Cormack
c6aa7b9053 Add onshutdown support to runc
This will run any containers in `/containers/onshutdown` on a clean
shutdown.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-18 11:54:58 +01:00
Justin Cormack
fe8f979da5 Add support for shutdown scripts
These can be added by other packages if they need to do something on
clean shutdown.

Crash only software can ignore this.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-18 11:54:58 +01:00
Justin Cormack
a4650b242f Use the upstream dind package to run docker
It is pretty close to our docker package, if we adjust the command
that is run to avoid the actual dind startup script. We can't use
the normal docker image as it does not have mkfs and so on.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-18 11:15:38 +01:00
Ian Campbell
fc20fb8a2c pkg/init: Do not run inits in the background
We want them to run in sequence. For example we want mounts to be done (done by
`pkg/runc/etc/init.d/010-onboot`) before we start services (done by
`pkg/containerd/etc/init.d/020-containerd`). This was most likely introduced by
28b4245b12 ("Move onboot startup script to runc package").

None of the initscripts in pkg/* block, but some in projects (selinux and
logging, not updated here) do.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-07-15 12:39:12 +01:00
Justin Cormack
07469ac60b Merge pull request #2207 from justincormack/getty-x
remove -x from getty script
2017-07-15 12:38:52 +01:00
Justin Cormack
98ed378dbd Add a securetty file
Do not copy host securetty file - this one should be comprehensive
or bind mount host one in yourself.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-14 18:34:52 +01:00
Riyaz Faizullabhoy
469ba103db Merge pull request #2209 from ijc/expurgate-set--x
pkg/mount: Remove set -x
2017-07-14 09:58:36 -07:00
Riyaz Faizullabhoy
66e328e38d Merge pull request #2211 from justincormack/runc-static-pie
Make runc a static pie executable
2017-07-14 09:56:56 -07:00
Justin Cormack
36df256177 Merge pull request #2202 from rn/lcow2
Add experimental support for LCOW
2017-07-14 16:58:01 +01:00
Justin Cormack
53d5ea42c4 Build runc as static pie
This was waiting on the runc update which added ability to add extra flags.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-14 15:38:32 +01:00
Ian Campbell
77ec9bced3 pkg/mount: Remove set -x
Noisy.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-07-14 15:21:01 +01:00
Rolf Neugebauer
cbc8d4a21c pkg: Add LCOW init package
This package build the init filesystem for LCOW (Linux Containers
on Windows) based on the instructions from:
https://github.com/Microsoft/opengcs.git

We also pull in a udhcpd config script from a specific version of
busybox which was the tip of master at the time this was added.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-07-14 15:02:42 +01:00
Justin Cormack
ecaa7f9c68 Remove the -x from the rungetty script
Its annoyingly verbose!

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-14 13:53:45 +01:00
Ian Campbell
538c8b5a7b containerd: Add a service command to cleanup stale containers on boot
This may occur if /var/lib/container happens to be on persistent storage.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-07-14 13:50:46 +01:00
Justin Cormack
28b4245b12 Move onboot startup script to runc package
As this does not use containerd at all, this means you can run very
minimal setups with just `runc` if you use no services, for example
most of our tests do not actually use services, or if you have other
similar very minimal use cases.

Move ulimit setup to `init` which makes more sense.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-14 11:04:35 +01:00
Ian Campbell
902e6182b0 Temporarily use https://github.com/containerd/containerd/pull/1141
We expect this (or something very similar) to be merged soon, it fixes #2131 so
moving ahead now.

The new alpine mirror is linuxkit/alpine:6832775a7e861ee2d7842e157688ece52d007142

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-07-13 10:42:55 +01:00
Ian Campbell
c5634c4650 Define CONTAINERD_REPO alongside CONTAINERD_COMMIT in tools/alpine
This will make it simpler to temporarily in PRs which are not yet merged (but
are expected to be soon).

Tools alpine is not rebuilt here since we are going to do just that in the next
commit.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-07-13 10:38:27 +01:00
Ian Campbell
a2d3be0e6f Update runc and containerd
pull in newer containerd v1.0.0-alpha0 via updated alpine base, update runc to
429a5387123625040bacfbb60d96b1cbd02293ab which is vendored by that version of
containerd (and also update alpine base for runc)

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-07-13 10:37:15 +01:00
Justin Cormack
f98a56fc2c Merge pull request #2179 from deitch/read-only-securetty
Use existing securetty
2017-07-11 23:19:43 +01:00
Justin Cormack
9b4bce1e40 Add forcetag to package build
Use this if you really need to rebuild a package, eg if you change `pakcage.mk`

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-11 16:35:58 +01:00
Justin Cormack
ed38adbc90 Add labels for repo and commit to packages
- use image spec metadata https://github.com/opencontainers/image-spec/blob/master/annotations.md
- omit commit if dirty

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-11 14:33:59 +01:00
Avi Deitcher
5d18cba75f Use existing securetty
Signed-off-by: Avi Deitcher <avi@deitcher.net>
2017-07-11 14:43:26 +03:00
Justin Cormack
298f4aab32 Consistently don't use quotes around image names
These are not needed, but we are inconsistent. Been waiting for a
quiet moment to fix this since I noticed while doing a presentation...

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-09 17:47:30 +01:00
Rolf Neugebauer
c3bc803445 Merge pull request #2164 from rn/meta
Allow metadata package to over write existing config files
2017-07-07 19:48:09 +01:00
Rolf Neugebauer
4e19cf363a Merge pull request #2119 from djs55/host-timesync-daemon
Add Docker for Mac host time sync daemon
2017-07-07 19:17:20 +01:00
Rolf Neugebauer
f021bb8aaa pkg: In the metadata package, don't error if a dir exists
Use os.Mkdirall() instead of os.Mkdir() as it does not
error if the path already exists.

This enables specifying a default config file with the image
and then over writing it with metadata.

While at it, also update to the latest alpine base image.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-07-07 18:29:02 +01:00
David Scott
d24d0bd559 Add pkg/host-timesync-daemon
Some hypervisors (e.g. hyperkit / xhyve) don't provide a good way to keep
the VM's clock in sync with the Host's clock. NTP will usually keep the
clocks together, but after a the host or VM is suspended and resumed the
clocks can be suddenly too far apart for NTP to work properly.

This simple daemon listens on an AF_VSOCK port and resynchronises the VM
clock from the virtualised hardware clock.

This is a Go conversion of original C code written by
Magnus Skjegstad <magnus@skjegstad.com>

Signed-off-by: David Scott <dave.scott@docker.com>
2017-07-07 11:35:36 +01:00
Ian Campbell
aee080f621 pkg: Reduce the number of times variables containing $(shell...) are invoked.
For DIRTY and RELEASED this means simply performing a static assignment with
`:=`. For HASH it is a little more complex since it is (and we want/need it to
be) a conditional assigment. However it is only used for defining TAG, so make
that a static assignment.

This reduces the number of times the complex DIRTY shell command in particular
is evaluated.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-07-06 16:33:08 +01:00
Ian Campbell
4fba9d8545 pkg: Update content trust handling.
Firstly add option to disable content trust, for the use of e.g. projects which
are pushing to the linuxkitprojects org (which has no trust setup) rather than
the main linuxkit org.

Secondly, when trust _is_ enabled then enable it globally, in particular it is
now active for the `docker build` and hence containers referenced in
Dockerfiles via "FROM" will be checked.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-07-06 16:20:34 +01:00
Magnus Skjegstad
ba8be57eef pkg: add acpid
Adds a new service for running `/sbin/acpid` that comes with busybox.
The VM will shut down if the power button is pressed.

Signed-off-by: Magnus Skjegstad <magnus@skjegstad.com>
2017-07-05 22:37:46 +02:00
Justin Cormack
f43e1cbf27 Move containerd init file from init package to containerd package
Otherwise it gets out of sync, and it makes little sense it being in
`init` now anyway.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-03 13:42:00 +01:00
Ian Campbell
fe918f8a31 pkg: Run git update-index --refresh before git diff-index.
Otherwise files which have an updated timestamp but no actual changes are
marked as changes because `git diff-index` only uses the `lstat` result and not
the actual file contents. Running `git update-index --refresh` updates the
cache.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-07-03 10:27:31 +01:00
Ian Campbell
7fd155b38d pkg: Do not mark user supplied HASH as dirty
If a user passes a HASH e.g. HASH=dev then assume they know what they are doing
and don't need dirty tracking.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-07-03 10:27:31 +01:00