428 Commits

Author SHA1 Message Date
Ian Campbell
538c8b5a7b containerd: Add a service command to cleanup stale containers on boot
This may occur if /var/lib/container happens to be on persistent storage.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-07-14 13:50:46 +01:00
Justin Cormack
28b4245b12 Move onboot startup script to runc package
As this does not use containerd at all, this means you can run very
minimal setups with just `runc` if you use no services, for example
most of our tests do not actually use services, or if you have other
similar very minimal use cases.

Move ulimit setup to `init` which makes more sense.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-14 11:04:35 +01:00
Ian Campbell
902e6182b0 Temporarily use https://github.com/containerd/containerd/pull/1141
We expect this (or something very similar) to be merged soon, it fixes #2131 so
moving ahead now.

The new alpine mirror is linuxkit/alpine:6832775a7e861ee2d7842e157688ece52d007142

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-07-13 10:42:55 +01:00
Ian Campbell
c5634c4650 Define CONTAINERD_REPO alongside CONTAINERD_COMMIT in tools/alpine
This will make it simpler to temporarily in PRs which are not yet merged (but
are expected to be soon).

Tools alpine is not rebuilt here since we are going to do just that in the next
commit.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-07-13 10:38:27 +01:00
Ian Campbell
a2d3be0e6f Update runc and containerd
pull in newer containerd v1.0.0-alpha0 via updated alpine base, update runc to
429a5387123625040bacfbb60d96b1cbd02293ab which is vendored by that version of
containerd (and also update alpine base for runc)

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-07-13 10:37:15 +01:00
Justin Cormack
f98a56fc2c Merge pull request #2179 from deitch/read-only-securetty
Use existing securetty
2017-07-11 23:19:43 +01:00
Justin Cormack
9b4bce1e40 Add forcetag to package build
Use this if you really need to rebuild a package, eg if you change `pakcage.mk`

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-11 16:35:58 +01:00
Justin Cormack
ed38adbc90 Add labels for repo and commit to packages
- use image spec metadata https://github.com/opencontainers/image-spec/blob/master/annotations.md
- omit commit if dirty

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-11 14:33:59 +01:00
Avi Deitcher
5d18cba75f Use existing securetty
Signed-off-by: Avi Deitcher <avi@deitcher.net>
2017-07-11 14:43:26 +03:00
Justin Cormack
298f4aab32 Consistently don't use quotes around image names
These are not needed, but we are inconsistent. Been waiting for a
quiet moment to fix this since I noticed while doing a presentation...

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-09 17:47:30 +01:00
Rolf Neugebauer
c3bc803445 Merge pull request #2164 from rn/meta
Allow metadata package to over write existing config files
2017-07-07 19:48:09 +01:00
Rolf Neugebauer
4e19cf363a Merge pull request #2119 from djs55/host-timesync-daemon
Add Docker for Mac host time sync daemon
2017-07-07 19:17:20 +01:00
Rolf Neugebauer
f021bb8aaa pkg: In the metadata package, don't error if a dir exists
Use os.Mkdirall() instead of os.Mkdir() as it does not
error if the path already exists.

This enables specifying a default config file with the image
and then over writing it with metadata.

While at it, also update to the latest alpine base image.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-07-07 18:29:02 +01:00
David Scott
d24d0bd559 Add pkg/host-timesync-daemon
Some hypervisors (e.g. hyperkit / xhyve) don't provide a good way to keep
the VM's clock in sync with the Host's clock. NTP will usually keep the
clocks together, but after a the host or VM is suspended and resumed the
clocks can be suddenly too far apart for NTP to work properly.

This simple daemon listens on an AF_VSOCK port and resynchronises the VM
clock from the virtualised hardware clock.

This is a Go conversion of original C code written by
Magnus Skjegstad <magnus@skjegstad.com>

Signed-off-by: David Scott <dave.scott@docker.com>
2017-07-07 11:35:36 +01:00
Ian Campbell
aee080f621 pkg: Reduce the number of times variables containing $(shell...) are invoked.
For DIRTY and RELEASED this means simply performing a static assignment with
`:=`. For HASH it is a little more complex since it is (and we want/need it to
be) a conditional assigment. However it is only used for defining TAG, so make
that a static assignment.

This reduces the number of times the complex DIRTY shell command in particular
is evaluated.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-07-06 16:33:08 +01:00
Ian Campbell
4fba9d8545 pkg: Update content trust handling.
Firstly add option to disable content trust, for the use of e.g. projects which
are pushing to the linuxkitprojects org (which has no trust setup) rather than
the main linuxkit org.

Secondly, when trust _is_ enabled then enable it globally, in particular it is
now active for the `docker build` and hence containers referenced in
Dockerfiles via "FROM" will be checked.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-07-06 16:20:34 +01:00
Magnus Skjegstad
ba8be57eef pkg: add acpid
Adds a new service for running `/sbin/acpid` that comes with busybox.
The VM will shut down if the power button is pressed.

Signed-off-by: Magnus Skjegstad <magnus@skjegstad.com>
2017-07-05 22:37:46 +02:00
Justin Cormack
f43e1cbf27 Move containerd init file from init package to containerd package
Otherwise it gets out of sync, and it makes little sense it being in
`init` now anyway.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-03 13:42:00 +01:00
Ian Campbell
fe918f8a31 pkg: Run git update-index --refresh before git diff-index.
Otherwise files which have an updated timestamp but no actual changes are
marked as changes because `git diff-index` only uses the `lstat` result and not
the actual file contents. Running `git update-index --refresh` updates the
cache.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-07-03 10:27:31 +01:00
Ian Campbell
7fd155b38d pkg: Do not mark user supplied HASH as dirty
If a user passes a HASH e.g. HASH=dev then assume they know what they are doing
and don't need dirty tracking.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-07-03 10:27:31 +01:00
Ian Campbell
bdd7e7163b pkg: Allow override of commit to calculate hash for show-tag
Useful to answer the question of what the hash was at the point of some older
commit.

$ make --no-print-directory -C pkg/init/ show-tag
linuxkit/init:36c56f0664d49c5a6adc1120d1bf5ba6ac30b389
$ make --no-print-directory -C pkg/init/ show-tag HASH_COMMIT=4699f80ef73141ee87a77d03f93065e541fab382
linuxkit/init:17693d233dd009b2a3a8d23673cb85969e1dce80

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-07-03 10:27:31 +01:00
Vincent Demeester
a315e0b681 Fix DIRS find command to work on more system
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2017-07-03 10:27:31 +01:00
Ian Campbell
1ea64d33f0 pkg: use Jessie-era compatible git command for calculating $(HASH)
The current rune for $(HASH) returns nothing with the Jessie version of git
(2.1.4). Using `--full-tree` works corectly but requires an absolute path (else
it produces e.g. "fatal: ../init: '../init' is outside repository").

Both `ls-tree` and `diff-index` are happy with a full absolute path, which
simplifies things since we can use `$(CURDIR)` directly.

Tested with a dirty `pkg/init` on both Jessie (git 2.1.4) and Stretch (git
2.11.0) with the following command, which produces identical output in both
cases:

$ for pkg in init containerd ; do make -C pkg/$pkg --no-print-directory show-tag; ( cd pkg/$pkg && make show-tag ); done
linuxkit/init:36c56f0664d49c5a6adc1120d1bf5ba6ac30b389-dirty
linuxkit/init:36c56f0664d49c5a6adc1120d1bf5ba6ac30b389-dirty
linuxkit/containerd:1e3e8f207421de8deac8cedc26a138d6b1661a0d
linuxkit/containerd:1e3e8f207421de8deac8cedc26a138d6b1661a0d

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-07-03 10:27:31 +01:00
Ian Campbell
66f9cc134f pkg: Consolidate $(TAG) to be the full tag
Saves lots of repetitions of `$(ORG)/$(IMAGE):$(HASH)$(DIRTY)` throughout the
file.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-07-03 10:27:31 +01:00
Ian Campbell
4a3d5acb88 pkg: Only consider the package directory itself for dirty tracking.
Since this is what is used in the hash.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-07-03 10:27:31 +01:00
Ian Campbell
b6728a85e9 pkg: Simplify dirty handling
By setting `DIRTY` to either "-dirty" or "" directly we can simply use
`$(HASH)$(DIRTY)` and avoid make adding a space as it does with `+=`.

For the push check we now block pushing if `$(DIRTY)` is non-empty.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-07-03 10:27:31 +01:00
Ian Campbell
55f537631a pkg: add target to print the tag
$ make --no-print-directory -C pkg/init/ show-tag
linuxkit/init:36c56f0664d49c5a6adc1120d1bf5ba6ac30b389

Useful for scripting etc.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-07-03 10:27:31 +01:00
Rolf Neugebauer
a148a8aa45 pkg: Fix docker-ce package
As of commit b2439251dd ("blueprints: update docker-for-mac.yml
to latest vpnkit packages") the docker-ce package does not need
the vpnkit-expose-port binary anymore. It was also removed from
the linuxkit/vpnkit-forwarder forwarder package with commit
f48a2bfe12 ("pkg: update vpnkit-forwarder") causing the docker-ce
package build to fail.

This commit fixes this by removing the linuxkit/vpnkit-forwarder
package from the docker-ce package build.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-07-01 19:24:16 +01:00
Rolf Neugebauer
4c1a45957e pkg: Add NETWORK to audit and rngd packages
This fixes a regression introduced in:
5a225b9d ("Makefile: Fix bug where network was not disabled")

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-07-01 18:11:45 +01:00
Rolf Neugebauer
699a864302 pkg: Update to new Alpine base
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-07-01 18:11:45 +01:00
Magnus Skjegstad
f48a2bfe12 pkg: update vpnkit-forwarder
Update to latest commit and don't include the binaries that are now in
vpnkit-expose-port.

Signed-off-by: Magnus Skjegstad <magnus@skjegstad.com>
2017-06-30 19:26:47 +02:00
Magnus Skjegstad
6c83b98a40 pkg: add vpnkit-expose-port
This package contains vpnkit binaries needed inside the dockerd
container that can be either copied into the container with a multistage
build or used as an init-container and bind-mounted to the right
locations.

vpnkit-expose-port is a userland proxy that interacts with vpnkit and
the vpnkit-forwarder service to open ports on the host.

vpnkit-iptables-wrapper wraps iptables to launch
vpnkit-expose-port automatically when ports are opened to a swarm
service.

Signed-off-by: Magnus Skjegstad <magnus@skjegstad.com>
2017-06-30 19:26:02 +02:00
Dave Tucker
5a225b9dc5 Makefile: Fix bug where network was not disabled
This commit moves the include statement to the bottom of the file to
ensure that all variables are set before conditionals are evaluated.

I also changed the ifndef NETWORK to ifdef NETWORK as the former was
incorrect. We want `NET_OPTS="--network=none"` in cases where NETWORK is
not defined.

Fixes: #2134

Signed-off-by: Dave Tucker <dt@docker.com>
2017-06-30 11:55:08 +01:00
Rolf Neugebauer
736713dc8f Merge pull request #2120 from dave-tucker/getty
Make it more obvious that getty/sshd are namespaced
2017-06-29 23:30:17 +01:00
Justin Cormack
fed24c4110 Update to Docker 17.06.0-ce
Note that the location of the static binaries changed.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-06-29 15:27:34 +01:00
Dave Tucker
5fb025824f getty: Make namespacing more obvious
Warn the user in the MOTD
Add "(ns: getty)" or "(ns: sshd)" to the PS1
Use `agetty` and `-a root` to ensure we get a login shell when insecure

Signed-off-by: Dave Tucker <dt@docker.com>
2017-06-29 14:58:44 +01:00
Tycho Andersen
740b97cc90 auditd: update image hash
Riyaz has actually pushed this hash, so let's use it.

Signed-off-by: Tycho Andersen <tycho@docker.com>
2017-06-28 16:44:54 -06:00
Tycho Andersen
b95e28f466 auditd: add missing build.sh file
Whoops :)

Signed-off-by: Tycho Andersen <tycho@docker.com>
2017-06-28 16:25:46 -06:00
Tycho Andersen
44cbd38650 add an auditd container
Two things to note here: we unfortunately can't just exec audit, because
something needs to load the rules in beforehand.

Second, it also dies if it can't re-nice itself, so we have to give it
CAP_SYS_NICE as well as the audit caps.

I didn't add this to the default linuxkit.yml because the linuxkit/audit
repo doesn't exist yet, but we should probably (?) do that. Additionally,
we should provide the kernel parameter audit=1, otherwise according to
auditd's man pages, we can't audit some early tasks.

Closes #52

Signed-off-by: Tycho Andersen <tycho@docker.com>
2017-06-28 09:15:51 -06:00
Riyaz Faizullabhoy
ec7f1df1b4 Merge pull request #2112 from djs55/trim-after-delete2
RFC: add "trim after image delete" to the Docker for Mac blueprint
2017-06-27 14:12:00 -07:00
Rolf Neugebauer
093dae22d5 Update YAML files
- Use the new style kernel tags with the full kernel version
- Update packages with new alpine base and new/simplified Makefiles.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-06-27 19:56:08 +01:00
Rolf Neugebauer
2fb65e8f1e doc: Remove explicit hashes from documentation in ./pkg
People typically update hashes in YAML files with
'git grep ... | sed ...' this will change the README.md files
in the ./pkg directory and thus change the git tree hash.

Remove the use of explicit hashes from those files to avoid
the obvious chicken and egg problem.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-06-27 18:01:48 +01:00
David Scott
38787b1ddd Add pkg/trim-after-delete
This adds a helper which monitors the Docker event stream and, when it
receives an image delete event, queues a call to `fstrim /var/lib/docker`
to trigger a space reclamation. Previously we would rely on a cron job
running every 15 minutes.

Signed-off-by: David Scott <dave.scott@docker.com>
2017-06-27 17:55:42 +01:00
Rolf Neugebauer
56d94318d9 pkg: Use curl instead of ADD to fetch rng-tools
There seems to be a change in docker 17.06-rc5 on how remote
URLs and tar archives work with ADD. Use curl instead.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-06-27 17:43:58 +01:00
Rolf Neugebauer
cab27698f4 pkg: Update all packages to the new alpine base
In a subsequent commit, all YAML files will be updated with
new package hashes since all packages needed rebuild due to
build system changes in commit adae27b8d1 ("Simplify
Makefiles for Packages"). So, we might as well bring all
packages up to the latest alpine base package.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-06-27 17:43:57 +01:00
Rolf Neugebauer
da24542d78 pkg: Improve package build
- Only build if the hash has changed. Previosuly
  we'd unconditionally build and not push of the tag existed.
- Add '.m' to the hash if the repository is dirty.
  This allows developers to build locally without committing
  but makes it clear that the hash is from a modified repository.
- Don't push to hub if the repository is dirty
- If the current commit has a tag, also push this to hub.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-06-27 17:43:57 +01:00
Riyaz Faizullabhoy
ac80cd2574 Merge pull request #2101 from furious-luke/master
Add a metadata provider for Vultr
2017-06-26 14:59:21 -07:00
Ian Campbell
bbbbf3cb5d sshd: sync bind mounts with getty container
This adds all the binds from pkg/getty/Dockerfile to pkg/sshd/Dockerfile and
retains the existing /root/.ssh and /etc/resolv.conf binds.

With this `dist` and `ctr` function in a ssh session.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-06-26 11:52:41 +01:00
Ian Campbell
2864f467b9 getty,sshd: Include ca-certificates.
Otherwise:

    dist pull docker.io/library/redis:alpine
    dist: failed to do request: Head https://registry-1.docker.io/v2/library/redis/manifests/alpine: x509: failed to load system roots and no roots provided

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-06-26 11:52:41 +01:00
Ian Campbell
34fbe42615 getty,sshd: Mount host /tmp into containers.
ctr (which runs in the getty or ssh container) relies on sharing files in /tmp
with containerd (which runs in the host mount namespace). Specifically it
currently uses paths under /tmp/containerd for the stdio FIFOs of containers, resulting in:

    # ctr run -t docker.io/library/redis:alpine test
    ctr: rpc error: code = Unknown desc = runtime create failed: runc create failed: container with id exists: test

Currently it is not possible to specify a non-existent source directory for a
bind mount, so we cannot easily bind just /tmp/containerd. Sharing all of /tmp
doesn't sound like a terrible idea anyway.

Defering updating the sha in *.yml until after some further changes to these packages.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-06-26 11:52:41 +01:00