This is an example external logging service which can be enabled by
adding it to the `init` section of the .yml, for example:
...
init:
- linuxkit/init:35866bb276c264a5f664bfac7456f4b9eeb87a4d
- linuxkit/runc:v0.4
- linuxkit/containerd:f2bc1bda1ab18146967fa1a149800aaf14bee81b
- linuxkit/ca-certificates:v0.4
- linuxkit/memlogd:cc035e5c9e4011ec1ba97a181a6689fc90965ce9
onboot:
...
Signed-off-by: David Scott <dave.scott@docker.com>
If external logging is enabled, this patch sets the stdout and stderr
of the `runc` invocations to one end of a socketpair and the other end is
sent to the logging service. Otherwise we log to files as before.
Signed-off-by: David Scott <dave.scott@docker.com>
An external logging system exists if the socket
/var/run/linuxkit-external-logging.sock
exists.
If an external logging system is enabled then create FIFOs for
containerd and send the other end of the FIFOs to the logging service.
Otherwise use /var/log files as before.
Signed-off-by: David Scott <dave.scott@docker.com>
These were being added to the incorrect directory.
Also move config file to /etc to be more standard.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
DNS lookups fail in qemu-user when it is built on Alpine: https://bugs.alpinelinux.org/issues/8131
Until this is resolved, we fetch the binaries from Debian and use those instead. The final stage
of the Dockerfile is still based on scratch.
We can revert this once the Alpine issue is fixed.
Signed-off-by: Justin Barrick <jbarrick@cloudflare.com>
For some reason, bind mounting does not always seem to work,
sometimes the filesystem is empty. Mounting a fresh copy seems
a better solution, and simplifies things. The container does
need `CAP_SYS_ADMIN` but only on boot.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
When busybox's reboot processing occurs in init, it runs all SHUTDOWN
actions that are defined in inittab. Once those are complete, it will
trigger either a halt, poweroff, or reboot, depending upon what signal
is received. The mechanism that's used to shell out through inittab
does not allow us to pass through exactly which invocation was
requested.
Due to the way that rc.shutdown works, it invokes the poweroff action
for any and all SHUTDOWN callbacks, whether they're a reboot, poweroff,
or halt. Instead of handling the reboot(2) syscall in rc.shutdown,
return after killing and unmounting and let busybox's init process
decide which reboot(2) action to use.
Signed-off-by: Krister Johansen <krister.johansen@oracle.com>
Since we are building containerd v1.1.0 with go 1.10 (as it requires) to the
same for init and runc too for consistency. In the case of init it is actually
required since we use the containerd client library there.
The subreaper interfaces have been removed from containerd and replaced with a
similar interface in runc/libcontainer, update init to use that now.
Signed-off-by: Ian Campbell <ijc@docker.com>
$ git diff linuxkit.yml
diff --git a/linuxkit.yml b/linuxkit.yml
index e2ec829db..21b84e4ad 100644
--- a/linuxkit.yml
+++ b/linuxkit.yml
@@ -1,6 +1,6 @@
kernel:
image: linuxkit/kernel:4.14.32
- cmdline: "console=tty0 console=ttyS0 console=ttyAMA0"
+ cmdline: "console=ttyS0 console=foobar"
init:
- linuxkit/init:v0.3
- linuxkit/runc:v0.3
$ linuxkit build linuxkit.yml
[...]
$ linuxkit run linuxkit
[...]
getty: cmdline has console=foobar but /dev/foobar is not a character device; not starting getty for foobar
linuxkit-2ae2c420a11c login: root (automatic login)
Welcome to LinuxKit!
NOTE: This system is namespaced.
The namespace you are currently in may not be the root.
(ns: getty) linuxkit-2ae2c420a11c:~# ls -l /proc/1/root/dev/foobar
-rw-r--r-- 1 root root 311 Apr 9 13:19 /proc/1/root/dev/foobar
(ns: getty) linuxkit-2ae2c420a11c:~# cat /proc/1/root/dev/foobar
Welcome to LinuxKit
## .
## ## ## ==
## ## ## ## ## ===
/"""""""""""""""""\___/ ===
{ / ===-
\______ O __/
\ \ __/
\____\_______/
Also added quotes around $tty for good measure.
Signed-off-by: Ian Campbell <ijc@docker.com>
Go commit https://github.com/golang/go/issues/23672 introduced a
whitelist ofr flags passed into gcc to prevent arbitrary code
execution (CVE-2018-6574). The x86 rngd code uses two CFLAGS
not on the whitelist. Add them to 'CGO_CFLAGS_ALLOW'.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@gmail.com>
Without the privileged flag, the tcsd daemon does not have
access to the mounted host device files, especially the tpm
device file.
Signed-off-by: Pratik Mallya <pratik.mallya@gmail.com>
We already had this increased above the defaults, but even larger values
seem to be recommended.
See https://blog.openai.com/scaling-kubernetes-to-2500-nodes/
"It’s common to tune this setting in HPC clusters, and is particularly
relevant in Kubernetes clusters since every pod has its own IP address
which consumes space in the ARP cache."
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
This package is primarily for users of custom or foreign
kernels which may have drivers enabled for which the
'firmware' package does not contain the firmware blobs.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
This new 'firmware' package contains the firmware blobs required
by the device drivers in a LinuxKit kernel. The list of required
blobs is determined by calling 'modinfo' on each module.
We also unconditionally include the AMD CPU microcode and the
licence files.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
go_vet (via https://goreportcard.com/report/github.com/linuxkit/linuxkit)
reported:
error: missing argument for Printf("%v"): format reads arg 2, have only 1 args (vet)
error: wrong number of args for format in Printf call: 2 needed but 3 args (vet)
Signed-off-by: Ian Campbell <ijc@docker.com>
This is similar to ae64ab6b82 from #2849 which
did the same for runtime.mkdir.
This makes it possible to specify both host (absolute) or container (relative)
paths.
Signed-off-by: Ian Campbell <ijc@docker.com>
This PR correctly plumbs a single context to propagate the containerd
namespace to the necessary commands. Services launched with containerd
after this change will now be in a default namespace of
`services.linuxkit`.
A top-level flag is added to the service command,
`--containerd-namespace` which can be used to change, if needed.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Use unix.Reboot from golang.org/x/sys/unix for poweroff and reboot
instead of relying on external commands.
Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
- Previously, KV lines which were commented would attempt to be set. Now any commented KV lines will also be ignored.
- Comments can start with a hash or semicolon
- Splitting KV on both period and forward slash
- Some kernels may not have certain features enabled (such as IPv6) in the default etc/sysctl.d/*.conf, and thus pkg/sysctl would only set the KV until the first failure, and then silently skip the rest of the KVs. Now any failure is logged as a WARN, and those lines can now be commented per the above change, as they will be identified.
Signed-off-by: Isaac Rodman <isaac@eyz.us>
On 4.9.x and 4.14.x kernels ebpf verifier bugs allow ebpf
programs to access (read/write) random memory. Setting
kernel.unprivileged_bpf_disabled=1 mitigates this somewhat
until it is fixed upstream.
See:
- https://lwn.net/Articles/742170
- https://lwn.net/Articles/742169
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
This was added to alpine since our package was created. Now we have upgraded we
can just use the binary.
The package contains an auditd.conf but we have a tweak local copy which writes
to stdio (which goes to /var/log/auditd.*.log already). The package doesn't
have an audit.rules so keep that here too.
Signed-off-by: Ian Campbell <ijc@docker.com>
containerd v1.0.0-rc.0 changed the subreaper config entry to the
negative given subreaper "on" is the default. However, linuxkit
does not need to change the default subreaper setting so removing
the now invalid config line.
Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
This is a `moby.ImageConfig` struct which is marshalled into JSON and added as
the `org.mobyproject.config` label on the built image.
Convert `pkg/sysctl` as PoC.
Signed-off-by: Ian Campbell <ijc@docker.com>
Implements https://github.com/moby/tool/pull/181
Design for things like Kubernetes setup that requires some cgroups to
exist when the service starts but it is not running in these, other
services are, so there would be a race if they are not created in each.
Essentially it is just a sugared `mkdir` in all the cgroup dirs.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
When waiting for devices to refresh a stat can fail with the
error message 'stat: /dev/vda1: stat: no such file or directory'.
This was observered with empty raw disks (*.img) that was being
created by the moby/hyperkit go library.
This commit restores the previous logic of refreshing with mdev
instead of returning an error.
Signed-off-by: dave protasowski <dprotaso@gmail.com>
The /sbin/ldconfig.real command in ubuntu for ppc64le cannot be executed
in docker images since the binfmt-misc configuration does not match the
executable file format. This patch is a port of
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=794737 to linuxkit
Signed-off-by: Olaf Flebbe <of@oflebbe.de>
By running:
./scripts/update-component-sha.sh --image linuxkit/alpine ad35b6ddbc70faa07e59a9d7dee7707c08122e8d
Signed-off-by: Ian Campbell <ijc@docker.com>
This was done with the following "script":
git rm pkg/{auditd,binfmt,init}/Makefile
sed -e 's/IMAGE=/image: /g' -i pkg/*/Makefile
sed -e 's/NETWORK=1/network: true/g' -i pkg/*/Makefile
sed -e 's/ARCHES=x86_64/arches:\n - amd64/g' -i pkg/*/Makefile
sed -e '/DEPS:\?=/d' -i pkg/*/Makefile
sed -e '/ARCHES=SKIP/d' -i pkg/node_exporter/Makefile
sed -e 's/include \.\.\/package.mk//g' -i pkg/*/Makefile
sed -e '/^$/d' -i pkg/*/Makefile
git mv pkg/node_exporter/Makefile pkg/node_exporter/build.yml-skip
for i in pkg/*/Makefile ; do git mv $i ${i%Makefile}build.yml ; done
and manual update of pkg/Makefile.
Signed-off-by: Ian Campbell <ijc@docker.com>
Using explicit targets is cleaner and easier to follow.
Also add `forcepush` to pkg and test/pkg and make `push` explicit.
The `dirs` phony was non-existent in all three cases. Likewise `$(DIR)` wasn't
used in that way in pkg or test/pkg and is nolonger used in that way in tools.
Finally, `clean` was bogus in all cases (tools cleaned a thing which never
exists and pkg and test/pkg were a syntax error).
Signed-off-by: Ian Campbell <ijc@docker.com>
This implements the proposal in #2564 and converts a handful of representative
or especially interesting (from a build PoV) packages to use it.
For now those pkg/* affected get a stub-`Makefile`, once all packages are
converted then `pkg/Makefile` can be adjusted and those stubs can be removed.
For now only `pkg/package.mk`'s functionality is implemented. In particular:
- `push-manifest.sh` remains a separate script, to enable calling it on systems
with just the LinuxKit tools installed arrange to install it under a less
generic name.
- `kernel` and `tools/alpine` do not use `pkg/package.mk` and those cases are
not yet fully considered/covered.
I have updated the documentation assuming that the existing uses of
`pkg/package.mk` will be removed quite soon in a follow up PR rather than
trying to document the situation which results after just this commit.
Due to `cmd/linuxkit` now gaining a library the build needs adjusting slightly to
allow both `make bin/linuxkit` and `go build` to work.
`go vet` has forced me to write some rather asinine comments for things that
are rather obvious from the name.
Signed-off-by: Ian Campbell <ijc@docker.com>
These packages include the wireguard tools which need to be
updated due to the wireguard kernel bump in:
43db718f14 ("wireguard: version bump").
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
- Added new argument -force to the format utility. This will force formatting of the specified single device if it exists and is a block device
- By default, the format package will no longer format the specified single device if content exists on that device
- Added new blkid-based check for content, in addition to the existing check for partitions on autoformat candidate devices.
- Cleanup of old unused code.
- Refactoring of block device verification.
- Added test/cases/040_packages/006_format_mount/005_by_device_force
- Added document of new arguments to /docs/external-disk.md. Also sorted the arguments in the doc alphabetically and added them as bullets so they do not run together on the page.
Signed-off-by: Isaac Rodman <isaac@eyz.us>
Some software (such as cri-containerd and kubelet) rely on the presence of
these links, which are usually created by udev.
Inspired by 74b145b5c7/sourceroot/functions.sh (L93...L124)
but rewritten in go.
Signed-off-by: Ian Campbell <ijc@docker.com>
golint on pkg/init now complains:
golint...
./init.go:199:2: redundant if ...; err != nil check, just return error instead.
Resulting in a change which doesn't seem like an improvement to me.
Signed-off-by: Ian Campbell <ijc@docker.com>
We always had 1G swap to work better with small memory setups, but this
was omitted in the update to LinuxKit.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
This removes more shell scripts to improve maintainability.
This now also works correctly in userspace, so it can be used for
running LinuxKit images in Docker and other such use cases.
It is a literal conversion of the shell scripts with a few small
tweaks.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
This removes things we do not need and expects the interface to
already be created.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Previously we would pass the path `/var/log/service.log` for both
stdout and stderr to containerd. containerd would construct a dict
with the paths as keys[1] and, due to the duplicate key, would only
open one of the files and start one `io.Copy` instance. Writes to
the other stream would be buffered by the pipe connected to
containerd-shim and would eventually block.
If we modified containerd to open the file twice and start 2
`io.Copy` instances, we would end up with the two streams interleaved
together. It seems cleaner to keep the streams separate; therefore
this patch logs stdout to `/var/log/service.out.log` and stderr to
`/var/log/service.err.log`.
[1]
49437711c3/linux/shim/io.go (L51)
Signed-off-by: David Scott <dave.scott@docker.com>
This removes all the code that had knowledge of how to do read only
and read write container mounts, and just uses the runtime config.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
This could be used in LinuxKit now, as there are some examples, eg
https://github.com/linuxkit/linuxkit/blob/master/blueprints/docker-for-mac/base.yml#L33
which are creating containers to do a mount.
The main reason though is to in future change the ad hoc code that generates
overlay mounts for writeable containers with a runtime config which does
the same thing; this code needs to create both tmpfs and overlay mounts.
See https://github.com/moby/tool/pull/145
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
The metadata package has binds
- /dev
- /var
- /sys
- /etc/resolv.conf
- /etc/ssl/certs
but unfortunately `/etc/ssl/certs` doesn't exist and this causes the
following commands:
cd blueprints/docker-for-mac # easy example
moby build -name docker-for-mac base.yml docker-17.06-ce.yml
linuxkit run hyperkit -networking=vpnkit -vsock-ports=2376 -disk size=500M docker-for-mac
to produce the following error on the VM console:
container_linux.go:265: starting container process caused "process_linux.go:348: container init caused \"rootfs_linux.go:57: mounting \\\"/etc/ssl/certs\\\" to rootfs \\\"/containers/onboot/000-metadata/rootfs\\\" at \\\"/etc/ssl/certs\\\" caused \\\"stat /etc/ssl/certs: no such file or directory\\\"\""
2017/08/21 16:39:40 Error creating 000-metadata: exit status 1
This patch creates /etc/ssl/certs in the `init` package. The metadata package
will now say things like
2017/08/21 16:44:39 No metadata/userdata found. Bye
Signed-off-by: David Scott <dave.scott@docker.com>
As discussed before, as we use this in three places, cloning in
base makes more sense.
Update base image.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Use the packet metadata to set up bonding, also get ssh keys and hostname.
This does not yet do anything with disk metadata.
Userdata is not used if it has been used for ipxe, but is otherwise available.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
This contans the upstream (non busybox) version of `ip` and also the
Wireguard tools as these aere not yet integrated into `ip`. It is designed
to be used for network configuration.
For example a Wireguard interface can be setup with something like:
```
- name: wg
image: linuxkit/ip
net: new
binds:
- /etc/wireguard:/etc/wireguard
command: ["sh", "-c", "ip link set dev wg0 up; ip address add dev wg0 192.168.2.1 peer 192.168.2.2; wg setconf wg0 /etc/wireguard/wg0.conf; wg show wg0"]
runtime:
interfaces:
- name: wg0
add: wireguard
createInRoot: true
bindNS:
net: /run/netns/wg
```
This binds the net namespace so other containers can use it later.
Note that `ip` supports batch scripts with `ip -b file` which is quite convenient for
this use case.
Alas `wg-quick` requires `bash` so this is currently a bit bloated, maybe @zx2c4
might be persuaded to rewrite it without bashisms...
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
This adds support for a runtime configuration file that can do:
- `mkdir` to make a directory at runtime, eg in `/var` or `/tmp`, to avoid workarounds
- `interface` that can create network interfaces in a container or move them
- `bindNS` that can bind mount namespaces of an `onboot` container to a file so a service can be started in that namespace.
It merges the `service` and `onboot` tools (in `init`) to avoid duplication. This also saves some size for
eg LCOW which did not use the `onboot` code in `runc`.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
During development I tend to do:
make -C pkg/foo tag HASH=dev ORG=ijc
and I expect to get an image `ijc/foo:dev` and not `ijc/foo:dev-amd64`. In
general I think if the HASH has been explicitly provided the build should
just honour that.
To compensate and avoid mistaken pushes adjust the check for dirtiness to also
insist on a non-empty suffix.
Signed-off-by: Ian Campbell <ijc@docker.com>
At least when running under Docker this seems to cause some issues,
possibly also explains some other oddnesses.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
This has a number of fixes/improvements
- console resize
- better error handling (pass stacktrace/errors back to docker
- fix for container images which have a resolve.conf file
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
Chown clears suid bits even for root on Linux.
Also move a few functions to x/sys/unix from syscall, to be
more arm64 friendly.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Runc is master as of this morning, slightly newer than what is vendored into
containerd v1.0.0-alpha3 but the differences[0] seemed benign and/or good to have.
Signed-off-by: Ian Campbell <ijc@docker.com>
[0] 429a538712...45bde006ca
This requires that users have `CAP_SYSLOG` in order to access `dmesg`.
This means that containers by default have no access to `dmesg` (which
can leak information about the host or other containers) unless they
have this capability added.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
They either won't work or the packages are specific
to x86_64 like the DfM or VMware packages.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
A package makefile can specify a list of architectures ('ARCHES')
it supports. If build on a unsupported architecture, a message is
printed and make exits without an error.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
This checks if any CDROM in the system has metadata. Useful if
you boot off CDROM and also have a metadata CDROM.
Also switch the metadata container to read only, as it is only
writing to bind mounted in directories, and /tmp.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Epoll in `syscall` package does not work on arm64. We do not have
arm64 yet here, but we will when we add tpm support etc.
Signed-off-by: Justin Cormack <justin@specialbusservice.com>
Only supports the use cases we currently need, currently support
for using Intel hardware rng to initialise and add entropy.
Supports oneshot and service mode. Call as `rngd -1` for one shot
mode.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Previously we were cheating and remounting /var `rw` but this does not
work if the filesystem is really read only. Nount a tmpfs, which may
be overmounted later by a persistent filesystem.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Previously we would sneakily remount as `rw` but of course you can't
really do that on a truly immutable filesystem.
See https://github.com/moby/tool/pull/129 for the `moby` side.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
We need to add 'ttyAMA0' console used on ARM64 platform into
securetty file to make it's possible to log into the system
as root. Also it will dismiss the below warning message before
login:
"getty: cmdline has console=ttyAMA0 but does not exist in
/etc/securetty; will not be able to log in as root on this tty ttyAMA0."
'ttyAMA1' is also added order to handle some system happens have that one
instead of 'ttyAMA0'
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Current package.mk will operate on the first dependence object
'check-dirty', which will result the make process abort earlier
with below message:
pkg/package.mk:60: *** Your repository is not clean. Will not push package image. Stop.
This is not expected behavior since we have no chance to build
a docker image locally in case of the user has no intention to
push that image.
This patch adjust the dependece order for the default target, thus
we can build the docker image locally but can't push that image
since we're in a dirty git repository.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
The logrus import path has changed, so adjust. Also there is a minor API change
to the containerd.IOCreation() function spec, it now takes a string id which we
can ignore.
Signed-off-by: Ian Campbell <ijc@docker.com>
At present they use a small shared function called "prepare"
that does the read-write remounts, that I will switch to doing overlay
mounts soon.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
This was split out from pkg/format into its own package.
It has the ability to extend ext4, btrfs and xfs partitions.
Signed-off-by: Dave Tucker <dt@docker.com>
This commit rewrites the mount package in Go.
It adds the ability to mount the by label, UUID or name.
It also fixes the automatic mount logic to check that a drive isn't
already mounted before attempting to mount it. This allows for multiple
uses of the mount pkg in a single YAML file.
Signed-off-by: Dave Tucker <dt@docker.com>
This commit re-writes the format package in Go and adds the ability to
add partition labels and also specify either ext4, btrfs or xfs
Signed-off-by: Dave Tucker <dt@docker.com>
This moves most of the initialisation of containers to the
service init in the `service` command.
Still leaves remounting root file systems read only but this
will go away shortly. Another step closer to removing shell
scripts in base system.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
This is like the `push` target but omits the pulls and depends on forcetag
instead. With the git commit now being embedded into the image this is now a
necessary part of rebasing a PR for which images have already been pushed.
Also adds PHONY targets for existing forcetag and push targets which were
missing.
NB $(error) appends a "." to omit the final one from the error message
Signed-off-by: Ian Campbell <ijc@docker.com>
The filesystem is supposed to be immutable, so do not try to make
a symlink; new versions of moby tool should add one anyway. But
try to make the directory a symlink points to, assuming that it
will be on a writeable filesystem.
fix#1920
see also #2288
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
This uses a more memory efficient copy, and gets us closer to
not having a shell in the base system if not required.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
People really want to play around with this, so adding them here makes
it possible. Just as iproute2 is part of these, so should
wireguard-tools.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
Since these are the user login containers, having the ability to add packages
is useful (e.g. I quite often find I want strace).
Doing this requires that we not share `/var` with the login containers since we
want the apk database therein. Previously it was thought that the containers
might need some parts of `/var` for `ctr` to work (e.g. `/var/lib/containerd`)
but this is not the case now (if it ever was) based on my testing.
Fixes#2206.
Signed-off-by: Ian Campbell <ijc@docker.com>
This is actually containerd#1141 rebased onto v1.0.0-alpha1.
The `dist` command has been integreated into `ctr` and so is removed, including
from the getty and sshd bind mounts and the test which uses it is updated..
There is no change to the version of runc vendored by containerd, so this is
unchanged.
Signed-off-by: Ian Campbell <ijc@docker.com>
If the configuration .json has contents like:
{
"etc": {
"ssl": {
"certs": {
"ca-certificates.crt": {
"perm": "0644",
"content": "large amount of certificate text"
}
}
}
},
...
}
then we print a warning because the node "ssl" has no "perm".
Previously the warning would include the contents of "ssl", which
would be large (and in theory could include secret information).
This patch modifies the warning print to only print the key and
not the value.
Without this patch, I see on the console:
2017/07/20 10:03:04 CDROM: Probe succeeded
2017/07/20 10:03:04 No permission provided ssl:map[certs:map[ca-certificates.crt:map[perm:0644 content:large amount of certificate text]]]
- 000-metadata
With this patch, I see on the console:
2017/07/20 09:54:18 CDROM: Probe succeeded
2017/07/20 09:54:18 No permission provided ssl
- 000-metadata
Signed-off-by: David Scott <dave.scott@docker.com>
The sample init script from opengcs is quite basic (and doesn't
set up several mounts and symlinks). Use the relevant portion
from rc.init from the LinuxKit init package instead.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
This updates the build of the two image caches to use the `pkg/package.mk`
infrastructure, albeit in a slightly (ok, very) atypical way.
In order to share the bulk of the build code (including the `Dockerfile` and
the `Makefile` machinery to download the images) we arrange for the necessary
bits to be copied at build time into distinct subdirectories and for the
`pkg/package.mk` to be aware of this possibility.
Since pkg/package.mk is only set up to build a single package we use a single
`image-cache/Makefile` to drive the whole process and recurse into
`Makefile.pkg` to build individual packages.
One particular subtlety is that the package hash is based on the `image-cache`
directory (which is in `git`) rather than the generated subdirectories (which
are not in `git`). Since all the generators (and their inputs) are in the
`image-cache` directory this is what we want. This means that the two images
are given the same tag, but this is deliberate and desirable.
The generated directories are completely temporary to avoid picking up stale
versions of images when versions are updated. Images are hardlinked into place.
The images are moved to the linuxkitprojects org. Using a dev tag for now, will
update once everything is in place.
Also use "tag" rather than "build" where appropriate in the Makefile.
There is no point in the .dockerignore now, but add a .gitignore.
Signed-off-by: Ian Campbell <ijc@docker.com>
These can be added by other packages if they need to do something on
clean shutdown.
Crash only software can ignore this.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
It is pretty close to our docker package, if we adjust the command
that is run to avoid the actual dind startup script. We can't use
the normal docker image as it does not have mkfs and so on.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
We want them to run in sequence. For example we want mounts to be done (done by
`pkg/runc/etc/init.d/010-onboot`) before we start services (done by
`pkg/containerd/etc/init.d/020-containerd`). This was most likely introduced by
28b4245b12 ("Move onboot startup script to runc package").
None of the initscripts in pkg/* block, but some in projects (selinux and
logging, not updated here) do.
Signed-off-by: Ian Campbell <ijc@docker.com>
Do not copy host securetty file - this one should be comprehensive
or bind mount host one in yourself.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
This package build the init filesystem for LCOW (Linux Containers
on Windows) based on the instructions from:
https://github.com/Microsoft/opengcs.git
We also pull in a udhcpd config script from a specific version of
busybox which was the tip of master at the time this was added.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
As this does not use containerd at all, this means you can run very
minimal setups with just `runc` if you use no services, for example
most of our tests do not actually use services, or if you have other
similar very minimal use cases.
Move ulimit setup to `init` which makes more sense.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
We expect this (or something very similar) to be merged soon, it fixes#2131 so
moving ahead now.
The new alpine mirror is linuxkit/alpine:6832775a7e861ee2d7842e157688ece52d007142
Signed-off-by: Ian Campbell <ijc@docker.com>
This will make it simpler to temporarily in PRs which are not yet merged (but
are expected to be soon).
Tools alpine is not rebuilt here since we are going to do just that in the next
commit.
Signed-off-by: Ian Campbell <ijc@docker.com>
pull in newer containerd v1.0.0-alpha0 via updated alpine base, update runc to
429a5387123625040bacfbb60d96b1cbd02293ab which is vendored by that version of
containerd (and also update alpine base for runc)
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
These are not needed, but we are inconsistent. Been waiting for a
quiet moment to fix this since I noticed while doing a presentation...
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Use os.Mkdirall() instead of os.Mkdir() as it does not
error if the path already exists.
This enables specifying a default config file with the image
and then over writing it with metadata.
While at it, also update to the latest alpine base image.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
Some hypervisors (e.g. hyperkit / xhyve) don't provide a good way to keep
the VM's clock in sync with the Host's clock. NTP will usually keep the
clocks together, but after a the host or VM is suspended and resumed the
clocks can be suddenly too far apart for NTP to work properly.
This simple daemon listens on an AF_VSOCK port and resynchronises the VM
clock from the virtualised hardware clock.
This is a Go conversion of original C code written by
Magnus Skjegstad <magnus@skjegstad.com>
Signed-off-by: David Scott <dave.scott@docker.com>
For DIRTY and RELEASED this means simply performing a static assignment with
`:=`. For HASH it is a little more complex since it is (and we want/need it to
be) a conditional assigment. However it is only used for defining TAG, so make
that a static assignment.
This reduces the number of times the complex DIRTY shell command in particular
is evaluated.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
Firstly add option to disable content trust, for the use of e.g. projects which
are pushing to the linuxkitprojects org (which has no trust setup) rather than
the main linuxkit org.
Secondly, when trust _is_ enabled then enable it globally, in particular it is
now active for the `docker build` and hence containers referenced in
Dockerfiles via "FROM" will be checked.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
Adds a new service for running `/sbin/acpid` that comes with busybox.
The VM will shut down if the power button is pressed.
Signed-off-by: Magnus Skjegstad <magnus@skjegstad.com>
Otherwise files which have an updated timestamp but no actual changes are
marked as changes because `git diff-index` only uses the `lstat` result and not
the actual file contents. Running `git update-index --refresh` updates the
cache.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
If a user passes a HASH e.g. HASH=dev then assume they know what they are doing
and don't need dirty tracking.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
Useful to answer the question of what the hash was at the point of some older
commit.
$ make --no-print-directory -C pkg/init/ show-tag
linuxkit/init:36c56f0664d49c5a6adc1120d1bf5ba6ac30b389
$ make --no-print-directory -C pkg/init/ show-tag HASH_COMMIT=4699f80ef73141ee87a77d03f93065e541fab382
linuxkit/init:17693d233dd009b2a3a8d23673cb85969e1dce80
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
The current rune for $(HASH) returns nothing with the Jessie version of git
(2.1.4). Using `--full-tree` works corectly but requires an absolute path (else
it produces e.g. "fatal: ../init: '../init' is outside repository").
Both `ls-tree` and `diff-index` are happy with a full absolute path, which
simplifies things since we can use `$(CURDIR)` directly.
Tested with a dirty `pkg/init` on both Jessie (git 2.1.4) and Stretch (git
2.11.0) with the following command, which produces identical output in both
cases:
$ for pkg in init containerd ; do make -C pkg/$pkg --no-print-directory show-tag; ( cd pkg/$pkg && make show-tag ); done
linuxkit/init:36c56f0664d49c5a6adc1120d1bf5ba6ac30b389-dirty
linuxkit/init:36c56f0664d49c5a6adc1120d1bf5ba6ac30b389-dirty
linuxkit/containerd:1e3e8f207421de8deac8cedc26a138d6b1661a0d
linuxkit/containerd:1e3e8f207421de8deac8cedc26a138d6b1661a0d
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
By setting `DIRTY` to either "-dirty" or "" directly we can simply use
`$(HASH)$(DIRTY)` and avoid make adding a space as it does with `+=`.
For the push check we now block pushing if `$(DIRTY)` is non-empty.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
$ make --no-print-directory -C pkg/init/ show-tag
linuxkit/init:36c56f0664d49c5a6adc1120d1bf5ba6ac30b389
Useful for scripting etc.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
As of commit b2439251dd ("blueprints: update docker-for-mac.yml
to latest vpnkit packages") the docker-ce package does not need
the vpnkit-expose-port binary anymore. It was also removed from
the linuxkit/vpnkit-forwarder forwarder package with commit
f48a2bfe12 ("pkg: update vpnkit-forwarder") causing the docker-ce
package build to fail.
This commit fixes this by removing the linuxkit/vpnkit-forwarder
package from the docker-ce package build.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
This fixes a regression introduced in:
5a225b9d ("Makefile: Fix bug where network was not disabled")
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
This package contains vpnkit binaries needed inside the dockerd
container that can be either copied into the container with a multistage
build or used as an init-container and bind-mounted to the right
locations.
vpnkit-expose-port is a userland proxy that interacts with vpnkit and
the vpnkit-forwarder service to open ports on the host.
vpnkit-iptables-wrapper wraps iptables to launch
vpnkit-expose-port automatically when ports are opened to a swarm
service.
Signed-off-by: Magnus Skjegstad <magnus@skjegstad.com>
This commit moves the include statement to the bottom of the file to
ensure that all variables are set before conditionals are evaluated.
I also changed the ifndef NETWORK to ifdef NETWORK as the former was
incorrect. We want `NET_OPTS="--network=none"` in cases where NETWORK is
not defined.
Fixes: #2134
Signed-off-by: Dave Tucker <dt@docker.com>
Warn the user in the MOTD
Add "(ns: getty)" or "(ns: sshd)" to the PS1
Use `agetty` and `-a root` to ensure we get a login shell when insecure
Signed-off-by: Dave Tucker <dt@docker.com>
Two things to note here: we unfortunately can't just exec audit, because
something needs to load the rules in beforehand.
Second, it also dies if it can't re-nice itself, so we have to give it
CAP_SYS_NICE as well as the audit caps.
I didn't add this to the default linuxkit.yml because the linuxkit/audit
repo doesn't exist yet, but we should probably (?) do that. Additionally,
we should provide the kernel parameter audit=1, otherwise according to
auditd's man pages, we can't audit some early tasks.
Closes#52
Signed-off-by: Tycho Andersen <tycho@docker.com>
- Use the new style kernel tags with the full kernel version
- Update packages with new alpine base and new/simplified Makefiles.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
People typically update hashes in YAML files with
'git grep ... | sed ...' this will change the README.md files
in the ./pkg directory and thus change the git tree hash.
Remove the use of explicit hashes from those files to avoid
the obvious chicken and egg problem.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
This adds a helper which monitors the Docker event stream and, when it
receives an image delete event, queues a call to `fstrim /var/lib/docker`
to trigger a space reclamation. Previously we would rely on a cron job
running every 15 minutes.
Signed-off-by: David Scott <dave.scott@docker.com>
There seems to be a change in docker 17.06-rc5 on how remote
URLs and tar archives work with ADD. Use curl instead.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
In a subsequent commit, all YAML files will be updated with
new package hashes since all packages needed rebuild due to
build system changes in commit adae27b8d1 ("Simplify
Makefiles for Packages"). So, we might as well bring all
packages up to the latest alpine base package.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
- Only build if the hash has changed. Previosuly
we'd unconditionally build and not push of the tag existed.
- Add '.m' to the hash if the repository is dirty.
This allows developers to build locally without committing
but makes it clear that the hash is from a modified repository.
- Don't push to hub if the repository is dirty
- If the current commit has a tag, also push this to hub.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
This adds all the binds from pkg/getty/Dockerfile to pkg/sshd/Dockerfile and
retains the existing /root/.ssh and /etc/resolv.conf binds.
With this `dist` and `ctr` function in a ssh session.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
Otherwise:
dist pull docker.io/library/redis:alpine
dist: failed to do request: Head https://registry-1.docker.io/v2/library/redis/manifests/alpine: x509: failed to load system roots and no roots provided
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
ctr (which runs in the getty or ssh container) relies on sharing files in /tmp
with containerd (which runs in the host mount namespace). Specifically it
currently uses paths under /tmp/containerd for the stdio FIFOs of containers, resulting in:
# ctr run -t docker.io/library/redis:alpine test
ctr: rpc error: code = Unknown desc = runtime create failed: runc create failed: container with id exists: test
Currently it is not possible to specify a non-existent source directory for a
bind mount, so we cannot easily bind just /tmp/containerd. Sharing all of /tmp
doesn't sound like a terrible idea anyway.
Defering updating the sha in *.yml until after some further changes to these packages.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
Vultr uses a very similar approach to AWS, including using the
same IP address for serving metadata. In fact, it seems
as though if AWS appears first in the list of providers, that
provider mistakenly believes to be running on AWS (hence the
insertion of `NewVultr` in between GCP and AWS. I don't believe
AWS servers will accidentally try to use the Vultr provider,
as it seems that the `/v1/` endpoint doesn't exist on AWS.
Signed-off-by: Luke Hodkinson <furious.luke@gmail.com>
These now inherit from a top-level package.mk
Options like use of the network can be enabled on a per package basis
This removes a lot of duplicate code and make the maintenace of these
Makefiles much easier
Signed-off-by: Dave Tucker <dt@docker.com>
This includes https://github.com/containerd/containerd/pull/994 and hence
requires updating the various instances of `/etc/containerd/config.toml`.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
Currently it supports only `service start <SERVICE>`, but it could grow e.g.
`stop`, `exec` etc in the future (although you can still use `ctr` for those).
In order to be able to use go-compile.sh the containerd build needs to move
from /root/go to /go as the GOPATH.
The vendoring situation is not ideal, but since this tool wants to be an exact
match for the containerd it seems tollerable to reuse its vendoring.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
This doesn't exist with newer ctr or in systems where service containers are
not started using the ctr tool. All it contains today are the stdio FIFOs,
which are not in general useful to access after container creation.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
vpnkit-expose-port is the dockerd userland proxy used to expose
forwarding ports with vpnkit. This adds the binary to the image in
/usr/bin/vpnkit-expose-port, but does not enable it by default.
Signed-off-by: Magnus Skjegstad <magnus@skjegstad.com>
