Reduce the number of packages to build for s390x. Firmware
is only used for physical devices, so disable it for s390x
where we mostly run in virtual machines.
Signed-off-by: Rolf Neugebauer <rn@rneugeba.io>
Some drivers offer mutliple firmwares with the WHENCE file
defining the default. Use the cope-firmware.sh script to
create a copy of the firmware repository with the defaults
copied in to the right place.
Signed-off-by: Rolf Neugebauer <rn@rneugeba.io>
This adds a new configuration provider that just reads a file.
This is needed for Docker Desktop, where we will run a LinuxKit distro in an isolated namespace within WSL 2.
In this scenario, the config will be accessible trough the WSL2 built-in 9p mount of the Windows filesystem.
Signed-off-by: Simon Ferquel <simon.ferquel@docker.com>
There were some mistakes made in the initial code where writes didn't work, this commit fixes that.
Signed-off-by: Simon Fridlund <simon@fridlund.email>
If the swap disk is larger than 1MiB, then use a 1MiB blocksize in `dd`
On my machine using a large block size speeds up swap file creation:
```
/ # time dd if=/dev/zero of=output bs=1024 count=1048576
1048576+0 records in
1048576+0 records out
real 0m 4.61s
user 0m 0.79s
sys 0m 3.77s
/ # time dd if=/dev/zero of=output bs=1048576 count=1024
1024+0 records in
1024+0 records out
real 0m 1.06s
user 0m 0.00s
sys 0m 1.04s
```
Signed-off-by: David Scott <dave.scott@docker.com>
This commit will update the Scaleway provider to fetch the cloud-init/cloud-config data from the user_data/cloud-init endpoint it will also make sure the whole public ssh key is fetched and no longer strip out the `ssh-rsa` part of the keys
Signed-off-by: Simon Fridlund <simon@fridlund.email>
It's now not needed to send a boot signal when booting an instance on
Scaleway, thus the method is not needed anymore.
Signed-off-by: Patrik Cyvoct <patrik@ptrk.io>
Copy firmaware files to the correct directory. Instead of
<vendor>/<fw-name>/<fw-name> copy it to <vendor>/<fw-name>.
Signed-off-by: Rolf Neugebauer <rn@rneugeba.io>
The Vultr provider currently never calls handleSSH, resulting in it
being impossible to bring up a LinuxKit image in vultr with the SSH
pubkey provided via the Vultr metadata API.
Signed-off-by: Sachi King <nakato@nakato.io>
Linux has documented but somewhat unusual behavior around
SIGSTOP/SIGCONT and certain syscalls, of which epoll_wait(2) is one. In
this particular case, rngd exited unexpectedly after getting ptrace'd
mid-epoll_wait. Fix this by handling EINTR from this syscall, and
continuing to add entropy and wait.
Signed-off-by: Krister Johansen <krister.johansen@oracle.com>
The compile fails with:
[ 30%] Building CXX object src/ast/CMakeFiles/ast.dir/codegen_llvm.cpp.o
[ 30%] Building CXX object src/ast/CMakeFiles/ast.dir/irbuilderbpf.cpp.o
[ 31%] Building CXX object src/ast/CMakeFiles/ast.dir/printer.cpp.o
[ 31%] Building CXX object src/ast/CMakeFiles/ast.dir/semantic_analyser.cpp.o
/bpftrace/src/ast/irbuilderbpf.cpp: In member function 'llvm::CallInst* bpftrace::ast::IRBuilderBPF::CreateProbeReadStr(llvm::AllocaInst*, size_t, llvm::Value*)':
/bpftrace/src/ast/irbuilderbpf.cpp:279:16: error: 'BPF_FUNC_probe_read_str' was not declared in this scope
getInt64(BPF_FUNC_probe_read_str),
^~~~~~~~~~~~~~~~~~~~~~~
/bpftrace/src/ast/irbuilderbpf.cpp: In member function 'llvm::CallInst* bpftrace::ast::IRBuilderBPF::CreateProbeReadStr(llvm::Value*, size_t, llvm::Value*)':
/bpftrace/src/ast/irbuilderbpf.cpp:294:16: error: 'BPF_FUNC_probe_read_str' was not declared in this scope
getInt64(BPF_FUNC_probe_read_str),
^~~~~~~~~~~~~~~~~~~~~~~
/bpftrace/src/ast/irbuilderbpf.cpp: In member function 'llvm::CallInst* bpftrace::ast::IRBuilderBPF::CreateGetCurrentCgroupId()':
/bpftrace/src/ast/irbuilderbpf.cpp:422:16: error: 'BPF_FUNC_get_current_cgroup_id' was not declared in this scope
getInt64(BPF_FUNC_get_current_cgroup_id),
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/bpftrace/src/ast/irbuilderbpf.cpp: In member function 'llvm::CallInst* bpftrace::ast::IRBuilderBPF::CreateGetCurrentTask()':
/bpftrace/src/ast/irbuilderbpf.cpp:461:16: error: 'BPF_FUNC_get_current_task' was not declared in this scope
getInt64(BPF_FUNC_get_current_task),
^~~~~~~~~~~~~~~~~~~~~~~~~
/bpftrace/src/ast/irbuilderbpf.cpp: In member function 'llvm::CallInst* bpftrace::ast::IRBuilderBPF::CreateGetStackId(llvm::Value*, bool)':
/bpftrace/src/ast/irbuilderbpf.cpp:497:16: error: 'BPF_FUNC_get_stackid' was not declared in this scope
getInt64(BPF_FUNC_get_stackid),
^~~~~~~~~~~~~~~~~~~~
/bpftrace/src/ast/semantic_analyser.cpp: In member function 'int bpftrace::ast::SemanticAnalyser::create_maps(bool)':
/bpftrace/src/ast/semantic_analyser.cpp:871:68: error: 'BPF_MAP_TYPE_STACK_TRACE' was not declared in this scope
bpftrace_.stackid_map_ = std::make_unique<bpftrace::FakeMap>(BPF_MAP_TYPE_STACK_TRACE);
^~~~~~~~~~~~~~~~~~~~~~~~
/bpftrace/src/ast/semantic_analyser.cpp:885:64: error: 'BPF_MAP_TYPE_STACK_TRACE' was not declared in this scope
bpftrace_.stackid_map_ = std::make_unique<bpftrace::Map>(BPF_MAP_TYPE_STACK_TRACE);
^~~~~~~~~~~~~~~~~~~~~~~~
make[2]: *** [src/ast/CMakeFiles/ast.dir/build.make:89: src/ast/CMakeFiles/ast.dir/irbuilderbpf.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[2]: *** [src/ast/CMakeFiles/ast.dir/build.make:115: src/ast/CMakeFiles/ast.dir/semantic_analyser.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:276: src/ast/CMakeFiles/ast.dir/all] Error 2
make: *** [Makefile:141: all] Error 2
Signed-off-by: Rolf Neugebauer <rn@rneugeba.io>
Needed for containerd v1.2.0 otherwise:
$ ctr run -t docker.io/library/hello-world@sha256:f3b3b28a45160805bb16542c9531888519430e9e6d6ffc09d72261b0d26ff74f test
[ 1311.667587] overlayfs: failed to resolve '/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/5/fs': -2
ctr: failed to mount /tmp/containerd-mount111658703: no such file or directory
Signed-off-by: Ian Campbell <ijc@docker.com>
In addition to bug fixes, this removes the special protocol used
for `shutdown` needed by old Windows builds < 14393.
Signed-off-by: David Scott <dave.scott@docker.com>
In e8786d73bb the logwrite package will
automatically append .log to every log.
In 5201049f2c the init package will send
stderr of a service `s` to a log named `s` and the stdout to `s.out`.
Therefore the files we create on disk are `s.log` and `s.out.log`.
This patch modifies the memlogd `logwrite` command-line wrapper to use
the same convention.
Note there is a confusing name clash between `pkg/logwrite` and `cmd/logwrite`
in `memlogd` modified here.
Signed-off-by: David Scott <dave.scott@docker.com>
- use the mkimage hashes that we had in LinuxKit as more up to date than tool.
- update docs
- move the code from moby under src/cmd/linuxkit
Signed-off-by: Justin Cormack <justin@specialbusservice.com>
When logging directly to files (the not-using-memlogd case) the onboot
services must log to /run/log because /var/log might be overmounted
by a persistent disk. Therefore we create a symlink at the end of
the onboot section.
When logging via memlogd, all logs are buffered until a logwrite service
starts, so no symlink is needed.
Signed-off-by: David Scott <dave.scott@docker.com>
Also simplify the code by directly storing the path to
the log file in the LogFile structure.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
Update the the firmware packages to the latest commit
of the upstream linux-firmware repository.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
This process connects to memlogd and streams logs to individual files,
one per log. It keeps track of how many bytes have been written to each
file and rotates when the file size exceeds a defined threshold.
By default the maximum size of each file before rotation is 1MiB and
we keep up to 10 files per log.
Signed-off-by: David Scott <dave.scott@docker.com>
Switch to a more formally-specified `kmsg`-style format for reading
the logs.
- update the spec in docs/logging.md
- check for bad names in pkg/memlogd with unit test
Signed-off-by: David Scott <dave.scott@docker.com>
- check writing to the log does not block
- check the log doesn't expand -- it should be finite
- check that client connections don't buffer arbitrary amounts of
data if the client is slow
Signed-off-by: David Scott <dave.scott@docker.com>
Previously we had a per-connection
bytes.Buffer // to be written to the connection
sync.Cond // to allow us to Wait for more data
This had the major disadvantage that the buffer was unbounded and so
a slow client could cause memory exhaustion in the server. This patch
replaces these with a single
chan *logEntry
which is naturally bounded and supports blocking read. We make write
non-blocking using select i.e. we drop messages rather than allocate
more space.
Signed-off-by: David Scott <dave.scott@docker.com>
This is an example external logging service which can be enabled by
adding it to the `init` section of the .yml, for example:
...
init:
- linuxkit/init:35866bb276c264a5f664bfac7456f4b9eeb87a4d
- linuxkit/runc:v0.4
- linuxkit/containerd:f2bc1bda1ab18146967fa1a149800aaf14bee81b
- linuxkit/ca-certificates:v0.4
- linuxkit/memlogd:cc035e5c9e4011ec1ba97a181a6689fc90965ce9
onboot:
...
Signed-off-by: David Scott <dave.scott@docker.com>
If external logging is enabled, this patch sets the stdout and stderr
of the `runc` invocations to one end of a socketpair and the other end is
sent to the logging service. Otherwise we log to files as before.
Signed-off-by: David Scott <dave.scott@docker.com>
An external logging system exists if the socket
/var/run/linuxkit-external-logging.sock
exists.
If an external logging system is enabled then create FIFOs for
containerd and send the other end of the FIFOs to the logging service.
Otherwise use /var/log files as before.
Signed-off-by: David Scott <dave.scott@docker.com>
These were being added to the incorrect directory.
Also move config file to /etc to be more standard.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
DNS lookups fail in qemu-user when it is built on Alpine: https://bugs.alpinelinux.org/issues/8131
Until this is resolved, we fetch the binaries from Debian and use those instead. The final stage
of the Dockerfile is still based on scratch.
We can revert this once the Alpine issue is fixed.
Signed-off-by: Justin Barrick <jbarrick@cloudflare.com>
For some reason, bind mounting does not always seem to work,
sometimes the filesystem is empty. Mounting a fresh copy seems
a better solution, and simplifies things. The container does
need `CAP_SYS_ADMIN` but only on boot.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
When busybox's reboot processing occurs in init, it runs all SHUTDOWN
actions that are defined in inittab. Once those are complete, it will
trigger either a halt, poweroff, or reboot, depending upon what signal
is received. The mechanism that's used to shell out through inittab
does not allow us to pass through exactly which invocation was
requested.
Due to the way that rc.shutdown works, it invokes the poweroff action
for any and all SHUTDOWN callbacks, whether they're a reboot, poweroff,
or halt. Instead of handling the reboot(2) syscall in rc.shutdown,
return after killing and unmounting and let busybox's init process
decide which reboot(2) action to use.
Signed-off-by: Krister Johansen <krister.johansen@oracle.com>
Since we are building containerd v1.1.0 with go 1.10 (as it requires) to the
same for init and runc too for consistency. In the case of init it is actually
required since we use the containerd client library there.
The subreaper interfaces have been removed from containerd and replaced with a
similar interface in runc/libcontainer, update init to use that now.
Signed-off-by: Ian Campbell <ijc@docker.com>
$ git diff linuxkit.yml
diff --git a/linuxkit.yml b/linuxkit.yml
index e2ec829db..21b84e4ad 100644
--- a/linuxkit.yml
+++ b/linuxkit.yml
@@ -1,6 +1,6 @@
kernel:
image: linuxkit/kernel:4.14.32
- cmdline: "console=tty0 console=ttyS0 console=ttyAMA0"
+ cmdline: "console=ttyS0 console=foobar"
init:
- linuxkit/init:v0.3
- linuxkit/runc:v0.3
$ linuxkit build linuxkit.yml
[...]
$ linuxkit run linuxkit
[...]
getty: cmdline has console=foobar but /dev/foobar is not a character device; not starting getty for foobar
linuxkit-2ae2c420a11c login: root (automatic login)
Welcome to LinuxKit!
NOTE: This system is namespaced.
The namespace you are currently in may not be the root.
(ns: getty) linuxkit-2ae2c420a11c:~# ls -l /proc/1/root/dev/foobar
-rw-r--r-- 1 root root 311 Apr 9 13:19 /proc/1/root/dev/foobar
(ns: getty) linuxkit-2ae2c420a11c:~# cat /proc/1/root/dev/foobar
Welcome to LinuxKit
## .
## ## ## ==
## ## ## ## ## ===
/"""""""""""""""""\___/ ===
{ / ===-
\______ O __/
\ \ __/
\____\_______/
Also added quotes around $tty for good measure.
Signed-off-by: Ian Campbell <ijc@docker.com>
Go commit https://github.com/golang/go/issues/23672 introduced a
whitelist ofr flags passed into gcc to prevent arbitrary code
execution (CVE-2018-6574). The x86 rngd code uses two CFLAGS
not on the whitelist. Add them to 'CGO_CFLAGS_ALLOW'.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@gmail.com>
Without the privileged flag, the tcsd daemon does not have
access to the mounted host device files, especially the tpm
device file.
Signed-off-by: Pratik Mallya <pratik.mallya@gmail.com>
We already had this increased above the defaults, but even larger values
seem to be recommended.
See https://blog.openai.com/scaling-kubernetes-to-2500-nodes/
"It’s common to tune this setting in HPC clusters, and is particularly
relevant in Kubernetes clusters since every pod has its own IP address
which consumes space in the ARP cache."
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
This package is primarily for users of custom or foreign
kernels which may have drivers enabled for which the
'firmware' package does not contain the firmware blobs.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
This new 'firmware' package contains the firmware blobs required
by the device drivers in a LinuxKit kernel. The list of required
blobs is determined by calling 'modinfo' on each module.
We also unconditionally include the AMD CPU microcode and the
licence files.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
go_vet (via https://goreportcard.com/report/github.com/linuxkit/linuxkit)
reported:
error: missing argument for Printf("%v"): format reads arg 2, have only 1 args (vet)
error: wrong number of args for format in Printf call: 2 needed but 3 args (vet)
Signed-off-by: Ian Campbell <ijc@docker.com>
This is similar to ae64ab6b82 from #2849 which
did the same for runtime.mkdir.
This makes it possible to specify both host (absolute) or container (relative)
paths.
Signed-off-by: Ian Campbell <ijc@docker.com>
This PR correctly plumbs a single context to propagate the containerd
namespace to the necessary commands. Services launched with containerd
after this change will now be in a default namespace of
`services.linuxkit`.
A top-level flag is added to the service command,
`--containerd-namespace` which can be used to change, if needed.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Use unix.Reboot from golang.org/x/sys/unix for poweroff and reboot
instead of relying on external commands.
Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
- Previously, KV lines which were commented would attempt to be set. Now any commented KV lines will also be ignored.
- Comments can start with a hash or semicolon
- Splitting KV on both period and forward slash
- Some kernels may not have certain features enabled (such as IPv6) in the default etc/sysctl.d/*.conf, and thus pkg/sysctl would only set the KV until the first failure, and then silently skip the rest of the KVs. Now any failure is logged as a WARN, and those lines can now be commented per the above change, as they will be identified.
Signed-off-by: Isaac Rodman <isaac@eyz.us>
On 4.9.x and 4.14.x kernels ebpf verifier bugs allow ebpf
programs to access (read/write) random memory. Setting
kernel.unprivileged_bpf_disabled=1 mitigates this somewhat
until it is fixed upstream.
See:
- https://lwn.net/Articles/742170
- https://lwn.net/Articles/742169
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
This was added to alpine since our package was created. Now we have upgraded we
can just use the binary.
The package contains an auditd.conf but we have a tweak local copy which writes
to stdio (which goes to /var/log/auditd.*.log already). The package doesn't
have an audit.rules so keep that here too.
Signed-off-by: Ian Campbell <ijc@docker.com>
containerd v1.0.0-rc.0 changed the subreaper config entry to the
negative given subreaper "on" is the default. However, linuxkit
does not need to change the default subreaper setting so removing
the now invalid config line.
Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
This is a `moby.ImageConfig` struct which is marshalled into JSON and added as
the `org.mobyproject.config` label on the built image.
Convert `pkg/sysctl` as PoC.
Signed-off-by: Ian Campbell <ijc@docker.com>
Implements https://github.com/moby/tool/pull/181
Design for things like Kubernetes setup that requires some cgroups to
exist when the service starts but it is not running in these, other
services are, so there would be a race if they are not created in each.
Essentially it is just a sugared `mkdir` in all the cgroup dirs.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
When waiting for devices to refresh a stat can fail with the
error message 'stat: /dev/vda1: stat: no such file or directory'.
This was observered with empty raw disks (*.img) that was being
created by the moby/hyperkit go library.
This commit restores the previous logic of refreshing with mdev
instead of returning an error.
Signed-off-by: dave protasowski <dprotaso@gmail.com>
The /sbin/ldconfig.real command in ubuntu for ppc64le cannot be executed
in docker images since the binfmt-misc configuration does not match the
executable file format. This patch is a port of
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=794737 to linuxkit
Signed-off-by: Olaf Flebbe <of@oflebbe.de>
By running:
./scripts/update-component-sha.sh --image linuxkit/alpine ad35b6ddbc70faa07e59a9d7dee7707c08122e8d
Signed-off-by: Ian Campbell <ijc@docker.com>
