Commit Graph

715 Commits

Author SHA1 Message Date
Rolf Neugebauer
4fdad53190 pkgs: Update packages to the latest linuxkit/alpine
except for bpftrace, since it does not compile:
https://github.com/linuxkit/linuxkit/pull/3230

Signed-off-by: Rolf Neugebauer <rn@rneugeba.io>
2019-03-03 12:25:40 +00:00
Ian Campbell
39ee6c0e9b Bump packages to new alpine w/ containerd v1.2.4
Signed-off-by: Ian Campbell <ijc@docker.com>
2019-02-14 10:01:33 +00:00
Ian Campbell
24c13f222c Bump runc to version from containerd v1.2.4
Signed-off-by: Ian Campbell <ijc@docker.com>
2019-02-14 10:01:33 +00:00
Ian Campbell
61ff76a7e3 Bump packages to new alpine w/ containerd v1.2.3
Signed-off-by: Ian Campbell <ijc@docker.com>
2019-02-11 09:47:37 +00:00
Ian Campbell
d56442cf2e Bump runc to version from containerd v1.2.3
Signed-off-by: Ian Campbell <ijc@docker.com>
2019-02-11 09:47:36 +00:00
Rolf Neugebauer
1d19a27e08
Merge pull request #3238 from kppk/master
Add support for named pipes to init
2019-02-08 10:04:28 +00:00
Tomas Knappek
63a6a60fed Fixes linuxkit#3237: add support for named pipes to init
Signed-off-by: Tomas Knappek <tomas.knappek@gmail.com>
2019-02-07 10:37:15 -08:00
Rolf Neugebauer
c44cf1293f
Merge pull request #3253 from johanneswuerbach/s390x-support
Add s390x support to binfmt_misc
2019-02-07 00:20:19 +00:00
Rolf Neugebauer
eeb2d546d1
Merge pull request #3249 from chriswue/master
Adding support to mount an encrypted filesystem
2019-01-18 01:28:50 +01:00
Christian Wuerdig
a9bc7372de Adding dm-crypt as core pkg
Signed-off-by: Christian Wuerdig <christian.wuerdig@gmail.com>
2019-01-18 11:56:18 +13:00
Christian Wuerdig
8cd16c4904 Adding losetup utility package to core pkg
Signed-off-by: Christian Wuerdig <christian.wuerdig@gmail.com>
2019-01-14 06:24:00 +13:00
Ian Campbell
7386a32afe pkg/runc: bump alpine base to newest one.
Corresponding to the similar changes in ed2dd46cd0.

Signed-off-by: Ian Campbell <ijc@docker.com>
2019-01-08 16:28:36 +00:00
Ian Campbell
ddd8c0c017 Bump runc to version from containerd v1.2.2
Signed-off-by: Ian Campbell <ijc@docker.com>
2019-01-08 11:52:42 +00:00
Ian Campbell
ed2dd46cd0 Bump packages to new alpine w/ containerd v1.2.2
Signed-off-by: Ian Campbell <ijc@docker.com>
2019-01-08 11:52:42 +00:00
Johannes Würbach
81d17a19c4
Add s390x support to binfmt_misc
Signed-off-by: Johannes Würbach <johannes.wuerbach@googlemail.com>
2018-12-22 22:50:37 +01:00
Simarpreet Singh
3e0686e5c2 pkg/bpftrace: Add bpftrace as a core pkg
Signed-off-by: Simarpreet Singh <simar@linux.com>
2018-12-07 14:31:43 -08:00
Ian Campbell
ce126caf3a Bump runc to version from containerd v1.2.0
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-10-30 10:53:47 +00:00
Ian Campbell
1292f7aa87 Bump packages to new alpine w/ containerd v1.2.0
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-10-30 10:53:47 +00:00
Ian Campbell
a4ca9fa674 getty&ssh: bind /var/lib/containerd
Needed for containerd v1.2.0 otherwise:

    $ ctr run -t docker.io/library/hello-world@sha256:f3b3b28a45160805bb16542c9531888519430e9e6d6ffc09d72261b0d26ff74f test
    [ 1311.667587] overlayfs: failed to resolve '/var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/snapshots/5/fs': -2
    ctr: failed to mount /tmp/containerd-mount111658703: no such file or directory

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-10-30 09:29:54 +00:00
Ian Campbell
2ddce9bdea Bump runc to version from containerd v1.1.4
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-09-25 15:55:08 +01:00
Ian Campbell
809d257502 Bump packages to new alpine w/ containerd v1.1.4
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-09-25 15:55:07 +01:00
Ian Campbell
e1c6ae1678 Bump runc to version from containerd v1.1.3
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-08-30 13:40:00 +01:00
Ian Campbell
ad556eab78 Bump packages to new alpine w/ containerd v1.1.3
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-08-30 13:40:00 +01:00
David Scott
24a6112548 pkg/host-timesync-daemon: update vendoring of linuxkit/virtsock
Signed-off-by: David Scott <dave.scott@docker.com>
2018-08-02 16:09:29 +01:00
David Scott
50199d5bd5 pkg/vsudd: update the vendoring of linuxkit/virtsock
In addition to bug fixes, this removes the special protocol used
for `shutdown` needed by old Windows builds < 14393.

Signed-off-by: David Scott <dave.scott@docker.com>
2018-08-02 16:09:24 +01:00
Rolf Neugebauer
21ff90c4c7 pkgs: Update packages to the latest linuxkit/alpine
Signed-off-by: Rolf Neugebauer <rn@rneugeba.io>
2018-07-26 20:03:09 +01:00
David Scott
ddee5f27f3 memlogd/logwrite: use the same naming convention as init
In e8786d73bb the logwrite package will
automatically append .log to every log.

In 5201049f2c the init package will send
stderr of a service `s` to a log named `s` and the stdout to `s.out`.
Therefore the files we create on disk are `s.log` and `s.out.log`.

This patch modifies the memlogd `logwrite` command-line wrapper to use
the same convention.

Note there is a confusing name clash between `pkg/logwrite` and `cmd/logwrite`
in `memlogd` modified here.

Signed-off-by: David Scott <dave.scott@docker.com>
2018-07-20 15:31:29 +01:00
Rolf Neugebauer
090f461e7f
Merge pull request #3117 from steeve/patch-1
Change GCP metadata SSH keys URL
2018-07-16 20:40:59 +01:00
Rolf Neugebauer
358f1a812a
Merge pull request #3115 from justincormack/merge-moby-tool
Merge moby tool back into LinuxKit
2018-07-16 17:52:37 +01:00
Justin Cormack
cf5cec204d
Update LinuxKit for merge of moby tool
- use the mkimage hashes that we had in LinuxKit as more up to date than tool.
- update docs
- move the code from moby under src/cmd/linuxkit

Signed-off-by: Justin Cormack <justin@specialbusservice.com>
2018-07-16 14:22:15 +01:00
Ian Campbell
eb48b1284c Bump packages to new alpine w/ containerd v1.1.2
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-07-16 10:29:41 +01:00
Steeve Morin
33b6fc77ea Change GCP metadata SSH keys URL
According to [1] the URL is not sshKeys but ssh-keys now.

[1] https://cloud.google.com/compute/docs/storing-retrieving-metadata

Signed-off-by: Steeve Morin <steeve.morin@gmail.com>
2018-07-15 01:29:31 +02:00
David Scott
637b1bdb3f pkg/kmsg: write to stderr (instead of stdout)
Part of #3111

Signed-off-by: David Scott <dave.scott@docker.com>
2018-07-13 11:48:16 +01:00
David Scott
7c243a8e8b pkg/init: only create /var/log/onboot symlink if not using memlogd
When logging directly to files (the not-using-memlogd case) the onboot
services must log to /run/log because /var/log might be overmounted
by a persistent disk. Therefore we create a symlink at the end of
the onboot section.

When logging via memlogd, all logs are buffered until a logwrite service
starts, so no symlink is needed.

Signed-off-by: David Scott <dave.scott@docker.com>
2018-07-13 11:47:43 +01:00
David Scott
5201049f2c pkg/init: write stderr to <name> rather than <name>.err
This will then be written to disk as <name>.log by logwrite.

Part of #3111

Signed-off-by: David Scott <dave.scott@docker.com>
2018-07-13 10:48:31 +01:00
Rolf Neugebauer
e8786d73bb pkg/logwrite: Append .log to the log files
Also simplify the code by directly storing the path to
the log file in the LogFile structure.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-07-11 15:55:27 +01:00
Rolf Neugebauer
b4c3f9db1f pkg/tss: Switch SF repository to our mirror hosted on github
sourceforge git access is not very reliable...

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-07-10 18:00:00 +01:00
Rolf Neugebauer
3672843dc9 pkg/node_exporter: Update to 0.16.0
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-07-10 16:53:49 +01:00
Rolf Neugebauer
1a6b6b2db9 pkg/firmware,firmware-all: Update to latest version
Update the the firmware packages to the latest commit
of the upstream linux-firmware repository.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-07-10 16:53:49 +01:00
Rolf Neugebauer
46184e258b pkg/cadvisor: Update to 0.30.2
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-07-10 16:53:49 +01:00
Rolf Neugebauer
c1e10002b0 pkgs: Update packages to latest alpine base
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-07-10 15:49:20 +01:00
Ian Campbell
18f11b2834 Bump packages to new alpine w/ containerd v1.1.1
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-07-10 09:53:11 +01:00
David Scott
e7fc5b59a3 pkg/kmsg: re-run vndr
Signed-off-by: David Scott <dave.scott@docker.com>
2018-07-09 20:36:58 +01:00
David Scott
793dd1b257 Add pkg/kmsg
This very simple program reads from `/dev/kmsg` and logs the output.

Signed-off-by: David Scott <dave.scott@docker.com>
2018-07-09 20:36:58 +01:00
David Scott
565f787bb3 Add pkg/logwrite which writes and rotates log files
This process connects to memlogd and streams logs to individual files,
one per log. It keeps track of how many bytes have been written to each
file and rotates when the file size exceeds a defined threshold.

By default the maximum size of each file before rotation is 1MiB and
we keep up to 10 files per log.

Signed-off-by: David Scott <dave.scott@docker.com>
2018-07-09 20:36:51 +01:00
Rolf Neugebauer
6045ddaf51 pkg: Use alpine 3.8 version of go (no need for go1.10)
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@gmail.com>
2018-07-07 18:24:56 +01:00
Rolf Neugebauer
a9acdd827e pkg: Update packages to latest alpine base
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@gmail.com>
2018-07-07 18:05:29 +01:00
David Scott
a07ecf60d5 memlogd: use kmsg format for reading the logs
Switch to a more formally-specified `kmsg`-style format for reading
the logs.

- update the spec in docs/logging.md
- check for bad names in pkg/memlogd with unit test

Signed-off-by: David Scott <dave.scott@docker.com>
2018-07-06 21:49:26 +01:00
David Scott
fe64f33430 memlogd: add test cases
- check writing to the log does not block
- check the log doesn't expand -- it should be finite
- check that client connections don't buffer arbitrary amounts of
  data if the client is slow

Signed-off-by: David Scott <dave.scott@docker.com>
2018-07-06 21:49:26 +01:00
David Scott
6aca715ad8 memlogd: replace an unbounded buffer with bounded channel
Previously we had a per-connection

  bytes.Buffer // to be written to the connection
  sync.Cond    // to allow us to Wait for more data

This had the major disadvantage that the buffer was unbounded and so
a slow client could cause memory exhaustion in the server. This patch
replaces these with a single

  chan *logEntry

which is naturally bounded and supports blocking read. We make write
non-blocking using select i.e. we drop messages rather than allocate
more space.

Signed-off-by: David Scott <dave.scott@docker.com>
2018-07-06 21:49:26 +01:00
David Scott
3e742018d6 memlogd: use net.Conn rather than *net.UnixConn
There's no need to be so specific about the connection implementation.

Signed-off-by: David Scott <dave.scott@docker.com>
2018-07-06 21:49:25 +01:00
David Scott
c92af038fb projects/logging: promote pkg/memlogd to a toplevel package
This is an example external logging service which can be enabled by
adding it to the `init` section of the .yml, for example:

    ...
    init:
      - linuxkit/init:35866bb276c264a5f664bfac7456f4b9eeb87a4d
      - linuxkit/runc:v0.4
      - linuxkit/containerd:f2bc1bda1ab18146967fa1a149800aaf14bee81b
      - linuxkit/ca-certificates:v0.4
      - linuxkit/memlogd:cc035e5c9e4011ec1ba97a181a6689fc90965ce9
    onboot:
    ...

Signed-off-by: David Scott <dave.scott@docker.com>
2018-07-06 21:49:25 +01:00
David Scott
f4bbce7a6c service: use the logging system with runc
If external logging is enabled, this patch sets the stdout and stderr
of the `runc` invocations to one end of a socketpair and the other end is
sent to the logging service. Otherwise we log to files as before.

Signed-off-by: David Scott <dave.scott@docker.com>
2018-07-06 21:49:21 +01:00
David Scott
4dc75bc67b service: use the logging system with containerd
An external logging system exists if the socket

  /var/run/linuxkit-external-logging.sock

exists.

If an external logging system is enabled then create FIFOs for
containerd and send the other end of the FIFOs to the logging service.
Otherwise use /var/log files as before.

Signed-off-by: David Scott <dave.scott@docker.com>
2018-07-06 21:48:57 +01:00
Brice Figureau
f7ff31382b Make sure motd contains the correct ctr command
The listed namespace wasn't correct, and thus the given `ctr` command
wasn't working.

Signed-off-by: Brice Figureau <brice@daysofwonder.com>
2018-07-04 11:32:36 +02:00
Rolf Neugebauer
9dac66b831
Merge pull request #3078 from justincormack/dhcp-hook-fix
Fix dhcpcd hooks config
2018-06-29 19:02:24 +01:00
Justin Cormack
157c73cfed
Fix dhcpcd hooks config
These were being added to the incorrect directory.

Also move config file to /etc to be more standard.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2018-06-29 16:31:44 +01:00
Ian Campbell
34666cd87d Bump packages to new alpine w/ containerd v1.1.0-rc.2
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-06-28 17:35:04 +01:00
Justin Cormack
561e9375e8
Merge pull request #3075 from justincormack/binfmt_mount
Binfmt fixes
2018-06-22 16:09:34 +01:00
Justin Barrick
14ba30f418
Workaround DNS issue with qemu-user-static on alpine by using Debian in the first stage #2710
DNS lookups fail in qemu-user when it is built on Alpine: https://bugs.alpinelinux.org/issues/8131

Until this is resolved, we fetch the binaries from Debian and use those instead. The final stage
of the Dockerfile is still based on scratch.

We can revert this once the Alpine issue is fixed.

Signed-off-by: Justin Barrick <jbarrick@cloudflare.com>
2018-06-22 14:10:50 +01:00
Justin Cormack
a6dbe8ceba
Rather than bind mounting binfmt_misc filesystem, mount a copy.
For some reason, bind mounting does not always seem to work,
sometimes the filesystem is empty. Mounting a fresh copy seems
a better solution, and simplifies things. The container does
need `CAP_SYS_ADMIN` but only on boot.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2018-06-22 14:10:43 +01:00
Ian Campbell
90f7d1bed9 Bump packages to new alpine w/ containerd v1.1.1-rc.1
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-06-19 16:01:20 +01:00
Patrik Cyvoct
5ab6b88029
Add Scaleway provider in metadata package
Signed-off-by: Patrik Cyvoct <patrik@ptrk.io>
2018-06-14 11:51:03 +02:00
Justin Barrick
9821180548 Add a restart and stop command to service.
Signed-off-by: justinbarrick <jbarrick@cloudflare.com>
2018-06-12 15:04:16 -07:00
Krister Johansen
16604bc039 Reboot should reboot instead of powering off.
When busybox's reboot processing occurs in init, it runs all SHUTDOWN
actions that are defined in inittab.  Once those are complete, it will
trigger either a halt, poweroff, or reboot, depending upon what signal
is received.  The mechanism that's used to shell out through inittab
does not allow us to pass through exactly which invocation was
requested.

Due to the way that rc.shutdown works, it invokes the poweroff action
for any and all SHUTDOWN callbacks, whether they're a reboot, poweroff,
or halt.  Instead of handling the reboot(2) syscall in rc.shutdown,
return after killing and unmounting and let busybox's init process
decide which reboot(2) action to use.

Signed-off-by: Krister Johansen <krister.johansen@oracle.com>
2018-06-11 08:29:08 -07:00
Avi Deitcher
643414f6f4 Add services.linuxkit warning to motd for sshd and getty
Signed-off-by: Avi Deitcher <avi@deitcher.net>
2018-06-06 14:32:12 +03:00
Ian Campbell
525ff1c1a8 Bump packages to new alpine w/ containerd v1.1.1-rc.0
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-06-01 14:09:05 +01:00
Rolf Neugebauer
1de059188b pkgs: Update packages to latest alpine base
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@gmail.com>
2018-05-12 13:18:03 +01:00
Ian Campbell
d563d78ac1 Bump packages to new alpine w/ containerd v1.1.0
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-04-25 09:58:31 +01:00
Robin Winkelewski
ffe9ac2319 move pkg config from Dockerfile to build.yml
Signed-off-by: Robin Winkelewski <w9ncontact@gmail.com>
2018-04-24 00:59:09 +02:00
Ian Campbell
2aea82683e containerd: disable cri plugin by default
It complains/fails if CNI is not available.

Also drop two obsolete keys.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-04-18 13:20:51 +01:00
Ian Campbell
e8e28c27e9 Bump packages to new alpine w/ containerd v1.1.0-rc.2
Since we are building containerd v1.1.0 with go 1.10 (as it requires) to the
same for init and runc too for consistency. In the case of init it is actually
required since we use the containerd client library there.

The subreaper interfaces have been removed from containerd and replaced with a
similar interface in runc/libcontainer, update init to use that now.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-04-18 11:27:03 +01:00
Ian Campbell
3301af529c Avoid writing to tty devices which don't actually exist.
$ git diff linuxkit.yml
    diff --git a/linuxkit.yml b/linuxkit.yml
    index e2ec829db..21b84e4ad 100644
    --- a/linuxkit.yml
    +++ b/linuxkit.yml
    @@ -1,6 +1,6 @@
     kernel:
       image: linuxkit/kernel:4.14.32
    -  cmdline: "console=tty0 console=ttyS0 console=ttyAMA0"
    +  cmdline: "console=ttyS0 console=foobar"
     init:
       - linuxkit/init:v0.3
       - linuxkit/runc:v0.3
    $ linuxkit  build linuxkit.yml
    [...]
    $ linuxkit run linuxkit
    [...]
    getty: cmdline has console=foobar but /dev/foobar is not a character device; not starting getty for foobar

    linuxkit-2ae2c420a11c login: root (automatic login)

    Welcome to LinuxKit!

    NOTE: This system is namespaced.
    The namespace you are currently in may not be the root.
    (ns: getty) linuxkit-2ae2c420a11c:~# ls -l /proc/1/root/dev/foobar
    -rw-r--r--    1 root     root           311 Apr  9 13:19 /proc/1/root/dev/foobar
    (ns: getty) linuxkit-2ae2c420a11c:~# cat /proc/1/root/dev/foobar

    Welcome to LinuxKit

                            ##         .
                      ## ## ##        ==
                   ## ## ## ## ##    ===
               /"""""""""""""""""\___/ ===
              {                       /  ===-
               \______ O           __/
                 \    \         __/
                  \____\_______/

Also added quotes around $tty for good measure.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-04-09 14:26:49 +01:00
Rolf Neugebauer
2d00440351 pkg: Update to latest alpine base
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-04-05 14:05:39 +01:00
Ian Campbell
62d8e43fe4 Bump runc to version from containerd v1.0.3
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-04-03 11:11:32 +01:00
Ian Campbell
98aed68d98 Bump packages to new alpine w/ containerd v1.0.3
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-04-03 11:11:32 +01:00
Rolf Neugebauer
998eaa7dad pkg: Fix cgo CFLAGS for rngd
Go commit https://github.com/golang/go/issues/23672 introduced a
whitelist ofr flags passed into gcc to prevent arbitrary code
execution (CVE-2018-6574). The x86 rngd code uses two CFLAGS
not on the whitelist. Add them to 'CGO_CFLAGS_ALLOW'.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@gmail.com>
2018-03-24 09:23:15 +00:00
Rolf Neugebauer
7459bd1546 pkg: Disable cadvisor for s390x
I get a build error:

patching file build/build.sh
>> building assets
>> building binaries
>> building cadvisor
vendor/github.com/klauspost/crc32/crc32.go:157:10: undefined: updateCastagnoli
vendor/github.com/klauspost/crc32/crc32.go:159:10: undefined: updateIEEE
vendor/github.com/klauspost/crc32/crc32.go:182:48: undefined: updateIEEE
make: *** [Makefile:40: build] Error 2

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@gmail.com>
2018-03-23 21:00:37 +00:00
Rolf Neugebauer
68fb6d0951 pkg: Update kernel used for the firmware package
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@gmail.com>
2018-03-23 20:54:42 +00:00
Rolf Neugebauer
eb9e6cda93 pkg: Update to latest alpine base
This should enable s390x support for all package

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@gmail.com>
2018-03-23 20:20:13 +00:00
Pratik Mallya
8333ea0b5e Update README to include privileged flag
Without the privileged flag, the tcsd daemon does not have
access to the mounted host device files, especially the tpm
device file.

Signed-off-by: Pratik Mallya <pratik.mallya@gmail.com>
2018-03-06 01:32:04 -06:00
Avi Deitcher
cc57288b64 Add logging to /var/log for onboot containers
Signed-off-by: Avi Deitcher <avi@deitcher.net>
2018-03-02 15:29:23 +02:00
Ian Campbell
1b07ca4d8c Bump runc to version from containerd v1.0.2
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-02-14 13:10:42 +00:00
Ian Campbell
b72df17cb3 Bump packages to new alpine w/ containerd v1.0.2
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-02-14 13:10:42 +00:00
Ian Campbell
3157ff9976 Bump packages to new alpine w/ containerd v1.0.2-rc.1
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-02-07 09:50:19 +00:00
Ian Campbell
46b0ab3d9b Bump runc to version from containerd v1.0.2-rc.0
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-01-31 09:43:25 +00:00
Ian Campbell
726f967f2d Bump packages to new alpine w/ containerd v1.0.2-rc.0
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-01-31 09:41:49 +00:00
Rolf Neugebauer
1fadf13ec4 pkg: Allow specifying addition options to 'linuxkit' invocations
This is useful for 'make OPTIONS="-release=v0.1" push'

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-01-25 13:50:28 +00:00
Rolf Neugebauer
7cf2c5abf7 pkg: Update all packages to the latest linuxkit/alpine
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-01-25 13:42:33 +00:00
Rolf Neugebauer
9b208c3772 pkg/ip: Update package with latest WireGuard tools
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-01-24 11:44:17 +00:00
Justin Cormack
d4b23a76de Increase size of ARP cache
We already had this increased above the defaults, but even larger values
seem to be recommended.

See https://blog.openai.com/scaling-kubernetes-to-2500-nodes/

"It’s common to tune this setting in HPC clusters, and is particularly
relevant in Kubernetes clusters since every pod has its own IP address
which consumes space in the ARP cache."

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2018-01-19 11:02:17 +00:00
Justin Cormack
4b9ffadb2e Add support for runtime containerd namespace configuration
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2018-01-18 12:40:56 +00:00
Ian Campbell
a811110ad8 Bump packages to new alpine w/ containerd v1.0.1
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-01-18 10:59:41 +00:00
Rolf Neugebauer
aa2791a6dd pkg/firmware-all: Add package containing all Linux Firmware blobs
This package is primarily for users of custom or foreign
kernels which may have drivers enabled for which the
'firmware' package does not contain the firmware blobs.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-01-16 19:08:19 +00:00
Rolf Neugebauer
1d9c4361c0 pkg/firmware: Add a package with firmware required by enabled device drivers
This new 'firmware' package contains the firmware blobs required
by the device drivers in a LinuxKit kernel. The list of required
blobs is determined by calling 'modinfo' on each module.

We also unconditionally include the AMD CPU microcode and the
licence files.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-01-16 19:08:18 +00:00
Avi Deitcher
c0ff9cf6f0
Merge pull request #2864 from deitch/fix-metadata-label
Add /run to binds and use newer yml label format
2018-01-16 11:36:49 +02:00
Avi Deitcher
0d4b2742e2 Add /run to binds and use newer yml label format
Signed-off-by: Avi Deitcher <avi@deitcher.net>
2018-01-15 19:51:35 +02:00
Ian Campbell
779be8cff1 mount: Add missing argument to Printf
go_vet (via https://goreportcard.com/report/github.com/linuxkit/linuxkit)
reported:

    error: missing argument for Printf("%v"): format reads arg 2, have only 1 args (vet)
    error: wrong number of args for format in Printf call: 2 needed but 3 args (vet)

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-01-15 17:24:36 +00:00
Ian Campbell
826ee28c8d metadata: print Entry objects with %+v not %s
go_vet (via https://goreportcard.com/report/github.com/linuxkit/linuxkit)
reported:

    error: arg current for printf verb %s of wrong type: main.Entry (vet)

For each of these.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-01-15 17:24:36 +00:00
Ian Campbell
a4230b74aa init: print integer with %d not %s
go_vet (via https://goreportcard.com/report/github.com/linuxkit/linuxkit)
reported:

    error: arg resource for printf verb %s of wrong type: int (vet)

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-01-15 17:24:36 +00:00
Avi Deitcher
2af908d1b7 Move metadata to /run instead of /var
Signed-off-by: Avi Deitcher <avi@deitcher.net>
2018-01-15 14:23:03 +02:00
Ian Campbell
e0b871560a Bump to runc 7f24b40cc5423 as used by containerd v1.0.1-rc.0
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-01-12 10:51:34 +00:00
Ian Campbell
913e6420ba Bump packages to new alpine w/ containerd v1.0.1-rc.0
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-01-12 10:51:01 +00:00
Ian Campbell
965c678f9c
Merge pull request #2845 from ijc/example-hostmount-writeable-overlay
Add an example of setting up a writeable overlay of a host path
2018-01-11 16:03:15 +00:00
Rolf Neugebauer
e3284d50e0
Merge pull request #2847 from w9n/nvme
format: fix for nvme
2018-01-11 14:21:03 +00:00
Ian Campbell
794ea7ed72 Make runtime.mounts[].destination relative to container rootfs
This is similar to ae64ab6b82 from #2849 which
did the same for runtime.mkdir.

This makes it possible to specify both host (absolute) or container (relative)
paths.

Signed-off-by: Ian Campbell <ijc@docker.com>
2018-01-11 12:58:09 +00:00
Robin Winkelewski
bd20f419bf format: fix for nvme
Signed-off-by: Robin Winkelewski <w9ncontact@gmail.com>
2018-01-11 11:43:53 +01:00
Justin Cormack
ae64ab6b82 Make relative paths in runtime mkdir be relative to container rootfs
Currently all paths were treated as absolute paths.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2018-01-10 14:58:49 +00:00
Justin Cormack
a4a332f39a Use logrus not log for logging consistently
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2018-01-10 14:28:50 +00:00
Stephen J Day
fbbab9eafd
pkg/init/cmd/service: plumb containerd namespace
This PR correctly plumbs a single context to propagate the containerd
namespace to the necessary commands. Services launched with containerd
after this change will now be in a default namespace of
`services.linuxkit`.

A top-level flag is added to the service command,
`--containerd-namespace` which can be used to change, if needed.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2018-01-05 14:45:19 -08:00
Tobias Klauser
642c6f23ac Use pure Go unix.Reboot function in rc.init
Use unix.Reboot from golang.org/x/sys/unix for poweroff and reboot
instead of relying on external commands.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2018-01-04 13:16:28 +01:00
isaac.rodman
6ac2ab8924 pkg/sysctl fixes: support commented KV lines, no post-crit KV set skips, support adding sysctl .conf files
- Previously, KV lines which were commented would attempt to be set. Now any commented KV lines will also be ignored.
- Comments can start with a hash or semicolon
- Splitting KV on both period and forward slash
- Some kernels may not have certain features enabled (such as IPv6) in the default etc/sysctl.d/*.conf, and thus pkg/sysctl would only set the KV until the first failure, and then silently skip the rest of the KVs. Now any failure is logged as a WARN, and those lines can now be commented per the above change, as they will be identified.

Signed-off-by: Isaac Rodman <isaac@eyz.us>
2018-01-02 07:33:13 -07:00
Rolf Neugebauer
720fb219ce pkg/sysctl: Prevent ebpf privilege escalation
On 4.9.x and 4.14.x kernels ebpf verifier bugs allow ebpf
programs to access (read/write) random memory. Setting
kernel.unprivileged_bpf_disabled=1 mitigates this somewhat
until it is fixed upstream.

See:
- https://lwn.net/Articles/742170
- https://lwn.net/Articles/742169

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-12-25 15:32:57 +01:00
Ian Campbell
abf0a5d098 auditd: Use package from alpine 3.7
This was added to alpine since our package was created. Now we have upgraded we
can just use the binary.

The package contains an auditd.conf but we have a tweak local copy which writes
to stdio (which goes to /var/log/auditd.*.log already). The package doesn't
have an audit.rules so keep that here too.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-12-15 11:07:55 +00:00
Ian Campbell
de242facca auditd: move config into build.yml
Signed-off-by: Ian Campbell <ijc@docker.com>
2017-12-15 10:16:37 +00:00
Rolf Neugebauer
fa51c915ba pkg: Update all package to the alpine:3.7 base
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-12-08 16:25:13 +00:00
Rolf Neugebauer
a41dd90a64 pkg/qemu-ga: Fix build issue with alpine:3.7
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-12-08 16:25:13 +00:00
Rolf Neugebauer
84b8bce809 pkg/audit: Fix build issue with alpine:3.7
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-12-08 16:25:13 +00:00
Ian Campbell
2561827100 Bump packages to new alpine w/ containerd v1.0.0
Signed-off-by: Ian Campbell <ijc@docker.com>
2017-12-05 09:55:32 +00:00
Phil Estes
42014e361f
remove subreaper config item
containerd v1.0.0-rc.0 changed the subreaper config entry to the
negative given subreaper "on" is the default. However, linuxkit
does not need to change the default subreaper setting so removing
the now invalid config line.

Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
2017-12-04 09:23:22 -05:00
Ian Campbell
28e22e40bb Bump packages to new alpine w/ containerd v1.0.0-rc.0
Minor updates due to renamings in containerd.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-12-01 13:36:16 +00:00
Ian Campbell
1f6c1a59ca linuxkit pkg: Add config field to build.yml
This is a `moby.ImageConfig` struct which is marshalled into JSON and added as
the `org.mobyproject.config` label on the built image.

Convert `pkg/sysctl` as PoC.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-11-28 12:01:39 +00:00
Marco Mariani
959b6dd96d pkg/cadvisor
Signed-off-by: Marco Mariani <marco.mariani@alterway.fr>
2017-11-21 13:46:42 +01:00
Justin Cormack
83522d81fd
Merge pull request #2761 from justincormack/restore-build
Restore linuxkit build
2017-11-21 10:21:39 +00:00
Justin Cormack
f8e352d375 Replace moby build with linuxkit build throughout
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-11-20 17:06:54 +00:00
Marco Mariani
01d0a1835c pkg/node_exporter from sources
Signed-off-by: Marco Mariani <marco.mariani@alterway.fr>
2017-11-20 12:13:26 +01:00
Rolf Neugebauer
0a2db0ac83
Merge pull request #2758 from rn/no-lcow
Remove LCOW
2017-11-17 17:23:34 +00:00
Justin Cormack
cad6527033
Merge pull request #2755 from justincormack/runtime-cgroups
Add support for creating cgroups in runtime section
2017-11-17 17:01:27 +00:00
Rolf Neugebauer
32fca2954f Remove LCOW
It is now under https://github.com/linuxkit/lcow

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-11-17 15:55:19 +00:00
Justin Cormack
6cb919b489 Add support for creating cgroups in runtime section
Implements https://github.com/moby/tool/pull/181

Design for things like Kubernetes setup that requires some cgroups to
exist when the service starts but it is not running in these, other
services are, so there would be a race if they are not created in each.

Essentially it is just a sugared `mkdir` in all the cgroup dirs.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-11-17 14:12:41 +00:00
dave
86f254850d After partitioning we now retry the stat prior to formatting
When waiting for devices to refresh a stat can fail with the
error message 'stat: /dev/vda1: stat: no such file or directory'.

This was observered with empty raw disks (*.img) that was being
created by the moby/hyperkit go library.

This commit restores the previous logic of refreshing with mdev
instead of returning an error.

Signed-off-by: dave protasowski <dprotaso@gmail.com>
2017-11-16 09:36:36 -05:00
Rolf Neugebauer
e145ef02b3 pkg/lcow: Update OpenGCS to latest
While at it might as well bump alpine base

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-11-15 10:57:55 +00:00
Ian Campbell
ca02060a4c Bump runc to 74a17296470088de3805e138d3d87c62e613dfc4
As recommended by containerd v1.0.0-beta.3

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-11-15 09:28:44 +00:00
Rolf Neugebauer
879ea03277
Merge pull request #2703 from guillaumerose/multi
metadata: handle json with more than 2 levels
2017-11-14 13:39:27 +00:00
Marco Mariani
d87e4f622b node_exporter 0.15.1; updated parameters
Signed-off-by: Marco Mariani <marco.mariani@alterway.fr>
2017-11-14 11:37:02 +01:00
Guillaume Rose
06e86154b6 metadata: handle json with more than 2 levels
Signed-off-by: Guillaume Rose <guillaume.rose@docker.com>
2017-11-13 14:35:17 +01:00
Ian Campbell
61ca751840 Bump packages to new alpine w/ containerd v1.0.0-beta.3
Signed-off-by: Ian Campbell <ijc@docker.com>
2017-11-09 10:16:33 +00:00
Rolf Neugebauer
7c5f797935 pkg: Update getty/ip/sshd package to pick up new wireguard
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-11-03 15:11:06 +00:00
Olaf Flebbe
6ec8e895a7 Support static ppc64le executables in qemu
The /sbin/ldconfig.real command in ubuntu for ppc64le cannot be executed
in docker images since the binfmt-misc configuration does not match the
executable file format. This patch is a port of
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=794737 to linuxkit

Signed-off-by: Olaf Flebbe <of@oflebbe.de>
2017-10-28 21:09:38 +02:00
Isaac Rodman
0ff1f84918 pkg/ip: added iptables, ebtables, ipvsadm, and bridge-utils
Signed-off-by: Isaac Rodman <isaac@eyz.us>
2017-10-24 16:45:38 -07:00
Ian Campbell
6c7b1c0251 Bump all packages to newest alpine.
Signed-off-by: Ian Campbell <ijc@docker.com>
2017-10-24 10:11:24 +01:00
Nick Jones
5663112822
Fix typo which broke metadata retrieval
Signed-off-by: Nick Jones <nick@dischord.org>
2017-10-18 22:47:03 +01:00
Ian Campbell
9facf7b585 ip: Update to newest alpine.
Rebuild to pickup newer go also picked up newer wireguard-tools,
0.0.20171005-r0 → 0.0.20171011-r0.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-10-12 16:51:37 +01:00
Ian Campbell
29fc12607f Update various other go base pkgs to newer alpine (go1.9)
These are those which set $GOPATH but don't use go-compile.sh.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-10-12 16:50:47 +01:00
Ian Campbell
a9b03c3f95 Update all users of go-compile.sh to new alpine (go1.9)
Signed-off-by: Ian Campbell <ijc@docker.com>
2017-10-12 16:47:35 +01:00
Ian Campbell
aba82533a0 Update go system components to new alpine (go1.9)
Signed-off-by: Ian Campbell <ijc@docker.com>
2017-10-12 16:45:01 +01:00
Ian Campbell
2875ef0e82 runc: Update to version used by containerd v1.0.0-beta.2
Signed-off-by: Ian Campbell <ijc@docker.com>
2017-10-12 13:01:06 +01:00
Ian Campbell
aeed5f6b40 init,containerd: bump to new alpine with v1.0.0-beta.2.
Signed-off-by: Ian Campbell <ijc@docker.com>
2017-10-12 12:57:08 +01:00
Ian Campbell
a237b51511 Bump alpine base across the board.
By running:

    ./scripts/update-component-sha.sh --image linuxkit/alpine ad35b6ddbc70faa07e59a9d7dee7707c08122e8d

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-10-10 09:55:45 +01:00
Ian Campbell
13d11f49ba Rename "tag*" Makefile targets to "build*"
This better matches the `linuxkit pkg build` nomenclature.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-10-09 16:48:48 +01:00
Ian Campbell
9bb87c0091 Remove unused pkg/package.mk
All uses now converted to `linuxkit pkg`

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-10-09 16:40:33 +01:00
Ian Campbell
fc3e659f0e Convert all of pkg/* to linuxkit pkg build
This was done with the following "script":

    git rm pkg/{auditd,binfmt,init}/Makefile
    sed -e 's/IMAGE=/image: /g' -i pkg/*/Makefile
    sed -e 's/NETWORK=1/network: true/g' -i pkg/*/Makefile
    sed -e 's/ARCHES=x86_64/arches:\n  - amd64/g' -i pkg/*/Makefile
    sed -e '/DEPS:\?=/d' -i pkg/*/Makefile
    sed -e '/ARCHES=SKIP/d' -i pkg/node_exporter/Makefile
    sed -e 's/include \.\.\/package.mk//g' -i pkg/*/Makefile
    sed -e '/^$/d' -i pkg/*/Makefile
    git mv pkg/node_exporter/Makefile pkg/node_exporter/build.yml-skip
    for i in pkg/*/Makefile ; do git mv $i ${i%Makefile}build.yml ; done

and manual update of pkg/Makefile.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-10-09 16:20:26 +01:00
Ian Campbell
ca70b7b6af Update pkg/Makefile and test/pkg/Makefile based on tools/Makefile
Using explicit targets is cleaner and easier to follow.

Also add `forcepush` to pkg and test/pkg and make `push` explicit.

The `dirs` phony was non-existent in all three cases. Likewise `$(DIR)` wasn't
used in that way in pkg or test/pkg and is nolonger used in that way in tools.

Finally, `clean` was bogus in all cases (tools cleaned a thing which never
exists and pkg and test/pkg were a syntax error).

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-10-09 16:20:08 +01:00
Justin Cormack
d0882b9674 Merge pull request #2571 from ijc/linuxkit-pkg
Implement package build, push and show-tag in linuxkit tool
2017-10-09 13:59:02 +01:00
Nick Jones
6deabe8a2f
[OpenStack] Metadata service support
This commit adds support for retrieving instance metadata on OpenStack
public clouds.

Signed-off-by: Nick Jones <nick@dischord.org>
2017-10-08 17:08:00 +01:00
Ian Campbell
ba60937754 Implement package build, push and show-tag in linuxkit tool
This implements the proposal in #2564 and converts a handful of representative
or especially interesting (from a build PoV) packages to use it.

For now those pkg/* affected get a stub-`Makefile`, once all packages are
converted then `pkg/Makefile` can be adjusted and those stubs can be removed.

For now only `pkg/package.mk`'s functionality is implemented. In particular:

- `push-manifest.sh` remains a separate script, to enable calling it on systems
  with just the LinuxKit tools installed arrange to install it under a less
  generic name.
- `kernel` and `tools/alpine` do not use `pkg/package.mk` and those cases are
  not yet fully considered/covered.

I have updated the documentation assuming that the existing uses of
`pkg/package.mk` will be removed quite soon in a follow up PR rather than
trying to document the situation which results after just this commit.

Due to `cmd/linuxkit` now gaining a library the build needs adjusting slightly to
allow both `make bin/linuxkit` and `go build` to work.

`go vet` has forced me to write some rather asinine comments for things that
are rather obvious from the name.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-10-06 12:59:25 +01:00
Ian Campbell
f8b80b6f00 pkg: correct tagging forced release
Signed-off-by: Ian Campbell <ijc@docker.com>
2017-10-06 12:56:33 +01:00
Rolf Neugebauer
d8283fc769 pkg: Update alpine base for getty/ip/sshd
These packages include the wireguard tools which need to be
updated due to the wireguard kernel bump in:
43db718f14 ("wireguard: version bump").

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-10-06 11:16:10 +01:00
Isaac Rodman
3de78f2109 format: blkid content check, format argument, and verbose argument
- Added new argument -force to the format utility. This will force formatting of the specified single device if it exists and is a block device
- By default, the format package will no longer format the specified single device if content exists on that device
- Added new blkid-based check for content, in addition to the existing check for partitions on autoformat candidate devices.
- Cleanup of old unused code.
- Refactoring of block device verification.
- Added test/cases/040_packages/006_format_mount/005_by_device_force
- Added document of new arguments to /docs/external-disk.md. Also sorted the arguments in the doc alphabetically and added them as bullets so they do not run together on the page.

Signed-off-by: Isaac Rodman <isaac@eyz.us>
2017-10-02 11:53:17 -07:00
Ian Campbell
2582614135 pkg/mount: also create /dev/disk/by-uuid links
Some software (such as cri-containerd and kubelet) rely on the presence of
these links, which are usually created by udev.

Inspired by 74b145b5c7/sourceroot/functions.sh (L93...L124)
but rewritten in go.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-09-27 16:39:58 +01:00
Rolf Neugebauer
5c37a154da Merge pull request #2536 from rn/lcow-kern
Update kernel and LCOW
2017-09-25 15:27:15 +01:00
Rolf Neugebauer
02f0049172 pkg/init-lcow: Update OpenGCS to latest version
While at it, might as well update alpine base

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-09-25 13:44:40 +01:00
Ian Campbell
09c89a923b Bump init and containerd package to v1.0.0-beta.1
golint on pkg/init now complains:

    golint...
    ./init.go:199:2: redundant if ...; err != nil check, just return error instead.

Resulting in a change which doesn't seem like an improvement to me.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-09-25 13:39:21 +01:00
Justin Cormack
b52fcb71a1 Add swap to the Docker for Mac blueprint
We always had 1G swap to work better with small memory setups, but this
was omitted in the update to LinuxKit.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-09-21 14:50:27 -07:00
Rolf Neugebauer
cc200d296a Merge pull request #2527 from eyz/linuxkit-dhcpcd-CAP_SYS_ADMIN
added capability to pkg/dhcpcd for sethostname
2017-09-21 20:53:16 +01:00
Isaac Rodman
0add33e288 added capability to pkg/dhcpcd for sethostname
Signed-off-by: Isaac Rodman <isaac@eyz.us>
2017-09-20 15:11:43 -07:00
Justin Cormack
498086dbb9 Update containerd to use pre-built version
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-09-20 14:24:10 -07:00
Rolf Neugebauer
d9ffc57532 pkg: Update the getty, ip, and sshd package to latest alpine base
These packages use the wireguard tools which have been updated
with the previous commit.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-09-20 11:46:48 +01:00
Justin Cormack
042cd763fe Add vendoring for rc.init
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-09-18 11:46:00 -07:00
Justin Cormack
bcfb760a1d Convert rc.init and rc.shutdown to Go
This removes more shell scripts to improve maintainability.

This now also works correctly in userspace, so it can be used for
running LinuxKit images in Docker and other such use cases.

It is a literal conversion of the shell scripts with a few small
tweaks.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-09-18 11:45:59 -07:00
Isaac Rodman
38452aa742 added poweroff.sh for pkg/open-vm-tools to allow shutdown from hypervisor
Signed-off-by: Isaac Rodman <isaac@eyz.us>
2017-09-16 10:39:23 -07:00
Rolf Neugebauer
6845f602dc Merge pull request #2506 from eyz/master
new pkg/modprobe from linuxkit/alpine base onto scratch
2017-09-16 17:21:15 +01:00
Rolf Neugebauer
d38ea16f4a Merge pull request #2513 from rn/lcow-kernel
Update 4.12.13 kernel patches and use it for LCOW
2017-09-15 21:41:44 +01:00
Isaac Rodman
0cce316caa pkg/modprobe from linuxkit/alpine base, using established multistage build onto scratch pattern
Signed-off-by: Isaac Rodman <isaac@eyz.us>
2017-09-15 12:38:40 -07:00
Justin Cormack
9d55b197c0 Merge pull request #2483 from zx2c4/custom-wg-script
wireguard: use custom wg-quick
2017-09-15 09:51:54 -07:00
Rolf Neugebauer
adb92ca672 pkg/init-lcow: Update to the latest version
This includes fixes to the intermittent on pull as well as
other fixes.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-09-15 14:47:12 +01:00
Jason A. Donenfeld
8f3847e985 wireguard: use custom wg-quick
This removes things we do not need and expects the interface to
already be created.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2017-09-15 01:01:55 +02:00
David Scott
9c35dbaac2 service: log stdout and stderr to separate files
Previously we would pass the path `/var/log/service.log` for both
stdout and stderr to containerd. containerd would construct a dict
with the paths as keys[1] and, due to the duplicate key, would only
open one of the files and start one `io.Copy` instance. Writes to
the other stream would be buffered by the pipe connected to
containerd-shim and would eventually block.

If we modified containerd to open the file twice and start 2
`io.Copy` instances, we would end up with the two streams interleaved
together. It seems cleaner to keep the streams separate; therefore
this patch logs stdout to `/var/log/service.out.log` and stderr to
`/var/log/service.err.log`.

[1]
49437711c3/linux/shim/io.go (L51)

Signed-off-by: David Scott <dave.scott@docker.com>
2017-09-14 11:15:04 +01:00
Rolf Neugebauer
36c84004a2 pkg/init-lcow: Update to latest version of OpenGCS
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-09-12 16:59:07 +01:00
Ian Campbell
6cbc3ed518 Bump init+containerd to new alpine base with containerd beta.0
Signed-off-by: Ian Campbell <ijc@docker.com>
2017-09-07 18:31:57 +01:00
Rolf Neugebauer
3ebec6f3e8 pkg/init-lcow: Update to latest OpenGCS version
While at it also update to latest alpine base

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-09-06 08:54:14 +01:00
Rolf Neugebauer
2cb0612011 pkg/lcow: Update OpenGCS to latest
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-30 19:41:02 +01:00
Ian Campbell
b0b08e18a2 Bump init and containerd packages to v1.0.0-alpha6
Signed-off-by: Ian Campbell <ijc@docker.com>
2017-08-29 01:02:20 +01:00
Justin Cormack
5773dfd844 Cleanup cleanup code
Only remove main mount at rootfs, as we do not really know
about the state of the rest.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-23 16:40:03 +01:00
Justin Cormack
49cf74f4f6 Switch to having the moby tool specify how to do mounts
This removes all the code that had knowledge of how to do read only
and read write container mounts, and just uses the runtime config.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-23 16:40:03 +01:00
Justin Cormack
c677b391fc Add support for mount in runtime config
This could be used in LinuxKit now, as there are some examples, eg
https://github.com/linuxkit/linuxkit/blob/master/blueprints/docker-for-mac/base.yml#L33
which are creating containers to do a mount.

The main reason though is to in future change the ad hoc code that generates
overlay mounts for writeable containers with a runtime config which does
the same thing; this code needs to create both tmpfs and overlay mounts.

See https://github.com/moby/tool/pull/145

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-23 16:40:03 +01:00
Justin Cormack
2129d1de21 Merge pull request #2450 from justincormack/improve-wireguard-example
Improve wireguard example
2017-08-22 17:44:01 +01:00
Justin Cormack
584613cddb Update ip package to latest Alpine base
There was a wireguard mismatch with the kernel module in the recent kernel
builds.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-22 11:50:20 +01:00
David Scott
da86df1ad6 init: add /etc/ssl/certs as required by the metadata package
The metadata package has binds
  - /dev
  - /var
  - /sys
  - /etc/resolv.conf
  - /etc/ssl/certs

but unfortunately `/etc/ssl/certs` doesn't exist and this causes the
following commands:

  cd blueprints/docker-for-mac # easy example
  moby build -name docker-for-mac base.yml docker-17.06-ce.yml
  linuxkit run hyperkit -networking=vpnkit -vsock-ports=2376 -disk size=500M docker-for-mac

to produce the following error on the VM console:

  container_linux.go:265: starting container process caused "process_linux.go:348: container init caused \"rootfs_linux.go:57: mounting \\\"/etc/ssl/certs\\\" to rootfs \\\"/containers/onboot/000-metadata/rootfs\\\" at \\\"/etc/ssl/certs\\\" caused \\\"stat /etc/ssl/certs: no such file or directory\\\"\""
  2017/08/21 16:39:40 Error creating 000-metadata: exit status 1

This patch creates /etc/ssl/certs in the `init` package. The metadata package
will now say things like

  2017/08/21 16:44:39 No metadata/userdata found. Bye

Signed-off-by: David Scott <dave.scott@docker.com>
2017-08-21 17:46:29 +01:00
Justin Cormack
90f905a799 Move containerd clone to tools/alpine
As discussed before, as we use this in three places, cloning in
base makes more sense.

Update base image.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-21 15:16:09 +01:00
Justin Cormack
a808808ce1 Add vendoring needed for netlink, packet
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-19 19:25:21 +01:00
Justin Cormack
92451cf2e4 Add packet metadata support, with bonding
Use the packet metadata to set up bonding, also get ssh keys and hostname.

This does not yet do anything with disk metadata.

Userdata is not used if it has been used for ipxe, but is otherwise available.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-19 19:25:14 +01:00
Rolf Neugebauer
5898bc9f0d kernel: Update to 4.9.44/4.4.83
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-18 18:08:32 +01:00
Justin Cormack
9506d45d49 Merge pull request #2413 from justincormack/runtime-config
Add a runtime config
2017-08-18 17:45:10 +01:00
Justin Cormack
9d8f1bd900 Allow specifying a set of metadata providers
run with `cmd: ["metadata", "aws"]` to just check for AWS metadata for example.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-17 14:50:50 +01:00
Justin Cormack
57512b3ec9 Add an ip package
This contans the upstream (non busybox) version of `ip` and also the
Wireguard tools as these aere not yet integrated into `ip`. It is designed
to be used for network configuration.

For example a Wireguard interface can be setup with something like:
```
  - name: wg
    image: linuxkit/ip
    net: new
    binds:
      - /etc/wireguard:/etc/wireguard
    command: ["sh", "-c", "ip link set dev wg0 up; ip address add dev wg0 192.168.2.1 peer 192.168.2.2; wg setconf wg0 /etc/wireguard/wg0.conf; wg show wg0"]
    runtime:
      interfaces:
        - name: wg0
          add: wireguard
          createInRoot: true
      bindNS:
          net: /run/netns/wg
```

This binds the net namespace so other containers can use it later.

Note that `ip` supports batch scripts with `ip -b file` which is quite convenient for
this use case.

Alas `wg-quick` requires `bash` so this is currently a bit bloated, maybe @zx2c4
might be persuaded to rewrite it without bashisms...

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-17 12:18:38 +01:00
Justin Cormack
c82b2fe13e Add vendoring for netlink
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-17 12:10:28 +01:00
Justin Cormack
0c81ce19e8 Add a runtime config
This adds support for a runtime configuration file that can do:
- `mkdir` to make a directory at runtime, eg in `/var` or `/tmp`, to avoid workarounds
- `interface` that can create network interfaces in a container or move them
- `bindNS` that can bind mount namespaces of an `onboot` container to a file so a service can be started in that namespace.

It merges the `service` and `onboot` tools (in `init`) to avoid duplication. This also saves some size for
eg LCOW which did not use the `onboot` code in `runc`.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-17 12:10:28 +01:00
Justin Cormack
df16f6fb9e Move init command into a subdirectory
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-17 12:10:28 +01:00
Rolf Neugebauer
f9f8b5bc42 pkg/lcow-init: Update to latest version of OpenGCS
While at it also update to latest alpine base

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-16 16:50:34 +01:00
Rolf Neugebauer
714dbcf0fe Merge pull request #2399 from ijc/pkg-honour-hash-override
package: Do not add an arch suffix if an explicit HASH is given
2017-08-14 10:14:04 +01:00
Ian Campbell
e11c1c0224 pkg: Bump to containerd v1.0.0-alpha4 and runc v1.0.0-rc4
Signed-off-by: Ian Campbell <ijc@docker.com>
2017-08-11 16:16:33 +01:00
Ian Campbell
0469901778 package: Do not add an arch suffix if an explicit HASH is given
During development I tend to do:
	make -C pkg/foo tag HASH=dev ORG=ijc
and I expect to get an image `ijc/foo:dev` and not `ijc/foo:dev-amd64`. In
general I think if the HASH has been explicitly provided the build should
just honour that.

To compensate and avoid mistaken pushes adjust the check for dirtiness to also
insist on a non-empty suffix.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-08-11 11:26:00 +01:00
Justin Cormack
c42ddc0af8 Fix root directory permissions with overlay
At least when running under Docker this seems to cause some issues,
possibly also explains some other oddnesses.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-10 14:22:12 +01:00
Rolf Neugebauer
46d1e568cc pkg/init-lcow: Update opengcs
This has a number of fixes/improvements
- console resize
- better error handling (pass stacktrace/errors back to docker
- fix for container images which have a resolve.conf file

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-08 17:53:27 +01:00
Justin Cormack
e61794a344 Merge pull request #2365 from deitch/test-dev-existence-getty
Test dev existence getty
2017-08-07 20:32:39 +01:00
Justin Cormack
76bb43027b Do an extra chmod after chown
Chown clears suid bits even for root on Linux.

Also move a few functions to x/sys/unix from syscall, to be
more arm64 friendly.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-07 14:49:43 +01:00
Ian Campbell
6ef10ae33b Update to containerd v1.0.0-alpha3 and runc master
Runc is master as of this morning, slightly newer than what is vendored into
containerd v1.0.0-alpha3 but the differences[0] seemed benign and/or good to have.

Signed-off-by: Ian Campbell <ijc@docker.com>

[0] 429a538712...45bde006ca
2017-08-07 11:47:44 +01:00
Avi Deitcher
cc174b4df0 test device exists before allowing getty to run
Signed-off-by: Avi Deitcher <avi@deitcher.net>
2017-08-06 10:23:12 +03:00
Rolf Neugebauer
dfa684c78f pkg/init-lcow: Update to tip of opengcs
Also adjust the building of the rootfs as this seem to have
changed.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-04 18:34:56 +01:00
Justin Cormack
0b2e34301b Merge pull request #2357 from justincormack/arm64-rngd
Support rngd on arm64
2017-08-04 16:13:46 +01:00
Justin Cormack
8b2327b0e2 Support rngd on arm64
Although it does not do anything, as there is no CPU rng on arm64 at present.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-04 14:01:18 +01:00
Justin Cormack
ce2347eda6 Enable sysctl kernel.dmesg_restrict
This requires that users have `CAP_SYSLOG` in order to access `dmesg`.
This means that containers by default have no access to `dmesg` (which
can leak information about the host or other containers) unless they
have this capability added.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-04 12:48:27 +01:00
Rolf Neugebauer
fee2a62f26 pkg: Disable some packages for arm64
They either won't work or the packages are specific
to x86_64 like the DfM or VMware packages.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-03 17:46:11 +01:00
Rolf Neugebauer
e44783fea1 pkg: Update alpine base for all packages
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-03 17:46:10 +01:00
Rolf Neugebauer
0ebb53c491 build: Add the ability to only build/push/sign for specific architectures
A package makefile can specify a list of architectures ('ARCHES')
it supports. If build on a unsupported architecture, a message is
printed and make exits without an error.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-03 17:36:22 +01:00
Rolf Neugebauer
981def7c7a build: Add show-tag and forcetag targets to pkg Makefile
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-03 17:36:22 +01:00
Rolf Neugebauer
c8e0d2ca28 pkg: Add invocation to push-manifest to generic package build
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-03 17:36:22 +01:00
Rolf Neugebauer
47af521e14 Merge pull request #2343 from justincormack/x-sys-unix-rngd
rngd improvements
2017-08-03 15:45:40 +01:00
Justin Cormack
4434ecc4af Support multiple CDROMS in metadata
This checks if any CDROM in the system has metadata. Useful if
you boot off CDROM and also have a metadata CDROM.

Also switch the metadata container to read only, as it is only
writing to bind mounted in directories, and /tmp.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-03 14:16:02 +01:00
Justin Cormack
f768ac4bfd Add flags to disable the different random sources
This makes testing the different options much easier.

Signed-off-by: Justin Cormack <justin@specialbusservice.com>
2017-08-02 20:55:36 +01:00
Justin Cormack
63798997f6 Convert rngd to use x/sys/unix
Epoll in `syscall` package does not work on arm64. We do not have
arm64 yet here, but we will when we add tpm support etc.

Signed-off-by: Justin Cormack <justin@specialbusservice.com>
2017-08-02 20:09:43 +01:00
Rolf Neugebauer
767ac54971 pkg/init-lcow: Update opengcs to the latest version
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-01 17:06:12 +01:00
Justin Cormack
4daf008956 Merge pull request #2331 from justincormack/rng-golang
Replace rngd with a Go version
2017-08-01 10:56:10 +01:00
Justin Cormack
b1c80b54b3 Revert #2317 remove kernel.random.write_wakeup_threshold from sysctl
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-31 23:05:46 +01:00
Justin Cormack
737fbcb3b6 Add standard directories in /var
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-31 21:09:48 +01:00
Justin Cormack
e0bc13451f Replace rngd with a Go version
Only supports the use cases we currently need, currently support
for using Intel hardware rng to initialise and add entropy.

Supports oneshot and service mode. Call as `rngd -1` for one shot
mode.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-31 20:16:26 +01:00
Justin Cormack
1412cf8835 Fix mode of created directories in mountie
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-31 17:31:40 +01:00
Justin Cormack
2c20ee2f8a Use a tmpfs for /var
Previously we were cheating and remounting /var `rw` but this does not
work if the filesystem is really read only. Nount a tmpfs, which may
be overmounted later by a persistent filesystem.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-31 15:25:54 +01:00
Justin Cormack
5ea2eaead1 Merge pull request #2309 from justincormack/ro-rw-setup
Use overlay for writeable containers
2017-07-31 15:25:09 +01:00
Justin Cormack
eacce1d52e Use overlay for writeable containers
Previously we would sneakily remount as `rw` but of course you can't
really do that on a truly immutable filesystem.

See https://github.com/moby/tool/pull/129 for the `moby` side.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-31 13:48:23 +01:00
Dennis Chen
bbfb7a7442 pkg/getty: Add ARM 'ttyAMA0' console into securetty
We need to add 'ttyAMA0' console used on ARM64 platform into
securetty file to make it's possible to log into the system
as root. Also it will dismiss the below warning message before
login:
"getty: cmdline has console=ttyAMA0 but does not exist in
/etc/securetty; will not be able to log in as root on this tty ttyAMA0."

'ttyAMA1' is also added order to handle some system happens have that one
instead of 'ttyAMA0'

Signed-off-by: Dennis Chen <dennis.chen@arm.com>
2017-07-31 10:27:03 +00:00
Riyaz Faizullabhoy
ade82574e6 Merge pull request #2317 from justincormack/sysctl-fix
Remove kernel.random.write_wakeup_threshold from sysctl
2017-07-28 13:22:38 -07:00
Justin Cormack
980a469bbc Remove kernel.random.write_wakeup_threshold from sysctl
3072 is the default value anyway, so no need to change.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-28 17:56:13 +01:00
Dennis Chen
d9e5e41df3 package.mk: Adjust the targe sequence to avoid early abort
Current package.mk will operate on the first dependence object
'check-dirty', which will result the make process abort earlier
with below message:
pkg/package.mk:60: *** Your repository is not clean. Will not push package image.  Stop.

This is not expected behavior since we have no chance to build
a docker image locally in case of the user has no intention to
push that image.

This patch adjust the dependece order for the default target, thus
we can build the docker image locally but can't push that image
since we're in a dirty git repository.

Signed-off-by: Dennis Chen <dennis.chen@arm.com>
2017-07-28 11:05:26 +00:00
Justin Cormack
9a823cf2e2 Add mountpoint to binfmt image
As this is read only it should be explicitly created.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-27 15:05:48 +01:00
Avi Deitcher
8f10f69fd3 Merge pull request #2233 from dave-tucker/ns-detect
getty: Detect if you are in a namespace
2017-07-27 16:50:02 +03:00
Avi Deitcher
543e61c643 Fix typo
Signed-off-by: Avi Deitcher <avi@deitcher.net>
2017-07-27 15:38:20 +03:00
Justin Cormack
16674a1557 Merge pull request #2306 from ijc/containerd-v1.0.0-alpha2
Containerd v1.0.0 alpha2
2017-07-27 13:30:24 +01:00
Dave Tucker
f41c976406 getty: Detect if you are in a namespace
This is an attempt at fixing #2213

If `INITGETTY` has been set then we are not in a namespace.

Signed-off-by: Avi Deitcher <avi@deitcher.net>
2017-07-27 14:55:28 +03:00
Ian Campbell
8eb6709a3c containerd: Bump to new alpine (with v1.0.0-alpha2)
The logrus import path has changed, so adjust. Also there is a minor API change
to the containerd.IOCreation() function spec, it now takes a string id which we
can ignore.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-07-27 11:07:03 +01:00
Rolf Neugebauer
a18f4403d0 pkg/init-lcow: Update opengcs to latest
While at it, might as well update linuxkit/alpine to the latest.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-07-27 10:47:19 +01:00
Justin Cormack
941e31a519 Merge pull request #2278 from deitch/tpm-tools
tpm tools
2017-07-26 22:01:51 +01:00
Justin Cormack
cb2ca4ef66 Switch runc and containerd startup to be entirely Go
At present they use a small shared function called "prepare"
that does the read-write remounts, that I will switch to doing overlay
mounts soon.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-26 16:32:09 +01:00
Justin Cormack
5194bf13d1 Merge pull request #2296 from justincormack/service-does-more
Make service start up containerd and services
2017-07-26 15:34:03 +01:00
Rolf Neugebauer
2b76e86500 Merge pull request #2152 from dave-tucker/multi-mount
Improve mount and format packages
2017-07-26 14:39:28 +01:00
Justin Cormack
b8f7b7aad0 Merge pull request #2294 from ijc/pkg-forcepush
pkg: Add forcepush target
2017-07-26 13:43:12 +01:00
Dave Tucker
8dcb57429a pkg: Add extend for extending partitions
This was split out from pkg/format into its own package.
It has the ability to extend ext4, btrfs and xfs partitions.

Signed-off-by: Dave Tucker <dt@docker.com>
2017-07-26 13:26:39 +01:00
Dave Tucker
a14a8be49e mount: Add support to mount by label, uuid or name
This commit rewrites the mount package in Go.
It adds the ability to mount the by label, UUID or name.
It also fixes the automatic mount logic to check that a drive isn't
already mounted before attempting to mount it. This allows for multiple
uses of the mount pkg in a single YAML file.

Signed-off-by: Dave Tucker <dt@docker.com>
2017-07-26 13:26:39 +01:00
Dave Tucker
5273ec1d33 format: Add partition labels and new filesystems
This commit re-writes the format package in Go and adds the ability to
add partition labels and also specify either ext4, btrfs or xfs

Signed-off-by: Dave Tucker <dt@docker.com>
2017-07-26 13:26:39 +01:00