Commit Graph

339 Commits

Author SHA1 Message Date
Rolf Neugebauer
adb92ca672 pkg/init-lcow: Update to the latest version
This includes fixes to the intermittent on pull as well as
other fixes.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-09-15 14:47:12 +01:00
David Scott
9c35dbaac2 service: log stdout and stderr to separate files
Previously we would pass the path `/var/log/service.log` for both
stdout and stderr to containerd. containerd would construct a dict
with the paths as keys[1] and, due to the duplicate key, would only
open one of the files and start one `io.Copy` instance. Writes to
the other stream would be buffered by the pipe connected to
containerd-shim and would eventually block.

If we modified containerd to open the file twice and start 2
`io.Copy` instances, we would end up with the two streams interleaved
together. It seems cleaner to keep the streams separate; therefore
this patch logs stdout to `/var/log/service.out.log` and stderr to
`/var/log/service.err.log`.

[1]
49437711c3/linux/shim/io.go (L51)

Signed-off-by: David Scott <dave.scott@docker.com>
2017-09-14 11:15:04 +01:00
Rolf Neugebauer
36c84004a2 pkg/init-lcow: Update to latest version of OpenGCS
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-09-12 16:59:07 +01:00
Ian Campbell
6cbc3ed518 Bump init+containerd to new alpine base with containerd beta.0
Signed-off-by: Ian Campbell <ijc@docker.com>
2017-09-07 18:31:57 +01:00
Rolf Neugebauer
3ebec6f3e8 pkg/init-lcow: Update to latest OpenGCS version
While at it also update to latest alpine base

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-09-06 08:54:14 +01:00
Rolf Neugebauer
2cb0612011 pkg/lcow: Update OpenGCS to latest
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-30 19:41:02 +01:00
Ian Campbell
b0b08e18a2 Bump init and containerd packages to v1.0.0-alpha6
Signed-off-by: Ian Campbell <ijc@docker.com>
2017-08-29 01:02:20 +01:00
Justin Cormack
5773dfd844 Cleanup cleanup code
Only remove main mount at rootfs, as we do not really know
about the state of the rest.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-23 16:40:03 +01:00
Justin Cormack
49cf74f4f6 Switch to having the moby tool specify how to do mounts
This removes all the code that had knowledge of how to do read only
and read write container mounts, and just uses the runtime config.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-23 16:40:03 +01:00
Justin Cormack
c677b391fc Add support for mount in runtime config
This could be used in LinuxKit now, as there are some examples, eg
https://github.com/linuxkit/linuxkit/blob/master/blueprints/docker-for-mac/base.yml#L33
which are creating containers to do a mount.

The main reason though is to in future change the ad hoc code that generates
overlay mounts for writeable containers with a runtime config which does
the same thing; this code needs to create both tmpfs and overlay mounts.

See https://github.com/moby/tool/pull/145

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-23 16:40:03 +01:00
Justin Cormack
2129d1de21 Merge pull request #2450 from justincormack/improve-wireguard-example
Improve wireguard example
2017-08-22 17:44:01 +01:00
Justin Cormack
584613cddb Update ip package to latest Alpine base
There was a wireguard mismatch with the kernel module in the recent kernel
builds.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-22 11:50:20 +01:00
David Scott
da86df1ad6 init: add /etc/ssl/certs as required by the metadata package
The metadata package has binds
  - /dev
  - /var
  - /sys
  - /etc/resolv.conf
  - /etc/ssl/certs

but unfortunately `/etc/ssl/certs` doesn't exist and this causes the
following commands:

  cd blueprints/docker-for-mac # easy example
  moby build -name docker-for-mac base.yml docker-17.06-ce.yml
  linuxkit run hyperkit -networking=vpnkit -vsock-ports=2376 -disk size=500M docker-for-mac

to produce the following error on the VM console:

  container_linux.go:265: starting container process caused "process_linux.go:348: container init caused \"rootfs_linux.go:57: mounting \\\"/etc/ssl/certs\\\" to rootfs \\\"/containers/onboot/000-metadata/rootfs\\\" at \\\"/etc/ssl/certs\\\" caused \\\"stat /etc/ssl/certs: no such file or directory\\\"\""
  2017/08/21 16:39:40 Error creating 000-metadata: exit status 1

This patch creates /etc/ssl/certs in the `init` package. The metadata package
will now say things like

  2017/08/21 16:44:39 No metadata/userdata found. Bye

Signed-off-by: David Scott <dave.scott@docker.com>
2017-08-21 17:46:29 +01:00
Justin Cormack
90f905a799 Move containerd clone to tools/alpine
As discussed before, as we use this in three places, cloning in
base makes more sense.

Update base image.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-21 15:16:09 +01:00
Justin Cormack
a808808ce1 Add vendoring needed for netlink, packet
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-19 19:25:21 +01:00
Justin Cormack
92451cf2e4 Add packet metadata support, with bonding
Use the packet metadata to set up bonding, also get ssh keys and hostname.

This does not yet do anything with disk metadata.

Userdata is not used if it has been used for ipxe, but is otherwise available.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-19 19:25:14 +01:00
Rolf Neugebauer
5898bc9f0d kernel: Update to 4.9.44/4.4.83
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-18 18:08:32 +01:00
Justin Cormack
9506d45d49 Merge pull request #2413 from justincormack/runtime-config
Add a runtime config
2017-08-18 17:45:10 +01:00
Justin Cormack
9d8f1bd900 Allow specifying a set of metadata providers
run with `cmd: ["metadata", "aws"]` to just check for AWS metadata for example.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-17 14:50:50 +01:00
Justin Cormack
57512b3ec9 Add an ip package
This contans the upstream (non busybox) version of `ip` and also the
Wireguard tools as these aere not yet integrated into `ip`. It is designed
to be used for network configuration.

For example a Wireguard interface can be setup with something like:
```
  - name: wg
    image: linuxkit/ip
    net: new
    binds:
      - /etc/wireguard:/etc/wireguard
    command: ["sh", "-c", "ip link set dev wg0 up; ip address add dev wg0 192.168.2.1 peer 192.168.2.2; wg setconf wg0 /etc/wireguard/wg0.conf; wg show wg0"]
    runtime:
      interfaces:
        - name: wg0
          add: wireguard
          createInRoot: true
      bindNS:
          net: /run/netns/wg
```

This binds the net namespace so other containers can use it later.

Note that `ip` supports batch scripts with `ip -b file` which is quite convenient for
this use case.

Alas `wg-quick` requires `bash` so this is currently a bit bloated, maybe @zx2c4
might be persuaded to rewrite it without bashisms...

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-17 12:18:38 +01:00
Justin Cormack
c82b2fe13e Add vendoring for netlink
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-17 12:10:28 +01:00
Justin Cormack
0c81ce19e8 Add a runtime config
This adds support for a runtime configuration file that can do:
- `mkdir` to make a directory at runtime, eg in `/var` or `/tmp`, to avoid workarounds
- `interface` that can create network interfaces in a container or move them
- `bindNS` that can bind mount namespaces of an `onboot` container to a file so a service can be started in that namespace.

It merges the `service` and `onboot` tools (in `init`) to avoid duplication. This also saves some size for
eg LCOW which did not use the `onboot` code in `runc`.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-17 12:10:28 +01:00
Justin Cormack
df16f6fb9e Move init command into a subdirectory
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-17 12:10:28 +01:00
Rolf Neugebauer
f9f8b5bc42 pkg/lcow-init: Update to latest version of OpenGCS
While at it also update to latest alpine base

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-16 16:50:34 +01:00
Rolf Neugebauer
714dbcf0fe Merge pull request #2399 from ijc/pkg-honour-hash-override
package: Do not add an arch suffix if an explicit HASH is given
2017-08-14 10:14:04 +01:00
Ian Campbell
e11c1c0224 pkg: Bump to containerd v1.0.0-alpha4 and runc v1.0.0-rc4
Signed-off-by: Ian Campbell <ijc@docker.com>
2017-08-11 16:16:33 +01:00
Ian Campbell
0469901778 package: Do not add an arch suffix if an explicit HASH is given
During development I tend to do:
	make -C pkg/foo tag HASH=dev ORG=ijc
and I expect to get an image `ijc/foo:dev` and not `ijc/foo:dev-amd64`. In
general I think if the HASH has been explicitly provided the build should
just honour that.

To compensate and avoid mistaken pushes adjust the check for dirtiness to also
insist on a non-empty suffix.

Signed-off-by: Ian Campbell <ijc@docker.com>
2017-08-11 11:26:00 +01:00
Justin Cormack
c42ddc0af8 Fix root directory permissions with overlay
At least when running under Docker this seems to cause some issues,
possibly also explains some other oddnesses.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-10 14:22:12 +01:00
Rolf Neugebauer
46d1e568cc pkg/init-lcow: Update opengcs
This has a number of fixes/improvements
- console resize
- better error handling (pass stacktrace/errors back to docker
- fix for container images which have a resolve.conf file

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-08 17:53:27 +01:00
Justin Cormack
e61794a344 Merge pull request #2365 from deitch/test-dev-existence-getty
Test dev existence getty
2017-08-07 20:32:39 +01:00
Justin Cormack
76bb43027b Do an extra chmod after chown
Chown clears suid bits even for root on Linux.

Also move a few functions to x/sys/unix from syscall, to be
more arm64 friendly.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-07 14:49:43 +01:00
Ian Campbell
6ef10ae33b Update to containerd v1.0.0-alpha3 and runc master
Runc is master as of this morning, slightly newer than what is vendored into
containerd v1.0.0-alpha3 but the differences[0] seemed benign and/or good to have.

Signed-off-by: Ian Campbell <ijc@docker.com>

[0] 429a538712...45bde006ca
2017-08-07 11:47:44 +01:00
Avi Deitcher
cc174b4df0 test device exists before allowing getty to run
Signed-off-by: Avi Deitcher <avi@deitcher.net>
2017-08-06 10:23:12 +03:00
Rolf Neugebauer
dfa684c78f pkg/init-lcow: Update to tip of opengcs
Also adjust the building of the rootfs as this seem to have
changed.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-04 18:34:56 +01:00
Justin Cormack
0b2e34301b Merge pull request #2357 from justincormack/arm64-rngd
Support rngd on arm64
2017-08-04 16:13:46 +01:00
Justin Cormack
8b2327b0e2 Support rngd on arm64
Although it does not do anything, as there is no CPU rng on arm64 at present.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-04 14:01:18 +01:00
Justin Cormack
ce2347eda6 Enable sysctl kernel.dmesg_restrict
This requires that users have `CAP_SYSLOG` in order to access `dmesg`.
This means that containers by default have no access to `dmesg` (which
can leak information about the host or other containers) unless they
have this capability added.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-04 12:48:27 +01:00
Rolf Neugebauer
fee2a62f26 pkg: Disable some packages for arm64
They either won't work or the packages are specific
to x86_64 like the DfM or VMware packages.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-03 17:46:11 +01:00
Rolf Neugebauer
e44783fea1 pkg: Update alpine base for all packages
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-03 17:46:10 +01:00
Rolf Neugebauer
0ebb53c491 build: Add the ability to only build/push/sign for specific architectures
A package makefile can specify a list of architectures ('ARCHES')
it supports. If build on a unsupported architecture, a message is
printed and make exits without an error.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-03 17:36:22 +01:00
Rolf Neugebauer
981def7c7a build: Add show-tag and forcetag targets to pkg Makefile
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-03 17:36:22 +01:00
Rolf Neugebauer
c8e0d2ca28 pkg: Add invocation to push-manifest to generic package build
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-03 17:36:22 +01:00
Rolf Neugebauer
47af521e14 Merge pull request #2343 from justincormack/x-sys-unix-rngd
rngd improvements
2017-08-03 15:45:40 +01:00
Justin Cormack
4434ecc4af Support multiple CDROMS in metadata
This checks if any CDROM in the system has metadata. Useful if
you boot off CDROM and also have a metadata CDROM.

Also switch the metadata container to read only, as it is only
writing to bind mounted in directories, and /tmp.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-08-03 14:16:02 +01:00
Justin Cormack
f768ac4bfd Add flags to disable the different random sources
This makes testing the different options much easier.

Signed-off-by: Justin Cormack <justin@specialbusservice.com>
2017-08-02 20:55:36 +01:00
Justin Cormack
63798997f6 Convert rngd to use x/sys/unix
Epoll in `syscall` package does not work on arm64. We do not have
arm64 yet here, but we will when we add tpm support etc.

Signed-off-by: Justin Cormack <justin@specialbusservice.com>
2017-08-02 20:09:43 +01:00
Rolf Neugebauer
767ac54971 pkg/init-lcow: Update opengcs to the latest version
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-08-01 17:06:12 +01:00
Justin Cormack
4daf008956 Merge pull request #2331 from justincormack/rng-golang
Replace rngd with a Go version
2017-08-01 10:56:10 +01:00
Justin Cormack
b1c80b54b3 Revert #2317 remove kernel.random.write_wakeup_threshold from sysctl
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-31 23:05:46 +01:00
Justin Cormack
737fbcb3b6 Add standard directories in /var
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-31 21:09:48 +01:00