Commit Graph

715 Commits

Author SHA1 Message Date
Justin Cormack
98dfe7ad55 Merge pull request #1721 from riyazdf/more-signing-targets
Signing targets for binfmt, rngd, compilers, toybox, tini
2017-04-27 18:53:49 +01:00
Riyaz Faizullabhoy
2c4c55859f Add separate sign-tag target, keep tool building offline
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-04-27 10:34:40 -07:00
Justin Cormack
dae0e4efae Fix typo "ndodev"
Not sure when this arrived but it was stopping anything running.
Appears not to be in the older test `init` containers.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-27 18:01:05 +01:00
Riyaz Faizullabhoy
38ad84bfbd pull base images with content trust for binfmt, rngd, tini, toybox-media
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-04-26 14:28:30 -07:00
Justin Cormack
1cd201aac5 Merge pull request #1715 from deitch/swap-module
First cut of swap image and example
2017-04-26 21:47:54 +01:00
Avi Deitcher
715cfbd02c First cut of swap image and example
Signed-off-by: Avi Deitcher <avi@deitcher.net>

Fix spaces after commas

Remove MAINTAINER

Signed-off-by: Avi Deitcher <avi@deitcher.net>

Simplify swap.yml example to remove files section

Signed-off-by: Avi Deitcher <avi@deitcher.net>

Switch swap.sh to sh from bash and remove bash from image

Signed-off-by: Avi Deitcher <avi@deitcher.net>

Replace fallocate with dd and update calculation function to support it

Signed-off-by: Avi Deitcher <avi@deitcher.net>

Fix indentation

Signed-off-by: Avi Deitcher <avi@deitcher.net>

Change link to swap to just swap.sh

Signed-off-by: Avi Deitcher <avi@deitcher.net>

Fix indent

Signed-off-by: Avi Deitcher <avi@deitcher.net>
2017-04-26 23:05:45 +03:00
Riyaz Faizullabhoy
78a0d018c1 enable DCT in buildchain for compiler images
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-04-26 12:09:33 -07:00
Riyaz Faizullabhoy
72e7945ec6 Add signing targets for binfmt, c-compile, go-compile, rngd, tini, toybox-media
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-04-26 10:55:20 -07:00
Justin Cormack
713046e158 Update ca-certificates to be based on Alpine and use nested build
We were using Debian but Alpine more consistent. Use nested build.

Currently extract the hash in a nasty way but this can be fixed later
when we switch over hashing method.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-26 15:24:26 +01:00
Justin Cormack
98159fd13f Merge pull request #1708 from rneugeba/binfmt
Move binfmt to a multi-stage build
2017-04-26 12:19:09 +01:00
Rolf Neugebauer
c6416ddecb pkg: Use multi-stage builds for the binfmt package
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-25 14:54:54 +01:00
Ian Campbell
20cfa2af2c Switch fallback hostname to linuxkit-* from moby-*
This was missed when things were renamed.

The intention with this code was (apparently) to provide a (pseudo)unique
hostname in the case where something more specific was not provided (e.g. by
DHCP). Make this a little clearer by using '(none)' rather than 'linuxkit' as
the default, in the normal case this will be overwritten by something more
specific and if it isn't we will change it to something somewhat unique derived
from the MAC address (as before). nb: '(none)' is already used by Debian so I
think it is a safe choice as the sentinel value.

The use of both CONFIG_DEFAULT_HOSTNAME and the explicit /etc/hostname from
mkimage.sh is likely to be redundant in some cases, but neither seems to
completely cover all cases so keep both.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-04-25 14:09:38 +01:00
Rolf Neugebauer
8d1eb49fa7 pkg: Convert rngd to a multi-stage build
This allows us to get rid of the ./base/rng-tools

Use c-compile to compile it

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-24 18:41:27 +01:00
Rolf Neugebauer
85bcee93dd tools: Move tini to ./tools and convert to multi-stage build
Previously the "tini" contained the entire build. Using the
multi-stage build creates a new "tini" base image, which only
contains the "/bin/tini". This image can be used as the base
image for packages requiring "tini".

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-24 18:41:27 +01:00
Justin Cormack
d616cea3e1 Enable Prometheus metrics on port 13337 on containerd
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-16 10:43:44 -05:00
Rolf Neugebauer
856a2651b0 pkg: Add a Prometheus node_exporter package
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-16 09:43:40 -05:00
Justin Cormack
e79e3a38ad Use containerd to run containers
Because we are not yet using the image store, this requires a
patched `ctr` that can run a specified rootfs rather than an
image.

We will switch to the image store later, but this requires
extensive build changes.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-14 15:35:29 -05:00
Justin Cormack
72c3f9cfa2 More renames to LinuxKit
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-14 11:47:24 -05:00
Riyaz Faizullabhoy
3d91a7be86 Bump go-compile with ineffassign
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-04-12 16:54:51 -07:00
Rolf Neugebauer
0a0786d49e pkg: Update packages to push to linuxkit hub org
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-12 21:28:17 +01:00
Rolf Neugebauer
e7314ff350 pkg: Use images from linuxkit hub org
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-12 21:28:17 +01:00
Rolf Neugebauer
36749ea5c4 build: Switch using c-compile from linuxkit hub org
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-12 21:28:17 +01:00
Rolf Neugebauer
a06e6bbb37 build: Switch using go-compile from linuxkit hub org
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-12 21:28:17 +01:00
Rolf Neugebauer
105cb48ad6 metadata: Simplify the GCP SSH code
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-12 16:19:24 +01:00
Rolf Neugebauer
432bdda272 pkg: Remove metadata-gcp
It's no longer used

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-12 15:42:48 +01:00
Rolf Neugebauer
a21c566777 metadata: Make sure the ssh directory exists before writing to it
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-12 15:39:08 +01:00
Rolf Neugebauer
ce52c28cf6 metadata: Write the provider to /var/config/provider
This will be useful for testing.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-12 15:39:08 +01:00
Rolf Neugebauer
b0a0b4ceb7 metadata: Make JSON parser more robust
Make sure the type conversions are OK.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-12 15:39:08 +01:00
Rolf Neugebauer
626f02def2 metadata: Improve logging
Implementing a String() interface for each provider makes it
easier for users to prefix log strings with the provider.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-12 15:39:08 +01:00
Justin Cormack
3c326bebdf Make / rshared
Previously only `/var` was `rshared` but some people need to share
mounts in `/opt` etc so let us make everything `rshared` for now.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-12 13:10:46 +01:00
Rolf Neugebauer
9cf63a4aaa Merge pull request #1584 from rneugeba/userdata
pkg: Add a generic metadata package
2017-04-11 15:45:34 +01:00
Rolf Neugebauer
766e1d95d3 pkg: Add a generic metadata package
This package handles meta and user data for different cloud
and other platforms. It should be easy to extend to new
platforms.

Currently, it handles GCP metadata and a simple CDROM userdata
provider.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-11 15:32:17 +01:00
Justin Cormack
2c7628c101 Support dhcpcd in one shot mode as well
This is needed for cloud environments that want to get their metadata in
the onboot phase over the network.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-11 14:58:57 +01:00
Justin Cormack
a3ec9e86df Add openntpd container
Added to Docker example as Swarm mode likes time sync.

Note uses pool.ntp.org at present, on some providers it
should use a different source.

Constraints not enabled as they seem to have an issue;
possibly trying to set before network is up.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-10 20:32:59 +01:00
Justin Cormack
3afe494ae9 Merge pull request #1567 from justincormack/sysfs
Add a sysfs container and fix memory hierarchy
2017-04-10 17:59:37 +01:00
Justin Cormack
bc23fde1c2 Use the real default containerd toml config
Rather than an empty one.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-10 16:25:13 +01:00
Justin Cormack
0ffc2867a9 Use hierarchy for memory cgroups
Container systems expect this...

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-10 14:36:06 +01:00
Justin Cormack
821fdaecc8 Remove SELinux setup until actually implemented
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-10 14:35:07 +01:00
Justin Cormack
efd1efe531 Add a sysfs container
However, do not try to change memory cgroups from it; this needs to be in `init`.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-10 14:33:38 +01:00
Justin Cormack
9ee52aa966 Rework how /var is mounted
Instead of mounting a new filesystem, revert to doing a `rw` bind.

However do not make `/` `rshared`, just `/var` as that is where we expect
filesystems to be mounted for persistence. Also only make the actual
container rootfs writeable, not the whole directory.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-10 11:28:00 +01:00
Justin Cormack
bfa76205d5 Change propagation for root
- make / rshared
- make /containers private
- make /var its own tmpfs mountpoint, shared

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-09 13:50:04 +01:00
Justin Cormack
3ffa912c41 Move mount back to its own package, not inside docker
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-09 12:53:46 +01:00
Justin Cormack
e0aced6be0 Merge pull request #1550 from justincormack/rootfs-mountpoint
Make each rootfs a mountpoint by binding
2017-04-09 09:52:23 +01:00
Justin Cormack
fb5d6a8fad Add an (empty) config file for containerd
It needs one now.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-08 21:38:31 +01:00
Justin Cormack
c40351a0a8 Make each rootfs a mountpoint by binding
Otherwise shared mounts do not work correctly with `runc`.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-08 21:10:30 +01:00
Justin Cormack
eb22d6909f system → onboot daemon → services
As suggested by @shykes these are clearer

- onboot for things that are run at boot time to completion
- services for persistent services

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-08 16:56:47 +01:00
Ilya Dmitrichenko
a3638a2b8f Make sshd example usefull for debugging
Signed-off-by: Ilya Dmitrichenko <errordeveloper@gmail.com>
2017-04-08 12:49:51 +01:00
Justin Cormack
a54a694772 Split out init to have standalone runc, containerd
Also add ca-certificates to base, needed to use `dist` to pull.

Make two stage builds for `containerd` and `runc` so they have a
from `scratch` second stage.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-07 16:25:18 +01:00
Thomas Gazagnaire
226d394d15 update runc to version ac50e77bbb440dcab354a328c79754e2502b79ca
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-07 11:46:34 +02:00
Riyaz Faizullabhoy
8ba64546eb Remount rootfs as read-only after init, /var and /containers mounted as
read-write

Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-04-06 09:30:46 -07:00
Justin Cormack
485452d9bd Merge pull request #1460 from justincormack/automount
Add a formatting container and persistent disk support for docker
2017-04-06 16:08:20 +01:00
Justin Cormack
0a030dc219 Clean up dhcpcd container
It is not necessary to bring up `eth0`, the program does it fine.

This means we can remove shell script, clean up build.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-06 15:32:22 +01:00
Justin Cormack
cf7b952995 Add persistent drive support to Docker container
This works and runs containers now, if you eg `runc exec` into it.
Needs a few tweaks for rlimits, but will pull and run containers.

Will integrate better with ssh/dev containers to make more usable.

For a simple test use
```
./bin/moby build examples/docker.yml
./bin/moby run hyperkit -disk-size 100 docker

```

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-06 14:25:42 +01:00
Justin Cormack
8bcb2c9e0e Add a formatting container
This is based on the code we used for Docker Editions, and
will format an external drive, to be used for example for
`/var` for Docker image persistence.

It does not `mount` the drive yet, as splitting format and mount
gives better modularity.

Example yaml fragment:
```
  - name: format
    image: "mobylinux/format:097d4f22b20f976b1f89d8f0b8a5d074d35b856c"
    binds:
     - /dev:/dev
    capabilities:
     - CAP_SYS_ADMIN
     - CAP_MKNOD
```

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-06 13:19:46 +01:00
Ian Campbell
2b54e18f9f Drop unnecessary use of start-stop-daemon with containerd.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-04-06 09:34:38 +01:00
Ian Campbell
8270bdfe33 Use exec in a couple of places to avoid needless lingering /bin/sh processes
```
$ apk -U add procps
$ ps xf
    1 ?        Ss     0:05 /sbin/init
  357 ?        Ss     0:00 /bin/sh /etc/init.d/containerd
  359 ?        Sl     0:00  \_ /usr/bin/containerd
  360 ?        Ss     0:00 /bin/sh /etc/init.d/containers
  432 ?        Sl     0:00  \_ /usr/bin/runc run --bundle /containers/daemon/swa
  466 ?        Ssl    0:00  |   \_ /usr/bin/swarmd --containerd-addr=/run/contai
  427 ?        Sl     0:00  \_ /usr/bin/runc run --bundle /containers/daemon/dhc
  457 ?        Ss     0:00  |   \_ bin/sh /usr/local/bin/start_dhcpcd.sh
  474 ?        S      0:00  |       \_ /sbin/dhcpcd --nobackground
  429 ?        Sl     0:00  \_ /usr/bin/runc run --bundle /containers/daemon/rng
  576 ?        Ss     0:00      \_ /bin/tini /usr/sbin/rngd -f
  580 ?        S      0:00          \_ /usr/sbin/rngd -f
```

becomes

```
$ ps xf
    1 ?        Ss     0:06 /sbin/init
  358 ?        Ss     0:00 /bin/sh /etc/init.d/containers
  426 ?        Sl     0:00  \_ /usr/bin/runc run --bundle /containers/daemon/dhc
  458 ?        Ss     0:00  |   \_ /sbin/dhcpcd --nobackground
  431 ?        Sl     0:00  \_ /usr/bin/runc run --bundle /containers/daemon/swa
  460 ?        Ssl    0:00  |   \_ /usr/bin/swarmd --containerd-addr=/run/contai
  428 ?        Sl     0:00  \_ /usr/bin/runc run --bundle /containers/daemon/rng
  574 ?        Ss     0:00      \_ /bin/tini /usr/sbin/rngd -f
  578 ?        S      0:00          \_ /usr/sbin/rngd -f
  356 ?        Ssl    0:00 /usr/bin/containerd

```

Specifically these are gone:
  357 ?        Ss     0:00 /bin/sh /etc/init.d/containerd
  457 ?        Ss     0:00  |   \_ bin/sh /usr/local/bin/start_dhcpcd.sh

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-04-06 09:34:38 +01:00
Riyaz Faizullabhoy
a33b9ff4b1 dhcpcd system container
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-04-04 08:23:47 -07:00
Ian Campbell
709402d780 Update to runc ef9a4b315558d31eae520725ff67383c2f79c3cb
This is compatible with containerd 8353da59c6ae7e1933aac2228df23541ef8b163f
which was picked up by d2caae4c1a.

This required jiggering with riddler output some more to update to new OCI
config.json format for capabilities.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-04-03 16:47:14 +01:00
Ian Campbell
31e0a848e1 pkg/init: ruinc and containerd binaries depend on Makefile
Since Makefile contains the hashes we need a dependency otherwise changing the
hashes does not necessarily result in the binaries getting updated.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-03-30 17:20:59 +01:00
Rolf Neugebauer
eac2fdd88d init: Enable tty? as console
Enable tty0 and friends as console and start getty on them.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-28 10:46:58 +01:00
Justin Cormack
7b79053306 Update containerd to current master
- now supports image pull and run end to end
- update runc to last version before spec update fix #1302
- remove ext2 utils from init

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-25 18:15:52 +00:00
Justin Cormack
905636d642 Move installable packages to pkg
Still leaves some intermediate repos in `base/`

See #1266

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-22 12:33:11 +00:00
Justin Cormack
e643f4a454 Move Go code to src/cmd
This does not get everything where we want it finally, see #1266
nor the optimal way of building, but it gets it out of top level.

Added instructions to build if you have a Go installation.

Not moving `vendor` yet.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-21 14:37:23 +00:00
Justin Cormack
159202416c Out with the old, in with the new Moby
- remove remainder of editions code
- add a new check container to run tests without Docker
- switch over `make test` to use new command to build tests

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-06 22:28:41 +00:00
Justin Cormack
4797088ec0 Rough first version of the moby tool
- terrible code
- lots needs changing
- can build a Moby from a config yaml that boots

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-02-20 21:44:23 +00:00