Commit Graph

676 Commits

Author SHA1 Message Date
Justin Cormack
713046e158 Update ca-certificates to be based on Alpine and use nested build
We were using Debian but Alpine more consistent. Use nested build.

Currently extract the hash in a nasty way but this can be fixed later
when we switch over hashing method.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-26 15:24:26 +01:00
Justin Cormack
98159fd13f Merge pull request #1708 from rneugeba/binfmt
Move binfmt to a multi-stage build
2017-04-26 12:19:09 +01:00
Rolf Neugebauer
cb732e7f80 YAML: Update files to use the new binfmt package
Checked that /proc/sys/fs/binfmt_misc/status is enabled for
architectures specified.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-25 15:55:10 +01:00
Ian Campbell
20cfa2af2c Switch fallback hostname to linuxkit-* from moby-*
This was missed when things were renamed.

The intention with this code was (apparently) to provide a (pseudo)unique
hostname in the case where something more specific was not provided (e.g. by
DHCP). Make this a little clearer by using '(none)' rather than 'linuxkit' as
the default, in the normal case this will be overwritten by something more
specific and if it isn't we will change it to something somewhat unique derived
from the MAC address (as before). nb: '(none)' is already used by Debian so I
think it is a safe choice as the sentinel value.

The use of both CONFIG_DEFAULT_HOSTNAME and the explicit /etc/hostname from
mkimage.sh is likely to be redundant in some cases, but neither seems to
completely cover all cases so keep both.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-04-25 14:09:38 +01:00
Rolf Neugebauer
db6d3d7ed2 docs: Consolidate mac tips & tricks in ./docs/mac.md
We had serveral files with instructions, in particular for
networking, for macOS/Docker for Mac. Let's have just one place.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-25 13:17:04 +01:00
Rolf Neugebauer
855f8f2722 demo: Move redis example to ./examples
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-25 13:17:04 +01:00
Rolf Neugebauer
3ba9d8064b YAML: Update standard yml files to use the new rngd image
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-24 18:41:27 +01:00
Justin Cormack
2709ee88bc Split build and push, and remove push code from run
This currently only changes the `gcp` target, but is the new
model - the `build` command will only do things locally, then
you need to `push` to an image store such as GCP or other ones
in order to `run` for platforms that cannot boot directly from
a local image.

Fix #1618

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-24 17:12:05 +01:00
Riyaz Faizullabhoy
d715b6114b Merge pull request #1673 from rneugeba/min
examples: Add a minimal YAML file
2017-04-23 13:15:37 -07:00
Rolf Neugebauer
e0eb0125ae examples: Add a minimal YAML file
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-22 17:30:30 +01:00
Thomas Shaw
465b513289 Typo in link to sshd.yml 2017-04-21 22:31:03 -05:00
Rolf Neugebauer
f2b9295081 examples: Add doc on how to use the SSHD example
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-21 14:12:02 -05:00
Justin Cormack
f8a6193845 Move more images to new hub org
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-16 20:28:33 -05:00
Rolf Neugebauer
0fb3544837 examples: Add a minimal node_exporter example
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-16 09:43:40 -05:00
Rolf Neugebauer
24ff9b75d8 examples: Don't make the metadata container readonly
It needs to create the /cdrom mountpoint

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-14 11:57:24 -05:00
Justin Cormack
72c3f9cfa2 More renames to LinuxKit
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-14 11:47:24 -05:00
Riyaz Faizullabhoy
9609010ea8 Also update ymls
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-04-13 09:17:14 -07:00
Rolf Neugebauer
a1ce71cd0a Update YAML files with new metadata package
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-12 16:20:41 +01:00
Rolf Neugebauer
27a3eee6dc gcp: Switch GCP to the new metadata package
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-12 15:39:08 +01:00
Justin Cormack
f603d37638 Merge pull request #1599 from avsm/packet-net-run
Add `moby run packet` to boot on baremetal Packet.net hosts
2017-04-12 13:40:50 +01:00
Justin Cormack
e4213f1582 Merge pull request #1602 from justincormack/slash-rshared
Make / rshared
2017-04-12 13:39:31 +01:00
Justin Cormack
3c326bebdf Make / rshared
Previously only `/var` was `rshared` but some people need to share
mounts in `/opt` etc so let us make everything `rshared` for now.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-12 13:10:46 +01:00
Anil Madhavapeddy
e61941999f Add moby run packet to boot on baremetal Packet.net hosts
This uses the Packet.net API and iPXE to boot a Moby host.
There are several enhancements coming soon, such as SSH key
customisation, but this PR is sufficient to boot a host and
then use the web interface to get console access.

The user must currently upload the built artefacts to a public
URL and specify it via --base-url, e.g.:
moby run packet --api-key <key> --project-id <id> \
  --base-url http://recoil.org/~avsm/ipxe --hostname test-moby packet

See #1424 #1245 for related issues.

Signed-off-by: Anil Madhavapeddy <anil@docker.com>
2017-04-12 12:59:05 +01:00
Rolf Neugebauer
5541d5840f gcp: Set up DHCP and resolv.conf bind mount properly
The DHCP client updates /tmp/etc/resolv.conf and this needs
to be bind mounted into the other containers.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-12 12:26:30 +01:00
Rolf Neugebauer
b0629176ab examples: Don't use a file section if you don't need it
Exception is ./moby.yaml where it serves as an example.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-12 12:26:30 +01:00
Rolf Neugebauer
a59e24b4fa example: Don't use the full @sha25t version of images
The sha1 tag should be sufficient to uniquely identify the image
and the sha256 versions are just very long...

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-12 12:24:02 +01:00
Justin Cormack
2c7628c101 Support dhcpcd in one shot mode as well
This is needed for cloud environments that want to get their metadata in
the onboot phase over the network.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-11 14:58:57 +01:00
Justin Cormack
f9a13a2fc3 Merge pull request #1576 from justincormack/openntpd
Add openntpd container
2017-04-10 21:38:46 +01:00
Justin Cormack
a3ec9e86df Add openntpd container
Added to Docker example as Swarm mode likes time sync.

Note uses pool.ntp.org at present, on some providers it
should use a different source.

Constraints not enabled as they seem to have an issue;
possibly trying to set before network is up.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-10 20:32:59 +01:00
Riyaz Faizullabhoy
5f90faeafa init: update main moby example and tests
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-04-10 11:59:23 -07:00
Justin Cormack
3afe494ae9 Merge pull request #1567 from justincormack/sysfs
Add a sysfs container and fix memory hierarchy
2017-04-10 17:59:37 +01:00
Justin Cormack
bc23fde1c2 Use the real default containerd toml config
Rather than an empty one.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-10 16:25:13 +01:00
Justin Cormack
efd1efe531 Add a sysfs container
However, do not try to change memory cgroups from it; this needs to be in `init`.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-10 14:33:38 +01:00
Justin Cormack
9ee52aa966 Rework how /var is mounted
Instead of mounting a new filesystem, revert to doing a `rw` bind.

However do not make `/` `rshared`, just `/var` as that is where we expect
filesystems to be mounted for persistence. Also only make the actual
container rootfs writeable, not the whole directory.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-10 11:28:00 +01:00
Justin Cormack
f079f7a7cd Update to new init container with mount changes
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-09 13:52:45 +01:00
Justin Cormack
b2a3215e5f Update Docker image to use mount image not do mount itself
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-09 13:50:04 +01:00
Rolf Neugebauer
733e8f3307 config: Remove unused binfmt container
A few YAML files include the binfmt container, where it's not really
needed. Remove it to make the samples simpler.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-04-09 10:09:00 +01:00
Justin Cormack
e0aced6be0 Merge pull request #1550 from justincormack/rootfs-mountpoint
Make each rootfs a mountpoint by binding
2017-04-09 09:52:23 +01:00
Justin Cormack
fb5d6a8fad Add an (empty) config file for containerd
It needs one now.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-08 21:38:31 +01:00
Justin Cormack
c40351a0a8 Make each rootfs a mountpoint by binding
Otherwise shared mounts do not work correctly with `runc`.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-08 21:10:30 +01:00
Justin Cormack
eb22d6909f system → onboot daemon → services
As suggested by @shykes these are clearer

- onboot for things that are run at boot time to completion
- services for persistent services

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-08 16:56:47 +01:00
Ilya Dmitrichenko
a3638a2b8f Make sshd example usefull for debugging
Signed-off-by: Ilya Dmitrichenko <errordeveloper@gmail.com>
2017-04-08 12:49:51 +01:00
Riyaz Faizullabhoy
a947ded0fb Enforce content trust with trust key and yaml subkeys for image and org
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-04-07 16:29:25 -07:00
Justin Cormack
a54a694772 Split out init to have standalone runc, containerd
Also add ca-certificates to base, needed to use `dist` to pull.

Make two stage builds for `containerd` and `runc` so they have a
from `scratch` second stage.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-07 16:25:18 +01:00
Justin Cormack
d9faecdee9 Make init accept a list of images not just a single one.
fix #1527

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-07 14:25:28 +01:00
Thomas Gazagnaire
226d394d15 update runc to version ac50e77bbb440dcab354a328c79754e2502b79ca
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
2017-04-07 11:46:34 +02:00
Justin Cormack
dbb67e969b Merge pull request #1298 from riyazdf/readonly-rootfs
Remount rootfs as read-only after init, /var and /containers mounted as rw
2017-04-06 18:35:44 +01:00
Riyaz Faizullabhoy
8ba64546eb Remount rootfs as read-only after init, /var and /containers mounted as
read-write

Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-04-06 09:30:46 -07:00
Justin Cormack
dc1818147c Merge pull request #1522 from dave-tucker/gce-gcp
Use GCP everywhere for consistency
2017-04-06 17:27:15 +01:00
Dave Tucker
df1c66dd04 moby: Replace references to GCE with GCP
Signed-off-by: Dave Tucker <dt@docker.com>
2017-04-06 17:00:53 +01:00
Justin Cormack
485452d9bd Merge pull request #1460 from justincormack/automount
Add a formatting container and persistent disk support for docker
2017-04-06 16:08:20 +01:00
Justin Cormack
0a030dc219 Clean up dhcpcd container
It is not necessary to bring up `eth0`, the program does it fine.

This means we can remove shell script, clean up build.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-06 15:32:22 +01:00
Justin Cormack
cf7b952995 Add persistent drive support to Docker container
This works and runs containers now, if you eg `runc exec` into it.
Needs a few tweaks for rlimits, but will pull and run containers.

Will integrate better with ssh/dev containers to make more usable.

For a simple test use
```
./bin/moby build examples/docker.yml
./bin/moby run hyperkit -disk-size 100 docker

```

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-06 14:25:42 +01:00
Ian Campbell
2b54e18f9f Drop unnecessary use of start-stop-daemon with containerd.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-04-06 09:34:38 +01:00
Ian Campbell
8270bdfe33 Use exec in a couple of places to avoid needless lingering /bin/sh processes
```
$ apk -U add procps
$ ps xf
    1 ?        Ss     0:05 /sbin/init
  357 ?        Ss     0:00 /bin/sh /etc/init.d/containerd
  359 ?        Sl     0:00  \_ /usr/bin/containerd
  360 ?        Ss     0:00 /bin/sh /etc/init.d/containers
  432 ?        Sl     0:00  \_ /usr/bin/runc run --bundle /containers/daemon/swa
  466 ?        Ssl    0:00  |   \_ /usr/bin/swarmd --containerd-addr=/run/contai
  427 ?        Sl     0:00  \_ /usr/bin/runc run --bundle /containers/daemon/dhc
  457 ?        Ss     0:00  |   \_ bin/sh /usr/local/bin/start_dhcpcd.sh
  474 ?        S      0:00  |       \_ /sbin/dhcpcd --nobackground
  429 ?        Sl     0:00  \_ /usr/bin/runc run --bundle /containers/daemon/rng
  576 ?        Ss     0:00      \_ /bin/tini /usr/sbin/rngd -f
  580 ?        S      0:00          \_ /usr/sbin/rngd -f
```

becomes

```
$ ps xf
    1 ?        Ss     0:06 /sbin/init
  358 ?        Ss     0:00 /bin/sh /etc/init.d/containers
  426 ?        Sl     0:00  \_ /usr/bin/runc run --bundle /containers/daemon/dhc
  458 ?        Ss     0:00  |   \_ /sbin/dhcpcd --nobackground
  431 ?        Sl     0:00  \_ /usr/bin/runc run --bundle /containers/daemon/swa
  460 ?        Ssl    0:00  |   \_ /usr/bin/swarmd --containerd-addr=/run/contai
  428 ?        Sl     0:00  \_ /usr/bin/runc run --bundle /containers/daemon/rng
  574 ?        Ss     0:00      \_ /bin/tini /usr/sbin/rngd -f
  578 ?        S      0:00          \_ /usr/sbin/rngd -f
  356 ?        Ssl    0:00 /usr/bin/containerd

```

Specifically these are gone:
  357 ?        Ss     0:00 /bin/sh /etc/init.d/containerd
  457 ?        Ss     0:00  |   \_ bin/sh /usr/local/bin/start_dhcpcd.sh

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-04-06 09:34:38 +01:00
Justin Cormack
c21996fc0b Fix indentation in examples/gcp.yml
Yaml is fussy...

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-05 16:24:03 +01:00
Ian Campbell
decc6b46ff Update example DHCP containers with oom_score_adj -> oomScoreAdj
Looks like this was missed when #1316 was rebased over #1474.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-04-05 10:28:13 +01:00
Riyaz Faizullabhoy
a33b9ff4b1 dhcpcd system container
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-04-04 08:23:47 -07:00
Justin Cormack
57c75741e9 Revert Command->Args but remove from yaml where not needed
In the riddler change I changed "command" in the yaml to "args"
but did not change the files. In fact we basically used the
default command everywhere so this did not actually break.

Remove the unnecessary "command" lines to simplify yaml.

Revert the command to args change for now as I think I prefer
command, but its easier to switch now. Need to think if the
entrypoint/command distinction matters before finalizing.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-04 09:51:17 +01:00
Justin Cormack
065af9707c Replace riddler with code that constructs config.json directly
Generated largely from the specified config; small parts taken from `docker image inspect`,
such as the command line.

Renamed some of the yaml keys to match the OCI spec rather than Docker Compose as
we decided they are more readable, no more underscores.

Add some extra functionality
- tmpfs specification
- fully general mount specification
- no new privileges can be specified now

For nostalgic reasons, using engine-api to talk to the docker cli as
we only need an old API version, and it is nice and easy to vendor...

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-04-03 23:28:55 +01:00
Ian Campbell
709402d780 Update to runc ef9a4b315558d31eae520725ff67383c2f79c3cb
This is compatible with containerd 8353da59c6ae7e1933aac2228df23541ef8b163f
which was picked up by d2caae4c1a.

This required jiggering with riddler output some more to update to new OCI
config.json format for capabilities.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-04-03 16:47:14 +01:00
Rolf Neugebauer
02146dcfc5 init: update yaml files with new init image
Also add tty0 as a console to default moby image

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-28 10:53:44 +01:00
Rolf Neugebauer
e9fbe43b34 Rename .yaml to .yml
docker-compose and other utilities use the .yml extension.
For consistency rename all .yaml to .yml

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-27 09:53:26 +02:00
Justin Cormack
7b79053306 Update containerd to current master
- now supports image pull and run end to end
- update runc to last version before spec update fix #1302
- remove ext2 utils from init

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-25 18:15:52 +00:00
dan
a75989f447 Added the capability to output moby images as vmdk files for use with VMware Workstation/Fusion
Corrected naming from vmware->vmdk and fixed Makfile

Fixed mistake outputting a vhd instead of a vmdk in output.go

Build vmdk image and added to Docker Hub, corrected link in output.go

Modified directories to confirm to standard mkimage-<imgType>

Signed-off-by: Dan Finneran <dan@thebsdbox.co.uk>
2017-03-23 11:16:40 +00:00
Justin Cormack
b3b5ec7fe5 Make sshd output to stderr not syslog
This way we get to see the logs.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-22 09:43:32 +00:00
Justin Cormack
a092d4352e Merge pull request #1323 from justincormack/gcp-metadata
Add a metadata handler for GCP which allows ssh login
2017-03-16 19:24:43 +00:00
Justin Cormack
80787f2e65 Merge pull request #1324 from justincormack/no-apk
Clean up apk cache in init
2017-03-16 19:22:14 +00:00
Justin Cormack
05123a7afa Clean up apk cache in init
See #1312

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-16 19:14:08 +00:00
Justin Cormack
73e30a757f Add a metadata handler for GCP which allows ssh login
- this needs improvements to make it more "platform native", in
  particular GCP supports multiple users and more ssh key mangement
  options.
- at present you can login as root with any platform ssh key
- add support for uts=host and ipc=host
- set the hostname from the metadata as well

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-16 18:45:44 +00:00
Justin Cormack
ff941778e4 GCP image handling enhancements
- the `public` option was not previously implemented
- add `replace` only for GCP images which will error otherwise. Only
  recommended for use in development, in production use the `--name` option
  to provide a different name eaxch time. Note only applies to GCP images,
  will document these options properly soon.
- add a `family` option; this allows you to upload many images and the
  user can select the latest using the `family` option instead of a specific
  image.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-16 13:20:21 +00:00
Riyaz Faizullabhoy
8d3a7ff351 logs: separate daemon container logs into separate files
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-03-15 12:24:10 +00:00
Justin Cormack
4142092024 Merge pull request #1299 from justincormack/gcp
Add upload to Google Cloud Storage and Google Compute Image creation
2017-03-14 14:55:16 +00:00
Justin Cormack
37c6169ab7 Add upload to Google Cloud Storage and Google Compute Image creation
- the image upload uses the cloud API
- currently auth and image creation need the `gcloud` CLI tool.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-14 14:45:05 +00:00
Rolf Neugebauer
3874a37824 kernel: Update YAML files with new kernel images
Use the 4.9.x (aka latest) in all files

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2017-03-14 10:54:44 +00:00
Justin Cormack
453bec56c5 Add a WIP example to run sshd
- Currently only works if you add your ssh key in the example yaml, but will replace
with metadata support shortly.
- sshd logging not yet configured (needs to share syslog socket).

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-03-13 15:19:54 +00:00