Commit Graph

267 Commits

Author SHA1 Message Date
Justin Cormack
2e22b3adc6 Merge pull request #83 from justincormack/hyperkit-support
Use hyperkit to make raw disks
2017-06-15 16:31:59 -07:00
Justin Cormack
13da5e08ea Use hyperkit to make raw disks
- enable the hyperkit option by default on MacOS
- use it for creating raw disk images

fix #68

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-06-15 15:10:55 -07:00
Justin Cormack
d8cc1b3f08 Merge pull request #82 from justincormack/source-ifpresent
Allow some files specified with `source` to be optional
2017-06-15 11:22:21 -07:00
Justin Cormack
4d64a46ae9 Allow some files specified with source to be optional
See discussion in https://github.com/linuxkit/linuxkit/pull/2027

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-06-15 11:07:58 -07:00
Justin Cormack
d534273407 Merge pull request #81 from ijc/no-default-hostname
Do not force default hostname `moby`.
2017-06-13 17:42:18 +02:00
Ian Campbell
755c6a0bf7 Do not force default hostname moby.
This disables the code in LinuxKit's `/bin/rc.init` which attempts to detect an
unconfigured hostname and generate a unique (ish) version from the MAC address.

Anyone who wants a specific fallback hostname can populate `etc/hostname`
through the `files` stanza in their `yml` file.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-06-13 16:37:42 +01:00
Justin Cormack
464396c63f Merge pull request #80 from justincormack/http-fetch-yaml
Add support to fetch yaml file with http
2017-06-13 16:58:51 +02:00
Justin Cormack
d7ca674ea6 Add support to fetch yaml file with http
For example you can do
```
moby build https://raw.githubusercontent.com/linuxkit/linuxkit/master/linuxkit.yml
```

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-06-12 18:11:21 +02:00
Justin Cormack
90fd3e7e41 Merge pull request #79 from justincormack/tilde
Allow ~ for home directory when specifying source of file
2017-06-12 16:43:28 +02:00
Justin Cormack
1988b312d2 Allow ~ for home directory when specifying source of file
Useful when adding ssh keys

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-06-12 16:37:00 +02:00
Justin Cormack
e94d0d8093 Merge pull request #78 from justincormack/add-initrd
Move the initrd library to this repo
2017-06-12 15:43:47 +02:00
Justin Cormack
322d06f906 Update vendoring to use initrd library from this repo
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-06-12 15:36:30 +02:00
Justin Cormack
528b430789 Clean up initrd creation library
Remove duplicated code and clean up file handling a little.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-06-12 15:34:30 +02:00
Justin Cormack
ddc92b8846 Code in the initrd generator to split a single tarball
In the WIP code in `moby` we now have a standard base tarball format,
that includes the kernel and cmdline as files in `/boot` so that the
entire output of the yaml file can default to a single tarball. Then
this can be split back up by LinuxKit into initrd, kernel and cmdline
as needed. This will probably become the only output of the `moby build`
stage, with a `moby package` stage dealing with output formats.

We may remove the output format specification from the yaml file as well,
and just have it in the command.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-06-12 15:34:22 +02:00
Justin Cormack
ecf14815f7 Update Go path to linuxkit/linuxkit for libraries
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-06-12 15:34:11 +02:00
Tycho Andersen
c313e39250 init: fake supporting hard links slightly differently
Instead, make a hard link a symlink. This isn't much better, but it allows
some cases (e.g. installing GCC on moby via alpine) to work.

Signed-off-by: Tycho Andersen <tycho@docker.com>
2017-06-12 15:34:02 +02:00
Justin Cormack
6bce274b81 Move Go code to src/cmd
This does not get everything where we want it finally, see #1266
nor the optimal way of building, but it gets it out of top level.

Added instructions to build if you have a Go installation.

Not moving `vendor` yet.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-06-12 15:33:53 +02:00
Justin Cormack
49002e7a1d Out with the old, in with the new Moby
- remove remainder of editions code
- add a new check container to run tests without Docker
- switch over `make test` to use new command to build tests

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-06-12 15:33:45 +02:00
Justin Cormack
ed832e185f Rough first version of the moby tool
- terrible code
- lots needs changing
- can build a Moby from a config yaml that boots

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-06-12 15:33:34 +02:00
Justin Cormack
6b4bae6832 Merge pull request #77 from justincormack/tar-abs
Remove absolute paths from files
2017-06-09 11:49:26 +01:00
Justin Cormack
796d943054 Remove absolute paths from files
tarballs should only have relative paths in.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-06-09 11:39:52 +01:00
Justin Cormack
e7fc277fd2 Merge pull request #76 from justincormack/docs
Add docs
2017-06-08 15:53:54 +01:00
Justin Cormack
eea5c6f126 Updates to the docs
Some of this was out of date or missing.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-06-08 15:45:49 +01:00
Justin Cormack
0eb4a49b24 Import yaml documentation from LinuxKit
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-06-08 14:58:29 +01:00
Justin Cormack
35ee128bfd Merge pull request #75 from justincormack/perm
Allow setting of mode in files section
2017-06-08 14:54:39 +01:00
Justin Cormack
83c166d3f2 Allow setting of mode in files section
Also keep track of directory creation there, so you can explicitly
set directory permissions if required, and to avoid duplicates.

We should really keep track of files created elsewhere in the build
as well as we still might create some extras, but at least you can
set the write permisisons.

We can add uid, gid support too if required...

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-06-08 14:44:02 +01:00
Justin Cormack
e0aac90f44 Merge pull request #74 from justincormack/raw-name
Fixes for output types
2017-06-07 16:52:35 +01:00
Justin Cormack
98732e8936 Fixes for output types
Make `raw` output type call its file `.raw` as well, other minor fixes.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-06-07 16:46:36 +01:00
Justin Cormack
68ffbb5e7e Merge pull request #72 from justincormack/rename-images
Rename img→raw and gcp-img→gcp
2017-06-07 16:38:29 +01:00
Justin Cormack
d906292096 Merge pull request #73 from justincormack/docker
Add an output format for running images with Docker
2017-06-07 16:38:14 +01:00
Justin Cormack
9a3ac36236 Add an example of building an application and running in Docker or LinuxKit
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-06-07 16:24:32 +01:00
Justin Cormack
dbe50976c3 Add suitable init for running in a container
This goes with the experimental `docker` output format.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-06-07 15:33:52 +01:00
Justin Cormack
8abeffde59 Add Docker output format
This will add a Dockerfile which will build the contents into an
image and then call `tinit` to start it.

This is fairly experimental, but is a prototype for other non
LinuxKit outputs. The container will need to run as `privileged`
as `runc` needs quite a few capabilities and `containerd` needs to
mount.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-06-07 15:32:02 +01:00
Justin Cormack
eabe03b702 Rename img→raw and gcp-img→gcp
- generally people refer to a plain disk image as `raw`
- `gcp` is shorter and it is the only image type supported
- remove `img-gz` as it is not needed. It does not really save space
  as you have to build the full image and compress it anyway. On
  many platforms the `raw` image will be a sparse file anyway,
  even on the Mac soon.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-06-07 14:43:43 +01:00
Justin Cormack
ca677b3c40 Merge pull request #71 from justincormack/output-file
Use streaming output where possible
2017-06-07 14:28:44 +01:00
Justin Cormack
c2806000e0 Use streaming APIs rather than buffers for building
This should lower memory usage a lot and should be faster.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-06-07 12:13:12 +01:00
Justin Cormack
b086231008 Allow streaming output for tar
This is a little ugly in terms of the validation now, but it is a move towards
splitting "build" and "package".

The "tar" output (and soon others) can output direct to a file or to stdout.
Obviously you can only build a single output format like this.

The LinuxKit output formats that build disk images cannot stream as they
have to build whole images. These allow multiple outputs.

In future we will probably change to
```
moby build | moby package
```

or similar, but that is a bit ugly, so currently have a compromise where
there are essentially two output types.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-06-06 14:50:28 +01:00
Justin Cormack
181c66d42c Merge pull request #70 from justincormack/multiple-yaml
Allow specification of multiple yaml files for a single build
2017-06-05 16:39:31 +01:00
Justin Cormack
3a16c02f10 Merge pull request #69 from justincormack/gcp-fix
Do not use Linuxkit to build GCP images for now
2017-06-05 15:37:48 +01:00
Justin Cormack
e879037646 Disable use of linuxkit for GCP for now
GCP does not recognise the images, even though they appear identical to those made
by libguestfs and work on qemu fine. Their validation code does not like them for some
reason.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-06-05 15:31:45 +01:00
Justin Cormack
503545a2de Merge pull request #67 from riyazdf/trust-caching
trust: check local image cache to not pull every time
2017-06-04 19:31:47 +01:00
Justin Cormack
54e58f2808 Allow specification of multiple yaml files for a single build
Each section will be appended in order of the CLI, other then
kernel where last specified one wins.

This is useful if you eg want to have a base version for (say)
AWS and GCP and then add your own image on top.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-06-04 17:50:13 +01:00
Riyaz Faizullabhoy
6c6190172c trust: include trust configuration in docker inspect for OCI config
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-06-02 15:18:55 -07:00
Justin Cormack
058ce768f4 Merge pull request #66 from justincormack/linuxkit-part2
Add the temporary directories under ~/.moby
2017-06-02 22:39:41 +01:00
Justin Cormack
5b73f53dcf Put tmp direcories under ~/.moby
Docker4Mac won't bind mount default tmp dir into containers

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-06-02 22:36:21 +01:00
Riyaz Faizullabhoy
6221552204 trust: check local image cache to not pull every time
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-06-02 13:03:47 -07:00
Justin Cormack
294389aef9 Merge pull request #49 from justincormack/linuxkit
Use linuxkit to build qcow2 and raw image disks, rather than docker containers with libguestfs
2017-06-02 16:39:28 +01:00
Justin Cormack
249bbc56ed Use linuxkit to build qcow2 and raw image disks, rather than docker containers with libguestfs
- does not require docker if user has qemu natively, will still fall back to docker
- allow specifying size for fixed size disk images
- add a raw disk output format
- more dogfooding
- marginally slower, but can be improved later

The images used to do the build are cached to make the process quicker.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-06-02 16:33:06 +01:00
Justin Cormack
d599aedb7f Merge pull request #62 from riyazdf/config-dir-trust
add default trust config and cache dir
2017-06-02 16:25:52 +01:00
Riyaz Faizullabhoy
46b77f699f add default config and cache dir, initially use for trust
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-06-02 08:14:03 -07:00