I am doing some upstream `runc` work with kernel keys and have
various other uses. No urgency so not updating the package
builds yet.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
This utility image takes a tarball as input and places the
contents into a read-only, compressed squashfs filesystem
which is produced on stdout.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
Most of the tools packages are not usable on s390x so
explicitly list them.
Also removed arm64 from mkimage-gcp as GCP does not
support arm machines and fixed a minor inconsistency
the way the architecture was specified in mkimage-raw-bios.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
- On macOS, docker-credential-osxkeychain.bin was renamed to
docker-credential-osxkeychain
- Pass --ignore-missing to the manifest-tool invocation.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
This is temporary to un-break the build until we have pushed
a alpine base image for s390x.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
Currently, there is a conflict in musl versions between stable
(used for tools/alpine) and edge (where wireguard-tools is).
This cased the tools/alpine build to fail.
With this commit we build our own wireguard-tools package,
using the APKBUILD file from edge, against the libraries
libraries from stable. We then add the wireguard-tools package
to the mirror.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
Add support for s390 architecture for linuxkit/alpine and the
other docker images in tools and pkg.
Signed-off-by: Alice Frosi <alice@linux.vnet.ibm.com>
Looks like bc791e60e7 ("Update docs and test cases to work with
latest format versions") accidentally replaced 'qcow2' with
'qcow2-bios' for two of the output formats. Fix it.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
This part reverts f7c50156c5 ("tools: pull linuxkit/guestfs by digest
only (not digest+tag)") as pulling by tag now seems to work and we
have content trust enabled, so pulling by digest should not be necessary.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
Moby is moving to using pigz instead of the the golang gzip
library (https://github.com/moby/moby/pull/35697)
Include pigz in the base for downstream projects to use.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
The iucode_tool is used to convert the Intel CPU microcode
binaries into a cpio archive. There is no alpine package
for it, so compile it from source.
This is for x86_64 only and on other archs we create a empty
file.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
Before the alpine base image is multi-arch and signed, the
DOCKER_CONTENT_TRUST=1 doesn't work on AArch64 for 'docker build'.
Now since the alpine base image is already multi-arch and signed,
also we've used 'push_manifest.sh' to push and sign linuxkit/image,
so we can remove this workaround.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Remove the grub2 patch from LinuxKit since it has already been merged
into the CoreOS grub2 mainline code.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
The package contains u-boot and the RPi firmware blobs. It expects
a tar ball of the root filesystem (including kernel and dtbs) on stdin
and produces a compressed tar ball on stdout with the files to copy to
a FAT32 formatted SD card.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
Note that a237b51511 ("Bump alpine base across the board.") inadvertently
corrupted `tools/alpine/versions.aarch64` (by updating it to the amd64 hash),
this commit corrects this as part of the rebuild.
Signed-off-by: Ian Campbell <ijc@docker.com>
Signing is not setup for these repositories. It seems they haven't been updated
since before trust was added since their previous Makefile metadata (which was
translated in build.yml) did not contain `NOTRUST=1`.
Signed-off-by: Ian Campbell <ijc@docker.com>
By running:
./scripts/update-component-sha.sh --image linuxkit/alpine ad35b6ddbc70faa07e59a9d7dee7707c08122e8d
Signed-off-by: Ian Campbell <ijc@docker.com>
tools/alpine and tools/guestfs are omitted since they do not currently use
pkg/package.mk and do their own thing. Slightly hacky arrangements are made for
these. Note that previously they were only recursed into for their default
target (push) and that behaviour is retained.
Signed-off-by: Ian Campbell <ijc@docker.com>
Current versions of Docker complain:
Sending build context to Docker daemon 5.12kB
Step 1/4 : FROM linuxkit/guestfs@sha256:62e810869e83dabe04af7ac8fcd81d9f177d986a3424dcb9c4eb5439993c962f
...
Successfully tagged linuxkit/mkimage-gcp:736880776838c8f41c081276d9c017ae6adbf1bb-amd64
Tagging linuxkit/guestfs@sha256:62e810869e83dabe04af7ac8fcd81d9f177d986a3424dcb9c4eb5439993c962f as linuxkit/guestfs:aecc40bf84ce4f4238e06934c46d7cc570a39eed@sha256:62e810869e83dabe04af7ac8fcd81d9f177d986a3424dcb9c4eb5439993c962f
refusing to create a tag with a digest reference
../../pkg/package.mk:82: recipe for target 'tag-y' failed
Just use the digest for now. Perhaps someone will have time to fixup this build
properly at some point.
Make the same change to test/pkg/ltp.
Signed-off-by: Ian Campbell <ijc@docker.com>
Using explicit targets is cleaner and easier to follow.
Also add `forcepush` to pkg and test/pkg and make `push` explicit.
The `dirs` phony was non-existent in all three cases. Likewise `$(DIR)` wasn't
used in that way in pkg or test/pkg and is nolonger used in that way in tools.
Finally, `clean` was bogus in all cases (tools cleaned a thing which never
exists and pkg and test/pkg were a syntax error).
Signed-off-by: Ian Campbell <ijc@docker.com>
On x86_64, the blkid package gets installed as a dependency, but
not on arm64. Explicitly add it as the new format package depends
on it.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
Currently projects/kubernetes pulls in the binaries, adding these dependencies
will allow us to build them as part of our packaging process.
Signed-off-by: Ian Campbell <ijc@docker.com>
This patch uses coreos grub2 instead of the built-in gummiboot
tool with Alpine distribution.
Coreos grub2 has the security feature such as TPM and kernel
verification, so we can setup a trust chain when loading
modules with grub2.
GNU grub2[1] also has the plan to add those security related
features, they have a 'verifiers' branch to do that, but
there're some build issue need to fix,so this patch use
coreos as an alternative.
This patch is used to address the #2359#2375.
Thanks Avi Deitcher <avi@deitcher.net> for the contribution
to build the GRUB2 from ubuntu 16.04 to alpine base image.
Change Log:
1.Address the comments raised by @rn
2.Change the '/dev/vda' as the device name point by kernel command
line 'root=' on arm64, '/dev/sr0' on amd64. As next plan, we can adapt
a more flexible method to get the dev name of the CDROM.
3.Switch the base image to build grub2 from ubuntu 16.04 to alpine.
4.'linux' as the grub2 menu entry on arm64, while 'linuxefi' on amd64.
[1] https://git.savannah.gnu.org/git/grub.git
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
This makes it easier to add the containerd testing tools like
`containerd-stress` to test packages, for example, and also at
some point move `ctr` out of the base image as it can be installed
from the alpine image.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
The first is needed by the okernel project and last two to run
configure on the zfs kernel modules.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
As discussed before, as we use this in three places, cloning in
base makes more sense.
Update base image.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
These are needed by Kubernetes which is currently not using the standard abse image.
Also sort it.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
In line with BIOS ISO changes previously.
Remove boot options, user can add if required, they made boot very slow.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Also adjust the 'linuxkit/alpine' script to follow the
same pattern.
The new version of the script extract username/password from
the credential helper (or docker) and build and 'expect'
script to feed the info to 'notary'.
They can be invoked by:
DOCKER_CONTENT_TRUST_REPOSITORY_PASSPHRASE="phrase" ./push-manifest.sh ...
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
Now that LinuxKit works on a truly immutable filesystem, for an
ISO it makes sense to use the ISO filesystem rather than using
an initrd. Only major difference is you may need to specify the
device for the root filesystem in the kernel command line, but
we set a sane default of `/dev/sr0` if unspecified.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
- append a architecture specific suffix (-amd64, -arm64) to the
image
- add a script which looks up the hashes from the 'versions' file
and creates a multi-arch manifest
- the manifest is pushed to hub and signed with notary
The new linuxkit/alpine multi-arch image is available with the
x86_64 hash without the '-amd64' suffix.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
This should reduce instability due to different people's environment. This
undoes the reordering done by "alpine: Update to containerd v1.0.0-alpha2",
leaving just the hash update (as expected).
Signed-off-by: Ian Campbell <ijc@docker.com>
The previous commit moved ovmf to the packages instead
of stashing it on the filesystem. This requires the
package to be installed here.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
We add the 'ovmf' to the mirror instead of installing
it in the rootfs. This way we record the version we are
using.
We already pull a few images (go, tini) from community
so may as well pull ovmf directly via 'packages.x86_64'.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
This makes the need to pull $(BASE) redundant.
Rebuild resulted in linuxkit/guestfs:aecc40bf84ce4f4238e06934c46d7cc570a39eed@sha256:62e810869e83dabe04af7ac8fcd81d9f177d986a3424dcb9c4eb5439993c962f
Signed-off-by: Ian Campbell <ijc@docker.com>
We should always pull and push linuxkit/alpine with content trust
irrespective of architecture, unless explicitly disabled.
Currently, we have to dance around various other issues on arm64,
which are now documented in the Makefile.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
Somewhere between the various updates yesterday the hash in
'versions.x86_64' went wrong and there is no image with hash
available on hub.
This commit updates the alpine base to the latest version and
thus rectifies the issue
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
This was added in 17.06 and allows us to avoid using `$(BASE):build` which is
not safe against parallel builds etc.
Having done this restructure the build to not always delete the built container
and to separate out the `hash` and `version` file rules so that they can be
included in both the `tag` and `push` targets.
Signed-off-by: Ian Campbell <ijc@docker.com>
Alpine is the base docker image for the LinuxKit, but currently
it only supports amd64 architecture. This patch is try to unify
the alpine tool docker image build process order to suport other
architectures, such as AArch64, by using '--build-arg' to override
the alpine base image specified by 'FROM' in the Dockerfile.
Also this patch splits the standalone packages into 2 parts:
one is common for all archs, another is arch-specific.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
This is actually containerd#1141 rebased onto v1.0.0-alpha1.
The `dist` command has been integreated into `ctr` and so is removed, including
from the getty and sshd bind mounts and the test which uses it is updated..
There is no change to the version of runc vendored by containerd, so this is
unchanged.
Signed-off-by: Ian Campbell <ijc@docker.com>
I noticed that running the runes in docs/vendoring.md made a bunch of files
which are omitted by modern vndr come back, presumably due to the old version
of vndr referenced there. So update the go-compile package to the current
version everywhere.
Also correct the use of `make` variable syntax (`$(PWD)`) where shell syntax
command substitution for `pwd` was desired.
Signed-off-by: Ian Campbell <ijc@docker.com>
tools/qemu and toos/go-compile define the IMAGE after the
package.mk, which result in below error if 'make ORG=other_org':
...
invalid argument "other_org/:2c6d9e1d9c52167f4f2b7a8fd235eda318175c99"for t: invalid reference format
See 'docker build --help'.
../../pkg/package.mk:47: recipe for target 'tag' failed
make: *** [tag] Error 125
This because '../../pkg/package.mk' need to use IMAGE variable first.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Otherwise the only record is if the pusher happens to remember to put it in the
commit message (tollerable) or the github PR (not great).
Also add the same show-tag target as `pkg/package.mk` supports, although with a
very different implementation.
The actual hash is unchanged from the previous commit.
Signed-off-by: Ian Campbell <ijc@docker.com>
If a user of linuxkit/alpine wants to produce an image with apk in it then it
is useful for them to have the original upstream repository list.
The new hash is linuxkit/alpine:3744607156e6b67e3e7d083b15be9e7722215e73
Signed-off-by: Ian Campbell <ijc@docker.com>
We expect this (or something very similar) to be merged soon, it fixes#2131 so
moving ahead now.
The new alpine mirror is linuxkit/alpine:6832775a7e861ee2d7842e157688ece52d007142
Signed-off-by: Ian Campbell <ijc@docker.com>
This will make it simpler to temporarily in PRs which are not yet merged (but
are expected to be soon).
Tools alpine is not rebuilt here since we are going to do just that in the next
commit.
Signed-off-by: Ian Campbell <ijc@docker.com>
We should use this from a container at top level instead; it makes it
much harder to port to other architectures if we have to have this as
it is written in Haskell making a multi arch build much harder.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
- iproute2 provides a more complete suite of 'ip' utilities than
the busybox implementation. Useful, for debugging.
- iperf3 for network testing
- tcpdump for network debugging
- expect is used in some tests. Adding it to the base image should
make it easier to create a LinuxKit image to self-host CI.
The new version of the base package is:
linuxkit/alpine:3d57ded3edd54e616210cf0c17e3bc15eed9d03a
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
Since we have to push all new packages due to 5a225b9d
("Makefile: Fix bug where network was not disabled") we
may as well update the Alpine base to the latest.
The new tag is: 9bcf61f605ef0ce36cc94d59b8eac307862de6e1
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
Commit 4699f80e ("test: add test-containerd") added
the containerd commit hash to the alpine base image.
Unfortunately, setting it as ARG does not work and it
needs to be set with ENV for other builds to pick it up.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
We'll use abuild in the next commit for building auditd from source.
Also add these audit deps:
* swig
* python3
* libcap-ng-dev
Signed-off-by: Tycho Andersen <tycho@docker.com>
A subsequent commit will update all packages, so we might as well
update all packages to the latest base.
The new hash is: linuxkit/alpine:c608d404c1cb36cce0c7d9303e30b52c9d81ccf0
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
Moved qemu-img to keep list fully alphabetical.
Rebuild pushed as linuxkit/alpine:c96f56ef989ccb31fd115d25c1b90f05b992271d
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
iAdd a bunch of packages which are useful for compiling
some of the tools shipped with the Linux kernel source code.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
- no longer uses several of the `mkimage-*` tools in favour of dogfooding
with `linuxkit` and using the `mkimage` package.
- fix the qemu docker container fallbacks to work better when multiple
paths are used for disks and the image.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Adding a mkimage package to create dynamic 25GB VHD images (static
VHD images are already supported by LinuxKit).
Signed-off-by: Dave Freitag <dcfreita@us.ibm.com>
This ovmf package is part of the community repository. We don't
want to "pollute" the Alpine base with it. Luckily it's juts a single
file which we can install and then copy to the base image.
The package is needed for qemu EFI boot.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
While not used anywhere, adding it to the based makes it easy
to add temporarily add it to init (or elsewhere) for debugging.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
The Dockerfile is now an input to the contents of the base image and
needs to be included in the hash calculation.
Also, make the Makefile, Dockerfile and pacakges file a dependency.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
We use the "official" Alpine based Docker image and extract
the binary and libraries from it and add to the base. Compiling
it from source would require a Haskell setup...
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
The added packages are necessary to create a base Alpine
container image and for converting more packages.
Update the versions file to the latest versions
Also make the HUB org/registry build time configurable.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
This is a fairly generic bootable disk with syslinux. Should
work if you `dd` it onto a USB stick, and should also work for AWS.
You need to uncompress it of course! Default size is 1G.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
It is not longer used by any package. Some projects
still use it and I haven't updated their builds but they
should continue to work.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
Directly install the required packages from the Alpine base
image to compile the toybox-media package.
Also use the git tree hash for the package hash while at it.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
Use the alpine base image and install the tools to compile
tini manually. This removes the dependency on c-compile.
Also use the git tree hash for the package hash while at it.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
This means that we limit to one place where we use network access
in building, and in future all other package builds can be deterministic.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Previously we hardcoded `bzImage` which is not used for all
use cases or architectures.
fix#1630
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Updated go-compile to be able to compile remotely. Note I
did not update the oter users of go-compile as it does not affect
them.
Update `go get` instructions to fetch new one, or `make && make install`
will still work.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
This was missed when things were renamed.
The intention with this code was (apparently) to provide a (pseudo)unique
hostname in the case where something more specific was not provided (e.g. by
DHCP). Make this a little clearer by using '(none)' rather than 'linuxkit' as
the default, in the normal case this will be overwritten by something more
specific and if it isn't we will change it to something somewhat unique derived
from the MAC address (as before). nb: '(none)' is already used by Debian so I
think it is a safe choice as the sentinel value.
The use of both CONFIG_DEFAULT_HOSTNAME and the explicit /etc/hostname from
mkimage.sh is likely to be redundant in some cases, but neither seems to
completely cover all cases so keep both.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
Previously the "tini" contained the entire build. Using the
multi-stage build creates a new "tini" base image, which only
contains the "/bin/tini". This image can be used as the base
image for packages requiring "tini".
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
Previously, the cmdline file was never sent in the build context which
caused the script to pick the default options.
Also, when mulitple options are specified (not the default) the right
escaping needs to be used.
Signed-off-by: Thomas Gazagnaire <thomas@gazagnaire.org>
This flags allows passing additional ldflags to the build.
It is primarily there to pass -C arguments to specify values
of variables at compile time.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
Generated largely from the specified config; small parts taken from `docker image inspect`,
such as the command line.
Renamed some of the yaml keys to match the OCI spec rather than Docker Compose as
we decided they are more readable, no more underscores.
Add some extra functionality
- tmpfs specification
- fully general mount specification
- no new privileges can be specified now
For nostalgic reasons, using engine-api to talk to the docker cli as
we only need an old API version, and it is nice and easy to vendor...
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
