Done as follows:
find -name build.yml | xargs dirname | while read d ; do
t=$(linuxkit pkg show-tag $d)
./scripts/update-component-sha.sh --image ${t%:*} ${t#*:}
done
git commit -s test pkg tools blueprints examples projects/kubernetes projects/swarmd docs linuxkit.yml Makefile src
This explicitly excludes projects/* which I did not know whether to update.
Then:
git reset --hard
for i in init runc containerd ca-certificates sysctl dhcpcd getty rngd ; do
o=$(git grep -h "\(image:\|-\) *linuxkit/$i:[0-9a-f]\{40\}" origin/master:linuxkit.yml | awk '// { print $2 }')
n=$(linuxkit pkg show-tag pkg/$i)
./scripts/update-component-sha.sh "$o" "$n"
done
git commit --amend projects
This updates any projects which were using components with the same hash as the
top-level linuxkit.yml.
Signed-off-by: Ian Campbell <ijc@docker.com>
Signing is not setup for this repositories. It seems it hasn't been updated
since before trust was added since the previous Makefile metadata (which was
translated in build.yml) did not contain `NOTRUST=1`.
Signed-off-by: Ian Campbell <ijc@docker.com>
By running:
./scripts/update-component-sha.sh --image linuxkit/alpine ad35b6ddbc70faa07e59a9d7dee7707c08122e8d
Signed-off-by: Ian Campbell <ijc@docker.com>
Current versions of Docker complain:
Sending build context to Docker daemon 5.12kB
Step 1/4 : FROM linuxkit/guestfs@sha256:62e810869e83dabe04af7ac8fcd81d9f177d986a3424dcb9c4eb5439993c962f
...
Successfully tagged linuxkit/mkimage-gcp:736880776838c8f41c081276d9c017ae6adbf1bb-amd64
Tagging linuxkit/guestfs@sha256:62e810869e83dabe04af7ac8fcd81d9f177d986a3424dcb9c4eb5439993c962f as linuxkit/guestfs:aecc40bf84ce4f4238e06934c46d7cc570a39eed@sha256:62e810869e83dabe04af7ac8fcd81d9f177d986a3424dcb9c4eb5439993c962f
refusing to create a tag with a digest reference
../../pkg/package.mk:82: recipe for target 'tag-y' failed
Just use the digest for now. Perhaps someone will have time to fixup this build
properly at some point.
Make the same change to test/pkg/ltp.
Signed-off-by: Ian Campbell <ijc@docker.com>
Using explicit targets is cleaner and easier to follow.
Also add `forcepush` to pkg and test/pkg and make `push` explicit.
The `dirs` phony was non-existent in all three cases. Likewise `$(DIR)` wasn't
used in that way in pkg or test/pkg and is nolonger used in that way in tools.
Finally, `clean` was bogus in all cases (tools cleaned a thing which never
exists and pkg and test/pkg were a syntax error).
Signed-off-by: Ian Campbell <ijc@docker.com>
In particular also fix the wireguard test whose kernel
tag hasn't been updated for quite some time...
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
- Added new argument -force to the format utility. This will force formatting of the specified single device if it exists and is a block device
- By default, the format package will no longer format the specified single device if content exists on that device
- Added new blkid-based check for content, in addition to the existing check for partitions on autoformat candidate devices.
- Cleanup of old unused code.
- Refactoring of block device verification.
- Added test/cases/040_packages/006_format_mount/005_by_device_force
- Added document of new arguments to /docs/external-disk.md. Also sorted the arguments in the doc alphabetically and added them as bullets so they do not run together on the page.
Signed-off-by: Isaac Rodman <isaac@eyz.us>
golint on pkg/init now complains:
golint...
./init.go:199:2: redundant if ...; err != nil check, just return error instead.
Resulting in a change which doesn't seem like an improvement to me.
Signed-off-by: Ian Campbell <ijc@docker.com>
- Extract common portions from the YAML file
This makes kernel/init/runc updates easier as only one
files needs changing
- Rename test-ns.yml to test.yml to make it more in line
with the other tests
- Use '-name' in the moby build
- Simplify cleanup
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
- Consistently use env variables for names
- Make sure temporary files are removed (the 'create'
images were left behind)
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
Build tests don't need the 'find' nor 'rm -r' as they don't
create directories.
There is no need to delete the 'test.yml' file in
the output tests.
Consistently use NAME as the name argument
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
This removes things we do not need and expects the interface to
already be created.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
This is less confusing as there is also an output option to set the file.
See https://github.com/moby/tool/pull/146
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
This is based on the example, but modified so that it can work as a test.
It is slightly less convenient running services as tests as the output is
sent to log files, so we have an `onshutdown` container that checks to see
if the test passed.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
As discussed before, as we use this in three places, cloning in
base makes more sense.
Update base image.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
- xfsprogs is required for mkfs.xfs for some tests.
- mount persistent disk on /var/lib instead of /var.
- include host /dev (for loop devices) in mounts.
- /tmp need not be exec.
Signed-off-by: Ian Campbell <ijc@docker.com>
`${CONTAINERD_COMMIT}` is only defined in the `FROM linuxkit/alpine` section
and not the in `FROM scratch` section.
This means we were always checking out the current master at the time of build
rather than the expected commit because `git checkout $CONTAINERD_COMMIT` was
expanding to `git checkout `.
Signed-off-by: Ian Campbell <ijc@docker.com>
In order to update these tests it required first running df -h to obtain
the new disk size and then adding that value to check.sh
This commit replaces the hardcoded values with a regex that checks that
the filesystem is somewhere between 400-599MB in size. Before being
extended this would have been somewhere in the region of 200-300MB.
Signed-off-by: Dave Tucker <dt@docker.com>
This commit removes the dependency on Docker by using raw disks and
appending from /dev/zero to extend them once initially formatted.
Additionally, the tests now use unique ${NAME} variables
Signed-off-by: Dave Tucker <dt@docker.com>
The following packages did not work on aarch64:
- linuxkit/test-docker-bench
- linuxkit/test-ltp
- linuxkit/test-ns
- linuxkit/test-virtsock
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
It is not in any wa=y a required container, and now that arm64
and other architecture machines are widely available we should
start to deprecate it, as it has many issues, eg requires patches
to qemu for Go support, will mislabel images etc.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Mounting a directory inside a read only container requires that to be
created in advance, but `runc` worked around that if the rootfs was not
originally read only.
You cannot even bind mount a file that does not exist into a
read only container.
The containerd test is given a disk, as running on an overlay does
not work; however it is also disabled as one of the parts of the test
is failing, needs investigation.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
It has been EOLed today and won't receive any further updates.
The images are still on hub so can be continued to be used
for the time being.
4.12 support is coming soon.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
The newest tests actually run containerd and therefore have some additional
requirements:
- containerd + shim + runc binaries are needed. We bind these in from the host.
The test code should, by design, be from matching containerd source, assuming
we remember to update test/pkg/container/Dockerfile when we bump
CONTAINERD_COMMIT. 5217b9973b added a reminder
to do so.
- the tests need networking (to pull images). So add dhcp to onboot and bind
/etc/resolv.conf into the test container.
- running containers requires a writeable cgroup mount.
- containerd wants /etc/localtime, so install the UTC one (as we do in
pkg/containerd).
The test image already has `net: host` and `capabilities: all`.
Signed-off-by: Ian Campbell <ijc@docker.com>
The logrus import path has changed, so adjust. Also there is a minor API change
to the containerd.IOCreation() function spec, it now takes a string id which we
can ignore.
Signed-off-by: Ian Campbell <ijc@docker.com>
with cwd of test this was done with:
for i in pkg/* ; do make --no-print-directory -C $i show-tag; done | ( IFS=: ; while read image hash ; do ../scripts/update-component-sha.sh --image $image $hash ; done )
Note that `linuxkit/test-virtsock` (built by `test/pkg/virtsock`) does not
appear to be referenced anywhere.
Signed-off-by: Ian Campbell <ijc@docker.com>
Makefile and Dockerfile are implicit from pacakge.mk.
Need to list the other files consumed by the Dockerfile though.
template.yml is only for manual testing and so is not a dependency of the
standard build.
Signed-off-by: Ian Campbell <ijc@docker.com>
This makes the package actually build reproducibly, with the downside that it
requires changing the hash. Perhaps this should move to tools/alpine.
Signed-off-by: Ian Campbell <ijc@docker.com>
This commit adds a myriad of test cases to ensure the format and mount
and extend packages are working as expected
Signed-off-by: Dave Tucker <dt@docker.com>
These tests run individual and a mix of namespace stress tests
mostly around networking and unix domain sockets where either
the client or the server of socket echo application is run inside
a container in different configurations:
- different protocols
- short or long lived connections
- different levels of concurrency
Tests are only run if the 'kernel' label is specified and more
detailed tests are run if the additional 'kernel-extra' label
is specified.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
The previous version just created a network name space which does
not allow us to also test additional namespaces, e.g. for unix
domain sockets.
This commit uses runc to create a fully namespaced container to
run a test in. It creates a container, configures the network
interfaces in the new network namespace before starting the
container.
A OCI config.json template is used and then customised for a
given test based on command line arguments.
Finally, instead of iperf, we use the socket stress test from
https://github.com/linuxkit/virtsock as it provides finer-grained
control over the traffic patterns (e.g. long lived vs lots of
short lived connections).
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
This is actually containerd#1141 rebased onto v1.0.0-alpha1.
The `dist` command has been integreated into `ctr` and so is removed, including
from the getty and sshd bind mounts and the test which uses it is updated..
There is no change to the version of runc vendored by containerd, so this is
unchanged.
Signed-off-by: Ian Campbell <ijc@docker.com>
This is what CONFIG_DEBUG_SET_MODULE_RONX was renamed to; since we want
that one, presumably we want this one too.
Signed-off-by: Tycho Andersen <tycho@docker.com>
It is pretty close to our docker package, if we adjust the command
that is run to avoid the actual dind startup script. We can't use
the normal docker image as it does not have mkfs and so on.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
It is a far smaller download. Also pin to a specific version to protect against
the possibility of the output string changing.
The new image name is a bit of a mouthful, put it (and the expected output)
into a variable.
Signed-off-by: Ian Campbell <ijc@docker.com>
This test was building all the supported output formats and so was taking
orders of magnitude longer than every other test. It was also obscuring which
output formats were slow and/or broken.
The vhd test remains skipped.
Signed-off-by: Ian Campbell <ijc@docker.com>
pull in newer containerd v1.0.0-alpha0 via updated alpine base, update runc to
429a5387123625040bacfbb60d96b1cbd02293ab which is vendored by that version of
containerd (and also update alpine base for runc)
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
These are not needed, but we are inconsistent. Been waiting for a
quiet moment to fix this since I noticed while doing a presentation...
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
These test specific combinations of TCP/UDP/IPv4/IPv6/veth/loopback
for all the kernels.
Tests take quite a while to run so there are two new labels:
'kernel' and 'kernel-extra'. 'kernel' runs the more important
tests for the two kernels we care most about. 'kernel-extra'
are additional tests for completeness.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
- Use the new style kernel tags with the full kernel version
- Update packages with new alpine base and new/simplified Makefiles.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
This is based on examples/getty.yml modified to drop console=tty0 from the
command line since we will be capturing/logging only the serial so want
everything to go there.
Also updates the getty example to pickup the latest containerd and
ca-certificates as used in the top level linuxkit.yml.
Fix a typo in the containerd test too.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
See #1872 this is very flaky and without a controlled known
working environment this is not helpful.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
The previous commit moved some filesystems to kernel modules.
modprobe them before checking if they are enabled.
Also update the YAML files for the tests.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
Adds a new test group "linuxkit.build.blueprints" and a test for
building blueprints/docker-for-mac.yml.
Signed-off-by: Magnus Skjegstad <magnus@skjegstad.com>
This includes https://github.com/containerd/containerd/pull/994 and hence
requires updating the various instances of `/etc/containerd/config.toml`.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
Currently it supports only `service start <SERVICE>`, but it could grow e.g.
`stop`, `exec` etc in the future (although you can still use `ctr` for those).
In order to be able to use go-compile.sh the containerd build needs to move
from /root/go to /go as the GOPATH.
The vendoring situation is not ideal, but since this tool wants to be an exact
match for the containerd it seems tollerable to reuse its vendoring.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
The new init adds the usermode helper which is needed with
the soon to be pushed new 4.11 kernel update.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
Without `-depth` find will first `rm -rf foo` and then try to recurse into
`foo` resulting in `No such file or directory`.
With this the various `|| true` suffixes should no longer be required.
Lastly, in test/cases/040_packages/013_mkimage/test.sh cleanup any
`disk.qcow2*` detritus as well as the image itself.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
Make sure we do not remove filesystems we expect to have.
Fix the failure cases for the kernel tests which were not working properly
due to shell code.
Fix some 4.11 kernel changes in config that show up once tests are fixed.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
If you also use console=tty0 you lose the output of commands running from
the system, as they run on the second console which is discarded.
Also drop page-poison as not relevant for tests.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
With the updated HyperKit go bindings we can redirect the VM output
and check for test results. Use this for all kernel tests as this
speeds up running the tests on OSX.
Also use 'set -x' instead of 'set -v' for consistency and don't fail
when the clean up code fails.
The mkimage package test currently doesn't work on HyperKit as it
requires support for multiple disks to be added.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
With the updated HyperKit go bindings we can redirect the VM output
and check for test results. Use this for all kernel tests as this
speeds up running the tests on OSX.
Also enable 'set -x' so we see the commands being executed and don't
fail the test if the cleanup failed.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
0eb21735ae accidentally broke some package
builds by switching linuxkit/alpine to linuxkit/containerd. Let's revert
the ones that shouldn't be there.
Closes#1991
Signed-off-by: Tycho Andersen <tycho@docker.com>
- rename img→raw and gcp-img→gcp
- support creating `docker` outputs, see https://github.com/moby/tool/tree/master/examples
- less memory usage via streaming outputs
- allow specification of multiple yaml files in a single command line, improves modularity
- notary fixes and local cache, so does not pull as much
- you now have to specify the full filename of the yaml file, you cannot omit the suffix
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Note that this is not the latest (which was 95efd45db073 at time of writing)
but the next commit 6428b4bad0c2 merges "Port ctr to use client package" breaks
the use of `ctr run --runtime-config` (by removing that option).
This contains https://github.com/containerd/containerd/pull/954 which was
causing some services to fail to start.
All previous uses of 15541037b9 are updated to
5749f2e9e6.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
- no longer uses several of the `mkimage-*` tools in favour of dogfooding
with `linuxkit` and using the `mkimage` package.
- fix the qemu docker container fallbacks to work better when multiple
paths are used for disks and the image.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
This changes the CLI specification for disks, as it needs to be able to
be repeated.
```
linuxkit run qemu -disk name,size=1G,format=qcow2 ...
```
Options may be omitted.
Currently other local backends may not support multiple disks, but this
can be added in future. Code for cloud backends has not changed as the
disk support is specific to the platform.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Update everything including the current
linuxkit/containerd:b1766e4c4c09f63ac4925a6e4612852a93f7e73b to the new
linuxkit/containerd:deaf5bf838bf7f131c2287ecff3ed9835b0497e2.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
This test check that the link is up, that we got a IP address
and the output of the dhcpcd daemon.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
Don't rely on systctl test package. Instead, add the check shell
script to the image and execute it with a standard Alpine image.
While at it, make the test more verbose.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
Use the index in the alphabet (a=1...) to group the tests
a little bit. The actual order does not matter.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
- Update to packages using the Alpine 3.6 base image
- Remove config for packages which now supply it
- Update/add trust section
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
The remaining packages will be updated with subsequent commits
to also include the config label.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
The latest version of the `moby` tool now requires that the output formats
be specified in the CLI not in the yaml file.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
