Commit Graph

133 Commits

Author SHA1 Message Date
James Hewitt
c582c4844f Add option to preserve digests on copy
When enabled, if digests can't be preserved an error will be raised.

Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2021-12-07 13:16:10 +00:00
James Hewitt
2046bfdaaa Add option to preserve digests on copy
When enabled, if digests can't be preserved an error will be raised.

Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2021-12-07 13:16:10 +00:00
Colin Walters
3606b2d1de proxy: Add a GetFullConfig method
Sadly...I swear I had tested this at one point, but it was
*definitely* not the intention that we just return the container
runtime configuration.

I need a method to return the full image configuration.  At some point
I must have accidentally added a redundant `.Config`.

This whole new method `GetFullConfig` is like `GetConfig` but
returns the whole image configuration.  A specific motivation
here is that it's only in the image configuration that we can
stick arbitrary metadata (labels) that will survive a round trip through
docker schema v2.
2021-12-06 17:15:46 -05:00
James Hewitt
d94015466f Add an option to allow copying image indexes alone
The new --multi-arch option allows the user to select between copying the
image associated with the system platform, all images in the index, or
just the index itself without attempting to copy the images.

Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2021-11-27 15:38:42 +00:00
James Hewitt
4dcd28df92 Use a dynamic temp dir for test
This test was incorrectly assuming that nothing would be made on disk,
but it was putting files into the source directory.

Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2021-11-27 13:44:28 +00:00
Colin Walters
e9535f868b tests: Add new "procutils" that exposes PDEATHSIG
To fix compilation on MacOS.

I think actually we want to use this pervasively in our tests
on Linux; it doesn't really matter when run inside a transient
container, but `PDEATHSIG` is useful for persistent containers (e.g.)
toolbox and when running outside of a pid namespace, e.g. on a host
system shell directly or in systemd.

Signed-off-by: Colin Walters <walters@verbum.org>
2021-11-15 21:02:14 -05:00
Colin Walters
fa86297c36 proxy_test: Test GetConfig
Now that we have a test suite, let's use it more.

Signed-off-by: Colin Walters <walters@verbum.org>
2021-11-15 21:02:14 -05:00
Colin Walters
2bb6f27d13 proxy_test: Add helper to read all from a reply
Prep for testing `GetConfig`.

Signed-off-by: Colin Walters <walters@verbum.org>
2021-11-15 21:02:14 -05:00
Colin Walters
f90725d80c proxy_test: Add a helper method to call without fd
To verify in one place.

Signed-off-by: Colin Walters <walters@verbum.org>
2021-11-15 21:02:14 -05:00
Colin Walters
644074cbb4 proxy: Add support for manifest lists
We need to support manifest lists. I'm not sure how I missed this
originally.  At least now we have integration tests that cover this.

The issue here is fairly subtle - the way c/image works right now,
`image.FromUnparsedImage` does pick a matching image from a list
by default.  But it also overrides `GetManifest()` to return the
original manifest list, which defeats our goal here.

Handle this by adding explicit manifest list support code.  We'll
want this anyways for future support for `GetRawManifest` or so
which exposes OCI manifest lists to the client.

Signed-off-by: Colin Walters <walters@verbum.org>
2021-11-15 21:02:14 -05:00
Colin Walters
83416068d3 tests/integration/proxy_test: New test that exercises proxy.go
I debated adding "reverse dependency testing" using
https://crates.io/crates/containers-image-proxy
but I think it's easier to reuse the test infrastructure here.

This also starts fleshing out a Go client for the proxy (not
that this is going to be something most Go projects would want
versus vendoring c/image...but hey, maybe it'll be useful).

Now what I hit in trying to use the main test images is currently
the proxy fails on manifest lists, so I'll need to fix that.

Signed-off-by: Colin Walters <walters@verbum.org>
2021-11-15 21:02:14 -05:00
Miloslav Trmač
0e87d4d1ca Run (gofmt -s -w)
Go 1.17 introduces a much more reasonable build constraint format, and gofmt now fails without using it.

Sadly we still need the old format as well, to support <1.17 builds.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2021-08-23 18:04:45 +02:00
Miloslav Trmač
c399909f04 Update non-module dependencies
Dependabot was apparently not picking these up (and
several haven't had a release for a long time anyway).

Also move from github.com/go-check/check to its newly
declared (and go.mod-enforced) name gopkg.in/check.v1.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2021-08-23 15:52:48 +02:00
Miloslav Trmač
ce6035b738 Add timeouts when waiting on OpenShift or the registry to start
... so that we terminate with the full context and pointing at
the relevant code, instead of relying
on the overall test suite timeout.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2021-07-30 12:36:54 +02:00
Miloslav Trmač
a1f9318e7b Fix two instances of unused err found by go-staticcheck
*cough* Go is an easy-to-use, hard-to-misuse language *cough*

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2021-07-28 21:46:07 +02:00
Chris Evich
285a5cb6a0 Fix using images from rate-limited docker hub
The necessary images have been manually copied over to quay.  Code was
updated with centralized constants for the utilized images.  Tests then
all reference the constants (in case the image locations need to change
again).

Signed-off-by: Chris Evich <cevich@redhat.com>
2021-07-22 16:28:20 -04:00
Miloslav Trmač
f80bf8a39f Revert "integration tests: disable ls for logs"
This reverts commit 65c5b0bf8d.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2021-07-09 13:15:13 +02:00
Daniel J Walsh
6d6c8b5609 Update tests for removal of error and Error from error messages
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2021-07-01 21:02:13 +02:00
Daniel J Walsh
b10d3e43a4 Merge pull request #1266 from cgwalters/copy-print-digest
copy: Add --digestfile
2021-05-07 05:17:35 -04:00
Colin Walters
a32be320cb copy: Add --digestfile
`podman push` gained this a while ago, and we want it here for the
same reason.

Motivated by closing a race condition in ostree-rs-ext:
17a991050c/lib/src/container/export.rs (L85)

Co-authored-by: Miloslav Trmač <mitr@redhat.com>
2021-05-06 09:19:16 -04:00
Brendan Aye
7898ffaf23 Added format parameter to sync command
Signed-off-by: Brendan Aye <brendan.aye@t-mobile.com>
2021-04-28 10:54:21 -07:00
Lokesh Mandvekar
b7bf15bc8b Dockerfile.build: switch to fedora:latest
CopySuite.TestCopySignatures: Use quay.io instead of docker.io.

Resolves: gh#174

Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
2021-02-24 15:49:14 +01:00
Daniel J Walsh
18e6c6f17b Fix Makefile to handle PREFIX correctly
PREFIX should default to /usr/local according to GNU coding standards.

DESTDIR should just be used for install

This will allow us to specify

make PREFIX=/usr DESTDIR=/tmp/build/ install

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-01-21 13:54:53 -05:00
Miloslav Trmač
3c33cb4556 Add a smoke test for signedIdentity:remapIdentity
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2021-01-08 17:20:09 +01:00
Miloslav Trmač
f94d85aa8e Split copyWithSignedIdentity from TestCopyVerifyingMirroredSignatures
... because we will add another user.

Also remove an obsolete FIXME.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2021-01-08 17:20:09 +01:00
Nalin Dahyabhai
6ee4b2dc84 integration test: sync k8s.gcr.io/pause instead of docker.io/alpine
Do repository synchronization tests using k8s.gcr.io/pause and
k8s.gcr.io/coredns/coredns instead of docker.io/alpine.
The k8s.gcr.io/pause repository includes multiple tags, at least one of
which is a single image ("1.0"), and at least one of which is a manifest
list ("3.2", "3.3"), and includes a "latest" tag.
The k8s.gcr.io/coredns/coredns repository includes multiple tags, at
least one of which is a single image ("v1.6.6"), and at least one of which
is a manifest list ("v1.8.0").

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2021-01-08 09:51:08 -05:00
Nalin Dahyabhai
b3a15e7288 integration test: use fedora-minimal for most manifest list tests
Switch most of our tests that exercise reading, copying from, and
inspecting tags that point to manifest lists from using
docker.io/estesp/busybox to using
registry.fedoraproject.org/fedora-minimal, which doesn't limit how often
we can pull the images.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2021-01-08 09:50:12 -05:00
Miloslav Trmač
c4fb93647a Fix reading the after-sync list of tags in SyncSuite.TestYamlUntagged
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2021-01-08 11:16:20 +01:00
Josh Soref
4987a67293 Spelling
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
2020-12-21 16:10:45 -05:00
Andrew DeMaria
873fbee01b Fix #858 Add --all sync flag to emulate copy --all
This replicates the --all copy flag to sync to perform the same
behavior. Namely, the default is CopySystemImage unless --all is passed
which changes the behavior to CopyAllImages. While it is probably
desirable for --all to be the default as there is no option to override
ones architecture with the sync command, --all can potentially break
existing sync incantations depending on registry support. Hence
CopySystemImage remains the default.

Signed-off-by: Andrew DeMaria <ademaria@cloudflare.com>
2020-11-16 15:18:56 -07:00
Andrew DeMaria
1a3eb478a7 Fix #858 Add support for digests in sync
Signed-off-by: Andrew DeMaria <ademaria@cloudflare.com>
2020-11-16 15:18:53 -07:00
Valentin Rothberg
65c5b0bf8d integration tests: disable ls for logs
For yet unknown reasons, Travis throws permission errors when trying to
recursively list the contents of a temp directory.  It passes locally,
so disable the logs to unblock CI.  Note, the reasons for the error are
yet to be revealed.

Related-issue: https://github.com/containers/skopeo/issues/1093
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-11-04 16:52:54 +01:00
Daniel J Walsh
62bafb102d Fix problems found by codespell
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-09-11 10:19:16 -04:00
Wong Hoi Sing Edison
0f458eec76 Add nix derivation for static builds
Signed-off-by: Wong Hoi Sing Edison <hswong3i@gmail.com>
2020-06-18 20:57:55 +08:00
Miloslav Trmač
322625eeca Use a separate field for the "sync images with tag matching regex" feature
Fields that magically change their behavior depending on type of the value
are too much hassle for no benefit.

For now, this just copies&pastes the full loop in imagesToCopyFromRegistry
to create another loop handling the new ImagesByTagRegex field. Simplifications
to reduce duplication will follow.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2020-05-25 11:58:19 +02:00
zhangguanzhang
bec7f6977e Add tags to support regular expressions in yaml conf
add relevant test cases and documentation
update the sync doc
2020-05-20 09:23:00 +08:00
Qi Wang
0ec2610f04 Add skopeo login&logout
Implements skopeo login&logout commands.

Signed-off-by: Qi Wang <qiwan@redhat.com>
2020-05-11 16:35:46 -04:00
Miloslav Trmač
a6ab2291ba Add tests for using signatures with mirrors
... to test the fix for https://github.com/containers/image/pull/912 .

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2020-05-11 14:37:43 +02:00
Qi Wang
46fbbbd282 Use cobra in skopeo
Use cobra in skopeo can help share code with podman/buildah(code for skopeo login/logout CLI).
(libpod issue  #839)

Signed-off-by: Qi Wang <qiwan@redhat.com>
2020-05-04 14:50:15 -04:00
Miloslav Trmač
02407d98a5 Fix TestCopyAtomicExtension
Actually test that X-R-S-S signatures are usable using atomic:,
as the comment says.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2020-05-01 01:27:03 +02:00
Valentin Rothberg
ed883c5230 Merge branch 'master' into feat-override-variant 2020-03-20 15:43:39 +01:00
Harshal Patil
8d1a4649f2 Partial image encryption support
Signed-off-by: Harshal Patil <harshal.patil@in.ibm.com>
2020-02-20 07:53:26 +05:30
Jirka Chadima
222beaf4c7 Adds the --override-variant command line flag
Signed-off-by: Jirka Chadima <chadima.jiri@gmail.com>
2020-01-09 15:13:28 +01:00
Flavio Castelli
9c402f3799 Introduce the sync command
The skopeo sync command can sync images between a SOURCE and a
destination.

The purpose of this command is to assist with the mirroring of
container images from different docker registries to a single
docker registry.

Right now the following source/destination locations are implemented:

  * docker -> docker
  * docker-> dir
  * dir -> docker

The dir location is supported to handle the use case
of air-gapped environments.
In this context users can perform an initial sync on a trusted machine
connected to the internet; that would be a `docker` -> `dir` sync.
The target directory can be copied to a removable drive that can then be
plugged into a node of the air-gapped environment. From there a
`dir` -> `docker` sync will import all the images into the registry serving
the air-gapped environment.

Notes when specifying the `--scoped` option:

The image namespace is changed during the  `docker` to `docker` or `dir` copy.
The FQDN of the registry hosting the image will be added as new root namespace
of the image. For example, the image `registry.example.com/busybox:latest`
will be copied to
`registry.local.lan/registry.example.com/busybox:latest`.

The image namespace is not changed when doing a
`dir:` -> `docker` sync operation.

The alteration of the image namespace is used to nicely scope images
coming from different registries (the Docker Hub, quay.io, gcr,
other registries). That allows all of them to be hosted on the same
registry without incurring in clashes and making their origin explicit.

Signed-off-by: Flavio Castelli <fcastelli@suse.com>
Co-authored-by: Marco Vedovati <mvedovati@suse.com>
2019-12-02 13:42:39 +01:00
Marco Vedovati
8ccd7b994d openshift cluster: remove .docker directory on teardown
Remove the $HOME/.docker directory when tearing down a cluster,
so that subsequent cluster creations can be carried out successfully.

Signed-off-by: Marco Vedovati <mvedovati@suse.com>
2019-12-02 13:42:39 +01:00
Harshal Patil
39ff039b3b Image encryption/decryption support in skopeo
Signed-off-by: Harshal Patil <harshal.patil@in.ibm.com>
Signed-off-by: Brandon Lum <lumjjb@gmail.com>
2019-11-26 15:17:38 +05:30
Miloslav Trmač
af0734832b Use a Windows Nano Server image instead of Server Core for multi-arch testing
This image is about 100 MB instead of about 2 GB for the Server Core,
decreasing disk requirements and hopefully significantly speeeding up
integration tests.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2019-11-21 21:25:35 +01:00
Valentin Rothberg
8f24d28130 vendor containers/image v5.0.0
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2019-10-28 10:37:46 +01:00
Nalin Dahyabhai
7d251f5a74 copy: add a --all/-a flag
Add a --all/-a flag to instruct us to attempt to copy all of the
instances in the source image, if the source image specified to "skopeo
copy" is actually a list of images.  Previously, we'd just try to locate
one for our preferred OS/arch combination.

Add a couple of tests to verify that we can copy an image into and then
back out of containers-storage.  The contents of an image that has been
copied out of containers-storage need a bit of tweaking to compensate
for containers-storage's habit of returning uncompressed versions of the
layer blobs that were originally written to it, in order to be
comparable to the image as it was when it was pulled from a registry.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2019-10-22 11:05:42 -04:00
Valentin Rothberg
7922028d7c Update to c/image v4.0.1
Update to use the correct c/image/v4 import path, work originally from
https://github.com/containers/skopeo/pull/733 by Valentin Rothberg <rothberg@redhat.com>.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2019-10-03 23:50:42 +02:00