Commit Graph

147 Commits

Author SHA1 Message Date
Miloslav Trmač
38f4b9d032 Enable schema1 support on the test registries
We expect schema1 images to work.  Also, docker/distribution
doesn't provide useful errors for rejected schema1 images
( https://github.com/distribution/distribution/issues/2925 ),
which makes it impractical for Skopeo to automatically convert
schema1 to schema2.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-06-16 20:27:05 +02:00
Miloslav Trmač
1b5fb465be Update for docker/distribution CLI change
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-06-16 20:27:05 +02:00
Miloslav Trmač
bbdabebd17 Change a repo used for sync tests
The k8s.gcr.io/coredns/coredns repo now contains an OCI
artifact, which we can't copy; so, use a different
repo to test syncing.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-06-11 03:15:14 +02:00
Miloslav Trmač
d9d3ceca45 Use filepath.WalkDir instead of filepath.Walk
... to optimize away some lstat(2) calls.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-04-13 20:14:04 +02:00
Miloslav Trmač
23a4605742 Extract four copies of the same loop into a function
Should not change behavior.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-04-13 20:14:04 +02:00
Miloslav Trmač
4811c07d71 Update users of deprecated io/ioutil
Mostly just name changes that should not change behavior, apart
from ioutil.ReadDir -> os.ReadDir avoiding per-item lstat(2) in
one case.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-04-13 20:13:52 +02:00
James Hewitt
4ef35a385a
Add option to specify the identity for signing
This enables pushing to registries where the push and pull uris may be
different, for example where pushed images are mirrored to a read only
replica for distribution.

Closes #1588

Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2022-03-30 22:02:43 +01:00
Miloslav Trmač
2019b79c7f Use check.C.MkDir() instead of manual ioutil.TempDir() calls
This saves us at least 2 lines (error check, and cleanup) on every
instance, or in some cases adds cleanup that we forgot.

This is inspired by, but not directly related to, Go 1.15's addition of
Testing.T.TempDir.

NOTE: This might significantly increase the tests' disk space requirements;
AFAICS the temporary directories are only cleaned up when a whole "suite
finishes running.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-03-16 16:05:08 +01:00
Miloslav Trmač
693de29e37 Add various missing error handling
... as found by (golangci-lint run).

Note: this does not add (golangci-lint run) to the Makefile
to ensure the coding standard.

(BTW golangci-lint currently fails on structcheck, which doesn't
handle embedded structs, and that's a years-long known unfixed
limitation.)

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-03-15 21:48:51 +01:00
Miloslav Trmač
f44ee2f80a Remove assignments to an unused variable
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-03-15 21:48:51 +01:00
Miloslav Trmač
a71900996f Rename "self" receiver
> receiver name should be a reflection of its identity; don't use generic names such as "this" or "self" (ST1006)

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-03-15 21:48:51 +01:00
Miloslav Trmač
23183072fb Work around systemd-resolved's handling of .invalid domains
... per https://github.com/containers/skopeo/pull/1558 .

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-02-15 16:54:51 +01:00
Miloslav Trmač
3be97ce281 Beautify a few calls
Use the sort-of-convention of keeping the output matching regex,
and the command, on separatel lines.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-02-15 16:54:51 +01:00
Daniel J Walsh
2858904e4b
Run codespell on code
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2022-01-21 07:49:49 -05:00
James Hewitt
c582c4844f Add option to preserve digests on copy
When enabled, if digests can't be preserved an error will be raised.

Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2021-12-07 13:16:10 +00:00
James Hewitt
2046bfdaaa Add option to preserve digests on copy
When enabled, if digests can't be preserved an error will be raised.

Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2021-12-07 13:16:10 +00:00
Colin Walters
3606b2d1de proxy: Add a GetFullConfig method
Sadly...I swear I had tested this at one point, but it was
*definitely* not the intention that we just return the container
runtime configuration.

I need a method to return the full image configuration.  At some point
I must have accidentally added a redundant `.Config`.

This whole new method `GetFullConfig` is like `GetConfig` but
returns the whole image configuration.  A specific motivation
here is that it's only in the image configuration that we can
stick arbitrary metadata (labels) that will survive a round trip through
docker schema v2.
2021-12-06 17:15:46 -05:00
James Hewitt
d94015466f Add an option to allow copying image indexes alone
The new --multi-arch option allows the user to select between copying the
image associated with the system platform, all images in the index, or
just the index itself without attempting to copy the images.

Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2021-11-27 15:38:42 +00:00
James Hewitt
4dcd28df92 Use a dynamic temp dir for test
This test was incorrectly assuming that nothing would be made on disk,
but it was putting files into the source directory.

Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
2021-11-27 13:44:28 +00:00
Colin Walters
e9535f868b tests: Add new "procutils" that exposes PDEATHSIG
To fix compilation on MacOS.

I think actually we want to use this pervasively in our tests
on Linux; it doesn't really matter when run inside a transient
container, but `PDEATHSIG` is useful for persistent containers (e.g.)
toolbox and when running outside of a pid namespace, e.g. on a host
system shell directly or in systemd.

Signed-off-by: Colin Walters <walters@verbum.org>
2021-11-15 21:02:14 -05:00
Colin Walters
fa86297c36 proxy_test: Test GetConfig
Now that we have a test suite, let's use it more.

Signed-off-by: Colin Walters <walters@verbum.org>
2021-11-15 21:02:14 -05:00
Colin Walters
2bb6f27d13 proxy_test: Add helper to read all from a reply
Prep for testing `GetConfig`.

Signed-off-by: Colin Walters <walters@verbum.org>
2021-11-15 21:02:14 -05:00
Colin Walters
f90725d80c proxy_test: Add a helper method to call without fd
To verify in one place.

Signed-off-by: Colin Walters <walters@verbum.org>
2021-11-15 21:02:14 -05:00
Colin Walters
644074cbb4 proxy: Add support for manifest lists
We need to support manifest lists. I'm not sure how I missed this
originally.  At least now we have integration tests that cover this.

The issue here is fairly subtle - the way c/image works right now,
`image.FromUnparsedImage` does pick a matching image from a list
by default.  But it also overrides `GetManifest()` to return the
original manifest list, which defeats our goal here.

Handle this by adding explicit manifest list support code.  We'll
want this anyways for future support for `GetRawManifest` or so
which exposes OCI manifest lists to the client.

Signed-off-by: Colin Walters <walters@verbum.org>
2021-11-15 21:02:14 -05:00
Colin Walters
83416068d3 tests/integration/proxy_test: New test that exercises proxy.go
I debated adding "reverse dependency testing" using
https://crates.io/crates/containers-image-proxy
but I think it's easier to reuse the test infrastructure here.

This also starts fleshing out a Go client for the proxy (not
that this is going to be something most Go projects would want
versus vendoring c/image...but hey, maybe it'll be useful).

Now what I hit in trying to use the main test images is currently
the proxy fails on manifest lists, so I'll need to fix that.

Signed-off-by: Colin Walters <walters@verbum.org>
2021-11-15 21:02:14 -05:00
Miloslav Trmač
0e87d4d1ca Run (gofmt -s -w)
Go 1.17 introduces a much more reasonable build constraint format, and gofmt now fails without using it.

Sadly we still need the old format as well, to support <1.17 builds.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2021-08-23 18:04:45 +02:00
Miloslav Trmač
c399909f04 Update non-module dependencies
Dependabot was apparently not picking these up (and
several haven't had a release for a long time anyway).

Also move from github.com/go-check/check to its newly
declared (and go.mod-enforced) name gopkg.in/check.v1.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2021-08-23 15:52:48 +02:00
Miloslav Trmač
ce6035b738 Add timeouts when waiting on OpenShift or the registry to start
... so that we terminate with the full context and pointing at
the relevant code, instead of relying
on the overall test suite timeout.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2021-07-30 12:36:54 +02:00
Miloslav Trmač
a1f9318e7b Fix two instances of unused err found by go-staticcheck
*cough* Go is an easy-to-use, hard-to-misuse language *cough*

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2021-07-28 21:46:07 +02:00
Chris Evich
285a5cb6a0
Fix using images from rate-limited docker hub
The necessary images have been manually copied over to quay.  Code was
updated with centralized constants for the utilized images.  Tests then
all reference the constants (in case the image locations need to change
again).

Signed-off-by: Chris Evich <cevich@redhat.com>
2021-07-22 16:28:20 -04:00
Miloslav Trmač
f80bf8a39f Revert "integration tests: disable ls for logs"
This reverts commit 65c5b0bf8d.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2021-07-09 13:15:13 +02:00
Daniel J Walsh
6d6c8b5609 Update tests for removal of error and Error from error messages
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2021-07-01 21:02:13 +02:00
Daniel J Walsh
b10d3e43a4
Merge pull request #1266 from cgwalters/copy-print-digest
copy: Add --digestfile
2021-05-07 05:17:35 -04:00
Colin Walters
a32be320cb copy: Add --digestfile
`podman push` gained this a while ago, and we want it here for the
same reason.

Motivated by closing a race condition in ostree-rs-ext:
17a991050c/lib/src/container/export.rs (L85)

Co-authored-by: Miloslav Trmač <mitr@redhat.com>
2021-05-06 09:19:16 -04:00
Brendan Aye
7898ffaf23
Added format parameter to sync command
Signed-off-by: Brendan Aye <brendan.aye@t-mobile.com>
2021-04-28 10:54:21 -07:00
Lokesh Mandvekar
b7bf15bc8b Dockerfile.build: switch to fedora:latest
CopySuite.TestCopySignatures: Use quay.io instead of docker.io.

Resolves: gh#174

Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
2021-02-24 15:49:14 +01:00
Daniel J Walsh
18e6c6f17b
Fix Makefile to handle PREFIX correctly
PREFIX should default to /usr/local according to GNU coding standards.

DESTDIR should just be used for install

This will allow us to specify

make PREFIX=/usr DESTDIR=/tmp/build/ install

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2021-01-21 13:54:53 -05:00
Miloslav Trmač
3c33cb4556 Add a smoke test for signedIdentity:remapIdentity
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2021-01-08 17:20:09 +01:00
Miloslav Trmač
f94d85aa8e Split copyWithSignedIdentity from TestCopyVerifyingMirroredSignatures
... because we will add another user.

Also remove an obsolete FIXME.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2021-01-08 17:20:09 +01:00
Nalin Dahyabhai
6ee4b2dc84 integration test: sync k8s.gcr.io/pause instead of docker.io/alpine
Do repository synchronization tests using k8s.gcr.io/pause and
k8s.gcr.io/coredns/coredns instead of docker.io/alpine.
The k8s.gcr.io/pause repository includes multiple tags, at least one of
which is a single image ("1.0"), and at least one of which is a manifest
list ("3.2", "3.3"), and includes a "latest" tag.
The k8s.gcr.io/coredns/coredns repository includes multiple tags, at
least one of which is a single image ("v1.6.6"), and at least one of which
is a manifest list ("v1.8.0").

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2021-01-08 09:51:08 -05:00
Nalin Dahyabhai
b3a15e7288 integration test: use fedora-minimal for most manifest list tests
Switch most of our tests that exercise reading, copying from, and
inspecting tags that point to manifest lists from using
docker.io/estesp/busybox to using
registry.fedoraproject.org/fedora-minimal, which doesn't limit how often
we can pull the images.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
2021-01-08 09:50:12 -05:00
Miloslav Trmač
c4fb93647a Fix reading the after-sync list of tags in SyncSuite.TestYamlUntagged
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2021-01-08 11:16:20 +01:00
Josh Soref
4987a67293 Spelling
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
2020-12-21 16:10:45 -05:00
Andrew DeMaria
873fbee01b
Fix #858 Add --all sync flag to emulate copy --all
This replicates the --all copy flag to sync to perform the same
behavior. Namely, the default is CopySystemImage unless --all is passed
which changes the behavior to CopyAllImages. While it is probably
desirable for --all to be the default as there is no option to override
ones architecture with the sync command, --all can potentially break
existing sync incantations depending on registry support. Hence
CopySystemImage remains the default.

Signed-off-by: Andrew DeMaria <ademaria@cloudflare.com>
2020-11-16 15:18:56 -07:00
Andrew DeMaria
1a3eb478a7
Fix #858 Add support for digests in sync
Signed-off-by: Andrew DeMaria <ademaria@cloudflare.com>
2020-11-16 15:18:53 -07:00
Valentin Rothberg
65c5b0bf8d integration tests: disable ls for logs
For yet unknown reasons, Travis throws permission errors when trying to
recursively list the contents of a temp directory.  It passes locally,
so disable the logs to unblock CI.  Note, the reasons for the error are
yet to be revealed.

Related-issue: https://github.com/containers/skopeo/issues/1093
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
2020-11-04 16:52:54 +01:00
Daniel J Walsh
62bafb102d
Fix problems found by codespell
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2020-09-11 10:19:16 -04:00
Wong Hoi Sing Edison
0f458eec76
Add nix derivation for static builds
Signed-off-by: Wong Hoi Sing Edison <hswong3i@gmail.com>
2020-06-18 20:57:55 +08:00
Miloslav Trmač
322625eeca Use a separate field for the "sync images with tag matching regex" feature
Fields that magically change their behavior depending on type of the value
are too much hassle for no benefit.

For now, this just copies&pastes the full loop in imagesToCopyFromRegistry
to create another loop handling the new ImagesByTagRegex field. Simplifications
to reduce duplication will follow.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2020-05-25 11:58:19 +02:00
zhangguanzhang
bec7f6977e Add tags to support regular expressions in yaml conf
add relevant test cases and documentation
update the sync doc
2020-05-20 09:23:00 +08:00