In bootc, we want the ability to assert that signature verification is
enforced.
Add a new top-level `--require-signed` switch. When passed, we use the
new `RequireSignatureVerification()` method to ensure that signature
verification is enforced.
Part of https://github.com/containers/skopeo/issues/1829.
Signed-off-by: Jonathan Lebon <jonathan@jlebon.com>
Renovate seems to have had a difficulty with that:
> Could not determine new digest for update (go package go.podman.io/common)
for some reason, let's do it manually now, hoping that
Renovate will catch up next week.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
`prepare-files` action was interfering with spec file update which caused
https://github.com/containers/skopeo/issues/2760 .
`post-modifications` needs to be limited to the propose_downstream job or
else it will interfere with upstream PR copr builds.
Also, s/PACKIT_PROJECT_TAG/PACKIT_PROJECT_VERSION/ .
Co-authored-by: Nikola Forró <nforro@redhat.com>
Signed-off-by: Lokesh Mandvekar <lsm5@redhat.com>
And also remove old, no longer needed validate-gofmt.sh script.
golangci-lint checks the formatting already for us.
And add a fmt make target that just runs golangci-lint fmt for easier
use.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
These happen after commit to upstream and don't affect upstream.
These notifications only end up adding unnecessary noise.
Overall build failures can happen for a variety of reasons like copr
infra, outdated toolchain on some environments etc.
Signed-off-by: Lokesh Mandvekar <lsm5@redhat.com>
This will ensure both `test-system` and `test-system-local` work.
The `test-system` target will use the skopeo binary at `./bin/skopeo`.
Setting SKOPEO_BINARY should have no effect on it.
The `test-system-local` (and `test-integration-local`) target can use
SKOPEO_BINARY at any location. This will be useful in CI where we need to test
skopeo installed by the package at /usr/bin.
Co-authored-by: Miloslav Trmač <mitr@redhat.com>
Signed-off-by: Lokesh Mandvekar <lsm5@redhat.com>
`hack/test-system.sh` earlier removed in commit 7e235ea contained
a storage.conf setup useful for running system tests in skopeo_cidev
container.
Getting rid of that also broke `make test-system` because the
systemtest: `copy: additional tag` started to break.
Running `make test-system-local` isn't ideal for local environments due
to creation of system-global containers with predictable names.
See: https://github.com/containers/skopeo/issues/2701#issuecomment-3415769846
So, this commit resurrects hack/test-system.sh. Followup commit will
modify Makefile targets to use this script again.
Signed-off-by: Lokesh Mandvekar <lsm5@redhat.com>
I want this for https://github.com/bootc-dev/bootc/issues/1686
so we can distinguish pulls there.
But more generally it's can be a good idea for people writing
scripts using skopeo to set custom user agents so that registries
can more easily trace which actors are performing tasks.
Assisted-by: Claude Code
Signed-off-by: Colin Walters <walters@verbum.org>
... and construct it from the primary inputs instead.
The name doesn't hurt in itself, but it ~forces us to use
multi-line test data, making this larger and harder to read than
necessary.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This changes the test, we drop the two inputs that can't happen
in practice; and, instead, test the default situation of no TLS
options provided anywhere.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
They use exactly the same body, and differ only in data
(and we are going to keep iterating on the body).
Should not change (test) behavior.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Else use ./bin/skopeo as the default binary.
This makes it a lot more flexible compared to just searching and using
the first skopeo in PATH. Also, allows to avoid any binary installation
to /usr/bin.
The SKOPEO_BINARY evaluation logic has been moved to a separate and
reusable Makefile target.
Co-authored-by: Miloslav Trmač <mitr@redhat.com>
Signed-off-by: Lokesh Mandvekar <lsm5@redhat.com>
NewEphemeralSigningMechanism() may, with Sequoia, return
a mechanism which !SupportsSigning(); so, to determine that,
test with a non-ephemeral mechanism instead. (That's likely
actually faster, because we create a GNUPGHOME in these tests
anyway, so we avoid creating an deleting a separate temporary
directory.)
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
1. Packit config includes a `get-current-version` action to fetch the
version from `version/version.go` and not from the default `git
describe`. So, the copr rpm version will always be ahead of what's in
the official repos.
2. The absurdly high RPM Epoch is now limited to builds from
rhcontainerbot/podman-next copr. So, copr builds on PR jobs will use
the default Epoch value.
These two changes will make it easier for testers to fetch the scratch copr builds from
upstream PRs and it should allow for seamless upgrades.
(official rpm -> test copr rpm for bugfix -> upgrade to new official rpm)
Signed-off-by: Lokesh Mandvekar <lsm5@redhat.com>
testing-farm public ranch now provides ELN composes which enables
super-early testing of the next RHEL.
Signed-off-by: Lokesh Mandvekar <lsm5@redhat.com>
With TMT being able to fetch and run tests using git url
and ref, there's no need to sync files on every propose_downstream
Packit action.
Removing files_to_sync should be safe as we only need to sync
`.packit.yaml` which is part of Packit's default behavior already.
Other files like gating.yaml only need to exist downstream and
shouldn't need any frequent manual changes, so
we can remove those from upstream and packit file-sync too.
New setup: We use a `prepare-files` action that will operate only on
`plans/main.fmf` in downstream dist-git and update the ref with the
tag from the latest release.
Signed-off-by: Lokesh Mandvekar <lsm5@redhat.com>
The sole purpose of skopeo-tests subpackage was to make system test
files readily available for CI on bodhi updates.
Given we can reuse test config from upstream via TMT, there's no
reason to continue shipping the test files. This subpackage can be
repurposed to only install test dependencies.
Signed-off-by: Lokesh Mandvekar <lsm5@redhat.com>
We will, temporarily, want to run from rawhide images
that include the rust-podman-sequoia RPM, as well as Fedora 43
images.
We have only one skopeo_cidev though, and we don't want to build
a rawhide variant; so, allow consuming it from rawhide.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Currently, if a key is not found, the GPG mechanism reports
> Invalid GPG signature: {$GoStructDump}
while the Sequoia one reports
> $keyFingerprint was not found
Accept both.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Bumping to v1.20 to align with Buildah v1.41 and Podman v5.6
now that Skopeo has the same c/* libraries vendored in.
Signed-off-by: tomsweeneyredhat <tsweeney@redhat.com>
Rely on "$@" to allow passing options with spaces, like SKOPEO_LDFLAGS.
Then pass SKOPEO_LDFLAGS.
Also fix the documentation of TESTFLAGS.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This updates github.com/opencontainers/image-spec/schema subpackage from an obsolete
.../jsonschema/v5 to /v6 .
We use these dependencies only in tests, so it doesn't matter that much,
but removing references to years-old unmaintained versions makes it easier to check
/ remove other such cases.
(This is test-only, so the added files in vendor/ do not mean a larger binary
size; it's almost exactly unchanged.)
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Add some test coverage, but not full - the previous code
did not have any.
Should not change behavior, other than some help texts.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
AddFlagSet silently ignores conflicting flags, whereas individual
flag definitions panic on (simple) conflicts; so, to detect ambiguities,
prefer to use AddFlagSet first.
Should not change behavior.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Sequoia-PGP does not accept SHA-1 by default; update
to avoid that.
Both the keys and the signatures come from c/image commit
70dbc81047f63a68130a5bdf1bfaee43db39746d .
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
We're currently not updating official CentOS Stream packages via Packit.
So, this commit disables the downstream PR update job.
No noticeable effect upstream. Safe to merge.
Signed-off-by: Lokesh Mandvekar <lsm5@redhat.com>
- Changed GitHub username from 'Neil-Smith' to 'actionmancan'
- Fixed GitHub URL from https://github.com/Neil-Smith to https://github.com/actionmancan
- Corrected column alignment for consistent formatting
- Maintains Neil Smith's role as Community Manager
Does this PR introduce a user-facing change?
```release-note
None
```
Signed-off-by: G A Neil Smith <nesmith@redhat.com>
bats isn't shipped on ELN, RHEL and CentOS Stream. Enabling EPEL is also
problematic on internal testing farm environments.
This commit fetches and installs bats from source on RHEL and CentOS
Stream so we don't have to worry about EPEL enablement anymore.
We also aren't doing any ELN testing on TMT currently, so we don't need
any additional configuration for it and the current config should work
across the board.
Signed-off-by: Lokesh Mandvekar <lsm5@redhat.com>
Unfortunately on a number of occasions, Skopeo has been released
officially with a `-dev` suffix in the version number. Assist in
catching this mistake at release time by the addition of a simple
conditional test. Note that it must be positively enabled by a
magic env. var. before executing the system tests.
Also (as suggested by @lsm5) update the TMT test to trigger the new
condition for future Fedora releases.
Signed-off-by: Chris Evich <cevich@redhat.com>
Dynamically link sqlite3 when installed, the main motivation is that we
reduce the podman binary size with that. I see about 3.3 MB savings.
But also dynamically linking it means if there a vulnerabilities only
the sqlite3 distro package needs updating and we don't have to make a
new podman release with the vendored update.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
- Don't claim that Windows uses XDG_RUNTIME_DIR, defer to the authoritative
man page.
- Consistently document the (regrettable?) REGISTRY_AUTH_FILE environment
variable.
- Consistently refer to "registry credentials file", not "authentication file".
- (Also deemphasizes (docker login), I'm not sure we want that.)
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Some container storage operations (e.g., mounting the home directory
for containers/storage) require CAP_SYS_ADMIN.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
I'm thinking about making it un-experimental. Having docs
is a step towards that.
Co-authored-by: Gemini Code Assist (which basically got this right)
Co-authored-by: Miloslav Trmač <mitr@redhat.com>
Signed-off-by: Colin Walters <walters@verbum.org>
F40 is now EOL and all current Fedora releases have Go 1.23. So, we're
safe to re-enable fedora-all.
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
The original model the idea here is the proxy centralizes
verification of things like digest. However in practice,
this causes reading to be seriously awkward; ref
https://github.com/containers/containers-image-proxy-rs/issues/79
(Basically `FinishPipe` blocks the metadata channel)
Also, I have a project to implement a registry frontend to
`containers-storage:` and a core problem with `GetBlob` right
now is it *requires* the blob size up front even though the
underlying Go logic doesn't.
Moving to a "raw" interface solves that too. In this new
raw API, we return two file descriptors, one for the data
and one for the error channel, which contains a JSON
serialization of an error.
For the error type we reuse the existing "is error retryable"
and expose that back to the client.
We also (backwards compatibly) add this new error code
for the existing APIs.
Signed-off-by: Colin Walters <walters@verbum.org>
This removes a safety mechanism, but it's better than
inventing a dummy tag value (both here and in c/image).
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Sometimes running golangci-lint with --tests=false helps to uncover some
unused code which was not removed because it has unit tests. Since
everything is already cached, this additional run doesn't take much
time.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Before commit d4bd787e this script used to contain some logic,
but now it's just a wrapper that does nothing useful and stands in the
way. Remove it, and call golangci-lint directly.
This slightly changes the way the linting is done, because BUILDTAGS
was empty before, and now they it contains libsubid.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Some files in integration did not have _test, resulting in lots of
complains when running golangci-lint with --tests=false.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
So I wondered why our email workflow only reported things for podman...
It seems the secrets: inherit is broken and no longer working, I see all
jobs on all repos failing with:
Error when evaluating 'secrets'. .github/workflows/check_cirrus_cron.yml (Line: 19, Col: 11): Secret SECRET_CIRRUS_API_KEY is required, but not provided while calling.
This makes no sense to me I doubled checked the names, nothing changed
on our side and it is consistent for all projects. Interestingly this
same thing passed on March 10 and 11 (on all repos) but failed before
and after this as well.
Per[1] we are not alone, anyway let's try to get this working again even
if it means more duplication.
[1] actions/runner#2709
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
This was added earlier as part of PR label-based triggering of Packit
jobs. But we decided not to go ahead with that approach, thus keeping
only a single set of tests. This file should've been removed during the
revert, but better late than never.
Ref: https://github.com/containers/skopeo/pull/2558
FWIW, this yaml file doesn't work by itself without the corresponding
GHA which was never included. So, this yaml config was pretty much a NOP
anyway.
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
Many of the people in OWNERS no longer contribute to the project,
so clean the file and restrict to those who are still active.
Alongside this, add our core maintainers who have merge authority
on all repos.
Governance is a simple link to the Podman governance model, and
MAINTAINERS.md mirrors the new OWNERS.
Signed-off-by: Matt Heon <mheon@redhat.com>
Switch to keeping TMT tests independent of PR labels for now.
In order to keep PR-label dependent tests, Packit UI would need
improvement making it clear that some tests are not meant to run,
perhaps also changing the status to `ignore` or `neutral`.
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
> QF1008: could remove embedded field "dockerImageOptions" from selector (staticcheck)
Should not change behavior.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
I was experimenting with images with lots of layers (> 200) and
this invocation was incorrectly adding the entire response
into what was intended as the metadata plane.
`GetManifest` and `GetConfig` (even those are relatively small)
still always return their data over a pipe, same as blobs.
Add a new `GetLayerInfoPiped` that does the same so we
can easily get this information for images with a lot of layers.
Signed-off-by: Colin Walters <walters@verbum.org>
This updates from a release to a commit, but the release
is very old, and we get to remove a technically-obsolete dependency.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
As pointed out in buildah[1] the action is broken in bad ways where it can
trigger 1000+ rerun wasting our cloud resources.
Get rid of it for now until we find something better or can properly
identify and fix the root cause.
[1] https://github.com/containers/buildah/issues/6035
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
This commit enables TMT jobs triggered by Packit to run system tests.
2 set of jobs `dev` and `release` have been added. `dev` jobs are meant
to run on main PRs with additional package updates fetched from
podman-next copr while `release` jobs are meant to run on release-
branch PRs using only the dependencies present in the official distro.
Packit checks PR labels (see previous commit) to filter out
the jobs that get run.
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
This will be useful in the followup commit that enables TMT test jobs on
PRs.
PRs on `main` branch should be tested with bleeding-edge dependencies
from the podman-next COPR while PRs on `release` branches should be
tested only with the official distro packages. Packit will run/skip the
relevant set of tests based on this label.
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
The default gobuild macro on CentOS Stream now accounts for `BUILDTAGS`,
so we don't need to redefine the macro in rpm spec.
The `libtrust_openssl` has been set in the spec for RHEL
environments.
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
- Add option to set a fixed delay between retries.
- Clarify that if delay is unset, skopeo uses an exponential delay for
retries.
Fixes#2476
Signed-off-by: Adam Kaplan <adam.kaplan@redhat.com>
The "2" tag is very old and not a multi arch manifest. As such testing
on aarch64 failed because it pulled and x86_64 image instead. This was
found in downstream RHEL testing.
The "2.8.2" is multi arch and used in podman testing were we
successfully run aarch64 based testing.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
RHEL targets are often out of date with respect to golang and other
important dependencies leading to frequent failures. CentOS Stream moves
faster and helps to ensure whatever eventually enters RHEL has been
tested upstream at some point.
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
As the title says. In preparation of Skopeo v1.17.0 to go
out with Podman v5.3, this gets the c/* projects to:
containers/storage: v1.56.0
containers/image: v5.33.0
containers/common: v0.61.0
Signed-off-by: tomsweeneyredhat <tsweeney@redhat.com>
dnf erase was removed from dnf5 in f41, AFAIK remove does the same thing
and erase was just a removed alias.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
* Digest file output would have docker reference of source
and sha of of the mainfest sync'd with the target. This
file would not be created if dry-run flag is enabled
* improved the sync document to include the correct output for manifest file.
* added new line for the manifest file once all images are sync'd
* Ensuring we log on manifest digest if the copy operation was successful.
* Check for errors if any once sync process is complete.
* Ensure to capture the failure when closing the manifest file.
* Ensure we are not writing manifest sha for failed copy of imagesand aborting the process in case write to file fails
Signed-off-by: Ankit Agarwal <aagarwal@cloudsmith.io>
This constraint helps to avoid duplicate Packit jobs. The effect will
only be seen at the time of Fedora koji build and not on upstream. This
doesn't affect upstream.
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
ELN is kind of a midway between Fedora and RHEL, so it's best to mention
ELN jobs separately. This will also allow reusing fedora targets using
YAML anchors for TMT tests.
This commit also mentions fedora-40 targets separately for copr_build
jobs so that once fedora-41 is released, fedora-40 jobs continue to
trigger.
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
Packit now has sidetag support for adding multiple builds into a single
bodhi update.
Since we release c/ccommon, skopeo, buildah and podman often
almoost simultaneously, we should release them to Fedora in a single
bodhi update using sidetags so all builds can be tested together.
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
Bump Skopeo to v1.16.0 in preparation for
RHEL 9.5/10.0 Beta and Podman's bump to v5.2
[NO NEW TESTS NEEDED]
Signed-off-by: tomsweeneyredhat <tsweeney@redhat.com>
All our supported distros except CentOS Stream 9 support autochangelog
and we currently build rpms for CentOS Stream 9 only on the podman-next
copr, where we don't really care much about changelog.
So, there's no need for this conditional to exist.
Podman Ref: https://github.com/containers/podman/pull/23110
Signed-off-by: Lokesh Mandvekar <lsm5@redhat.com>
Previously this task ran using the Cirrus-CI compute service, consuming
compute credits. However, since podman is already using a persistent
worker pool for CI, it can also be leveraged for use here. Since Mac
resources are relatively expensive, it also makes financial sense to
re-use infrastructure where possible.
Notes:
- This change also restricts the cross-build task to only running on
the default branch. This is necessary because the PW Pool environment
is always rolling forward and may not match what was in place at the
time a release-branch was created.
- This changes the test from an install + run, to just a build + run.
This is required because the task is running as a regular user w/o
permissions to install anything system-wide, on a host shared with
other CI tasks.
Signed-off-by: Chris Evich <cevich@redhat.com>
Without this change, packit will trigger duplicate jobs on podman-next
for `skopeo-fedora`, `skopeo-centos` etc, which essentially point to the
same file.
1 job triggered builds packages for all environments enabled on the
COPR, so simply `skopeo-fedora` should suffice.
Signed-off-by: Lokesh Mandvekar <lsm5@redhat.com>
Until now `cc' was hard-coded as the only compiler used. Supporting selecting
the compiler and preprocessor to be used via environment variables makes life
easier for distributors, so this commit mimics how podman does it in its hack/*
scripts.
Signed-off-by: Tomas Volf <~@wolfsden.cz>
The bats package is not available on RHEL. It's in Fedora and EPEL.
Having bats as a requirement for the `skopeo-tests` subpackage will
cause installibility test failures for RHEL 10 / C10S gating tests.
This commit makes `bats` a requirement only on Fedora. RHEL and CentOS
Stream gating will need to fetch bats through separately enabling EPEL
or other means.
Podman PR with similar change: https://github.com/containers/podman/pull/22632
Signed-off-by: Lokesh Mandvekar <lsm5@redhat.com>
This commit will enable downstream syncing to CentOS Stream 10. The
centos maintainer will need to manually run `packit propose-downstream`
and `centpkg build` until better centos integration is in place.
This commit also builds both rhel9 and centos9 copr rpms so we can check
for things like differences in golang compiler.
Signed-off-by: Lokesh Mandvekar <lsm5@redhat.com>
In the Buildah repo, we had a request for a ptr to the container image
location in the docs. We've added similar to Podman and Buildah.
Signed-off-by: tomsweeneyredhat <tsweeney@redhat.com>
Correct the summary for standalone-sign. The old summary appears to be a
bad copy-and-paste from "skopeo copy".
Clarify that both of these tools are debugging tools, only operating on
local files.
Signed-off-by: Ken Dreyer <kdreyer@ibm.com>
This is needed on the off-chance that some tool or a human suggests
updating the minimum version to 1.21 or later. Since doing so would
cause Fedora and Debian to start behaving differently WRT builds.
Signed-off-by: Chris Evich <cevich@redhat.com>
This commit makes skopeo return a different exit code when an
input is not found. The use case is `osbuild` which uses skopeo
to inspect images and it would be nice to differenciate between
an image that is not found and general skopeo errors (or errors
like network issues etc).
I picked exit code `2` for `not found` because it is also the value
of `ENOENT`.
Man page and a test are added.
Signed-off-by: Michael Vogt <mvogt@redhat.com>
This removes the need for any `Provides: bundled()` we needed in spec
files.
The updated Provides will be visible in the build logs and can also be
verified with `rpm -q --provides $RPM_FILE`.
Signed-off-by: Lokesh Mandvekar <lsm5@redhat.com>
Given the big changes in c/common, c/image and c/storage, I think
it makes sense to create Skopeo v1.15.0 now to go out with Podman v5.0
and Buildah v1.35.
[NO NEW TESTS NEEDED]
Signed-off-by: tomsweeneyredhat <tsweeney@redhat.com>
Move docs under "Building without a container".
Check instead of setting env variable.
Provide alternative approach.
Signed-off-by: Miltiadis Kallianis <miltiadis.kallianis@asml.com>
... and c/image/v5 to main
Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
As the title says. Bumping to v1.15.0-dev. We'll be using 1.14 in
RHEL 8.10/9.4, and v1.15 of Skopeo will be targeted for a later RHEL
version.
[NO NEW TESTS NEEDED]
Signed-off-by: tomsweeneyredhat <tsweeney@redhat.com>
As the title says. Bumping now in preparation for RHEL 8.10/9.4.
Once merged, I will create release-1.14 branch based on this commit.
[NO NEW TESTS NEEDED]
Signed-off-by: tomsweeneyredhat <tsweeney@redhat.com>
ENTRYPOINT was incorrectly documented to be set to /
(which doesn't even make sense).
Stop mentioning PATH and WORKDIR in the top-level README,
typical users of the container shouldn't need to care,
and it's already somewhat implied by "built using the latest Fedora".
Fixes#2134.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Some other containers/* repos use these values in test names;
we don't, so remove them so that we don't have to worry
about keeping them up to date.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
These jobs have been failing since early August due to
technical/scripting problems. Disable/remove entirely since a fix
is unlikely to be implemented anytime soon.
Signed-off-by: Chris Evich <cevich@redhat.com>
The lowercase `and` in the License field isn't compatible with spdx
license format.
This commit replaces all `and` with `AND` in the License field in spec.
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
Building all PRs of all container projects into the same COPR does not
properly isolate PRs from each other.
To avoid that, change the copr_build configuration to use the packit
default COPRs, which are specific to the particular PR, and disappear
after a few weeks. Depending projects should only run against what
landed in skopeo/main i.e. the podman-next COPR.
Signed-off-by: Martin Pitt <mpitt@redhat.com>
We already require it, because docker/credential-helpers uses Go 1.19
os/exec.Cmd.Environ(). So make that official.
> go mod tidy -go=1.19
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
> go get github.com/containers/image/v5@main
> go mod tidy && go mod vendor
This updates c/image with a new version of x/exp.
That package has changed API in an incompatible way,
so just bumping x/exp (as in https://github.com/containers/skopeo/pull/2060 )
would break Skopeo builds.
This updates both c/image and x/exp in lockstep (and nothing
needs updating in Skopeo itself for the x/exp breakage).
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This automatically the global --policy-path and --insecure-policy options,
which don't affect h.sysctx.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
The pre-sync action constantly breaks and is currently not possible to
reliably test until the subsequent upstream release due to limitations
in packit.
The lines being added by the action script to the downstream Fedora spec
were only meant to keep Fedora happy. But given that they provide
no tangible benefit as github notifies us of security
issues in libraries mentioned in go.mod and go.sum, along with redhat
prodsec's own magic for creating security alerts, there's absolutely
no point to having the pre-sync action run and add a layer of uncertainty.
This commit removes the pre-sync action and
`rpm/update-spec-provides.sh`.
Ref: https://github.com/containers/podman/issues/19232
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
In PR review for a different issue, the question of what happens
if we hit overflow for the imageid serial was hit. This feels
pretty unlikely; if I did the math right, it'd require opening
an average of 136 images per second to overflow it in a year.
Nevertheless, in practice what we're sending on the wire is just a JSON
number, and if we extend this to the "max safe JSON number" of 2^53,
it'd take 285,616,414 images per second to overflow in a year, going
from implausible to probably impossible.
With a bit more work of course, we could make this a sparse mapping
and reuse freed numbers, but eh.
Signed-off-by: Colin Walters <walters@verbum.org>
The default Packit sandbox environment that runs Packit tasks for
downstream Fedora does not have golist installed by default and can't
run superuser tasks.
This commit will download and extract the golist binary from the Fedora
rpm and use it to provide golist.
The GOPATH mention in `rpm/update-spec-provides.sh` is only required for
golist to generate the gopaths and doesn't affect upstream or the rpm spec.
Currently, the only way to reliably test this is on an open github issue by running
`/packit propose-downstream`. This can't be run on an open PR.
The job-specific packit actions can only be tested via the packit
service and not via packit cli.
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
Distro conditionals have been rewritten in a cleaner
and easier to read way.
All `bcond` macros have been replaced with friendlier alternatives.
Also removed macros related to `import_path` as they are no longer
necessary.
update-spec-provides.sh has been updated to account for possible build
issues in downstream tasks which won't be noticed until a downstream
release.
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
It's a bit cumbersome to manage a tooling version buried deep in a
command, let alone one also buried deep in a `Makefile`. Add a
variable to hold the version number so renovate can easily manage it.
This happens via a `regex` manager in the shared configuration
include `containers/automation//renovate/defaults.json5`. Also add a
helpful note/reminder to humans who may want to manually change the
version for some reason.
Depends on: https://github.com/containers/automation/pull/145
Signed-off-by: Chris Evich <cevich@redhat.com>
Assigning to a group isn't supported in github. Renove this section and
the golang section (all options moved into defaults.json5).
Signed-off-by: Chris Evich <cevich@redhat.com>
Get rid of `skopeo.spec.rpkg` in favour of
`rpm/skopeo.spec` which gets synced with fedora dist-git on every
upstream release. The version in the new spec file is set to `0` by
default and gets updated by packit automatically on every packit task.
Packit will automatically create a PR on fedora dist-git on every new
upstream release. A sample PR will look like:
https://src.fedoraproject.org/rpms/container-selinux/pull-request/10#
A dry run for this can be triggered using:
`$ packit propose-downstream --local-content`
To run this command locally, you would need to have your packit
user-configuration-file set.
Ref: https://packit.dev/docs/configuration/#user-configuration-file
along with a fedora api key created at:
https://src.fedoraproject.org/settings#nav-api-tab with sufficient ACLs.
This patch incurs zero additional cost to upstream as the downstream
tasks occur only after upstream release and do not block anything
upstream.
[NO NEW TESTS NEEDED]
Co-authored-by: Miloslav Trmač <mitr@redhat.com>
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
Currently it uses golang-1.18.7-1.fc36.x86_64 ; we need 1.19,
and the current version of the base image has golang-1.20.3-1.fc38.x86_64
.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
- Accurately document the effect of GO_DYN_FLAGS
- Fix a typo
- Remove a shell script example; removes a docker command,
we don't care to maintain it to run correctly,
and anyone knowledgeable enough to understand the
trade-offs of static linking can almost certainly create that from scratch.
Fixes#1700
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
More template functions available in (skopeo inspect --format)
Adds new ways to supply trusted keys to (skopeo standalone-verify).
Now requires Go 1.18.
- [CI:DOCS] Fix up language in README
- Add unit tests for tlsVerifyConfig's yaml.Unmarshaler
- Cirrus: Use human-readable CI VM Images
- [CI:BUILD] copr: fix el8 build and enable debuginfo
- [CI:BUILD] enable debuginfo for el8 copr builds
- Update to use, and benefit from, Go 1.18
- [CI:DOCS] Disable dependabot
- Renovate: c/common rule moved to defaults
- [CI:BUILD] Packit: initial enablement
- Replace gopkg.in/check.v1 by github.com/stretchr/testify/suite/
- Corrected typo in skopeo-sync and updated description
- Fix tabelating output in (skopeo inspect --format)
- Use common library reporter
- Fix formatting of inspect examples
- Use io.WriteString
- Factor out the output of data in (skopeo inspect)
- Simplify inspectOptions.writeOutput a bit more
- Cirrus: Update CI VM images
- Make the installation instructions more prominent in README.md
- [CI:BUILD] Packit: trigger builds on commit to main branch
- systemtests: Fix 040-local-registry-auth about XDG_RUNTIME_DIR
- Verify signatures from a trust store
- Rename argument. Only use any with public key file. Double check fingerprint is in public key file.
- Use multiple fingerprint function Allow comma separated fingerprint list
- Avoid use of a deprecated capability.NewPid
- Fix error handling of signature.NewEphemeralGPGSigningMechanism
- Cross-link the top-level and subcommand option lists
- Use golangci-lint instead of golint
- Add (make tools) to install (for now only) golangci-lint, use it in Cirrus
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
The only thing hack/make.sh is now really doing is the
warning + sleep without SKOPEO_CONTAINER_TESTS .
So, make that a separate script, and eliminate the
hack/make directory.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This is simpler to do, cheap enough for our repo size, and it
does not require a network access to see which files to check.
And it's the last user of hack/make/.validate, which I wanted to
remove in the first place.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
- Always lint everything, not just changed files;
that means that if we upgrade the linter, we will
need to clean everything up, but that's a good thing
for contributors who come after that linter upgrade.
- Don't skip linting the integration tests, there's no
good reason to skip them.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
We now require Go 1.18. As of that version:
- GO111MODULE=on is implied by having a go.mod file
- -mod=vendor is implied by having a vendor directory
so just remove both options everywhere
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
signature.NewEphemeralGPGSigningMechanism is called in an if branch
where the previous err := introduces a "new" err variable, which means
the failure isn't visible after the if.
So, do the dumb thing and just check on both branches explicitly.
(We still need to worry about correctly setting "mech" and
"publicKeyfingerprints" to persist after the if.)
How I hate Go sometimes. And this shows we really should update
the linter.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Add the ability to use an on-disk trust store to verify signatures. Also allow the user to trust any known fingerprint instead of having to specify one.
Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
This commit lets packit trigger builds on
`rhcontainerbot/podman-next` copr after a commit to the main branch
instead of the current github webhook trigger.
The builds triggered via packit also provide more information in their
`version-release`:
Current webhook triggered build:
`101:0.0.git.2460.cfd6f20f-1`.
Ref: https://copr.fedorainfracloud.org/coprs/rhcontainerbot/podman-next/package/skopeo/
Packit triggered build for another package (netavark) on podman-next:
101:1.6.0~dev-1.20230321121647013339.main.61.gd6f0352
Ref: https://copr.fedorainfracloud.org/coprs/rhcontainerbot/podman-next/package/netavark/
The packit triggered build correctly shows the upstream branch name,
commit id, timestamp as well as the upstream version.
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
... mostly so that I get practice and remember this exists in the future.
(This saves one allocation & copy when the target implements
io.StringWriter. And that makes absolutely no relevant difference
on this path.)
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
tabwriter buffers lines that contain \t in memory, and only
writes them out on a .Flush(). So actually call that.
Without this, things like
> --format 'name\tdigest\tlabels\n{{.Name}}\t{{.Digest}}\t{{.Labels}}\n'
result in no output at all.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
gopkg.in/check.v1 hasn't had any commit since Nov 2020.
That's not a immediate issue for a test-only dependency, but
because it hides access to the standard library *testing.T,
eventually it will become limiting.
Also, using the same framework for unit and integration tests
seems practical.
This is mostly a batch copy&paste job, with a fairly high risk
of unexpected breakage.
Also, I didn't take much time at all to carefully choose between
assert.* and require.*; we can tune that as failures show up.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This commit will run COPR builds on every PR against all active
releases of CentOS Stream and Fedora, thus allowing buildability checks before the
PR merges.
Builds are done on a custom COPR project:
`rhcontainerbot/packit-builds`.
Ref: https://copr.fedorainfracloud.org/coprs/rhcontainerbot/packit-builds/
The build targets are set in the copr itself, so we don't need to
explicitly mention them in `.packit.yaml`, making upstream configuration
a lot simpler.
The `spec.rpkg` file meant for rpm builds post-pr-merge at
`rhcontainerbot/podman-next` copr gets reused for packit builds, so the
packit jobs are independent of Fedora / CentOS dist-git.
NOTE: The Packit copr_build tasks help to check if every commit builds on
supported Fedora and CentOS Stream arches. They do not block the current
Cirrus-based workflow.
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
Fedora 35 builds are disabled, so remove fedora 35
conditionals while we're at it.
Bump containers-common dependency to match with that in
podman.spec.rpkg.
TODO: fix debuginfo for rhel8
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
Image content hasn't changed much, the biggest thing here is the
`$IMAGE_SUFFIX` value. This new schema is also fully manageable
by renovate. Allowing a tag-push to c/automation_images to create image
update PRs in all repos automatically.
Signed-off-by: Chris Evich <cevich@redhat.com>
- Add a prompt to the skopeo commands.
- Add a "console" identifier to fenced code
blocks which has a prompt, not "sh".
Signed-off-by: Toshiki Sonoda <sonoda.toshiki@fujitsu.com>
Use (podman unshare) as already suggested, it is necessary for an unprivileged
user to remove the temporary c/storage state. OTOH it doesn't work with Docker at all.
Don't use the - prefix, it only works at the _start_ of a rule, not in the middle of
a multi-line shell script.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
- Don't do it at all for the CI VM: We can use the
VM's global Podman configuration, and use faster overlay
instead of vfs, so let's do that.
- For the developer-run (make test-system):
- Add graphroot and runroot paths to make the configuration minimally valid
- Explicitly point CONTAINERS_STORAGE_CONF at the configutation
to be certain it will get used.
Then drop the (podman pull ...) in runner.sh:_podman_reset that seemed to
previously workaround the invalid /etc/containers/storage.conf .
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
The containers/image CI setup reuses the runner script from this repo to
execute the skopeo tests. However, an env. var. is being taken out of
context in that environment, leading to failure. Fix this by
hard-coding an image-name which will always be available in both
environments.
Signed-off-by: Chris Evich <cevich@redhat.com>
This is necessary, since running the skopeo tests modifies the host
environment. This can result in some warning messages the first time
a container is started. These messages can interfere with tests which
are sensitive to stdout/stderr. Since many/most tests require a local
image registry, launch it with `/bin/true` after doing a system reset
to clear away any pesky warning messages.
Signed-off-by: Chris Evich <cevich@redhat.com>
For whatever reasons, the podman configuration in CI results in the
inspect test throwing the following error:
```
not ok 4 inspect: image manifest list w/ diff platform
125
configuration is unset - using hardcoded default graph root
\"/var/lib/containers/storage\""
configuration is unset - using hardcoded default graph root
\"/var/lib/containers/storage\""
StoreOptions
```
Fix this by not using `podman`. It's unnecessary, since all the test
needs is the golang-flavor of the current system's architecture name.
That can easily be obtained by asking the go tool directly.
Signed-off-by: Chris Evich <cevich@redhat.com>
These are already present in the VM images. These instructions only
cause the DNF cache to be refreshed, wasting precious developer time.
Signed-off-by: Chris Evich <cevich@redhat.com>
(skopeo copy) and (skopeo sync) now support --sign-by-sigstore=param-file,
using the containers-sigstore-signing-params.yaml(5) file format.
That notably adds support for Fulcio and Rekor signing.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This task does not make sense to maintain long-term on release
branches. Its intent is always/only to test the latest/greatest code
and environment. After release, it's simply too difficult to maintain
functioning CI with a constantly changing (Cirrus-managed) OSX environment.
Ensure the task only runs for PRs targeted at the default branch, or if
the current branch is the default branch.
Signed-off-by: Chris Evich <cevich@redhat.com>
Should not change behavior - it would matter for unit tests
which don't exist.
Also, promptForPassphrase must continue to hard-code "real" os.Stdin and os.Stdout.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Missing `.Close()` on the blob currently leaks a temporary
file. Noticed this when doing repeated pulls.
Signed-off-by: Colin Walters <walters@verbum.org>
Fairly universally, the last Cirrus-Cron job is set to fire off at
22:22 UTC. However, the re-run of failed jobs GHA workflow was
scheduled for 22:05, meaning it will never re-run the last cirrus-cron
job should it fail.
Re-arrange the execution order so as to give plenty of time between the
last cirrus-cron job starting, the auto-re-run attempt, and the final
failure-check e-mail.
Signed-off-by: Chris Evich <cevich@redhat.com>
Patch created by re-running the two example commands and manually
abbreviating long lists in the output.
Fixes#1766.
Signed-off-by: Fabian P. Schmidt <kerel@mailbox.org>
Since d9dfc44 the 'skopeo inspect' command exposes the LayerData
which often contains the layer size. This is a very useful feature
so we mentioned it in the README now.
Signed-off-by: Fabian P. Schmidt <kerel@mailbox.org>
This does reverse-dependency testing, verifying `proxy.go` using
the ostree-rs-ext Rust code's unit tests.
Based on #1781 by @cgwalters
Signed-off-by: Chris Evich <cevich@redhat.com>
The `behind-base-branch` setting means:
Renovate will rebase whenever the branch falls 1 or more
commit behind its base branch
Signed-off-by: Chris Evich <cevich@redhat.com>
> go get github.com/containers/image/v5@main
> make vendor
... to make sure that we don't regress against Skopeo 1.9.3.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
In some code I'm writing I want to be able to cleanly test if an
image exists, as distinguished from other errors like authentication
problems, network flakes etc.
As best I can tell, the containers/image abstraction doesn't
offer a clean way to do this.
For now, I chose the route of adding the ugly string error matching
here for the two cases I care about (docker v2s2 registry and oci
directories), so my Rust code can operate in terms of clean
`Option<Image>`.
Signed-off-by: Colin Walters <walters@verbum.org>
FIXME: This is not actually tested against a representative
error; we basically assume generic "scope is not sufficient" handling.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
AFAICT, “IBM Bluemix” has become “IBM Cloud”, and the “Bluemix” registry
is now (somehow related to?) icr.io; e.g.
https://cloud.ibm.com/docs/Registry?topic=Registry-registry_overview
lists bluemix.net and icr.io host names.
Randomly looking for a public image hosted on that registry, at least
> skopeo list-tags docker://icr.io/codeengine/firstjob
now succeeds.
So I’m assuming that at least the current cloud deployment now allows
listing tags, and does not need special handling. (It's unclear if
that is true for all existing deployments.)
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
- It's unclear why it exists in the first place
- Looking at callers of imagesToCopyFromRepo, the only caller of this:
either the input is a single repo, in which case the failure to
list tags clearly results in a no-op and a "No images to sync" fatal
failure ...
- ... or the input is YAML, and in that case the caller is already
skipping the repo on a failure.
Either way, it's unclear why we would have a special "Registry disallows
tag retrieval" error special case instead of the generic text.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
They were useful before assertSkopeoSucceeds/assertSkopeoFails,
when they were used multiple times. Now, they don't
make the code any shorter.
Should not change (test) behavior.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
(skopeo inspect) now provides more information about individual layers.
The default /etc/containers/registries.d/default.yaml now has all entries
commented-out, to use built-in defaults; that can change the default for lookaside-staging
to use an unprivileged users' home directory instead of a path in /var/.
- GHA: Re-use identical workflow from buildah repo
- Optimize upstream skopeo container image build
- Fix running tests on macOS
- Reformat with Go 1.19's gofmt
- Fix a comment
- Fix looking for commands with GNU make 4.2.1
- Talk about "registry repositories" in (skopeo sync) documentation
- Point at --all in the --preserve-digests option documentation
- Remove unused GIT_BRANCH definition
- Don't include git commit from a parent directory in the --version output
- Update for c/image's update of github.com/gobuffalo/pop
- Merge pull request #1737 from mtrmac/pop-v5-override
- Stop using docker/docker/pkg/homedir in tests
- add inspect layersData
- Don't abort sync if the registry returns invalid tags
- warn users about --dest-compress and --dest-decompress misuse
- document imageDestOptions.warnAboutIneffectiveOptions()
- warn about ineffective destination opts in sync cmd
- default.yaml should have all options commented
- Fix documentation in the default registries.d content.
- [CI:DOCS] Add quay-description update reminder
- Revert addition of -compat=1.17 to (go mod tidy)
- Update for https://github.com/klauspost/pgzip/pull/50
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
... to fix reads of compressed data by docker-archive:
> go get github.com/klauspost/pgzip@master
> make vendor
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Typically, the compat with earlier versions causes us to use
newer versions of dependencies, which can only be a good thing.
Over time, the 1.17 version reference is just going to become obsolete.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
The user is not very likely to be able to do anything about that,
and we have no other way to read those images - so just skip them;
we already skip image copies in much more directly user-caused
situations, including invalid user-provided strings.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
> go get github.com/containers/image/v5@main
> go mod tidy -go=1.16 && go mod tidy -go=1.17
> make vendor
The (go mod tidy) pair is necessary to keep c/image CI working.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This can happen when building RPMs out of tarballs (which don't contain
the .git repository).
To test:
> make -n /bin/skopeo; mv .git ../.git ; make -n bin/skopeo
Fixes#1707 .
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Running cross-platform compiles using emulation is a painfully
slow process. Since CI-runtime is limited, improve image build time
by leveraging the automatic RPM builds occurring for the podman-next
COPR repo. This adds build-time efficiency by offloading the
compilation task. Note: These RPMs are built any time the 'main'
branch changes, so they'll still be very recent.
https://copr.fedorainfracloud.org/coprs/rhcontainerbot/podman-next/
Signed-off-by: Chris Evich <cevich@redhat.com>
It's a PITA to maintain duplicate code across repos. Relatively
github-actions added a feature that allows re-using workflows
from other repos. Use that here to reduce duplication.
Signed-off-by: Chris Evich <cevich@redhat.com>
- [CI:DOCS] Cirrus: Use the latest imgts container
- Cirrus: Update CI VM images to match podman CI
- Bump github.com/containers/common from 0.49.0 to 0.49.1
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Note: Removed disused `PRIOR_FEDORA*` and `UBUNTU_*` references since
they're not actually used in this CI. Further, F35 VM images were not
built as part of `c6013173500215296` due to a missing golang 1.18
requirement for podman.
Signed-off-by: Chris Evich <cevich@redhat.com>
- Bump github.com/sirupsen/logrus from 1.8.1 to 1.9.0
- Bump github.com/containers/storage from 1.41.0 to 1.42.0
- Update to github.com/containers/image/v5 v5.22.0
- Update to github.com/containers/common v0.49.0
- Stop using deprecated names from c/common/pkg/retry
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Adds support for copying non-image OCI artifacts, and for
creating and enforcing sigstore signatures.
Shell autocompletions are now auto-generated, adding support
for zsh, fish and PowerShell.
Now requires Go 1.17.
- Bump github.com/docker/docker
- Config files live in /usr/local/etc on FreeBSD
- Avoid hard-coding the location of bash
- Bump github.com/containers/storage from 1.40.2 to 1.41.0
- Bump github.com/docker/docker
- add completion command to generate shell completion scripts
- Remove cgo_pthread_ordering_workaround.go
- Update c/image
- Stop calling gpgme-config
- shell completion: add Makefile target
- shell completion: add install instructions docs
- shell completion: add completion for transports names
- [CI:DOCS] Pin actions to a full length commit SHA
- Updated skopeo logo with new artwork
- Update to gopkg.in/yaml.v3 v3.0.0
- fix make completions for all POSIX shells
- Update to github.com/opencontainers/runc >= 1.1.2
- Cirrus: use Ubuntu 22.04 LTS
- Bump github.com/containers/ocicrypt from 1.1.4 to 1.1.5
- Bump github.com/stretchr/testify from 1.7.1 to 1.7.2
- Bump github.com/docker/docker
- Update go.mod to Go 1.17
- Use testing.T.Setenv instead of os.Setenv in tests
- Change a repo used for sync tests
- Use an updated CI image
- Update for docker/distribution CLI change
- Enable schema1 support on the test registries
- CoPR: Autobuild rpm on rhcontainerbot/podman-next
- [CI:DOCS] Makefile: include cautionary note for rpm target
- [CI:DOCS] skopeo.spec.rpkg: Fix syntax highlighting
- Bump github.com/spf13/cobra from 1.4.0 to 1.5.0
- Bump github.com/stretchr/testify from 1.7.2 to 1.7.4
- Bump github.com/stretchr/testify from 1.7.4 to 1.7.5
- Cirrus: Migrate multiarch build off github actions
- Update & fix skopeo multiarch image Containerfiles
- Use bytes.ReplaceAll instead of bytes.Replace(..., -1)
- Update IRC information
- Bump github.com/stretchr/testify from 1.7.5 to 1.8.0
- Introduce noteCloseFailure, use it for reporting of cleanup errors
- Modify error messages on failures to close
- Remove uses of pkg/errors
- Use errors.As() instead of direct type checks
- Vendor unreleased c/image with OCI artifact support
- Revert "Change a repo used for sync tests"
- Vendor in c/image with sigstore support
- Add --sign-by-sigstore-private-key to (skopeo copy) and (skopeo sync)
- Update for the renames of sigstore to lookaside
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
If a passphrase is not provided, prompt for one.
Outstanding:
- Should have integration tests.
- The signing options shared between copy and sync should live in utils.go.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
... to be a bit more robust against unexpected error wrapping.
Also be a little more idiomatic on the sync error handling path.
Should not change behavior, assuming the previous code was correct.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This is clearly safe because the changes are
mostly top-level CLI where nothing is checking
the type of the error.
Even in that case, use %w for idiomatic consistency
(and to make it easier to possibly move some code into a Go library.)
Mostly mechanical, but note the changes to error handling of .Close():
we use %w for the primary error, not for the close error.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
- Use a wrapping wording similar to c/image; it's slightly
awkward at the start of the error message, but those should
hopefully be rare.
- Notably, distinguish the three failure paths in (skopeo layers).
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Note that this is a behavior change: we used to do
retErr = errors.Wrapf(retErr, ..., closeErr)
which doesn't record closeErr if retErr was nil.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
These changes substantially mirror similar updates made recently to both
podman and buildah. Besides renaming `Dockerfile` -> `Containerfile`,
there are much needed updates to docs, and the build instructions.
Signed-off-by: Chris Evich <cevich@redhat.com>
The github actions workflow for this operation is complex and difficult
to maintain. For several months now a replacement has been running well
in the podman repository. It's scripts/components are centralized,
versioned, unit, and integration tested. Add cirrus tasks to run the
build, and another task to allow test builds in a PR.
This also adds support for a new magic CI string: `[CI:BUILD]`.
With this string in the PR title, automation will only do basic build
verification, and enable testing of the multi-arch build process.
Otherwise, many tasks were updated to not be created when running the
cirrus-cron multi-arch image builds, since this would simply be a waste
of time and invitation for flakes.
Lastly, since only native tooling is used in the new build process,
rename all the recipes to `Containerfile`.
Signed-off-by: Chris Evich <cevich@redhat.com>
For whatever reason, the comment rearrangement is
required for vim rpm synatx highlighting to work.
Also added a comment pointing out where additional comments
should go. :)
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
The new file `skopeo.spec.rpkg` along with a webhook will automatically
build rpms on every PR merge on the main branch.
Run `rpkg local` or `make rpm` to generate the rpm.
Known issue: Doesn't yet build for EL8 environments.
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
We expect schema1 images to work. Also, docker/distribution
doesn't provide useful errors for rejected schema1 images
( https://github.com/distribution/distribution/issues/2925 ),
which makes it impractical for Skopeo to automatically convert
schema1 to schema2.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
The k8s.gcr.io/coredns/coredns repo now contains an OCI
artifact, which we can't copy; so, use a different
repo to test syncing.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
... to silence Dependabot alerts about CVE-2022-29162 = GHSA-f3fp-gc8g-vw66.
Note that the vulnerable code is not actually included in Skopeo at all,
this is purely to silence imprecise vulnerability checkers.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
The {a,b} syntax is not POSIX compatible. The Makefile should run with
all POSIX shells so we cannot use shell specific features like this.
Fixes#1657
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
... to include a fix for CVE-2022-28948 = GHSA-hp87-p4gw-j4gq .
Note that the package is only used for Skopeo's tests, so
Skopeo's users can't reach the vulnerable code.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Make sure skopeo copy/inspect/delete show the transport names when shell
completion is used to not regress compared to the old bash completion
script.
In theory I would highly recommend to set completion functions for
every flag and command. This can be ensured with a test like this:
https://github.com/containers/podman/blob/main/cmd/podman/shell_completion_test.go
But this is a lot of work to get right and I am neither a skopeo user or
maintainer so I am missing a lot of context for most options. I think
this would be better handled by a person who knows skopeo better.
Normally options should either use AutocompleteNone() or
AutocompleteDefault() from c/common/pkg/completion.
Even better would be to add custom completion functions for arguments
that only accept fixed values, see AutocompleteSupportedTransports() in
this commit.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
As of the just-updated github.com/proglottis/gpgme 0.1.2,
the gpgme subpackage uses CGo's native #cgo pkg-config support
to find the relevant libraries, and we no longer need to manually set
CGO_CFLAGS and CGO_LDFLAGS. So stop doing that.
Note that the proglottis/gpgme update (implied by vendoring c/image)
means the minimal supported version of GPGME is 1.13.0.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Use the cobra lib to automatically generate shell completion scripts.
We can use the completion command which is automatically added, since it
is not importent for most users we hide it.
To test the new script on bash you can use `source <(bin/skopeo completion bash)`
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
On FreeBSD, bash lives in /usr/local/bin/bash. These scripts don't
really depend on bash so could be changed to /bin/sh.
Signed-off-by: Doug Rabson <dfr@rabson.org>
Mostly just name changes that should not change behavior, apart
from ioutil.ReadDir -> os.ReadDir avoiding per-item lstat(2) in
one case.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This enables pushing to registries where the push and pull uris may be
different, for example where pushed images are mirrored to a read only
replica for distribution.
Closes#1588
Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
skopeo list-tags docker-archive:... is now available.
- Improve a comment in the 010-inspect.bats test
- do not recommend upgrading all packages
- Bump github.com/containers/image/v5 from 5.19.1 to 5.20.0
- Update github.com/containerd/containerd
- Bump github.com/docker/docker
- Bump github.com/spf13/cobra from 1.3.0 to 1.4.0
- Add support for docker-archive: to skopeo list-tags
- Rename "self" receiver
- Remove assignments to an unused variable
- Add various missing error handling
- Simplify the proxy server a bit
- Bump github.com/stretchr/testify from 1.7.0 to 1.7.1
- Use assert.ErrorContains
- Update to Go 1.14 and revendor
- Use check.C.MkDir() instead of manual ioutil.TempDir() calls
- Formally record that we require Go 1.15
- Update the command to install golint
- Bump github.com/containers/ocicrypt from 1.1.2 to 1.1.3
- Bump github.com/docker/docker
- Bump github.com/containers/storage from 1.38.2 to 1.39.0
- Bump github.com/containers/common from 0.47.4 to 0.47.5
- Bump github.com/prometheus/client_golang to v1.11.1
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Resolves: CVE-2022-21698
Skopeo isn't actually impacted by the CVE unless a Prometheus listener
is set up, which is not a part of Skopeo's default behavior.
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
Actually add a DESCRIPTION heading.
Warn about deleting by digest - it affects an unknown set of tags.
Warn about deleting by tag - it currently works by the resolved digest.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
We already do in practice:
> vendor/golang.org/x/net/http2/transport.go:417:45: undefined: os.ErrDeadlineExceeded
so make that official.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This saves us at least 2 lines (error check, and cleanup) on every
instance, or in some cases adds cleanup that we forgot.
This is inspired by, but not directly related to, Go 1.15's addition of
Testing.T.TempDir.
NOTE: This might significantly increase the tests' disk space requirements;
AFAICS the temporary directories are only cleaned up when a whole "suite
finishes running.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Move JSON parsing into the request processing handler
so that we can consolidate the two instances of the response sending code.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
... as found by (golangci-lint run).
Note: this does not add (golangci-lint run) to the Makefile
to ensure the coding standard.
(BTW golangci-lint currently fails on structcheck, which doesn't
handle embedded structs, and that's a years-long known unfixed
limitation.)
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
> receiver name should be a reflection of its identity; don't use generic names such as "this" or "self" (ST1006)
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
The command to install skopeo for Ubuntu 20.04 includes a forced upgrade step for all packages.
Installing skopeo does not require the upgrade step, and it could lead to possible issues completely unrelated to the project.
Signed-off-by: Werner Robitza <werner.robitza@gmail.com>
Mainly this is to confirm some changes needed for the podman-py CI
setup don't disrupt operations here. Ref:
https://github.com/containers/automation_images/pull/111
Note: Glibc resolver configuration has changed from previous images. An
additional setup command was added to remove systemd-resolved from the
chain.
Signed-off-by: Chris Evich <cevich@redhat.com>
Highlights:
- A new sif: transport
- New options --multi-arch, --preserve-digests, --sign-passphrase-file
- Use a dynamic temp dir for test
- Add an option to allow copying image indexes alone
- proxy: Add a GetFullConfig method
- proxy: Also bump compatible semver
- Add option to preserve digests on copy
- Run codespell on code
- prompt-less signing via passphrase file
- add a SIF systemtest
- Merge pull request #1550 from vrothberg/sif-test
- Improve the documentation of the argument to (skopeo inspect)
- Document where various fields of (skopeo inspect) come from
- Improve the documentation of boolean flags
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
The Go behavior of boolean flags is as follows:
Accepted values are --flag, which is the same as --flag=true, and --flag=false,
which is the default (except for OptionalBoolFlag).
--flag {false,true} is parsed as --flag=true with a non-option {false,true} argument.
So, for almost all flags, document them just as --flag, not
mentioning the [={false,true}] part, because users can just
omit =true, or the whole flag instead of =false.
OTOH, for tls-verify, document only the tls-verify={true,false}
variant, because the primary use is tls-verify=false, and because
tls-verify is not "the default", but equivalent to an explicit
tls-verify=true (overriding registries.conf).
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
To support signing images without prompting the user, add CLI flags for
providing a passphrase file.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
> go get github.com/containerd/containerd@latest
> make vendor
... because 1.5.9 contains a vulnerability fix, and we
want to silence scanners.
NOTE: Skopeo DOES NOT use the vulnerable code that
was fixed in containerd 1.5.9, so it is NOT vulnerable to
GHSA-mvff-h3cj-wj9c .
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
> go get github.com/opencontainers/image-spec@a5463b7f9c8451553af3adcba2cab538469df00c
> make vendor
Primarily we want to use a 1.0.3-0... version rather than 1.0.2-0..., so that
dependencies on 1.0.2 don't cause Skopeo to use 1.0.2 instead of
the later main-branch code.
Go has some logic to prevent using pseudo-version that don't follow
a released version (which is the case here, where 1.0.2 is on a branch,
and we want to use a main-branch commit instead); luckily some later
PRs on the main branch include the full contents of the 1.0.2 branch.
So, update a bit further along the main branch.
This particular commit corresponds to the choice in
https://github.com/containers/image/pull/1433 .
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
... to silence warnings about CVE-2021-43784
/ GHSA-v95c-p5hm-xq8f .
NOTE: The vulnerable code was not used in this package,
so Skopeo is has not been vulnerable to this issue.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Sadly...I swear I had tested this at one point, but it was
*definitely* not the intention that we just return the container
runtime configuration.
I need a method to return the full image configuration. At some point
I must have accidentally added a redundant `.Config`.
This whole new method `GetFullConfig` is like `GetConfig` but
returns the whole image configuration. A specific motivation
here is that it's only in the image configuration that we can
stick arbitrary metadata (labels) that will survive a round trip through
docker schema v2.
just to keep various dependency checkers happy.
> go get github.com/containerd/containerd@v1.5.8
NOTE: This is NOT a fix for CVE-2021-41190 / GHSA-77vh-xpmg-72qh ,
that was fixed in Skopeo 1.5.2.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
... to a version past 1.0.2, just to keep various
dependency checkers happy.
> go get github.com/opencontainers/image-spec@v1.0.2-0.20211123152302-43a7dee1ec31
The commit is intended to match https://github.com/containers/image/pull/1419
to minimize churn.
NOTE: This is NOT a fix for CVE-2021-41190 / GHSA-77vh-xpmg-72qh ,
that was fixed in Skopeo 1.5.2.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
The new --multi-arch option allows the user to select between copying the
image associated with the system platform, all images in the index, or
just the index itself without attempting to copy the images.
Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
This test was incorrectly assuming that nothing would be made on disk,
but it was putting files into the source directory.
Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
Includes a fix for CVE-2021-41190 / GHSA-77vh-xpmg-72qh .
- use fedora:latest in contrib/skopeoimage/*/Dockerfile
- Fix test bug that prevented useful diagnostics on registry fail
- proxy: Add an API to fetch the config upconverted to OCI
- proxy: Add support for manifest lists
- proxy: Uncapitalize all errors
- Cirrus: Bump Fedora to release 35 & Ubuntu to 21.10
- Update to c/image v5.17.0
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
The Fedora 35 cloud images have switched to UEFI boot with a GPT
partition. Formerly, all Fedora images included support for runtime
re-partitioning. However, the requirement to test alternate storage
has since been dropped/removed. Rather than maintain a disused
feature, and supporting scripts, these Fedora VM images have reverted
to the default: Automatically resize to 100% on boot.
Signed-off-by: Chris Evich <cevich@redhat.com>
To fix compilation on MacOS.
I think actually we want to use this pervasively in our tests
on Linux; it doesn't really matter when run inside a transient
container, but `PDEATHSIG` is useful for persistent containers (e.g.)
toolbox and when running outside of a pid namespace, e.g. on a host
system shell directly or in systemd.
Signed-off-by: Colin Walters <walters@verbum.org>
We need to support manifest lists. I'm not sure how I missed this
originally. At least now we have integration tests that cover this.
The issue here is fairly subtle - the way c/image works right now,
`image.FromUnparsedImage` does pick a matching image from a list
by default. But it also overrides `GetManifest()` to return the
original manifest list, which defeats our goal here.
Handle this by adding explicit manifest list support code. We'll
want this anyways for future support for `GetRawManifest` or so
which exposes OCI manifest lists to the client.
Signed-off-by: Colin Walters <walters@verbum.org>
I debated adding "reverse dependency testing" using
https://crates.io/crates/containers-image-proxy
but I think it's easier to reuse the test infrastructure here.
This also starts fleshing out a Go client for the proxy (not
that this is going to be something most Go projects would want
versus vendoring c/image...but hey, maybe it'll be useful).
Now what I hit in trying to use the main test images is currently
the proxy fails on manifest lists, so I'll need to fix that.
Signed-off-by: Colin Walters <walters@verbum.org>
While the caller could fetch this today as a blob, it'd be in
either docker or oci schema. In keeping with the model of having
this proxy only expose OCI, add an API which uses the c/image logic
to do the conversion.
This is necessary for callers to get the diffIDs, and in general
to implement something like an external `skopeo copy`.
Signed-off-by: Colin Walters <walters@verbum.org>
Sigh. 'expr 1 - 1' yields 0 (correctly) but also exits 1. This
is even documented in the man page, but I didn't know it. And
thus, on the final iteration, when timeout reached 0, BATS
errored out on the expr instead of continuing to the 'podman logs'
or the 'die' message.
Solution is super trivial: use $(( ... )) instead of expr.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Surprisingly, the spf13/cobra CLI parsing logic, when presented
with an unknown subcommand outputs usage to stdout
and *exits successfully*.
This is bad for both users and scripts. Cargo cult some code
I found in podman to handle this.
Motivated by https://github.com/containers/containers-image-proxy-rs/pull/1
Signed-off-by: Colin Walters <walters@verbum.org>
This ensures layers are not uploaded that already exist on the
destination registry, in exchange for streaming layers to temporary
files when digests are unknown (ex. compressing "on the fly").
Signed-off-by: Paul Fisher <pfisher@lyft.com>
Following the discussion in #1478, we don't want to provide
(and maintain) static binaries, but giving instructions to
produce such builds (with appropriate warnings around these
instructions) was considered acceptable, so - here we go!
This imports the code from https://github.com/cgwalters/container-image-proxy
First, assume one is operating on a codebase that isn't Go, but wants
to interact with container images - we can't just include the Go containers/image
library.
The primary intended use case of this is for things like
[ostree-containers](https://github.com/ostreedev/ostree-rs-ext/issues/18)
where we're using container images to encapsulate host operating system
updates, but we don't want to involve the [containers/image](github.com/containers/image/)
storage layer.
Vendoring the containers/image stack in another project is a large lift; the stripped
binary for this proxy standalone weighs in at 16M (I'm sure the lack
of LTO and the overall simplicity of the Go compiler is a large factor).
Anyways, I'd like to avoid shipping another copy.
This command is marked as experimental, and hidden. The goal is
just to use it from the ostree stack for now, ideally shipping at least
in CentOS 9 Stream relatively soon. We can (and IMO should)
change and improve it later.
A lot more discussion in https://github.com/cgwalters/container-image-proxy/issues/1
By default skopeo checks to see if the user actually uses one of the
--*tls-verify flags. Their initial value is ignored. Setting the
initial value to false causes Cobra to not display the default value on
the screen when the user runs a `skopeo --help` command.
If the user does not specify a --*tls-verify option, it falls back to
using the value specified in the registries.conf file.
Fixes: https://github.com/containers/skopeo/issues/1383
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Go 1.17 introduces a much more reasonable build constraint format, and gofmt now fails without using it.
Sadly we still need the old format as well, to support <1.17 builds.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Dependabot was apparently not picking these up (and
several haven't had a release for a long time anyway).
Also move from github.com/go-check/check to its newly
declared (and go.mod-enforced) name gopkg.in/check.v1.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
the 'signing' test is flaking; symptom is that we can never
connect to the port on the registry:
https://api.cirrus-ci.com/v1/task/6208385738604544/logs/system.log
By all indications, the registry is up, i.e., the 'podman rm -f reg'
in teardown() succeeds, as shown by the 53c (CID) in the log. (It
bothers me that the FAIL message from die() does not appear in the
log, and I can't figure out why).
To try to diagnose this, run 'podman logs' on the registry upon
failure.
Signed-off-by: Ed Santiago <santiago@redhat.com>
In order to meet achievable deadlines converting from Travis to Cirrus
CI, one significant artifact was carried forward (instead of fixing):
Depending on a `--privileged` container to execute all/most automated
checks/tests.
Prior attempts to remove this aspect resulted in several test failures.
Fixing the problems was viewed as more time-consuming than simply
preserving this runtime environment.
Time has passed, and the code has since moved on. This commit removes
the legacy need to execute CI operations in a `--privileged`
container, instead running them directly on the host. At the same time,
the necessary test binaries are obtained from the same container used
for development/local testing purposes. This ensures the two
experiences are virtually always identical.
Signed-off-by: Chris Evich <cevich@redhat.com>
The Cirrus-CI configuration for this repository is setup to execute test
builds on certain important release branches. There is no built-in way
to monitor these for success or failure. This commit adds a
Github-Actions Workflow to e-mail the podman-monitor list if any fail.
Otherwise it will take no action if everything is successful.
Note: This duplicates 99.999% of the same YAML used for the Buildah
repository. The only changes were for the settings URL and
mentioning "skopeo" in a comment. A similar workflow is also in use
on the Podman repository.
Signed-off-by: Chris Evich <cevich@redhat.com>
We now synchronize the behavior with Podman and accept repositories
during login and logout per default.
Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
... so that we terminate with the full context and pointing at
the relevant code, instead of relying
on the overall test suite timeout.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Differentiate, again, between (skopeo --tls-verify subcommand)
and (skope subcommand --tls-verify), by
- using a "local" Corba flag for the (skopeo --tls-verify ...) variant
- adding separate --tls-verify flags to subcommands that only accept
them as legacy, available through deprecatedTLSVerifyFlags
(unlike the non-legacy path of dockerImageFlags());
- using TraverseChildren: true; this causes the global and
per-subcommand flags to be treated separately by Corba,
i.e. they no longer happen to share the "Hidden" flag
and Corba actually sets the right flag variable now.
So, we can now warn on (skopeo --tls-verify command) again,
and --help lists the flag correctly (it is hidden at the
global level, and in subcommands like copy that deprecated it,
but visible in subcommands like inspect where it's not deprecated).
NOTE: This removes --tls-verify from (skopeo manifest-digest) and
the three signing commands; it never made sense there. This change
could, in principle, break some users.
Also update man pages to match.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
It will get bigger, and we will also want to test imageDestOptions
for extra confidence.
Only moves the code, should not change (test) behavior.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
The current implementation can actually contact the registry (if
logout fails with "not logged in" but there are .docker/config.json
credentials present), so provide a non-deprecated way to disable TLS.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
The necessary images have been manually copied over to quay. Code was
updated with centralized constants for the utilized images. Tests then
all reference the constants (in case the image locations need to change
again).
Signed-off-by: Chris Evich <cevich@redhat.com>
This is the script that runs 'skopeo COMMAND --help' and
cross-checks that all the option flags are documented
in man pages, and vice-versa (all options listed in man
pages appear in COMMAND's --help message).
Copied from podman, with changes for skopeo-land (removing
the rst checks, and conforming to skopeo conventions).
Signed-off-by: Ed Santiago <santiago@redhat.com>
This duplicates the change from
https://github.com/containers/buildah/pull/3385
Since this workflow is duplicated across three repositories, maintaining
changes becomes onerous if the item contents vary between
implementations in any way. Improve this situation by encoding the
repository-specific details into env. vars. then referencing those vars
throughout. This way, a meaningful diff can be worked with to compare
the contents across repositories.
Also included are abstractions for the specific command used to obtain
the project version, and needed details for filtering the output. Both
of these vary across the Buildah, Skopeo, and Podman repos.
NOTE: This change requires the names of two github action secrets
to be updated: SKOPEO_QUAY_USERNAME -> REPONAME_QUAY_USERNAME
(and *PASSWORD).
Signed-off-by: Chris Evich <cevich@redhat.com>
... because our unit tests use createApp, so the current
main()-only edit is not visible to unit tests.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Add new script, hack/man-page-checker, copied from podman. Run it
in 'make validate-local' target.
This is NOT the checker requested in #1332 (verify that flags
listed in 'skopeo foo --help' are documented in man pages and
vice-versa). This is a much simpler script that merely looks
for very basic typos or discrepancies between skopeo.1.md
and skopeo-foo.1.md.
The next part (cross-checking flags) is in progress but will
require a huge number of changes to the man pages. I'm submitting
this now because it's easy to review.
Signed-off-by: Ed Santiago <santiago@redhat.com>
This partially reverts commit a81cd74734
so that most path variables like PREFIX and BINDIR refer to paths on
the installed system, and DESTDIR is used only in (make install), following
the philosophy of the GNU coding standards for path variables. (But not precisely
the variable names, which are lowercase in the standard, nor the principle that
even SYSCONFDIR should be under $PREFIX.
Keep the use of ?= instead of = because it somewhat better expresses the idea
that the values can be overridden.
Use ${DESTDIR}${BINDIR} instead of ${DESTDIR}/${BINDIR} etc, so that a plain
(make install) does not use paths like //usr/bin/... ; strictly speaking they
are IIRC reserved by POSIX, and more importantly it just looks untidy :)
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Normally it's a best-practice to only use short-names in Dockerfiles.
However, this image is really only ever used for testing/CI purposes.
Side-step the docker hub rate limits entirely by simply hard-coding the
FQIN and hope that's also "okay" for non-CI usage.
Signed-off-by: Chris Evich <cevich@redhat.com>
Bump up the global timeout due to some (possibly) temporary failures in
the 'cross' task. Also fix build-failure in Dockerfile related to use
of pre-module golang packages.
Signed-off-by: Chris Evich <cevich@redhat.com>
This was copy-pasted from buildah and podman, unfortunately the
Dockerfile entrypoint is different for skopeo. Fix it.
Signed-off-by: Chris Evich <cevich@redhat.com>
- Bugfix `make nixpkgs` which pin with branch `nixos-21.05`
- Code lint with `nixpkgs-fmt`
- Code sync between x86\_64 and aarch64
Signed-off-by: Wong Hoi Sing Edison <hswong3i@pantarei-design.com>
Github checks the `workflows` sub-directory for github actions workflow
files. Since it was called `workflow`, nothing was running. Fix this
by renaming the directory.
Signed-off-by: Chris Evich <cevich@redhat.com>
This is a port from the podman and buildah repository workflows.
It's purpose is to build and push multi-arch images containing the
latest upstream, testing, and stable versions of skopeo. It fully
replaces the last remaining use of Travis in this repo, for
substantially the same purpose.
In a future commit, I intend to de-duplicate this workflow from
podman and buildah, such that all three share a common set of details.
Until then, any changes will need to be manually duplicated across
all three repos.
Signed-off-by: Chris Evich <cevich@redhat.com>
This commit removes installation steps for older distros:
1. Amazon Linux 2
2. CentOS 7
3. Debian 10
4. Raspbian 10
5. Raspberry Pi OS armhf
We're no longer building new packages for these distros and the
installation steps for older packages are no longer guaranteed to work.
So, these are best removed from the official docs.
Related blog post (also applies to skopeo):
https://podman.io/blogs/2021/03/02/podman-support-for-older-distros.html
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
The `test_skopeo_task` must be substantially duplicated between this
repository and containers/image. Make this easier to maintain by
implementing a runner script to contain any/all differences. This
allows the `test_skopeo_task` definition to remain 100%
identical across the two repositories.
Signed-off-by: Chris Evich <cevich@redhat.com>
In 18e6c6f17b there where changes to
better handle PREFIX and DESTDIR in the Makefile. "CONTAINERSCONFIGDIR"
was renamed to "CONTAINERSCONFDIR" without ensuring it was correct in
the install block. This results in policy.json being isntalled into
"${DESTDIR}/" which is "/policy.json" for Linux distros.
Signed-off-by: Morten Linderud <morten@linderud.pw>
This won't be useful unless/until there are `.cirrus.yml` tasks which
actually utilize VMs. Once there are, or to support their development,
this script may be helpful.
Signed-off-by: Chris Evich <cevich@redhat.com>
- Bugfix `make nixpkgs` which pin with branch `nixos-20.09`
- Code lint with `nixpkgs-fmt`
- Code sync between x86\_64 and aarch64
Signed-off-by: Wong Hoi Sing Edison <hswong3i@pantarei-design.com>
The Skopeo images set `REGISTRY_AUTH_FILE=/auth.json` which is breaking
non-root users inside the container from logging in (`/` is writable by
root only). Setting it to `/tmp/auth.json` will support running
non-root users inside the container.
Fixes: #1233
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
This updates the indirect dependency we have on dsnet/compress (via
opencontainer/image-tools) to the latest master commit, which includes
an update to a non-vulnerable version of xz.
Without this env. var. being set from hack/make.sh, many/most
integration tests will `SKIP`. Fix this by notifying the user
and setting the magic `SKOPEO_CONTAINER_TESTS` variable.
Signed-off-by: Chris Evich <cevich@redhat.com>
These tests need to operate as part of the c/image repository CI to
verify downstream-usage. That environment is already inside the
container built from the Dockerfile (here). Support this use-case by
adding 'local' targets which bypass the container build. Also,
simplify the "in-container" check to more specifically verify the exact
container image it's operating under.
Signed-off-by: Chris Evich <cevich@redhat.com>
The quay.io/skopeo/testing:latest image was showing v1.2.0 for a version when it
should have been showing at least v1.2.1. The issue was the Fedora tag in the
Dockerfiles used to build the images was set to 32 and not the later 33.
Addresses: #1204
Signed-off-by: TomSweeneyRedHat <tsweeney@redhat.com>
By popular demand, add the stale bot which we'are already using at
Podman. The bot will remind us every 30 days of inactivity on specific
issues or pull requests and has turned into a useful tool to keep things
on the radar.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
When copying to an OCI layout destination, forcing zstd compression,
check that the manifest correctly describes the type of the layer blob.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Prior to this, the User-Agent used by containers/image would default to
the default User-Agent for golang, which makes it difficult to
distinguish skopeo from any other golang binaries in registry logs.
Signed-off-by: Jon Johnson <jonjohnson@google.com>
Switch to using images from quay.io/libpod instead, where
we're not (yet) subject to rate limiting.
Completely rewrite one unclear test. The purpose of the
test was to test #708, in which 'skopeo inspect --raw'
was fixed to be able to inspect images that do not
match the current host's os+arch. We now use a fixed
test image on quay.io, generated by a new script,
whose manifest is pretty unlikely to match our host.
Signed-off-by: Ed Santiago <santiago@redhat.com>
PREFIX should default to /usr/local according to GNU coding standards.
DESTDIR should just be used for install
This will allow us to specify
make PREFIX=/usr DESTDIR=/tmp/build/ install
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Do repository synchronization tests using k8s.gcr.io/pause and
k8s.gcr.io/coredns/coredns instead of docker.io/alpine.
The k8s.gcr.io/pause repository includes multiple tags, at least one of
which is a single image ("1.0"), and at least one of which is a manifest
list ("3.2", "3.3"), and includes a "latest" tag.
The k8s.gcr.io/coredns/coredns repository includes multiple tags, at
least one of which is a single image ("v1.6.6"), and at least one of which
is a manifest list ("v1.8.0").
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Switch most of our tests that exercise reading, copying from, and
inspecting tags that point to manifest lists from using
docker.io/estesp/busybox to using
registry.fedoraproject.org/fedora-minimal, which doesn't limit how often
we can pull the images.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
We are preparing for RHEL 8.4 release and want to make
sure all container tools have the same containers suppackages.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Pass down the creds from yaml file only if the values are not empty.
Enables to use credentials from other authfiles alternatively.
Signed-off-by: Qi Wang <qiwan@redhat.com>
Homebrew:
> Warning: You are using macOS 10.13.
> We (and Apple) do not provide support for this old version.
> You will encounter build failures with some formulae.
So, update to the 10.14 major version, fully-updated.
Also remove the Xcode update attempt, it was added before
Homebrew was warning about the Xcode version, but updating only
after running Homebrew does not help, and anyway it does not
complain anymore after the update.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Also update to the correct one information about required env variables
for multi-arch build
Signed-off-by: Yulia Gaponenko <yulia.gaponenko1@de.ibm.com>
Service accounts (a.k.a. robots) in `quay.io` are forcably namespaced
to the user or orginization under which they are created. Therefore,
it is impossible to use a common login/password to push images for
both `skopeo` and `containers` namespaces. Worse, because the
authentication is recorded against `quay.io`, multiple login sessions
are required.
Fix this by adding a function definition which verifies non-empty
username/password arguments, before logging in. Call this function
as needed from relevant targets, prior to pushing images.
Signed-off-by: Chris Evich <cevich@redhat.com>
This replicates the --all copy flag to sync to perform the same
behavior. Namely, the default is CopySystemImage unless --all is passed
which changes the behavior to CopyAllImages. While it is probably
desirable for --all to be the default as there is no option to override
ones architecture with the sync command, --all can potentially break
existing sync incantations depending on registry support. Hence
CopySystemImage remains the default.
Signed-off-by: Andrew DeMaria <ademaria@cloudflare.com>
Travis is used, as it has native hardware to run the build for many
architectures (amd64, s390x, ppc64le). Docker is used as build and
manifest tool. `quay.io/skopeo/upstream:master`, `quay.io/skopeo/stable:v1.2.0`
and `quay.io/containers/skopeo:v1.2.0` are specified as target multi-arch
upstream image.
Travis config file has 3 stages:
- local-build to do the local test for linux/amd64 and osx, as it was in
the initial code
- image-build-push to build and push images for specific architectures
(amd64, s390x, ppc64le)
- manifest-multiarch-push to create and push manifest for multi-arch
image - `quay.io/skopeo/upstream:master`, `quay.io/skopeo/stable:v1.2.0`
and `quay.io/containers/skopeo:v1.2.0`
last stage amnd image push step are not done for pull request.
2 env variables specified in Travis settings are expected - QUAY_USERNAME and
QUAY_PASSWORD to push the images to quay.io.
As a result multi-arch images for 3 architectures are published.
README about build setup id prepared
Signed-off-by: Yulia Gaponenko <yulia.gaponenko1@de.ibm.com>
For yet unknown reasons, Travis throws permission errors when trying to
recursively list the contents of a temp directory. It passes locally,
so disable the logs to unblock CI. Note, the reasons for the error are
yet to be revealed.
Related-issue: https://github.com/containers/skopeo/issues/1093
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
We now use the osusergo build tag to not use the glibc functions which
occur in the warnings but them from golang the os/user package.
Signed-off-by: Sascha Grunert <sgrunert@suse.com>
By default we should build bin/skopeo locally
and build docs locally.
Show output when doing make docs.
Add description in `make help` to explain default
behaviour.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Go 1.15 deprecates checking CN; this broke gating tests:
Get "https://localhost:5000/v2/": x509: certificate relies on legacy Common Name field, use SANs or temporarily enable Common Name matching with GODEBUG=x509ignoreCN=0
Easy two-line solution in the 'openssl' invocation. Huge
thanks to Nalin for tracking down and fixing while I was
still getting started:
https://github.com/containers/buildah/pull/2595
Copied from 0f2892a5b021de3b1cf273f5679fda8298b57c02 in buildah
Signed-off-by: Ed Santiago <santiago@redhat.com>
Add a make target that cross-compiles for a handful of the possible
targets that `go tool dist list` can tell us about.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
... which no longer works after #932.
This does not add documentation for the current static build approach,
nor does it add any other place where DISABLE_CGO is documented;
both are not tested by CI, and discouraged due to bad integration
with the rest of the system.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
because that's what users are looking for, and instead of using
a containers-storage: source, which might not even work all that
well with all the automatic defaults Podman sets up.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Enables to retry skopeo inspect. Add `--retry-times` to set the number of times to retry. Use exponential backoff and 1s as default initial retry delay.
Signed-off-by: Qi Wang <qiwan@redhat.com>
... which are currently failing with
> Error: The `brew link` step did not complete successfully
> The formula built, but is not symlinked into /usr/local
> Could not symlink Frameworks/Python.framework/Headers
> Target /usr/local/Frameworks/Python.framework/Headers
> is a symlink belonging to python@2. You can unlink it:
> brew unlink python@2
because the Travis-installed machine apparently has quite a few
Homebrew formulae installed, with an old version of Homebrew,
including a now-removed python@2, and that prevents updates of
python@3.
Remove the obsolete motivation for running (brew update), and replace it
with a similarly-good motivation that the Travis images are just too old
to be relevant to users.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
registry:2 no longer contains htpasswd.
Also don't use log_and_run ... >> $file
because that will cause the command to be logged to $file.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Add `-y` options to yum clean all
Only delete below /var/cache/dnf so that I can use the
-v /var/cache/dnf:/var/cache/dnf:O option when building
to speed up builds.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Say that the regex is the cause, include it in the error message,
and don't continue as if the compilation succeeded.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
... instead of manually parsing strings.
Should not change behavior, except maybe error messages if the
registry returns invalid tags.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This removes another string formatting use, and removes the
last recently introduced docker.Reference->reference.Named
redundancy.
Should not change behavior, apart from error messages.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Right now that only complicates code by going through a
types.ImageReference->reference.Named->types.ImageReference sequence,
but that will change as we modify the callers as well.
Should not change behavior.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
It is redundant, only used to form a tagged reference,
which can be done more safely using reference.WithTag.
Also move the *types.SystemContext parameter to the front,
as is usual.
Should not change behavior, apart from a few error messages.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
- Improve the language
- Be consistent with the previous example about a trailing slash
- Don't unnecessarily quote :, it is not a shell metacharacter.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Fields that magically change their behavior depending on type of the value
are too much hassle for no benefit.
For now, this just copies&pastes the full loop in imagesToCopyFromRegistry
to create another loop handling the new ImagesByTagRegex field. Simplifications
to reduce duplication will follow.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This PR adds the Dockerfiles necessary to create the upstream
and testing variants of the Skopeo container images that will
reside in quay.io/skopeo/upstream and quay.io/skopeo/testing
repositories. The only difference in the Dockerfile between
the stable and testing image is the option `--enablerepo updates-testing`
was added. The testing variant is relatively the same, but
I'd to clone and install Skopeo in the container.
I've also added a README.md which explains all of the varities
of images and includes some sample usage.
Signed-off-by: TomSweeneyRedHat <tsweeney@redhat.com>
Adds the Dockerfile for building the Skopeo container
image on quay.io. Once merged, this image will be
built automatically upon any merge into the master
branch. The images will live at:
quay.io/containers/skopeo:latest
quay.io/skopeo/stable:latest
I've built an image using this Dockerfile and have pushed
it to both repositories if you want to play with that.
Once merged, I'll create similar Dockerfiles for
quay.io/skopeo/testing and quay.io/skopeo/upstream.
Signed-off-by: TomSweeneyRedHat <tsweeney@redhat.com>
Only bumps the version number after the recent vendoring
from master, but Dependabot seems to be confused by that;
so, update to the final release to hopefully un-confuse it.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This is an unreleased version of c/image, but it is important to
to have the test added in in the next commit enforcing as soon as
possible.
> go get github.com/containers/image/v5@HEAD
> make vendor
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Use cobra in skopeo can help share code with podman/buildah(code for skopeo login/logout CLI).
(libpod issue #839)
Signed-off-by: Qi Wang <qiwan@redhat.com>
'podman info' changed format, again, without preserving backward
compatibility. Basically, some keys that used to be lower-case
are now upper-case-first-letter.
These tests need to work with new podman on rawhide, and
old podman on f31/f32 and possibly RHEL. We must therefore
add a revolting workaround for the change.
Signed-off-by: Ed Santiago <santiago@redhat.com>
This is conceptually consistent: First change the set of
dependencies, then update the vendored copy.
(Due to (go mod verify) afterwards, and CI running this again,
this should not make a difference in practice, so this is just
a clean-up.)
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
(export a=b command args) does not run (command args) with a=b,
it sets $a to b, and marks variables $a $command $args as exported,
i.e. (command args) is not run.
So, before https://github.com/containers/skopeo/pull/888 we were not actually
running (go mod tidy), and now we are not running (go mod vendor).
Just use $(GO), which already sets GO111MODULE=on, without the extra export.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
PR 834 broke Fedora gating tests, because "--runtime runc"
doesn't work so well on Rawhide. Let's try to be smarter
about when we add that override.
Signed-off-by: Ed Santiago <santiago@redhat.com>
We currently need it to drag in recent versions of other dependencies,
per https://github.com/containers/skopeo/issues/796 .
I'll work to update the relevant dependencies in c/image, but that will
only propagate to skopeo in the next c/image release; in the meantime,
this at least undoes the downgrades.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Replace shortnames with FQINs; this should allow tests to
run regardless of the state of registries.conf.
And, fix one broken new test that invoked 'jq' (without dot).
This usage works in Fedora, but not in RHEL.
Signed-off-by: Ed Santiago <santiago@redhat.com>
crun had a regression running on cgroupsv1 in containers. It has been
fixed upstream but did not yet bubble up into the packages. Force using
runc to unblock Skopeo's CI.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Fix cli to use REGISTRY_AUTH_FILE if set and to display the
default location to use for authfiles in the `skopeo copy --help`
Modify tests to verify the different settings.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
* Bump github.com/containers/image/v5 from 5.2.0 to 5.2.1
* Bump gopkg.in/yaml.v2 from 2.2.7 to 2.2.8
* Bump github.com/containers/common from 0.0.7 to 0.1.4
* Remove the reference to openshift/api
* vendor github.com/containers/image/v5@v5.2.0
* Manually update buildah to v1.13.1
* add specific authfile options to copy (and sync) command.
* Bump github.com/containers/buildah from 1.11.6 to 1.12.0
* Add context to --encryption-key / --decryption-key processing failures
* Bump github.com/containers/storage from 1.15.2 to 1.15.3
* Bump github.com/containers/buildah from 1.11.5 to 1.11.6
* remove direct reference on c/image/storage
* Makefile: set GOBIN
* Bump gopkg.in/yaml.v2 from 2.2.2 to 2.2.7
* Bump github.com/containers/storage from 1.15.1 to 1.15.2
* Introduce the sync command
* openshift cluster: remove .docker directory on teardown
* Bump github.com/containers/storage from 1.14.0 to 1.15.1
* document installation via apk on alpine
* Fix typos in doc for image encryption
* Image encryption/decryption support in skopeo
* make vendor-in-container
* Bump github.com/containers/buildah from 1.11.4 to 1.11.5
* Travis: use go v1.13
* Use a Windows Nano Server image instead of Server Core for multi-arch
testing
* Increase test timeout to 15 minutes
* Run the test-system container without --net=host
* Mount /run/systemd/journal/socket into test-system containers
* Don't unnecessarily filter out vendor from (go list ./...) output
* Use -mod=vendor in (go {list,test,vet})
* Bump github.com/containers/buildah from 1.8.4 to 1.11.4
* Bump github.com/urfave/cli from 1.20.0 to 1.22.1
* skopeo: drop support for ostree
* Don't critically fail on a 403 when listing tags
* Revert "Temporarily work around auth.json location confusion"
* Remove references to atomic
* Remove references to storage.conf
* Dockerfile: use golang-github-cpuguy83-go-md2man
* bump version to v0.1.41-dev
* systemtest: inspect container image different from current platform
arch
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
The referenced tag has been removed, which breaks dependabot (#791).
This is another attempt to fix it, by removing an explicit reference
(which was added when updating Buildah, because the version seemed newer than
Buildah's v0.0.0 with a newer commit).
The referenced package is never even physically vendored in here, so remove the
reference:
> go mod edit -droprequire=github.com/openshift/api
> make vendor
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
With additional prefixed flags for authfiles, it is possible to override the shared authfile flag to use different authfiles for src and dest registries. This is an important feature if the two registries have the same domain (but different paths) and require separate credentials.
Closes#773.
Signed-off-by: Daniel Strobusch <1847260+dastrobu@users.noreply.github.com>
Remove a direct reference on c/image/v5/storage which breaks the build
when using the `containers_image_storage_stub`. The reference is only
used to get the storage tranport string, which is now hard-coded; this
is fine as the transport will not change for backwards compat.
Fixes: #771
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
The skopeo sync command can sync images between a SOURCE and a
destination.
The purpose of this command is to assist with the mirroring of
container images from different docker registries to a single
docker registry.
Right now the following source/destination locations are implemented:
* docker -> docker
* docker-> dir
* dir -> docker
The dir location is supported to handle the use case
of air-gapped environments.
In this context users can perform an initial sync on a trusted machine
connected to the internet; that would be a `docker` -> `dir` sync.
The target directory can be copied to a removable drive that can then be
plugged into a node of the air-gapped environment. From there a
`dir` -> `docker` sync will import all the images into the registry serving
the air-gapped environment.
Notes when specifying the `--scoped` option:
The image namespace is changed during the `docker` to `docker` or `dir` copy.
The FQDN of the registry hosting the image will be added as new root namespace
of the image. For example, the image `registry.example.com/busybox:latest`
will be copied to
`registry.local.lan/registry.example.com/busybox:latest`.
The image namespace is not changed when doing a
`dir:` -> `docker` sync operation.
The alteration of the image namespace is used to nicely scope images
coming from different registries (the Docker Hub, quay.io, gcr,
other registries). That allows all of them to be hosted on the same
registry without incurring in clashes and making their origin explicit.
Signed-off-by: Flavio Castelli <fcastelli@suse.com>
Co-authored-by: Marco Vedovati <mvedovati@suse.com>
Remove the $HOME/.docker directory when tearing down a cluster,
so that subsequent cluster creations can be carried out successfully.
Signed-off-by: Marco Vedovati <mvedovati@suse.com>
Add a vendor-in-container make target to allow for executing make vendor
in a golang:1.13 container. The CI is currently enforcing golang 1.13
which has a different vendoring behavior than previous versions which
can lead to failing tests as some files might be added or deleted. The
new make target will help users who are not using 1.13 to vendor their
changes.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
This image is about 100 MB instead of about 2 GB for the Server Core,
decreasing disk requirements and hopefully significantly speeeding up
integration tests.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Experimentally, this seems to help with localhost access inside that
container (but I have no idea what's the reason for that).
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
The nested podman tries to write to it. This primarily only
removes noise from logs, it does not seem to significantly change
behavior.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This allows using the vendored dependencies instead of
searching for them in $GOPATH and elsewhere.
This does not necessarily matter for skopeo itself, but
the test-skopeo Makefile target in containers/image uses
(go mod edit -replace) to replace the vendored c/image with
a locally-edited copy; skopeo's (make check) then runs tests in
a container which does not have access to this locally-edited
copy, and since Go 1.13 this causes (go {list,test,vet})
to fail if -mod=vendor is not used.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Move signature yaml file to point at /var/lib/containers/sigstore.
Change skopeo-copy.1 to use containers-storage and docker transports
rather then atomic.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
These are getting out of date and should be left in containers/storage.
If packagers need it then then should get it from that repo.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
When --raw is provided, can inspect show the raw manifest list, w/o
requiring any particular platform to be present, this test case is
used for make sure inspect command w/ --raw option works well for
container image is different from current platform arch.
Signed-off-by: Alex Jia <chuanchang.jia@gmail.com>
Add a --all/-a flag to instruct us to attempt to copy all of the
instances in the source image, if the source image specified to "skopeo
copy" is actually a list of images. Previously, we'd just try to locate
one for our preferred OS/arch combination.
Add a couple of tests to verify that we can copy an image into and then
back out of containers-storage. The contents of an image that has been
copied out of containers-storage need a bit of tweaking to compensate
for containers-storage's habit of returning uncompressed versions of the
layer blobs that were originally written to it, in order to be
comparable to the image as it was when it was pulled from a registry.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
- zstd test - give unique name.
a36d81c copy/pasted an existing test but didn't give
the new test a new name, leading to bats warning:
duplicate test name(s) in [...]/020-copy.bats
- start_registry() - use bash builtins, not curl, to test
if registry port is open.
curl on Fedora now barfs with "Received HTTP/0.9 when not
allowed" when the registry is run with SSL, because the
response is not valid HTTP. One workaround would be 'curl
--http0.9' but (surprise) that option doesn't exist on rhel8;
and even with that option we would need --output /dev/null
to silence a different curl warning. Curl is overkill
for this purpose anyway, all we really need is netcat
or some simple binary is-port-listening-or-not test.
Fortunately, bash provides a /dev/tcp/<host>/<port>
emulator that does the right thing and works on Fedora
as well as RHEL8.
- new log_and_run() helper
This is the noisiest yet least critical part of this PR.
I'm sorry. It's motivated by my frustration in trying
to reproduce the curl problem above: getting just the
right incantation of openssl + podman-run cost me time.
With this enhancement, important commands are logged
as part of the output of failing tests, making it
easy[*] for maintenance programmers to figure out a
recipe for reproducing the failure.
[*] "easy" as long as the test-writing developer
uses log_and_run() wisely.
Signed-off-by: Ed Santiago <santiago@redhat.com>
Add a systemtest copying an image from docker to storage and then to an
oci-archive. There are other ways to trigger the same code paths, but
this one has caught a regression in c/image in libpod's.
Fixes: #734
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
The "inspect: env" test started failing since the environment in the
`fedora:latest` image has changed. Hence, only check for `PATH` in
the image's environment which is a defacto standard.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Skip the NewImage() step if we're just inspecting the raw manifest, so
that if the tag or digest being inspected resolves to a manifest list,
the local arch/OS combination doesn't need to be found in it to avoid an
error.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
This mainly pulls in the latest support for zstd-compressed layers and
eases testing of containers/image.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
The PR #700 replaced ostree buildtag with containers_image_ostree.
However specifying the ostree buildtag is needed by containers/storage
vfs driver.
Signed-off-by: Marco Vedovati <mvedovati@suse.com>
Starting from 9b902d0, the ostree transport is disabled by default,
and ostree is enabled with the tag containers_image_ostree.
Signed-off-by: Marco Vedovati <mvedovati@suse.com>
* vendor github.com/containers/image@v3.0.0
* enforce blocking of registries
* Fix lowest possible go version to be 1.9
* man pages: add --dest-oci-accept-uncompressed-layers
* bash completion: add --dest-oci-accept-uncompressed-layers
* README.md: skopeo on openSUSE
* copy: add a CLI flag for OCIAcceptUncompressedLayers
* migrate to go modules
* README: Clarify use of `libbtrfs-dev` on Ubuntu
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Go 1.13.x isn't sensitive to the `GO111MODULE` environment variable
causing `make binary-local` to not use the vendored sources in
`./vendor`. Force builds of module-supporting go versions to use the
vendored sources by setting `-mod=vendor`.
Verified in a `fedora:rawhide` container.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Vendor in the latest c/image to enforce blocking of registries when
creating a c/image/docker.dockerClient. Add integration tests to
avoid regressions.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
* progress bar: use spinners for unknown blob sizes
* improve README.md and the review of the changes
* use 'containers_image_ostree' as build tag
* ostree: default is no OStree support
* Add "Env" to ImageInspectInfo
* config.go: improve debug message
* config.go: log where credentials come from
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
containers/storage needs math/bits which has been added in go 1.9, so
this is now the lowest possible go version to build skopeo. We can also
remove the GO15VENDOREXPERIMENT variable since this has been enabled in
go 1.6 per default and removed in go 1.7.
Signed-off-by: Sascha Grunert <sgrunert@suse.com>
Turn of go modules to avoid breaking build environments to accidentally
try pulling the dependencies instead of using the ./vendor directory.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Adds a simple documentation how to install skopeo and its build dependencies
on an openSUSE distribution
Signed-off-by: José Guilherme Vanz <jguilhermevanz@suse.com>
Don't get tricked by the v1.5.2-0.20190620105408-93b1deece293 reference
in the go.mod file. The upper commit is *after* v2.0.0 and go simply
has a bug in dealing with git tags.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
There are cases where we want to pass this flag to the actual copy engine,
so let's add a CLI flag for it.
Signed-off-by: Tycho Andersen <tycho@tycho.ws>
Using `go get` with go modules has side-effects that we can avoid by
installing golint from the Fedora repositories.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
This adds the mirror-by-digest-only option to mirrors, and moves the search
order to an independent list.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This does not happen in this repo's tests, but containers/image's
(make test-skopeo) fails in the containers_image_openpgp configuration with
> not ok 10 signing
> ...
> # time="2019-06-11T20:59:32Z" level=fatal msg="Signing not supported: signing is not supported in github.com/containers/image built with the containers_image_openpgp build tag"
To reproduce/test this:
> make test-system BUILDTAGS='ostree containers_image_openpgp'
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
The usual 'podman run -d' race condition: we've been forking
off the container but not actually making sure it's up; this
leads to flakes in which we try (and fail) to access it.
Solution: use curl to check the port; we will expect a zero
exit status once we can connect. Time out at ten seconds.
Resolves: #675
Signed-off-by: Ed Santiago <santiago@redhat.com>
Since GPG 2.1, GPG asks for a passphrase by default; opt out when
generating test keys to avoid
> gpg: agent_genkey failed: No pinentry
> gpg: key generation failed: No pinentry
which happens otherwise (and we can't use an interactive pinentry
in a batch process anyway).
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Skopeo CI tests run under podman; hence the registries
run in the tests will be podman-in-podman. This requires
complex muckery to make work:
- install bats, jq, and podman in the test image
- add new test-system Make target. It runs podman
with /var/lib/containers bind-mounted to a tmpdir
and with other necessary options; and invokes a
test script that hack-edits /etc/containers/storage.conf
before running podman for the first time.
- add --cgroup-manager=cgroupfs option to podman
invocations in BATS: without this, podman-in-podman
fails with:
systemd cgroup flag passed, but systemd support for managing cgroups is not available
Also: gpg --pinentry-mode option is not available on all
our test platforms. Check for it before using.
Signed-off-by: Ed Santiago <santiago@redhat.com>
- Got TLS registry working, and test enabled. The trick was to
copy the .crt file to a separate directory *without* the .key
- auth test - set up a private XDG_RUNTIME_DIR, in case tests
are being run by a real user.
- signing test - remove FIXME comments; questions answered.
- helpers.bash - document start_registries(); save a .crt file,
not .cert; and remove unused stop_registries() - it's too hard
to do right, and very easy for individual tests to 'podman rm -f'
- run-tests - remove SKOPEO_BINARY definition, it's inconsistent
with the one in helpers.bash
Signed-off-by: Ed Santiago <santiago@redhat.com>
We need to verfy that the user entered a valid transport before attempting
to see if the transport exists, otherwise skopeo segfaults.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
This change fixes skopeo usage in restricted environment such as
bubblewrap where it doesn't need extra capabilities or user namespace
to perform its action.
Close#649
Signed-off-by: Tristan Cacqueray <tdecacqu@redhat.com>
Add a --config option to "skopeo inspect" to dump an image's
configuration blob in the OCI format, or the original format
if --config and --raw are specified.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Follow PR #433Close#421
Currently skopeo inspect allows to:
Use the default credentials in $HOME/.docker.config
Explicitly define credentials via de --creds flag
This implements a --no-creds flag which will query docker registries anonymously.
Signed-off-by: Qi Wang <qiwan@redhat.com>
overlay: propagate errors from mountProgram
utils: root in a userns uses global conf file
Fix handling of additional stores
Correctly check permissions on rootless directory
Fix possible integer overflow on 32bit builds
Evaluate device path for lvm
lockfile test: make concurrent RW test determinisitc
lockfile test: make concurrent read tests deterministic
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
This commit contains the necessary split-up between buildah/pkg and
buildah/util to avoid dependency breaks.
Signed-off-by: Sascha Grunert <sgrunert@suse.com>
This commit simply bumps containers/storage to the latest version to
unblock the containers/image integration test runs.
Signed-off-by: Sascha Grunert <sgrunert@suse.com>
Currently we are only installing the skopeo.1 man page. This
change will generate and install all man pages.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Since both of tabs and spaces in indentation were used and
there were tabs expected 4 spaces width and 8 spaces width,
only spaces use in indentation.
Signed-off-by: ERAMOTO Masaya <eramoto.masaya@jp.fujitsu.com>
After a global option was specified, a following string for global
options, commands, and command options was not complemented.
Signed-off-by: ERAMOTO Masaya <eramoto.masaya@jp.fujitsu.com>
Add checks to Tarvis to make sure that the vendor.conf is in sync with
the code and the dependencies in ./vendor. Do this by first running
`make vendor` followed by running `./hack/tree_status.sh` to check if
any file in the tree has been changed.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Most of the dependencies have been copied from libpod's vendor.conf
where such a cleanup has been executed recently.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
This script is meant to be used in CI after a `make vendor` run. It's
sole purpose is to execute a `git status --porcelain` and fail with the
list of files reported by it.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
containers/image moved to a new progress-bar library to fix various
issues related to overlapping bars and redundant entries.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Create a different man page for each of the subcommands.
Also replace some krufty references to kpod with podman
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Since the string of options variable as pattern in the case statement has
not been delimited and it does not match the value of prev variable,
bash completions tries to complement any option even when a specified
option requires a argument.
This fix stops complementing options when a option requires a argument.
Signed-off-by: ERAMOTO Masaya <eramoto.masaya@jp.fujitsu.com>
Move documentation about dependencies management from README.md to
CONTRIBUTING.md.
Closes#583
Signed-off-by: Silvano Cirujano Cuesta <silvano.cirujano-cuesta@siemens.com>
When copying images and the output is not a tty (e.g., when piping to a
file) print single lines instead of using progress bars. This avoids
long and hard to parse output.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Vendor the latest containers/image 50e5e55e46a391df8fce1291b2337f1af879b822
to enable parallel copying of layers.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
some tests I've done to try out the difference in performance:
I am using a directory repository so to not depend on the network.
User time (seconds): 39.40
System time (seconds): 6.83
Percent of CPU this job got: 121%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:38.07
User time (seconds): 8.32
System time (seconds): 1.62
Percent of CPU this job got: 128%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:07.72
User time (seconds): 42.68
System time (seconds): 6.64
Percent of CPU this job got: 162%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:30.44
User time (seconds): 8.94
System time (seconds): 1.51
Percent of CPU this job got: 178%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:05.85
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
That in turn makes sure that the cli.String() etc. flag access functions
are not used, and all flag handling is done using the *Options structures
and the Destination: members of cli.Flag.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
We no longer need it for handling flags.
Also, require the caller to explicitly pass an image name to parseImage
instead of, horribly nontransparently, using the first CLI option.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
It was not really any clearer when broken out. We already have
a pair of trivial src/dest API calls before this, so adding
a similar src/dest call for SystemContext follows the pattern.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
We no longer need the *cli.Context parameter, and at that point
it looks much cleaner to make this a method (already individually;
it will be even cleaner after a similar imageDestOptions conversion).
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
contextFromImageOptions is finally not using any string-based lookup
in cli.Context, so we don't need to record this value any more.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This introduces YET ANOTHER *Options structure, only to share this
option between copy source and destination. (We do need to do this,
because the libraries, rightly, refuse to work with source and
destination declaring its own versino of the --authfile flag.)
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This is an extension of imageOptions that carries destination-specific
flags.
This will allow us to handle --dest-* flags without also exposing
pointless --src-* flags.
(This is, also, where the type-safety somewhat breaks down;
after all the work to make the data flow and availability explicit,
everything ends up in an types.SystemContext, and it's easy enough
to use a destination-specific one for sources. OTOH, this is
not making the situation worse in any sense.)
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This is one of the ugliest parts; we need an extra parameter to support
the irregular screds/dcreds aliases.
This was previously unsupported by (skopeo layers).
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
We don't want to worry about mismatch of the flagPrefix value
between imageFlags() and contextFromImageOptions(). For now,
record it in imageOptions; eventually we will stop using it in
contextFromImageOptions and remove it again.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This is similar to the previous *Options structures, but this one
will support differing sets of options, in particular for the
copy source/destination.
The way the return values of imageFlags() are integrated into
creation of a cli.Command forces fakeContext() in tests to do
very ugly filtering to have a working *imageOptions available
without having a copyCmd() cooperate to give it to us. Rather
than extend copyCmd(), we do the filtering, because the reliance
on copyCmd() will go away after all flags are migrated, and so
will the filtering and fakeContext() complexity.
Finally, rename contextFromGlobalOptions to not lie about only
caring about global options.
This only introduces the infrastructure, all flags continue
to be handled in the old way.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
contextFromGlobalOptions now uses globalOptions instead
of cli.Context.Global* . That required passing globalOptions
through a few more functions.
Now, "all" that is left are all the non-global options
handled in contextFromGlobalOptions.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Replace commandTimeoutContextFromGlobalOptions with
globalOptions.commandTimeoutContext. This requires passing
globalOptions to more per-command *Options state.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This works just like the command-specific options. Handles only
the single flag for now, others will be added as the infrastructure
is built.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This works just like the command-specific options. Also
moves the "Before:" handler into a separate method.
Does not change behavior.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Use Destionation: &opts.flag in the flag definition
instead of c.String("flag-name") and the like in the hadler and
matching only by strings.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This is a big diff, but it really only replaces a few global variables
with functions returning a structure.
The ultimate goal of this patch set is to replace option handling using
> cli.StringFlag{Name:"foo", ...}
> ...
> func somethingHandler(c *cli.Context) error {
> c.String("foo")
> }
where the declaration and usage are connected only using a string constant,
and it's difficult to notice that one or the other is missing or that the
types don't match, by
> struct somethingOptions {
> foo string
> }
> ...
> cli.StringFlag{Name:"foo", Destination:&foo}
> ...
> func (opts *somethingOptions) run(c *cli.Context) error {
> opts.foo
> }
As a first step, this commit ONLY introduces the *Options structures,
but for now empty; nothing changes in the existing implementations.
So, we go from
> func somethingHandler(c *cli.Context error {...}
>
> var somethingCmd = cli.Command {
> ...
> Action: somethingHandler
> }
to
> type somethingOptions struct{
> } // empty for now
>
> func somethingCmd() cli.Command {
> opts := somethingOptions{}
> return cli.Command {
> ... // unchanged
> Action: opts.run
> }
> }
>
> func (opts *somethingOptions) run(c *cli.context) error {...} // unchanged
Using the struct type has also made it possible to place the definition of
cli.Command in front of the actual command handler, so do that for better
readability.
In a few cases this also broke out an in-line lambda in the Action: field
into a separate opts.run method. Again, nothing else has changed.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
It's probably not strictly necessary, but let's work with the current
implementation before worrying about possible idiosyncracies.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Before we use "go get" in CI, run "go version" so that we can be sure of
which version of the toolchain we're using.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
github.com/containers/image/copy.Image() now returns the copied
manifest, so we at least need to ignore it.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Bump github.com/containers/image to version
5e5b67d6b1cf43cc349128ec3ed7d5283a6cc0d1, which modifies copy.Image() to
add the new image's manifest to the values that it returns.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
... which has, apparently, never worked, because the golang image
has neither the GOPATH nor the working directory the Makefile expects.
Rather than move all this configuration into the Makefile to be able
to work with the golang images, just always use the skopeobuildimage
path, and only override the tags, to minimize divergence.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
Instead, use DockerReference() to obtain the repository name (which
also makes it work for other transports that support Docker references),
and a check for docker.Transport + docker.GetRepositoryTags.
This will allow dropping docker.Image from containers/image, and maybe
even all of ImageReference.NewImage (forcing callers to think about
manifest lists, among other things).
Minor change to allow passing the env TESTFLAGS to make. That's pretty
convenient to filter what tests to run.
E.g. run integration tests containing the substring `Copy`:
make test-integration TESTFLAGS="-check.f Copy"
Signed-off-by: Marco Vedovati <mvedovati@suse.com>
Replace the occurrences of `github.com/projectatomic` with
`github.com/containers` to ensure clean clones of the project are
building, travis badges on the README work as expected and other minor
things.
Signed-off-by: Flavio Castelli <fcastelli@suse.com>
These targets produce a pure-Go binary, without the following features:
* ostree
* devicemapper
* btrfs
* gpgme
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
https://github.com/projectatomic/skopeo/pull/519 made (skopeo copy)
suceed and print nothing to stderr; that could lead to hard-to-diagnose
failures in rare corner cases, e.g. shell scripts which do
(skopeo copy $src $dst) (as opposed to the correct
(skopeo copy "$src" "$dst") ) if $src and $dst are empty due to
a previous failure.
Needed to pick up this change:
ostree: use the same thread for ostree operations
Since https://github.com/ostreedev/ostree/pull/1555, locking is
enabled by default in OSTree. Unfortunately it uses thread-private
data and it breaks the Golang bindings. Force the same thread for the
write operations to the OSTree repository.
Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
skopeo is failing to build now on 32 bit systems. go-selinux update
should fix this. Also container/storage has had some cleanup fixes
to devicemapper support.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
The goal is to include the c/image documentation in a skopeo release,
so that RPMs and other distribution mechanisms can ship the c/image
documentation without having to create a separate package for c/image
(which would not otherwise be needed because it is vendored in users).
So, unify the updates of the "vendor" subdirectory as (make vendor),
and document it in README.md. Also drop hack/vendor.sh, we neither
use nor document it, so updating it as well seems pointless.
containers/storage and storage.conf now support flags to allow users
to setup containers/storage to run on devicemapper.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
docker-archive and oci-archive now allow the image reference
for the destination to be empty.
Update tests for this new change.
Signed-off-by: umohnani8 <umohnani@redhat.com>
- _Start_ with installing distribution packages, instead of
mentioning it after the user has already built everything from source.
- Note that both the binary and documentation needs to be built
for (make install) to work.
Add multitag support when generating docker-archive tarballs via the
newly added '--aditional-tag' option, which can be specified multiple
times to add more than one tag. All specified tags will be added to the
RepoTags field in the docker-archive's manifest.json file.
This change requires to vendor the latest containers/image with
commit a1a9391830fd08637edbe45133fd0a8a2682ae75.
Signed-off-by: Valentin Rothberg <vrothberg@suse.com>
Apparently, it was never documented to use (go vet $somefile.go)
(but (go tool vet $somefile.go) was).
go 1.10 seems to do more checks within packages, and $somefile.go
is interpreted as a package with only that file (even if other files
from that package are in the same directory), leading to spurious
"undefined: $symbol" errors.
So, just run (go vet) on ./... (explicitly excluding skopeo/vendor for the
benefit of Go 1.8). We only have three subpackages, so the savings, if any,
from running (go vet) only on the modified subpackages would be small.
More importantly, on a toolchain update, ./... allows us to see the newly
detected issues all at once, instead of randomly waiting for a commit that
changes one of the affected files for the failure to show up.
The hack/common.sh script contains
local go_version
go_version=($(go version))
if [[ "${go_version[2]}" < "go1.5" ]]; then
# fail
fi
which does a lexicographic string comparison, and fails with 1.10.
Just drop it, the fedora:latest image is not likely to revert to 1.5.
containers/image returns a more detailed error message for oci and
oci-archive transports when the syntax given by the user is incorrect
Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
In addition to the minimum necessary to update the API, also rename some
parameters/variables for consistency:
c *cli.Context
ctx context.Context
sys *types.SystemContext
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
This PR adds CLI support for overriding the default docker daemon host when using the
`docker-daemon` transport.
Fixes#244
Signed-off-by: Justin Lewis Salmon <justin.lewis.salmon@gmail.com>
These files are used by deb and rpm packages, so I'd rather have them
upstream than maintain in 2 separate places.
Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
The dir transport has been changed to save the blobs without the .tar extension
Fixes the skopeo tests failing due to this change
Signed-off-by: umohnani8 <umohnani@redhat.com>
Anyone running (vndr) currently ends up with failing tests in OCI schema
validation because gojsonschema has fixed its "$ref" interpretation, exposing
inconsistent URI usage inside image-spec/schema.
So, this runs (vndr), and uses mtrmac/image-spec:id-based-loader
( https://github.com/opencontainers/image-spec/pull/739 ) to make the tests pass
again. As soon as that PR is merged we should revert to using the upstream
image-spec repo again.
Re-vendor containers/storage to current revision
0d32dfce498e06c132c60dac945081bf44c22464, and containers/image to
current revision c8bcd6aa11c62637c5a7da1420f43dd6a15f0e8d.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
skopeo copy, delete, and inspect can now use credentials stored in the auth file
by the kpod login command
e.g kpod login docker.io -> skopeo copy dir:mydir docker://username/image
Signed-off-by: Urvashi Mohnani <umohnani@redhat.com>
The skopeo project, as part of Podman Container Tools, follows the [CNCF Code of Conduct](https://github.com/cncf/foundation/blob/main/code-of-conduct.md).
to see if someone else has already reported it. If so, feel free to add
your scenario, or additional information, to the discussion. Or simply
"subscribe" to it to be notified when it is updated.
@@ -115,16 +115,51 @@ Use your real name (sorry, no pseudonyms or anonymous contributions.)
If you set your `user.name` and `user.email` git configs, you can sign your
commit automatically with `git commit -s`.
### Dependencies management
Dependencies are managed via [standard go modules](https://golang.org/ref/mod).
In order to add a new dependency to this project:
- use `go get -d path/to/dep@version` to add a new line to `go.mod`
- run `make vendor`
In order to update an existing dependency:
- use `go get -d -u path/to/dep@version` to update the relevant dependency line in `go.mod`
- run `make vendor`
When new PRs for [containers/container-libs](https://github.com/containers/container-libs) break `skopeo` (i.e. `containers/container-libs` tests fail in `make test-skopeo`):
- create out a new branch in your `skopeo` checkout and switch to it
- find out the version of `containers/container-libs` you want to use and note its commit ID. You might also want to use a fork of `containers/container-libs`, in that case note its repo
- use `go get -d github.com/$REPO/container-libs/image/v5@$COMMIT_ID` to download the right version. The command will fetch the dependency and then fail because of a conflict in `go.mod`, this is expected. Note the pseudo-version (eg. `v5.13.1-0.20210707123201-50afbf0a326`)
- use `go mod edit -replace=go.podman.io/image/v5=github.com/$REPO/container-libs/image/v5@$PSEUDO_VERSION` to add a replacement line to `go.mod` (e.g. `replace go.podman.io/image/v5 => github.com/moio/container-libs/image/v5 v5.13.1-0.20210707123201-50afbf0a3262`)
- run `make vendor`
- make any other necessary changes in the skopeo repo (e.g. add other dependencies now required by `containers/container-libs`, or update skopeo for changed `containers/container-libs` API)
- optionally add new integration tests to the skopeo repo
- submit the resulting branch as a skopeo PR, marked “DO NOT MERGE”
- iterate until tests pass and the PR is reviewed
- then the original `containers/container-libs` PR can be merged, disregarding its `make test-skopeo` failure
- as soon as possible after that, in the skopeo PR, use `go mod edit -dropreplace=go.podman.io/image/v5` to remove the `replace` line in `go.mod`
- run `make vendor`
- update the skopeo PR with the result, drop the “DO NOT MERGE” marking
- after tests complete successfully again, merge the skopeo PR
## Communications
For general questions, or discussions, please use the
IRC group on `irc.freenode.net` called `container-projects`
that has been setup.
For general questions, or discussions, please use the
[#podman](https://app.slack.com/client/T08PSQ7BQ/C08MXJLCFCN) channel on the [CNCF
Slack](https://cloud-native.slack.com).
For development related discussions, please use the
[#podman-dev](https://app.slack.com/client/T08PSQ7BQ/C08NTKCDC1W) channel on the CNCF
Slack.
For discussions around issues/bugs and features, you can use the github
The Skopeo project, as part of Podman Container Tools, follows the [Podman Project Governance](https://github.com/containers/podman/blob/main/GOVERNANCE.md)
except sections found in this document, which override those found in Podman's Governance.
---
# Maintainers File
The definitive source of truth for maintainers of this repository is the local [MAINTAINERS.md](./MAINTAINERS.md) file. The [MAINTAINERS.md](https://github.com/containers/podman/blob/main/MAINTAINERS.md) file in the main Podman repository is used for project-spanning roles, including Core Maintainer and Community Manager. Some repositories in the project will also have a local [OWNERS](./OWNERS) file, which the CI system uses to map users to roles. Any changes to the [OWNERS](./OWNERS) file must make a corresponding change to the [MAINTAINERS.md](./MAINTAINERS.md) file to ensure that the file remains up to date. Most changes to [MAINTAINERS.md](./MAINTAINERS.md) will require a change to the repository’s [OWNERS](.OWNERS) file (e.g., adding a Reviewer), but some will not (e.g., promoting a Maintainer to a Core Maintainer, which comes with no additional CI-related privileges).
Any Core Maintainers listed in Podman’s [MAINTAINERS.md](https://github.com/containers/podman/blob/main/MAINTAINERS.md) file should also be added to the list of “approvers” in the local [OWNERS](./OWNERS) file and as a Core Maintainer in the list of “Maintainers” in the local [MAINTAINERS.md](./MAINTAINERS.md) file.
| Neil Smith | [actionmancan](https://github.com/actionmancan) | Community Manager | [Red Hat](https://github.com/RedHatOfficial) |
| Tom Sweeney | [TomSweeneyRedHat](https://github.com/TomSweeneyRedHat/) | Maintainer and Community Manager | [Red Hat](https://github.com/RedHatOfficial) |
The structure of this document was based off of the equivalent one in the [CRI-O Project](https://github.com/cri-o/cri-o/blob/main/MAINTAINERS.md).
## Note
If there is a discrepancy between the [MAINTAINERS.md](https://github.com/containers/podman/blob/main/MAINTAINERS.md) file in the main Podman repository and this file regarding Core Maintainers or Community Managers, the file in the Podman Repository is considered the source of truth.
[](https://www.bestpractices.dev/projects/10516)
`skopeo` is a command line utility that performs various operations on container images and image repositories. Skopeo works with API V2 registries such as Docker registries, the Atomic registry, private registries, local directories and local OCI-layout directories. Skopeo does not require a daemon to be running to perform these operations which consist of:
`skopeo` is a command line utility that performs various operations on container images and image repositories.
`skopeo` does not require the user to be running as root to do most of its operations.
`skopeo` does not require a daemon to be running to perform its operations.
`skopeo` can work with [OCI images](https://github.com/opencontainers/image-spec) as well as the original Docker v2 images.
Skopeo works with API V2 container image registries such as [docker.io](https://docker.io) and [quay.io](https://quay.io) registries, private registries, local directories and local OCI-layout directories. Skopeo can perform operations which consist of:
* Inspecting an image showing its properties including its layers.
* Copying an image from and to various storage mechanisms.
For example you can copy images from one registry to another, without requiring privilege.
* Inspecting a remote image showing its properties including its layers, without requiring you to pull the image to the host.
* Deleting an image from an image repository.
* Syncing an external image repository to an internal registry for air-gapped deployments.
* When required by the repository, skopeo can pass the appropriate credentials and certificates for authentication.
Skopeo operates on the following image and repository types:
* containers-storage:docker-reference
An image located in a local containers/storage image store. Location and image store specified in /etc/containers/storage.conf
An image located in a local containers/storage image store. Both the location and image store are specified in /etc/containers/storage.conf. (This is the backend for [Podman](https://podman.io), [CRI-O](https://cri-o.io), [Buildah](https://buildah.io) and friends)
* dir:path
An existing local directory path storing the manifest, layer tarballs and signatures as individual files. This is a non-standardized format, primarily useful for debugging or noninvasive container inspection.
* docker://docker-reference
An image in a registry implementing the "Docker Registry HTTP API V2". By default, uses the authorization state in $HOME/.docker/config.json, which is set e.g. using (docker login).
An image in a registry implementing the "Docker Registry HTTP API V2". By default, uses the authorization state in `$XDG_RUNTIME_DIR/containers/auth.json`, which is set using `skopeo login`.
* docker-archive:path[:docker-reference]
An image is stored in the`docker save`formated file. docker-reference is only used when creating such a file, and it must not contain a digest.
An image is stored in a`docker save`-formatted file. docker-reference is only used when creating such a file, and it must not contain a digest.
* docker-daemon:docker-reference
An image docker-reference stored in the docker daemon internal storage. docker-reference must contain either a tag or a digest. Alternatively, when reading images, the format can also be docker-daemon:algo:digest (an image ID).
An image tag in a directory compliant with "Open Container Image Layout Specification" at path.
* ostree:image[@/absolute/repo/path]
An image in local OSTree repository. /absolute/repo/path defaults to /ostree/repo.
Inspecting a repository
[Obtaining skopeo](./install.md)
-
`skopeo` is able to _inspect_ a repository on a Docker registry and fetch images layers.
For a detailed description how to install or build skopeo, see
[install.md](./install.md).
Skopeo is also available as a Container Image on [quay.io](https://quay.io/skopeo/stable). For more information, see the [Skopeo Image](https://github.com/containers/image_build/blob/main/skopeo/README.md) page.
## Inspecting a repository
`skopeo` is able to _inspect_ a repository on a container registry and fetch images layers.
The _inspect_ command fetches the repository's manifest and it is able to show you a `docker inspect`-like
json output about a whole repository or a tag. This tool, in contrast to `docker inspect`, helps you gather useful information about
a repository or a tag before pulling it (using disk space). The inspect command can show you which tags are available for the given
repository, the labels the image has, the creation date and operating system of the image and more.
`skopeo` can copy container images between various storage mechanisms, including:
* Container registries
- The Quay, Docker Hub, OpenShift, GCR, Artifactory ...
* Container Storage backends
- [github.com/containers/storage](https://github.com/containers/storage) (Backend for [Podman](https://podman.io), [CRI-O](https://cri-o.io), [Buildah](https://buildah.io) and friends)
When interacting with private registries, `skopeo` first looks for `--creds` (for `skopeo inspect|delete`) or `--src-creds|--dest-creds` (for `skopeo copy`) flags. If those aren't provided, it looks for the Docker's cli config file (usually located at `$HOME/.docker/config.json`) to get the credentials needed to authenticate. The ultimate fallback, as Docker does, is to provide an empty authentication when interacting with those registries.
## Syncing registries
```console
$ skopeo sync --src docker --dest dir registry.example.com/busybox /media/usb
```
Examples:
```sh
$ cat /home/runcom/.docker/config.json
{
"auths": {
"myregistrydomain.com:5000": {
"auth": "dGVzdHVzZXI6dGVzdHBhc3N3b3Jk",
"email": "stuf@ex.cm"
}
}
}
## Authenticating to a registry
# we can see I'm already authenticated via docker login so everything will be fine
#### Private registries with authentication
skopeo uses credentials from the --creds (for skopeo inspect|delete) or --src-creds|--dest-creds (for skopeo copy) flags, if set; otherwise it uses configuration set by skopeo login, podman login, buildah login, or docker login.
```console
$ skopeo login --username USER myregistrydomain.com:5000
$ cd$GOPATH/src/github.com/projectatomic/skopeo && make binary-local
```
### Building in a container
Building in a container is simpler, but more restrictive:
- It requires the `docker` command and the ability to run Linux containers
- The created executable is a Linux executable, and depends on dynamic libraries which may only be available only in a container of a similar Linux distribution.
```sh
$ make binary # Or (make all) to also build documentation, see below.
```
### Building documentation
To build the manual you will need go-md2man.
```sh
Debian$ sudo apt-get install go-md2man
Fedora$ sudo dnf install go-md2man
```
Then
```sh
$ make docs
```
Installing
-
If you built from source:
```sh
$ sudo make install
```
`skopeo` is also available from Fedora 23 (and later):
```sh
$ sudo dnf install skopeo
```
TODO
-
- list all images on registry?
- registry v2 search?
- show repo tags via flag or when reference isn't tagged or digested
- support rkt/appc image spec
NOT TODO
-
- provide a _format_ flag - just use the awesome [jq](https://stedolan.github.io/jq/)
CONTRIBUTING
Contributing
-
### Dependencies management
Please read the [contribution guide](CONTRIBUTING.md) if you want to collaborate in the project.
`skopeo` uses [`vndr`](https://github.com/LK4D4/vndr) for dependencies management.
In order to add a new dependency to this project:
- add a new line to `vendor.conf` according to `vndr` rules (e.g. `github.com/pkg/errors master`)
- run `vndr github.com/pkg/errors`
In order to update an existing dependency:
- update the relevant dependency line in `vendor.conf`
- run `vndr github.com/pkg/errors`
When new PRs for [containers/image](https://github.com/containers/image) break `skopeo` (i.e. `containers/image` tests fail in `make test-skopeo`):
- create out a new branch in your `skopeo` checkout and switch to it
- update `vendor.conf`. Find out the `containers/image` dependency; update it to vendor from your own branch and your own repository fork (e.g. `github.com/containers/image my-branch https://github.com/runcom/image`)
- run `vndr github.com/containers/image`
- make any other necessary changes in the skopeo repo (e.g. add other dependencies now requied by `containers/image`, or update skopeo for changed `containers/image` API)
- optionally add new integration tests to the skopeo repo
- submit the resulting branch as a skopeo PR, marked “DO NOT MERGE”
- iterate until tests pass and the PR is reviewed
- then the original `containers/image` PR can be merged, disregarding its `make test-skopeo` failure
- as soon as possible after that, in the skopeo PR, restore the `containers/image` line in `vendor.conf` to use `containers/image:master`
- run `vndr github.com/containers/image`
- update the skopeo PR with the result, drop the “DO NOT MERGE” marking
- after tests complete succcesfully again, merge the skopeo PR
Skopeo intends to mostly continue to be a very thin CLI wrapper over the [https://github.com/containers/image](containers/image) library, with most features being added there, not to this repo. A typical new Skopeo feature would only add a CLI for a recent containers/image feature.
## Future feature focus (most of the work must be done in the containers/image library)
* OCI artifact support.
* Integration of composefs.
* Partial pull support (zstd:chunked).
* Performance and stability improvements.
* Reductions to the size of the Skopeo binary.
*`skopeo sync` exists, and bugs in it should be fixed, but we don’t have much of an ambition to compete with much larger projects like [https://github.com/openshift/oc-mirror](oc-mirror).
## Security and Disclosure Information Policy for the skopeo Project
The skopeo Project follows the [Security and Disclosure Information Policy](https://github.com/containers/common/blob/main/SECURITY.md) for the Containers Projects.
// Both directory and oci should have ShellCompDirectiveFilterDirs to complete only directories, but it doesn't currently work in bash: https://github.com/spf13/cobra/issues/2242
flags.BoolVarP(&opts.quiet,"quiet","q",false,"Suppress output information when copying images")
flags.BoolVarP(&opts.all,"all","a",false,"Copy all images if SOURCE-IMAGE is a list")
flags.Var(commonFlag.NewOptionalStringValue(&opts.multiArch),"multi-arch",`How to handle multi-architecture images (system, all, or index-only)`)
flags.StringVar(&opts.signIdentity,"sign-identity","","Identity of signed image, must be a fully specified docker reference. Defaults to the target docker reference.")
flags.StringVar(&opts.digestFile,"digestfile","","Write the digest of the pushed image to the specified file")
flags.StringSliceVar(&opts.encryptionKeys,"encryption-key",[]string{},"*Experimental* key with the encryption protocol to use needed to encrypt the image (e.g. jwe:/path/to/key.pem)")
flags.IntSliceVar(&opts.encryptLayer,"encrypt-layer",[]int{},"*Experimental* the 0-indexed layer indices, with support for negative indexing (e.g. 0 is the first layer, -1 is the last layer)")
flags.StringSliceVar(&opts.decryptionKeys,"decryption-key",[]string{},"*Experimental* key needed to decrypt the image")
flags.UintVar(&opts.imageParallelCopies,"image-parallel-copies",0,"Maximum number of image layers to be copied (pulled/pushed) simultaneously. Not setting this field will fall back to containers/image defaults.")
returncmd
}
// parseMultiArch parses the list processing selection
// It returns the copy.ImageListSelection to use with image.Copy option
flags.StringVarP(&opts.format,"format","f","","Format the output to a Go template")
flags.BoolVarP(&opts.doNotListTags,"no-tags","n",false,"Do not list the available tags from the repository in the output")
flags.Var(newAlgorithmValue(&opts.manifestDigest),"manifest-digest","Algorithm to use for computing manifest digest (sha256, sha512); defaults to algorithm used in config digest")
returncmd
}
See skopeo(1) section "IMAGE NAMES" for the expected format
`,strings.Join(transports.ListNames(),", ")),
ArgsUsage:"IMAGE-NAME",
Flags:[]cli.Flag{
cli.StringFlag{
Name:"cert-dir",
Value:"",
Usage:"use certificates at `PATH` (*.crt, *.cert, *.key) to connect to the registry",
},
cli.BoolTFlag{
Name:"tls-verify",
Usage:"require HTTPS and verify certificates when talking to container registries (defaults to true)",
},
cli.BoolFlag{
Name:"raw",
Usage:"output raw manifest",
},
cli.StringFlag{
Name:"creds",
Value:"",
Usage:"Use `USERNAME[:PASSWORD]` for accessing the registry",
ifassert.NoError(t,err,"Could not parse, got error on %v",test[0])&&assert.NoError(t,err2,"Could not parse with regular parser, got error on %v",test[0]){
returnfmt.Errorf("Unrecognized command `%[1]s %[2]s`\nTry '%[1]s --help' for more information",cmd.CommandPath(),args[0])
}
returnfmt.Errorf("Unrecognized command `%[1]s %[2]s`\n\nDid you mean this?\n\t%[3]s\n\nTry '%[1]s --help' for more information",cmd.CommandPath(),args[0],strings.Join(suggestions,"\n\t"))
}
returnfmt.Errorf("Missing command '%[1]s COMMAND'\nTry '%[1]s --help' for more information",cmd.CommandPath())
}
// createApp returns a cobra.Command, and the underlying globalOptions object, to be run or tested.
funccreateApp()(*cobra.Command,*globalOptions){
opts:=globalOptions{}
rootCommand:=&cobra.Command{
Use:"skopeo",
Long:"Various operations with container images and container image registries",
RunE:requireSubcommand,
PersistentPreRunE:opts.before,
SilenceUsage:true,
SilenceErrors:true,
// Hide the completion command which is provided by cobra
rootCommand.PersistentFlags().StringVar(&opts.policyPath,"policy","","Path to a trust policy file")
rootCommand.PersistentFlags().BoolVar(&opts.insecurePolicy,"insecure-policy",false,"run the tool without any policy check")
rootCommand.PersistentFlags().BoolVar(&opts.requireSigned,"require-signed",false,"require any pulled image to be signed")
rootCommand.PersistentFlags().StringVar(&opts.registriesDirPath,"registries.d","","use registry configuration files in `DIR` (e.g. for container signature storage)")
rootCommand.PersistentFlags().StringVar(&opts.overrideArch,"override-arch","","use `ARCH` instead of the architecture of the machine for choosing images")
rootCommand.PersistentFlags().StringVar(&opts.overrideOS,"override-os","","use `OS` instead of the running OS for choosing images")
rootCommand.PersistentFlags().StringVar(&opts.overrideVariant,"override-variant","","use `VARIANT` instead of the running architecture variant for choosing images")
rootCommand.PersistentFlags().DurationVar(&opts.commandTimeout,"command-timeout",0,"timeout for the command execution")
rootCommand.PersistentFlags().StringVar(&opts.registriesConfPath,"registries-conf","","path to the registries.conf file")
logrus.Fatal("unable to mark registries-conf flag as hidden")
}
app.Before=func(c*cli.Context)error{
ifc.GlobalBool("debug"){
logrus.SetLevel(logrus.DebugLevel)
rootCommand.PersistentFlags().StringVar(&opts.tmpDir,"tmpdir","","directory used to store temporary files")
rootCommand.PersistentFlags().StringVar(&opts.userAgentPrefix,"user-agent-prefix","","prefix to add to the user agent string")
flag:=commonFlag.OptionalBoolFlag(rootCommand.Flags(),&opts.tlsVerify,"tls-verify","Require HTTPS and verify certificates when accessing the registry")
flag.Hidden=true
rootCommand.AddCommand(
copyCmd(&opts),
deleteCmd(&opts),
generateSigstoreKeyCmd(),
inspectCmd(&opts),
layersCmd(&opts),
loginCmd(&opts),
logoutCmd(&opts),
manifestDigestCmd(),
proxyCmd(&opts),
syncCmd(&opts),
standaloneSignCmd(),
standaloneVerifyCmd(),
tagsCmd(&opts),
untrustedSignatureDumpCmd(),
)
returnrootCommand,&opts
}
// gitCommit returns the git commit for this codebase, if we are built from a git repo; "" otherwise.
funcgitCommit()string{
bi,ok:=debug.ReadBuildInfo()
if!ok{
logrus.Fatal("runtime.ReadBuildInfo failed")
}
for_,e:=rangebi.Settings{
ife.Key=="vcs.revision"{
returne.Value
}
ifc.GlobalIsSet("tls-verify"){
logrus.Warn("'--tls-verify' is deprecated, please set this on the specific subcommand")
}
returnnil
}
app.Commands=[]cli.Command{
copyCmd,
inspectCmd,
layersCmd,
deleteCmd,
manifestDigestCmd,
standaloneSignCmd,
standaloneVerifyCmd,
untrustedSignatureDumpCmd,
return""
}
// before is run by the cli package for any command, before running the command-specific handler.
scopedbool// When true, namespace copied images at destination using the source repository name
allbool// Copy all of the images if an image in the source is a list
dryRunbool// Don't actually copy anything, just output what it would have done
keepGoingbool// Whether or not to abort the sync if there are any errors during syncing the images
appendSuffixstring// Suffix to append to destination image tag
}
// repoDescriptor contains information of a single repository used as a sync source.
typerepoDescriptorstruct{
DirBasePathstring// base path when source is 'dir'
ImageRefs[]types.ImageReference// List of tagged image found for the repository
Context*types.SystemContext// SystemContext for the sync command
}
// tlsVerifyConfig is an implementation of the Unmarshaler interface, used to
// customize the unmarshaling behaviour of the tls-verify YAML key.
typetlsVerifyConfigstruct{
skiptypes.OptionalBool// skip TLS verification check (false by default)
}
// registrySyncConfig contains information about a single registry, read from
// the source YAML file
typeregistrySyncConfigstruct{
Imagesmap[string][]string// Images map images name to slices with the images' references (tags, digests)
ImagesByTagRegexmap[string]string`yaml:"images-by-tag-regex"`// Images map images name to regular expression with the images' tags
ImagesBySemvermap[string]string`yaml:"images-by-semver"`// ImagesBySemver maps a repository to a semver constraint (e.g. '>=3.14') to match images' tags to
Credentialstypes.DockerAuthConfig// Username and password used to authenticate with the registry
TLSVerifytlsVerifyConfig`yaml:"tls-verify"`// TLS verification mode (enabled by default)
CertDirstring`yaml:"cert-dir"`// Path to the TLS certificates of the registry
}
// sourceConfig contains all registries information read from the source YAML file
flags.StringVarP(&opts.source,"src","s","","SOURCE transport type")
flags.StringVarP(&opts.destination,"dest","d","","DESTINATION transport type")
flags.BoolVar(&opts.scoped,"scoped",false,"Images at DESTINATION are prefix using the full source image path as scope")
flags.StringVar(&opts.appendSuffix,"append-suffix","","String to append to DESTINATION tags")
flags.StringVar(&opts.digestFile,"digestfile","","Write the digests and Image References of the resulting images to the specified file, separated by newlines")
flags.BoolVarP(&opts.all,"all","a",false,"Copy all images if SOURCE-IMAGE is a list")
flags.BoolVar(&opts.dryRun,"dry-run",false,"Run without actually copying data")
flags.BoolVarP(&opts.keepGoing,"keep-going","",false,"Do not abort the sync if any image copy fails")
returncmd
}
// UnmarshalYAML is the implementation of the Unmarshaler interface method
// for the tlsVerifyConfig type.
// It unmarshals the 'tls-verify' YAML key so that, when they key is not
fs.StringVar(&opts.authFilePath,"authfile",os.Getenv("REGISTRY_AUTH_FILE"),"path of the registry credentials file. Default is ${XDG_RUNTIME_DIR}/containers/auth.json")
returnfs,&opts
}
// dockerImageOptions collects CLI flags specific to the "docker" transport, which are
// the same across subcommands, but may be different for each image
// (e.g. may differ between the source and destination of a copy)
typedockerImageOptionsstruct{
global*globalOptions// May be shared across several imageOptions instances.
shared*sharedImageOptions// May be shared across several imageOptions instances.
deprecatedTLSVerify*deprecatedTLSVerifyOption// May be shared across several imageOptions instances, or nil.
authFilePathcommonFlag.OptionalString// Path to a */containers/auth.json (prefixed version to override shared image option).
credsOptioncommonFlag.OptionalString// username[:password] for accessing a registry
userNamecommonFlag.OptionalString// username for accessing a registry
passwordcommonFlag.OptionalString// password for accessing a registry
registryTokencommonFlag.OptionalString// token to be used directly as a Bearer token when accessing the registry
dockerCertPathstring// A directory using Docker-like *.{crt,cert,key} files for connecting to a registry or a daemon
tlsVerifycommonFlag.OptionalBool// Require HTTPS and verify certificates (for docker: and docker-daemon:)
noCredsbool// Access the registry anonymously
}
// imageOptions collects CLI flags which are the same across subcommands, but may be different for each image
// (e.g. may differ between the source and destination of a copy)
typeimageOptionsstruct{
dockerImageOptions
sharedBlobDirstring// A directory to use for OCI blobs, shared across repositories
dockerDaemonHoststring// docker-daemon: host to connect to
}
// dockerImageFlags prepares a collection of docker-transport specific CLI flags
// writing into imageOptions, and the managed imageOptions structure.
// the non-prefixed flag is handled by a shared flag.
fs.Var(commonFlag.NewOptionalStringValue(&flags.authFilePath),flagPrefix+"authfile","path of the registry credentials file. Default is ${XDG_RUNTIME_DIR}/containers/auth.json")
}
fs.Var(commonFlag.NewOptionalStringValue(&flags.credsOption),flagPrefix+"creds","Use `USERNAME[:PASSWORD]` for accessing the registry")
fs.Var(commonFlag.NewOptionalStringValue(&flags.userName),flagPrefix+"username","Username for accessing the registry")
fs.Var(commonFlag.NewOptionalStringValue(&flags.password),flagPrefix+"password","Password for accessing the registry")
ifcredsOptionAlias!=""{
// This is horribly ugly, but we need to support the old option forms of (skopeo copy) for compatibility.
// Don't add any more cases likethis.
f:=fs.VarPF(commonFlag.NewOptionalStringValue(&flags.credsOption),credsOptionAlias,"","Use `USERNAME[:PASSWORD]` for accessing the registry")
f.Hidden=true
}
fs.Var(commonFlag.NewOptionalStringValue(&flags.registryToken),flagPrefix+"registry-token","Provide a Bearer token for accessing the registry")
fs.StringVar(&flags.dockerCertPath,flagPrefix+"cert-dir","","use certificates at `PATH` (*.crt, *.cert, *.key) to connect to the registry or daemon")
commonFlag.OptionalBoolFlag(&fs,&flags.tlsVerify,flagPrefix+"tls-verify","require HTTPS and verify certificates when talking to the container registry or daemon")
fs.BoolVar(&flags.noCreds,flagPrefix+"no-creds",false,"Access the registry anonymously")
returnfs,&flags
}
// imageFlags prepares a collection of CLI flags writing into imageOptions, and the managed imageOptions structure.
fs.BoolVar(&opts.dirForceCompression,flagPrefix+"compress",false,"Compress tarball image layers when saving to directory using the 'dir' transport. (default is same compression type as source)")
fs.BoolVar(&opts.dirForceDecompression,flagPrefix+"decompress",false,"Decompress tarball image layers when saving to directory using the 'dir' transport. (default is same compression type as source)")
fs.BoolVar(&opts.ociAcceptUncompressedLayers,flagPrefix+"oci-accept-uncompressed-layers",false,"Allow uncompressed image layers when saving to an OCI image using the 'oci' transport. (default is to compress things that aren't compressed)")
fs.StringVar(&opts.compressionFormat,flagPrefix+"compress-format","","`FORMAT` to use for the compression")
fs.Var(commonFlag.NewOptionalIntValue(&opts.compressionLevel),flagPrefix+"compress-level","`LEVEL` to use for the compression")
fs.BoolVar(&opts.precomputeDigests,flagPrefix+"precompute-digests",false,"Precompute digests to prevent uploading layers already on the registry using the 'docker' transport.")
fs.BoolVar(&opts.forceCompressionFormat,flagPrefix+"force-compress-format",false,"Force exclusive use of the compression algorithm set in --dest-compress-format")
returnfs,&opts
}
// newSystemContext returns a *types.SystemContext corresponding to opts.
// It is guaranteed to return a fresh instance, so it is safe to make additional updates to it.
fs.BoolVar(&opts.removeSignatures,"remove-signatures",false,"Do not copy signatures from source")
fs.StringVar(&opts.signByFingerprint,"sign-by","","Sign the image using a GPG key with the specified `FINGERPRINT`")
fs.StringVar(&opts.signBySequoiaFingerprint,"sign-by-sq-fingerprint","","Sign the image using a Sequoia-PGP key with the specified `FINGERPRINT`")
fs.StringVar(&opts.signBySigstoreParamFile,"sign-by-sigstore","","Sign the image using a sigstore parameter file at `PATH`")
fs.StringVar(&opts.signBySigstorePrivateKey,"sign-by-sigstore-private-key","","Sign the image using a sigstore private key at `PATH`")
fs.StringVar(&opts.signPassphraseFile,"sign-passphrase-file","","Read a passphrase for signing an image from `PATH`")
fs.VarP(commonFlag.NewOptionalStringValue(&opts.format),"format","f",`MANIFEST TYPE (oci, v2s1, or v2s2) to use in the destination (default is manifest type of source, with fallbacks)`)
fs.BoolVar(&opts.preserveDigests,"preserve-digests",false,"Preserve digests of images and lists")
returnfs,&opts
}
// copyOptions interprets opts, returns a partially-filled *copy.Options,
// and a function that should be called to clean up.
// c/image/copy.Image does allow creating both simple signing and sigstore signatures simultaneously,
// with independent passphrases, but that would make the CLI probably too confusing.
// For now, use the passphrase with either, but only one of them.
ifopts.signPassphraseFile!=""{
count:=0
ifopts.signByFingerprint!=""{
count++
}
ifopts.signBySequoiaFingerprint!=""{
count++
}
ifopts.signBySigstorePrivateKey!=""{
count++
}
ifcount>1{
returnnil,nil,fmt.Errorf("Only one of --sign-by, --sign-by-sq-fingerprint and --sign-by-sigstore-private-key can be used with --sign-passphrase-file")
}
}
// Simple signing does not really allow empty but present passphrases — but for sigstore, cosign does support creating keys encrypted with an empty passphrase;
// so, at least for that case, we must track the distinction between an empty and a missing passphrase precisely.
}// opts.signByFingerprint triggers a GPG-agent passphrase prompt, possibly using a more secure channel, so we usually shouldn’t prompt ourselves if no passphrase was explicitly provided.
// With opts.signBySequoiaFingerprint, we don’t prompt for a passphrase (for now??): We don’t know whether the key requires a passphrase.
skopeo-experimental-image-proxy - API server for fetching container images (EXPERIMENTAL)
# SYNOPSIS
**skopeo experimental-image-proxy** [*options*]
# DESCRIPTION
**EXPERIMENTAL COMMAND**: This command is experimental, and its API is subject to change. It is currently hidden from the main help output and not supported on Windows.
`skopeo experimental-image-proxy` exposes core container image fetching APIs via custom JSON+fd-passing protocol. This provides a lightweight way to fetch container image content (manifests and blobs). This command is primarily intended for programs that want to operate on a storage type that skopeo doesn't natively handle. For example, the bootc project currently has a custom ostree-based container storage backend.
The client process that invokes `skopeo experimental-image-proxy` is responsible for creating a socket pair and passing one of the file descriptors to the proxy. By default, the proxy expects this file descriptor to be its standard input (fd 0), but a different fd can be specified using the **--sockfd** option.
**Protocol Overview**
The protocol requires a `socketpair(2)` of type `SOCK_SEQPACKET`, over which a single JSON message is sent per packet. Large data payloads, such as image manifests and blobs, are transferred over separate pipes (`pipe(2)`), with the read-ends of these pipes passed to the client via file descriptor (FD) passing.
*`success`: `true` if the call succeeded, `false` otherwise.
*`value`: The return value of the method, if any.
*`pipeid`: An integer identifying a pipe for data transfer. This ID is used with the `FinishPipe` method.
*`error_code`: A string indicating the type of error if `success` is `false` (e.g., "EPIPE", "retryable", "other"). (Introduced in protocol version 0.2.8)
*`error`: A string describing the error if `success` is `false`.
The current protocol version is `0.2.8`.
**Supported Protocol Methods**
The server supports the following methods:
***Initialize**: Initializes the proxy. This method must be called before any other method.
* Args: `[]` (empty array)
* Returns: `string` (the protocol version, e.g., "0.2.8")
***OpenImage**: Opens an image reference (e.g., `docker://quay.io/example/image:latest`).
* Args: `[string imageName]`
* Returns: `uint64` (an opaque image ID to be used in subsequent calls)
***OpenImageOptional**: Similar to `OpenImage`, but if the image is not found, it returns `0` (a sentinel image ID) instead of an error.
* Args: `[string imageName]`
* Returns: `uint64` (opaque image ID, or `0` if the image is not found)
***CloseImage**: Closes a previously opened image, releasing associated resources.
* Args: `[uint64 imageID]`
* Returns: `null`
***GetManifest**: Retrieves the image manifest. If the image is a manifest list, it is resolved to an image matching the proxy's current OS and architecture. The manifest is converted to OCI format if it isn't already. The `value` field in the reply contains the original digest of the manifest (if the image is a manifest list, this is the digest of the list, not the per-platform instance). The manifest content is streamed over a pipe.
* Args: `[uint64 imageID]`
* Returns: `string` (manifest digest in `value`), manifest data via pipe.
***GetFullConfig**: Retrieves the full image configuration, conforming to the OCI Image Format Specification. Configuration data is streamed over a pipe.
* Args: `[uint64 imageID]`
* Returns: `null`, configuration data via pipe.
***GetBlob**: Fetches an image blob (e.g., a layer) by its digest and expected size. The proxy performs digest verification on the blob data. The `value` field in the reply contains the blob size. Blob data is streamed over a pipe.
* Returns: `int64` (blob size in `value`, `-1` if unknown), blob data via pipe.
***GetRawBlob**: Fetches an image blob by its digest. Unlike `GetBlob`, this method does not perform server-side digest verification. It returns two file descriptors to the client: one for the blob data and another for reporting errors that occur during the streaming. This method does not use the `FinishPipe` mechanism. The `value` field in the reply contains the blob size. (Introduced in protocol version 0.2.8)
* Args: `[uint64 imageID, string digest]`
* Returns: `int64` (blob size in `value`, `-1` if unknown), and *two* file descriptors: one for the blob data, one for errors. The error is a `ProxyError` type, see below.
***GetLayerInfoPiped**: Retrieves information about image layers. This replaces `GetLayerInfo`. Layer information data is streamed over a pipe, which makes it more reliable for images with many layers that would exceed message size limits with `GetLayerInfo`. The returned data is a JSON array of `{digest: string, size: int64, media_type: string}`. (Introduced in protocol version 0.2.7)
* Args: `[uint64 imageID]`
* Returns: `null`, layer information data via pipe.
***FinishPipe**: Signals that the client has finished reading all data from a pipe associated with a `pipeid` (obtained from methods like `GetManifest` or `GetBlob`). This allows the server to close its end of the pipe and report any pending errors (e.g., digest verification failure for `GetBlob`). This method **must** be called by the client after consuming data from a pipe, except for pipes from `GetRawBlob`.
* Args: `[uint32 pipeID]`
* Returns: `null`
***Shutdown**: Instructs the proxy server to terminate gracefully.
* Args: `[]` (empty array)
* Returns: `null`
The following methods are deprecated:
***GetConfig**: (deprecated) Retrieves the container runtime configuration part of the image (the OCI `config` field). **Note**: This method returns only a part of the full image configuration due to a historical oversight. Use `GetFullConfig` for the complete image configuration. Configuration data is streamed over a pipe.
* Args: `[uint64 imageID]`
* Returns: `null`, configuration data via pipe.
***GetLayerInfo**: (deprecated) Retrieves an array of objects, each describing an image layer (digest, size, mediaType). **Note**: This method returns data inline and may fail for images with many layers due to message size limits. Use `GetLayerInfoPiped` for a more robust solution.
* Args: `[uint64 imageID]`
* Returns: `array` of `{digest: string, size: int64, media_type: string}`.
**Data Transfer for Pipes**
When a method returns a `pipeid`, the server also passes the read-end of a pipe via file descriptor (FD) passing. The client reads the data (e.g., manifest content, blob content) from this FD. After successfully reading all data, the client **must** call `FinishPipe` with the corresponding `pipeid`. This signals to the server that the transfer is complete, allows the server to clean up resources, and enables the client to check for any errors that might have occurred during the data streaming process (e.g., a digest mismatch during `GetBlob`). The `GetRawBlob` method is an exception; it uses a dedicated error pipe instead of the `FinishPipe` mechanism.
**ProxyError**
`GetBlobRaw` returns a JSON object of the following form in the error pipe where:
```
{
"code": "EPIPE" | "retryable" | "other",
"message": "error message"
}
```
- EPIPE: The client closed the pipe before reading all data.
- retryable: The operation failed but might succeed if retried.
- other: A generic error occurred.
# OPTIONS
**--sockfd**=*fd*
Serve on the opened socket passed as file descriptor *fd*. Defaults to 0 (standard input).
The command also supports common skopeo options for interacting with image registries and local storage. These include:
**--authfile**=*path*
Path of the primary registry credentials file. On Linux, the default is ${XDG\_RUNTIME\_DIR}/containers/auth.json.
See **containers-auth.json**(5) for more details about the credential search mechanism and defaults on other platforms.
Use `skopeo login` to manage the credentials.
The default value of this option is read from the `REGISTRY\_AUTH\_FILE` environment variable.
**--cert-dir**=*path*
Use certificates at *path* (\*.crt, \*.cert, \*.key) to connect to the registry.
**--creds** _username[:password]_
Username and password for accessing the registry.
**--daemon-host** _host_
Use docker daemon host at _host_ (`docker-daemon:` transport only)
**--no-creds**
Access the registry anonymously.
**--password**=*password*
Password for accessing the registry. Use with **--username**.
**--registry-token**=*token*
Provide a Bearer *token* for accessing the registry.
**--shared-blob-dir** _directory_
Directory to use to share blobs across OCI repositories.
**--tls-verify**=_bool_
Require HTTPS and verify certificates when talking to the container registry or daemon. Default to registry.conf setting.
**--username**=*username*
Username for accessing the registry. Use with **--password**.
# REFERENCE CLIENT LIBRARIES
- Rust: The [containers-image-proxy-rs project](https://github.com/containers/containers-image-proxy-rs) serves
as the reference Rust client.
# PROTOCOL HISTORY
- 0.2.1: Initial version
- 0.2.2: Added support for fetching image configuration as OCI
- 0.2.3: Added GetFullConfig
- 0.2.4: Added OpenImageOptional
- 0.2.5: Added LayerInfoJSON
- 0.2.6: Policy Verification before pulling OCI
- 0.2.7: Added GetLayerInfoPiped
- 0.2.8: Added GetRawBlob and error_code to replies
Copy an image (manifest, filesystem layers, signatures) from one location to another.
Uses the system's trust policy to validate images, rejects images not trusted by the policy.
_source-image_ use the "image name" format described above
_destination-image_ use the "image name" format described above
_source-image_ and _destination-image_ are interpreted completely independently; e.g. the destination name does not
automatically inherit any parts of the source name.
## OPTIONS
See also [skopeo(1)](skopeo.1.md) for options placed before the subcommand name.
**--additional-tag**=_strings_
Additional tags (supports docker-archive).
**--all**, **-a**
If _source-image_ refers to a list of images, instead of copying just the image which matches the current OS and
architecture (subject to the use of the global --override-os, --override-arch and --override-variant options), attempt to copy all of
the images in the list, and the list itself.
**--authfile** _path_
Path of the primary registry credentials file. On Linux, the default is ${XDG\_RUNTIME\_DIR}/containers/auth.json.
See **containers-auth.json**(5) for more details about the credential search mechanism and defaults on other platforms.
Use `skopeo login` to manage the credentials.
The default value of this option is read from the `REGISTRY\_AUTH\_FILE` environment variable.
**--src-authfile** _path_
Path of the primary registry credentials file for the source registry. Uses path given by `--authfile`, if not provided.
**--dest-authfile** _path_
Path of the primary registry credentials file for the destination registry. Uses path given by `--authfile`, if not provided.
**--dest-shared-blob-dir** _directory_
Directory to use to share blobs across OCI repositories.
**--digestfile** _path_
After copying the image, write the digest of the resulting image to the file.
**--preserve-digests**
Preserve the digests during copying. Fail if the digest cannot be preserved.
This option does not change what will be copied; consider using `--all` at the same time.
**--encrypt-layer** _ints_
*Experimental* the 0-indexed layer indices, with support for negative indexing (e.g. 0 is the first layer, -1 is the last layer)
**--format**, **-f** _manifest-type_
MANIFEST TYPE (oci, v2s1, or v2s2) to use in the destination (default is manifest type of source, with fallbacks)
**--help**, **-h**
Print usage statement
**--multi-arch** _option_
Control what is copied if _source-image_ refers to a multi-architecture image. Default is system.
Options:
- system: Copy only the image that matches the system architecture
- all: Copy the full multi-architecture image
- index-only: Copy only the index
The index-only option usually fails unless the referenced per-architecture images are already present in the destination, or the target registry supports sparse indexes.
**--quiet**, **-q**
Suppress output information when copying images.
**--remove-signatures**
Do not copy signatures, if any, from _source-image_. Necessary when copying a signed image to a destination which does not support signatures.
**--sign-by** _key-id_
Add a “simple signing” signature using that key ID for an image name corresponding to _destination-image_
**--sign-by-sigstore** _param-file_
Add a sigstore signature based on the options in the specified containers sigstore signing parameter file, _param-file_.
See containers-sigstore-signing-params.yaml(5) for details about the file format.
**--sign-by-sigstore-private-key** _path_
Add a sigstore signature using a private key at _path_ for an image name corresponding to _destination-image_
**--sign-by-sq-fingerprint** _fingerprint_
Add a “simple signing” signature using a Sequoia-PGP key with the specified _fingerprint_.
**--sign-passphrase-file** _path_
The passphrase to use when signing with `--sign-by`, `--sign-by-sigstore-private-key` or `--sign-by-sq-fingerprint`.
Only the first line will be read. A passphrase stored in a file is of questionable security if other users can read this file. Do not use this option if at all avoidable.
**--sign-identity** _reference_
The identity to use when signing the image. The identity must be a fully specified docker reference. If the identity is not specified, the target docker reference will be used.
**--src-shared-blob-dir** _directory_
Directory to use to share blobs across OCI repositories.
**--encryption-key** _protocol:keyfile_
Specifies the encryption protocol, which can be JWE (RFC7516), PGP (RFC4880), and PKCS7 (RFC2315) and the key material required for image encryption. For instance, jwe:/path/to/key.pem or pgp:admin@example.com or pkcs7:/path/to/x509-file.
**--decryption-key** _key[:passphrase]_
Key to be used for decryption of images. Key can point to keys and/or certificates. Decryption will be tried with all keys. If the key is protected by a passphrase, it is required to be passed in the argument and omitted otherwise.
**--src-creds** _username[:password]_
Credentials for accessing the source registry.
**--dest-compress**
Compress tarball image layers when saving to directory using the 'dir' transport. (default is same compression type as source).
**--dest-decompress**
Decompress tarball image layers when saving to directory using the 'dir' transport. (default is same compression type as source).
**--dest-oci-accept-uncompressed-layers**
Allow uncompressed image layers when saving to an OCI image using the 'oci' transport. (default is to compress things that aren't compressed).
**--dest-creds** _username[:password]_
Credentials for accessing the destination registry.
**--src-cert-dir** _path_
Use certificates at _path_ (*.crt, *.cert, *.key) to connect to the source registry or daemon.
**--src-no-creds**
Access the registry anonymously.
**--src-tls-verify**=_bool_
Require HTTPS and verify certificates when talking to container source registry or daemon. Default to source registry setting.
**--dest-cert-dir** _path_
Use certificates at _path_ (*.crt, *.cert, *.key) to connect to the destination registry or daemon.
**--dest-no-creds**
Access the registry anonymously.
**--dest-tls-verify**=_bool_
Require HTTPS and verify certificates when talking to container destination registry or daemon. Default to destination registry setting.
**--src-daemon-host** _host_
Copy from docker daemon at _host_. If _host_ starts with `tcp://`, HTTPS is enabled by default. To use plain HTTP, use the form `http://` (default is `unix:///var/run/docker.sock`).
**--dest-daemon-host** _host_
Copy to docker daemon at _host_. If _host_ starts with `tcp://`, HTTPS is enabled by default. To use plain HTTP, use the form `http://` (default is `unix:///var/run/docker.sock`).
Existing signatures, if any, are preserved as well.
**--dest-compress-format** _format_
Specifies the compression format to use. Supported values are: `gzip`, `zstd` and `zstd:chunked`.
`zstd:chunked` is incompatible with encrypting images,
and will be treated as `zstd` with a warning in that case.
**--dest-compress-level** _format_
Specifies the compression level to use. The value is specific to the compression algorithm used, e.g. for zstd the accepted values are in the range 1-20 (inclusive), while for gzip it is 1-9 (inclusive).
**--dest-force-compress-format**
Ensures that the compression algorithm set in --dest-compress-format is used exclusively.
**--src-registry-token** _token_
Bearer token for accessing the source registry.
**--dest-registry-token** _token_
Bearer token for accessing the destination registry.
**--dest-precompute-digests**
Precompute digests to ensure layers are not uploaded that already exist on the destination registry. Layers with initially unknown digests (ex. compressing "on the fly") will be temporarily streamed to disk.
**--retry-times**
The number of times to retry. By default, no retries are attempted.
**--retry-delay**
Fixed delay between retries. If not set (or set to 0s), retry wait time will be exponentially increased based on the number of failed attempts.
**--src-username**
The username to access the source registry.
**--src-password**
The password to access the source registry.
**--dest-username**
The username to access the destination registry.
**--dest-password**
The password to access the destination registry.
**--image-parallel-copies** _n_
Maximum number of image layers to be copied (pulled/pushed) simultaneously. Not setting this field will fall back to containers/image defaults.
## EXAMPLES
To just copy an image from one registry to another:
Container images can also be partially encrypted by specifying the index of the layer. Layers are 0-indexed indices, with support for negative indexing. i.e. 0 is the first layer, -1 is the last layer.
Let's say out of 3 layers that the image `docker.io/library/nginx:1.17.8` is made up of, we only want to encrypt the 2nd layer,
skopeo\-delete - Mark the _image-name_ for later deletion by the registry's garbage collector.
## SYNOPSIS
**skopeo delete** [*options*] _image-name_
## DESCRIPTION
Mark _image-name_ for deletion.
The effect of this is registry-specific; many registries don’t support this operation, or don’t allow it in some circumstances / configurations.
**WARNING**: If _image-name_ contains a digest, this affects the referenced manifest, and may delete all tags (within the current repository?) pointing to that manifest.
**WARNING**: If _image-name_ contains a tag (but not a digest), in the current version of Skopeo this resolves the tag into a digest, and then deletes the manifest by digest, as described above (possibly deleting all tags pointing to that manifest, not just the provided tag). This behavior may change in the future.
When using the github.com/distribution/distribution registry server:
To release the allocated disk space, you must login to the container registry server and execute the container registry garbage collector. E.g.,
skopeo\-inspect - Return low-level information about _image-name_ in a registry.
## SYNOPSIS
**skopeo inspect** [*options*] _image-name_
## DESCRIPTION
Return low-level information about _image-name_ in a registry.
See [skopeo(1)](skopeo.1.md) for the format of _image-name_.
The default output includes data from various sources: user input (**Name**), the remote repository, if any (**RepoTags**), the top-level manifest (**Digest**),
and a per-architecture/OS image matching the current run-time environment (most other values).
To see values for a different architecture/OS, use the **--override-os** / **--override-arch** options documented in [skopeo(1)](skopeo.1.md).
## OPTIONS
See also [skopeo(1)](skopeo.1.md) for options placed before the subcommand name.
**--authfile** _path_
Path of the primary registry credentials file. On Linux, the default is ${XDG\_RUNTIME\_DIR}/containers/auth.json.
See **containers-auth.json**(5) for more details about the credential search mechanism and defaults on other platforms.
Use `skopeo login` to manage the credentials.
The default value of this option is read from the `REGISTRY\_AUTH\_FILE` environment variable.
**--cert-dir** _path_
Use certificates at _path_ (\*.crt, \*.cert, \*.key) to connect to the registry.
**--config**
Output configuration in OCI format, default is to format in JSON format.
**--creds** _username[:password]_
Username and password for accessing the registry.
**--daemon-host** _host_
Use docker daemon host at _host_ (`docker-daemon:` transport only)
**--format**, **-f**=*format*
Format the output using the given Go template.
The keys of the returned JSON can be used as the values for the --format flag (see examples below).
Supports the Go templating functions available at https://pkg.go.dev/github.com/containers/common/pkg/report#hdr-Template_Functions
**--help**, **-h**
Print usage statement
**--no-creds**
Access the registry anonymously.
**--raw**
Output raw manifest or config data depending on --config option.
The --format option is not supported with --raw option.
**--registry-token** _Bearer token_
Registry token for accessing the registry.
**--retry-times**
The number of times to retry. By default, no retries are attempted.
**--retry-delay**
Fixed delay between retries. If not set (or set to 0s), retry wait time will be exponentially increased based on the number of failed attempts.
**--shared-blob-dir** _directory_
Directory to use to share blobs across OCI repositories.
**--tls-verify**=_bool_
Require HTTPS and verify certificates when talking to the container registry or daemon. Default to registry.conf setting.
**--username**
The username to access the registry.
**--password**
The password to access the registry.
**--no-tags**, **-n**
Do not list the available tags from the repository in the output. When `true`, the `RepoTags` array will be empty. Defaults to `false`, which includes all available tags.
skopeo\-list\-tags - List image names in a transport-specific collection of images.
## SYNOPSIS
**skopeo list-tags** [*options*] _source-image_
Return a list of tags from _source-image_ in a registry or a local docker-archive file.
_source-image_ name of the repository to retrieve a tag listing from or a local docker-archive file.
## OPTIONS
See also [skopeo(1)](skopeo.1.md) for options placed before the subcommand name.
**--authfile** _path_
Path of the updated registry credentials file. On Linux, the default is ${XDG\_RUNTIME\_DIR}/containers/auth.json.
See **containers-auth.json**(5) for more details about the credential search mechanism and defaults on other platforms.
Use `skopeo login` to manage the credentials.
The default value of this option is read from the `REGISTRY\_AUTH\_FILE` environment variable.
**--creds** _username[:password]_ for accessing the registry.
**--cert-dir** _path_
Use certificates at _path_ (\*.crt, \*.cert, \*.key) to connect to the registry.
**--help**, **-h**
Print usage statement
**--no-creds**
Access the registry anonymously.
**--registry-token** _Bearer token_
Bearer token for accessing the registry.
**--retry-times**
The number of times to retry. By default, no retries are attempted.
**--retry-delay**
Fixed delay between retries. If not set (or set to 0s), retry wait time will be exponentially increased based on the number of failed attempts.
**--tls-verify**=_bool_
Require HTTPS and verify certificates when talking to the container registry or daemon. Default to registry.conf setting.
**--username**
The username to access the registry.
**--password**
The password to access the registry.
## REPOSITORY NAMES
Repository names are transport-specific references as each transport may have its own concept of a "repository" and "tags".
This commands refers to repositories using a _transport_`:`_details_ format. The following formats are supported:
**docker://**_docker-repository-reference_
A repository in a registry implementing the "Docker Registry HTTP API V2".
A _docker-repository-reference_ is of the form: **registryhost:port/repositoryname** which is similar to an _image-reference_ but with no tag or digest allowed as the last component (e.g no `:latest` or `@sha256:xyz`)
more than one images were stored in a docker save-formatted file.
## EXAMPLES
### Docker Transport
To get the list of tags in the "fedora" repository from the docker.io registry (the repository name expands to "library/fedora" per docker transport canonical form):
```console
$ skopeo list-tags docker://docker.io/fedora
{
"Repository": "docker.io/library/fedora",
"Tags": [
"20",
"21",
"22",
"23",
"24",
"25",
"26-modular",
"26",
"27",
"28",
"29",
"30",
"31",
"32",
"branched",
"heisenbug",
"latest",
"modular",
"rawhide"
]
}
```
To list the tags in a local host docker/distribution registry on port 5000, in this case for the "fedora" repository:
This is primarily a debugging tool, useful for special cases, and usually should not be a part of your normal operational workflow; use `skopeo copy --sign-by` instead to publish and sign an image in one step.
_manifest_ Path to a file containing the image manifest
_docker-reference_ A docker reference to identify the image with
_key-fingerprint_ Key identity to use for signing
## OPTIONS
See also [skopeo(1)](skopeo.1.md) for options placed before the subcommand name.
**--help**, **-h**
Print usage statement
**--output**, **-o** _output file_
Write signature to _output file_.
**--passphrase-file**=_path_
The passphare to use when signing with the key ID from `--sign-by`. Only the first line will be read. A passphrase stored in a file is of questionable security if other users can read this file. Do not use this option if at all avoidable.
This command is intended for use with local signatures e.g. OpenPGP ( other signature formats may be added in the future ), as per containers-signature(5). Furthermore, this command does **not** interact with the artifacts generated by Docker Content Trust (DCT). For more information, please see [containers-signature(5)](https://github.com/containers/image/blob/main/docs/containers-signature.5.md).
This command is intended for use with local signatures e.g. OpenPGP ( other signature formats may be added in the future ), as per containers-signature(5). Furthermore, this command does **not** interact with the artifacts generated by Docker Content Trust (DCT). For more information, please see [containers-signature(5)](https://github.com/containers/image/blob/main/docs/containers-signature.5.md).
Synchronize images between registry repositories and local directories. Synchronization is achieved by copying all the images found at _source_ to _destination_ - useful when synchronizing a local container registry mirror or for populating registries running inside of air-gapped environments.
Differently from other skopeo commands, skopeo sync requires both source and destination transports to be specified separately from _source_ and _destination_.
One of the problems of prefixing a destination with its transport is that, the registry `docker://hostname:port` would be wrongly interpreted as an image reference at a non-fully qualified registry, with `hostname` and `port` the image name and tag.
Available _source_ transports:
- _docker_ (i.e. `--src docker`): _source_ is a repository hosted on a container registry (e.g.: `registry.example.com/busybox`).
If no image tag is specified, skopeo sync copies all the tags found in that repository.
- _dir_ (i.e. `--src dir`): _source_ is a local directory path (e.g.: `/media/usb/`). Refer to skopeo(1) **dir:**_path_ for the local image format.
- _yaml_ (i.e. `--src yaml`): _source_ is local YAML file path.
The YAML file should specify the list of images copied from different container registries (local directories are not supported). Refer to EXAMPLES for the file format.
Available _destination_ transports:
- _docker_ (i.e. `--dest docker`): _destination_ is a container registry (e.g.: `my-registry.local.lan`).
- _dir_ (i.e. `--dest dir`): _destination_ is a local directory path (e.g.: `/media/usb/`).
One directory per source 'image:tag' is created for each copied image.
When the `--scoped` option is specified, images are prefixed with the source image path so that multiple images with the same
name can be stored at _destination_.
## OPTIONS
See also [skopeo(1)](skopeo.1.md) for options placed before the subcommand name.
**--all**, **-a**
If one of the images in __src__ refers to a list of images, instead of copying just the image which matches the current OS and
architecture (subject to the use of the global --override-os, --override-arch and --override-variant options), attempt to copy all of
the images in the list, and the list itself.
**--authfile** _path_
Path of the primary registry credentials file. On Linux, the default is ${XDG\_RUNTIME\_DIR}/containers/auth.json.
See **containers-auth.json**(5) for more details about the credential search mechanism and defaults on other platforms.
Use `skopeo login` to manage the credentials.
The default value of this option is read from the `REGISTRY\_AUTH\_FILE` environment variable.
**--src-authfile** _path_
Path of the primary registry credentials file for the source registry. Uses path given by `--authfile`, if not provided.
**--dest-authfile** _path_
Path of the primary registry credentials file for the destination registry. Uses path given by `--authfile`, if not provided.
**--dry-run**
Run the sync without actually copying data to the destination.
**--src**, **-s** _transport_ Transport for the source repository.
**--format**, **-f** _manifest-type_ Manifest Type (oci, v2s1, or v2s2) to use when syncing image(s) to a destination (default is manifest type of source, with fallbacks).
**--help**, **-h**
Print usage statement.
**--scoped** Prefix images with the source image path, so that multiple images with the same name can be stored at _destination_.
**--append-suffix** _tag-suffix_ String to append to destination tags.
**--digestfile** _path_
After copying the images from source, write the digest of the resulting images along with Image Reference.
Preserve the digests during copying. Fail if the digest cannot be preserved.
This option does not change what will be copied; consider using `--all` at the same time.
**--remove-signatures** Do not copy signatures, if any, from _source-image_. This is necessary when copying a signed image to a destination which does not support signatures.
**--sign-by** _key-id_
Add a “simple signing” signature using that key ID for an image name corresponding to _destination-image_
**--sign-by-sigstore** _param-file_
Add a sigstore signature based on the options in the specified containers sigstore signing parameter file, _param-file_.
See containers-sigstore-signing-params.yaml(5) for details about the file format.
**--sign-by-sigstore-private-key** _path_
Add a sigstore signature using a private key at _path_ for an image name corresponding to _destination-image_
**--sign-by-sq-fingerprint** _fingerprint_
Add a “simple signing” signature using a Sequoia-PGP key with the specified _fingerprint_.
**--sign-passphrase-file** _path_
The passphrase to use when signing with `--sign-by`, `--sign-by-sigstore-private-key` or `--sign-by-sq-fingerprint`.
Only the first line will be read. A passphrase stored in a file is of questionable security if other users can read this file. Do not use this option if at all avoidable.
**--src-creds** _username[:password]_ for accessing the source registry.
**--dest-creds** _username[:password]_ for accessing the destination registry.
**--src-cert-dir** _path_ Use certificates (*.crt, *.cert, *.key) at _path_ to connect to the source registry or daemon.
**--src-no-creds** Access the registry anonymously.
**--src-tls-verify**=_bool_ Require HTTPS and verify certificates when talking to a container source registry or daemon. Default to source registry entry in registry.conf setting.
**--dest-cert-dir** _path_ Use certificates (*.crt, *.cert, *.key) at _path_ to connect to the destination registry or daemon.
**--dest-no-creds** Access the registry anonymously.
**--dest-tls-verify**=_bool_ Require HTTPS and verify certificates when talking to a container destination registry or daemon. Default to destination registry entry in registry.conf setting.
**--src-registry-token** _Bearer token_ for accessing the source registry.
**--dest-registry-token** _Bearer token_ for accessing the destination registry.
**--retry-times**
The number of times to retry. By default, no retries are attempted.
**--retry-delay**
Fixed delay between retries. If not set (or set to 0s), retry wait time will be exponentially increased based on the number of failed attempts.
**--keep-going**
If any errors occur during copying of images, those errors are logged and the process continues syncing rest of the images and finally fails at the end.
**--src-username**
The username to access the source registry.
**--src-password**
The password to access the source registry.
**--dest-username**
The username to access the destination registry.
**--dest-password**
The password to access the destination registry.
## EXAMPLES
### Synchronizing to a local directory
```console
$ skopeo sync --src docker --dest dir registry.example.com/busybox /media/usb
```
Images are located at:
```
/media/usb/busybox:1-glibc
/media/usb/busybox:1-musl
/media/usb/busybox:1-ubuntu
...
/media/usb/busybox:latest
```
### Synchronizing to a container registry from local
Images are located at:
```
/media/usb/busybox:1-glibc
```
Sync run
```console
$ skopeo sync --src dir --dest docker /media/usb/busybox:1-glibc my-registry.local.lan/test/
```
Destination registry content:
```
REPO TAGS
my-registry.local.lan/test/busybox 1-glibc
```
### Synchronizing to a local directory, scoped
```console
$ skopeo sync --src docker --dest dir --scoped registry.example.com/busybox /media/usb
- Repository `registry.example.com/busybox`: all images, as no tags are specified.
- Repository `registry.example.com/redis`: images tagged "1.0" and "2.0" along with image with digest "sha256:0000000000000000000000000000000011111111111111111111111111111111".
- Repository `registry.example.com/nginx`: images tagged "1.13.1-alpine-perl" and "1.13.2-alpine-perl".
- Repository `registry.example.com/alpine`: all images with tags match the semantic version constraint ">= 3.12.0" ("3.12.0, "3.12.1", ... ,"4.0.0", ...)
The full list of possible semantic version comparisons can be found in the
Version ordering and precedence is understood as defined here:
https://semver.org/#spec-item-11.
For the registry `registry.example.com`, the "john"/"this is a secret" credentials are used, with server TLS certificates located at `/home/john/certs`.
TLS verification is normally enabled, and it can be disabled setting `tls-verify` to `false`.
In the above example, TLS verification is enabled for `registry.example.com`, while is
`skopeo` is a command line utility providing various operations with container images and container image registries. For example, it is able to inspect a repository on a container registry and fetch image. It fetches the repository's manifest and it is able to show you a `docker inspect`-like json output about a whole repository or a tag. This tool, in contrast to `docker inspect`, helps you gather useful information about a repository or a tag without requiring you to run `docker pull` - e.g. - which tags are available for the given repository? which labels the image has?
## NAME
skopeo -- Command line utility used to interact with local and remote container images and container image registries
`skopeo` is a command line utility providing various operations with container images and container image registries.
`skopeo` can copy container images between various containers image stores, converting them as necessary. For example you can use `skopeo` to copy container images from one container registry to another.
`skopeo` can convert a Docker schema 2 or schema 1 container image to an OCI image.
`skopeo` can inspect a repository on a container registry without needlessly pulling the image. Pulling an image from a repository, especially a remote repository, is an expensive network and storage operation. Skopeo fetches the repository's manifest and displays a `docker inspect`-like json output about the repository or a tag. `skopeo`, in contrast to `docker inspect`, helps you gather useful information about a repository or a tag without requiring you to run `docker pull` - e.g. - Which tags are available for the given repository? Which labels does the image have?
`skopeo` can sign and verify container images.
`skopeo` can delete container images from a remote container registry.
Note: `skopeo` does not require any container runtimes to be running, to do most of
its functionality. It also does not require root, unless you are copying images into a container runtime storage backend, like the docker daemon or github.com/containers/storage.
It also allows you to copy container images between various registries, possibly converting them as necessary, and to sign and verify images.
## IMAGE NAMES
Most commands refer to container images, using a _transport_`:`_details_ format. The following formats are supported:
**containers-storage:**_docker-reference_
An image located in a local containers/storage image store. Location and image store specified in /etc/containers/storage.conf
An image located in a local containers/storage image store. Both the location and image store are specified in /etc/containers/storage.conf. (Backend for Podman, CRI-O, Buildah and friends)
**dir:**_path_
An existing local directory _path_ storing the manifest, layer tarballs and signatures as individual files. This is a non-standardized format, primarily useful for debugging or noninvasive container inspection.
**docker://**_docker-reference_
An image in a registry implementing the "Docker Registry HTTP API V2". By default, uses the authorization state in `$HOME/.docker/config.json`, which is set e.g. using `(docker login)`.
An image in a registry implementing the "Docker Registry HTTP API V2".
Credentials are typically managed using `(skopeo login)`;
see **containers-auth.json**(5) for more details about the credential search mechanism.
An image is stored in the `docker save` formated file. _docker-reference_ is only used when creating such a file, and it must not contain a digest.
An image is stored in the `docker save` formatted file. _docker-reference_ is only used when creating such a file, and it must not contain a digest.
**docker-daemon:**_docker-reference_
An image _docker-reference_ stored in the docker daemon internal storage. _docker-reference_ must contain either a tag or a digest. Alternatively, when reading images, the format can also be docker-daemon:algo:digest (an image ID).
An image _docker-reference_ stored in the docker daemon internal storage. _docker-reference_ must contain either a tag or a digest. Alternatively, when reading images, the format can be docker-daemon:algo:digest (an image ID).
**oci:**_path_**:**_tag_
An image _tag_ in a directory compliant with "Open Container Image Layout Specification" at _path_.
**ostree:**_image_[**@**_/absolute/repo/path_]
An image in local OSTree repository. _/absolute/repo/path_ defaults to _/ostree/repo_.
**oci-archive:**_path_**:**_tag_
An image _tag_ in a tar archive compliant with "Open Container Image Layout Specification" at _path_.
# OPTIONS
See [containers-transports(5)](https://github.com/containers/image/blob/main/docs/containers-transports.5.md) for details.
**--debug** enable debug output
## OPTIONS
**--policy** _path-to-policy_ Path to a policy.json file to use for verifying signatures and deciding whether an image is trusted, overriding the default trust policy file.
These options should be placed before the subcommand name.
Individual subcommands have their own options.
**--insecure-policy** Adopt an insecure, permissive policy that allows anything. This obviates the need for a policy file.
**--command-timeout** _duration_
**--registries.d** _dir_ use registry configuration files in _dir_ (e.g. for container signature storage), overriding the default path.
Copy an image (manifest, filesystem layers, signatures) from one location to another.
**--insecure-policy**
Uses the system's trust policy to validate images, rejects images not trusted by the policy.
Adopt an insecure, permissive policy that allows anything. This obviates the need for a policy file.
_source-image_ use the "image name" format described above
**--override-arch** _arch_
_destination-image_ use the "image name" format described above
Use_arch_ instead of the architecture of the machine for choosing images.
**--format, -f** _manifest-type_ Manifest type (oci, v2s1, or v2s2) to use when saving image to directory using the 'dir:' transport (default is manifest type of source)
**--override-os** _os_
**--remove-signatures** do not copy signatures, if any, from _source-image_. Necessary when copying a signed image to a destination which does not support signatures.
Use_OS_ instead of the running OS for choosing images.
**--sign-by=**_key-id_ add a signature using that key ID for an image name corresponding to _destination-image_
**--override-variant** _variant_
**--src-creds** _username[:password]_ for accessing the source registry
Use_variant_ instead of the running architecture variant for choosing images.
**--dest-compress** _bool-value_ Compress tarball image layers when saving to directory using the 'dir' transport. (default is same compression type as source)
**--policy** _path-to-policy_
**--dest-creds** _username[:password]_ for accessing the destination registry
Path to a policy.json file to use for verifying signatures and deciding whether an image is trusted, overriding the default trust policy file.
**--src-cert-dir** _path_ Use certificates at _path_ (*.crt, *.cert, *.key) to connect to the source registry
**--registries.d** _dir_
**--src-tls-verify** _bool-value_ Require HTTPS and verify certificates when talking to container source registry (defaults to true)
Use registry configuration files in _dir_ (e.g. for container signature storage), overriding the default path.
**--dest-cert-dir** _path_ Use certificates at _path_ (*.crt, *.cert, *.key) to connect to the destination registry
**--require-signed**
**--dest-ostree-tmp-dir** _path_ Directory to use for OSTree temporary files.
Require that any pulled image must be signed regardless of what the default or provided trust policy file says.
**--dest-tls-verify** _bool-value_ Require HTTPS and verify certificates when talking to container destination registry (defaults to true)
**--tmpdir** _dir_
Existing signatures, if any, are preserved as well.
Directory used to store temporary files. Defaults to /var/tmp.
## skopeo delete
**skopeo delete** _image-name_
**--user-agent-prefix** _prefix_
Mark _image-name_ for deletion. To release the allocated disk space, you need to execute the container registry garabage collector. E.g.,
Prefix to add to the user agent string. The resulting user agent will be in the format "_prefix_ skopeo/_version_".
Verify a signature using local files, digest will be printed on success.
_manifest_ Path to a file containing the image manifest
_docker-reference_ A docker reference expected to identify the image in the signature
_key-fingerprint_ Expected identity of the signing key
_signature_ Path to signature file
**Note:** If you do use this, make sure that the image can not be changed at the source location between the times of its verification and use.
## skopeo help
show help for `skopeo`
# FILES
## FILES
**/etc/containers/policy.json**
Default trust policy file, if **--policy** is not specified.
The policy format is documented in https://github.com/containers/image/blob/master/docs/policy.json.md .
The policy format is documented in [containers-policy.json(5)](https://github.com/containers/image/blob/main/docs/containers-policy.json.5.md) .
**/etc/containers/registries.d**
Default directory containing registry configuration, if **--registries.d** is not specified.
The contents of this directory are documented in https://github.com/containers/image/blob/master/docs/registries.d.md .
The contents of this directory are documented in [containers-registries.d(5)](https://github.com/containers/image/blob/main/docs/containers-registries.d.5.md).
<pathinkscape:export-ydpi="96.181694"inkscape:export-xdpi="96.181694"sodipodi:nodetypes="cc"inkscape:connector-curvature="0"id="path81576"d="m 61.405599,166.31541 v 5.83669"style="fill:none;fill-opacity:1;stroke:#9dc6e7;stroke-width:0.79375;stroke-linecap:square;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1"inkscape:export-filename="/home/duffy/Documents/Projects/Favors/Buildah logo/final/color-not-color.png"/>
echo"There is trace of conflict(s) in the following files :"
for f in "${badFiles[@]}";do
echo" - $f"
done
echo
echo'Please fix the conflict(s) commit the result.'
echo
} >&2
exit1
fi
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.