Compare commits

..

24 Commits

Author SHA1 Message Date
Chris Evich
85ce427969 [release-1.11] Bump to release version 1.11.3
This branch is utilized for RHEL releases and therefore should never
ever represent a `-dev` development release.  Bump the version
number to account for the change.

Resolves: RHEL-97092 RHEL-97090

Signed-off-by: Chris Evich <cevich@redhat.com>
2025-08-11 15:38:31 -04:00
Miloslav Trmač
c8ef2dcce3 Merge pull request #2643 from cevich/release-1.11_add_release_test
[release-1.11] Add conditional release-checking system test
2025-07-04 17:13:02 +02:00
Chris Evich
b5b0a9cd81 [release-1.11] Add conditional release-checking system test
Unfortunately on a number of occasions, Skopeo has been released
officially with a `-dev` suffix in the version number.  Assist in
catching this mistake at release time by the addition of a simple
conditional test.  Note that it must be positively enabled by a
magic env. var. before executing the system tests.

Original PR: https://github.com/containers/skopeo/pull/2631

Signed-off-by: Chris Evich <cevich@redhat.com>
2025-07-02 14:30:14 -04:00
Miloslav Trmač
e0171abca9 Merge pull request #2611 from cevich/release-1.11-multiarch_registry
[release-1.11] Support CI testing on non-x86_64
2025-05-28 20:43:09 +02:00
Chris Evich
4773bf1895 Support CI testing on non-x86_64
Previously, internal CI gating tests sometimes fail because the required
registry container image only supports x86_64.  Update to the `2.8.2`
image tag with support for all primary architectures.

Signed-off-by: Chris Evich <cevich@redhat.com>
2025-05-28 14:25:03 -04:00
Miloslav Trmač
7602ac68f8 Merge pull request #2424 from TomSweeneyRedHat/dev/tsweeney/v1.11-cve-2024-3727
[release-1.11] CVE-2024-3727
2024-09-19 21:34:15 +02:00
tomsweeneyredhat
7f996f3bdb [release-1.11] CVE-2024-3727
Addresses CVE-2024-3727 by bumping c/common to v0.51.4 and c/image
to v5.24.3

Fixes: https://issues.redhat.com/browse/OCPBUGS-37020
https://issues.redhat.com/browse/OCPBUGS-37022
https://issues.redhat.com/browse/OCPBUGS-37023

Signed-off-by: tomsweeneyredhat <tsweeney@redhat.com>
2024-09-19 14:11:24 -04:00
Colin Walters
78dc389125 Merge pull request #2359 from mtrmac/k8s.gcr.io-11
[release-1.11] Refer to registry.k8s.io instead of k8s.gcr.io
2024-06-19 19:41:22 -04:00
Miloslav Trmač
34ed1100de Refer to registry.k8s.io instead of k8s.gcr.io
... per https://kubernetes.io/blog/2023/02/06/k8s-gcr-io-freeze-announcement/ .

We are seeing intermittent failures (sufficient to reliably cause a test suite failure)
pulling from k8s.gcr.io, let's see if using the newer one improves things.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2024-06-19 17:50:41 +02:00
Miloslav Trmač
051898442c Merge pull request #2292 from TomSweeneyRedHat/dev/tsweeney/cve-jose-1.11
[release-1.11] Bump ocicrypt and go-jose CVE-2024-28180
2024-04-18 00:58:42 +02:00
tomsweeneyredhat
89cd9b89b6 [release-1.11] Bump ocicrypt and go-jose CVE-2024-28180
Bump github.com/go-jose/go-jose to v3.0.0 and
github.com/containers/ocicrypt to v1.1.10

Addresses: CVE-2024-28180
https://issues.redhat.com/browse/OCPBUGS-30789
https://issues.redhat.com/browse/OCPBUGS-30790
https://issues.redhat.com/browse/OCPBUGS-30791

Signed-off-by: tomsweeneyredhat <tsweeney@redhat.com>
2024-04-17 18:15:23 -04:00
Miloslav Trmač
df2b9aedc8 Merge pull request #2286 from mtrmac/integration-update-1.11
[release-1.11] Backport #2280
2024-04-10 20:01:14 +02:00
Miloslav Trmač
6f884cd817 Freeze the fedora-minimal image reference at Fedora 38
... because the tests are assuming a v2s2 image, but
as of Fedora 39, the image uses the OCI format.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2024-04-08 19:55:14 +02:00
Miloslav Trmač
7e11ab4ada Merge pull request #1991 from cevich/release_1.11_add_self_destruct
[release-1.11] Cirrus: Add CI self-destruct condition on EOL date
2023-05-09 16:09:47 +02:00
Chris Evich
9b087c653c [release-1.11] Cirrus: Add CI self-destruct condition on EOL date
This branch will never receive any security-backports when the
associated RHEL release reaches EOL.  Add a condition to force CI to
break with a helpful message, after this RHEL EOL date.

Signed-off-by: Chris Evich <cevich@redhat.com>
2023-05-03 11:18:19 -04:00
Miloslav Trmač
d79588e6c1 Bump to v1.11.3-dev
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2023-04-03 07:51:38 -04:00
Miloslav Trmač
dc1e14f7a7 Release 1.11.2
Updates golang.org/x/net to v0.7.0 to resolve CVE-2022-41723.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2023-04-03 07:51:38 -04:00
Miloslav Trmač
8191ef3ea1 Merge pull request #1948 from lsm5/release-1.11-CVE-2022-41723
[release-1.11] bump golang.org/x/net to v0.7.0
2023-03-24 22:44:49 +01:00
Lokesh Mandvekar
902506dd73 bump golang.org/x/net to v0.7.0
Resolves: CVE-2022-41723
Ref: https://cve.mitre.org/cgi-bin/cvename.cgi?name=2022-41723

Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
2023-03-24 09:54:45 +05:30
Miloslav Trmač
3f98753bfd Merge pull request #1912 from TomSweeneyRedHat/dev/tsweeney/1.11.1
[release-1.11] Bump to v1.11.1
2023-02-16 23:21:39 +01:00
tomsweeneyredhat
b2884205e7 [release-1.11] Bump to v1.11.2-dev
As the title says

[NO NEW TESTS NEEDEDED]

Signed-off-by: tomsweeneyredhat <tsweeney@redhat.com>
2023-02-16 15:16:10 -05:00
tomsweeneyredhat
fb1ade6d9e [release-1.11] Bump to v1.11.1
As the title says.  To ready for RHEL 8.8/9.2

[NO NEW TESTS NEEDED]

Signed-off-by: tomsweeneyredhat <tsweeney@redhat.com>
2023-02-15 17:27:51 -05:00
Valentin Rothberg
0d212fc3b5 Merge pull request #1902 from mtrmac/c-image-eof-1.11
[release-1.11] Update to c/image 5.24.1
2023-02-13 09:01:03 +01:00
Miloslav Trmač
40dd6507df Update to c/image 5.24.1
... to include an unexpected EOF workaround.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2023-02-09 21:12:48 +01:00
1513 changed files with 86617 additions and 340828 deletions

View File

@@ -20,8 +20,13 @@ env:
# Save a little typing (path relative to $CIRRUS_WORKING_DIR)
SCRIPT_BASE: "./contrib/cirrus"
####
#### Cache-image names to test with (double-quotes around names are critical)
####
FEDORA_NAME: "fedora-37"
# Google-cloud VM Images
IMAGE_SUFFIX: "c20240102t155643z-f39f38d13"
IMAGE_SUFFIX: "c6300530360713216"
FEDORA_CACHE_IMAGE_NAME: "fedora-${IMAGE_SUFFIX}"
# Container FQIN's
@@ -47,9 +52,7 @@ validate_task:
image: '${SKOPEO_CIDEV_CONTAINER_FQIN}'
cpu: 4
memory: 8
setup_script: |
make tools
test_script: |
script: |
make validate-local
make vendor && hack/tree_status.sh
@@ -72,13 +75,14 @@ doccheck_task:
"${SKOPEO_PATH}/${SCRIPT_BASE}/runner.sh" doccheck
osx_task:
# Don't run for docs-only builds.
# Don't run for docs-only or multi-arch image builds.
# Also don't run on release-branches or their PRs,
# since base container-image is not version-constrained.
only_if: &not_docs_or_release_branch >-
($CIRRUS_BASE_BRANCH == $CIRRUS_DEFAULT_BRANCH ||
$CIRRUS_BRANCH == $CIRRUS_DEFAULT_BRANCH ) &&
$CIRRUS_CHANGE_TITLE !=~ '.*CI:DOCS.*'
$CIRRUS_CHANGE_TITLE !=~ '.*CI:DOCS.*' &&
$CIRRUS_CRON != 'multiarch'
depends_on:
- validate
macos_instance:
@@ -87,7 +91,7 @@ osx_task:
export PATH=$GOPATH/bin:$PATH
brew update
brew install gpgme go go-md2man
make tools
go install golang.org/x/lint/golint@latest
test_script: |
export PATH=$GOPATH/bin:$PATH
go version
@@ -100,7 +104,8 @@ osx_task:
cross_task:
alias: cross
only_if: >-
$CIRRUS_CHANGE_TITLE !=~ '.*CI:DOCS.*'
$CIRRUS_CHANGE_TITLE !=~ '.*CI:DOCS.*' &&
$CIRRUS_CRON != 'multiarch'
depends_on:
- validate
gce_instance: &standardvm
@@ -138,7 +143,7 @@ ostree-rs-ext_task:
dockerfile: contrib/cirrus/ostree_ext.dockerfile
docker_arguments: # required build-args
BASE_FQIN: quay.io/coreos-assembler/fcos-buildroot:testing-devel
CIRRUS_IMAGE_VERSION: 2
CIRRUS_IMAGE_VERSION: 1
env:
EXT_REPO_NAME: ostree-rs-ext
EXT_REPO_HOME: $CIRRUS_WORKING_DIR/../$EXT_REPO_NAME
@@ -163,10 +168,11 @@ ostree-rs-ext_task:
#####
test_skopeo_task:
alias: test_skopeo
# Don't test for [CI:DOCS], [CI:BUILD].
# Don't test for [CI:DOCS], [CI:BUILD], or 'multiarch' cron.
only_if: >-
$CIRRUS_CHANGE_TITLE !=~ '.*CI:BUILD.*' &&
$CIRRUS_CHANGE_TITLE !=~ '.*CI:DOCS.*'
$CIRRUS_CHANGE_TITLE !=~ '.*CI:DOCS.*' &&
$CIRRUS_CRON != 'multiarch'
depends_on:
- validate
gce_instance:
@@ -199,6 +205,49 @@ test_skopeo_task:
"${SKOPEO_PATH}/${SCRIPT_BASE}/runner.sh" system
image_build_task: &image-build
name: "Build multi-arch $CTXDIR"
alias: image_build
# Some of these container images take > 1h to build, limit
# this task to a specific Cirrus-Cron entry with this name.
only_if: $CIRRUS_CRON == 'multiarch'
timeout_in: 120m # emulation is sssllllooooowwww
gce_instance:
<<: *standardvm
image_name: build-push-${IMAGE_SUFFIX}
# More muscle required for parallel multi-arch build
type: "n2-standard-4"
matrix:
- env:
CTXDIR: contrib/skopeoimage/upstream
- env:
CTXDIR: contrib/skopeoimage/testing
- env:
CTXDIR: contrib/skopeoimage/stable
env:
SKOPEO_USERNAME: ENCRYPTED[4195884d23b154553f2ddb26a63fc9fbca50ba77b3e447e4da685d8639ed9bc94b9a86a9c77272c8c80d32ead9ca48da]
SKOPEO_PASSWORD: ENCRYPTED[36e06f9befd17e5da2d60260edb9ef0d40e6312e2bba4cf881d383f1b8b5a18c8e5a553aea2fdebf39cebc6bd3b3f9de]
CONTAINERS_USERNAME: ENCRYPTED[dd722c734641f103b394a3a834d51ca5415347e378637cf98ee1f99e64aad2ec3dbd4664c0d94cb0e06b83d89e9bbe91]
CONTAINERS_PASSWORD: ENCRYPTED[d8b0fac87fe251cedd26c864ba800480f9e0570440b9eb264265b67411b253a626fb69d519e188e6c9a7f525860ddb26]
main_script:
- source /etc/automation_environment
- main.sh $CIRRUS_REPO_CLONE_URL $CTXDIR
test_image_build_task:
<<: *image-build
alias: test_image_build
# Allow this to run inside a PR w/ [CI:BUILD] only.
only_if: $CIRRUS_PR != '' && $CIRRUS_CHANGE_TITLE =~ '.*CI:BUILD.*'
# This takes a LONG time, only run when requested. N/B: Any task
# made to depend on this one will block FOREVER unless triggered.
# DO NOT ADD THIS TASK AS DEPENDENCY FOR `success_task`.
trigger_type: manual
# Overwrite all 'env', don't push anything, just do the build.
env:
DRYRUN: 1
# This task is critical. It updates the "last-used by" timestamp stored
# in metadata for all VM images. This mechanism functions in tandem with
# an out-of-band pruning operation to remove disused VM images.
@@ -237,6 +286,7 @@ success_task:
- cross
- proxy_ostree_ext
- test_skopeo
- image_build
- meta
container: *smallcontainer
env:

10
.github/dependabot.yml vendored Normal file
View File

@@ -0,0 +1,10 @@
version: 2
updates:
- package-ecosystem: gomod
directory: "/"
schedule:
interval: daily
time: "10:00"
timezone: Europe/Berlin
open-pull-requests-limit: 10

View File

@@ -49,4 +49,26 @@
/*************************************************
*** Repository-specific configuration options ***
*************************************************/
// Don't leave dep. update. PRs "hanging", assign them to people.
"assignees": ["containers/image-maintainers"], // same for skopeo
/*************************************************
***** Golang-specific configuration options *****
*************************************************/
"golang": {
// N/B: LAST MATCHING RULE WINS
// https://docs.renovatebot.com/configuration-options/#packagerules
"packageRules": [
// Package version retraction (https://go.dev/ref/mod#go-mod-file-retract)
// is broken in Renovate
// ref: https://github.com/renovatebot/renovate/issues/13012
{
"matchPackageNames": ["github.com/containers/common"],
// Both v1.0.0 and v1.0.1 should be ignored.
"allowedVersions": "!/v((1.0.0)|(1.0.1))$/"
},
],
},
}

View File

@@ -1,20 +0,0 @@
---
# See also:
# https://github.com/containers/podman/blob/main/.github/workflows/discussion_lock.yml
on:
schedule:
- cron: '0 0 * * *'
# Debug: Allow triggering job manually in github-actions WebUI
workflow_dispatch: {}
jobs:
# Ref: https://docs.github.com/en/actions/using-workflows/reusing-workflows
closed_issue_discussion_lock:
uses: containers/podman/.github/workflows/discussion_lock.yml@main
secrets: inherit
permissions:
contents: read
issues: write
pull-requests: write

View File

@@ -17,7 +17,7 @@ jobs:
pull-requests: write # for actions/stale to close stale PRs
runs-on: ubuntu-latest
steps:
- uses: actions/stale@v9
- uses: actions/stale@v7
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
stale-issue-message: 'A friendly reminder that this issue had no activity for 30 days.'

View File

@@ -1,3 +0,0 @@
---
run:
timeout: 5m

View File

@@ -1,24 +0,0 @@
---
# See the documentation for more information:
# https://packit.dev/docs/configuration/
# NOTE: The Packit copr_build tasks help to check if every commit builds on
# supported Fedora and CentOS Stream arches.
# They do not block the current Cirrus-based workflow.
specfile_path: rpm/skopeo.spec
upstream_tag_template: v{version}
srpm_build_deps:
- make
jobs:
- job: copr_build
trigger: pull_request
notifications:
failure_comment:
message: "Ephemeral COPR build failed. @containers/packit-build please check."
enable_net: true
targets:
- epel-9-x86_64
- epel-9-aarch64

View File

@@ -24,11 +24,6 @@ GOBIN := $(shell $(GO) env GOBIN)
GOOS ?= $(shell go env GOOS)
GOARCH ?= $(shell go env GOARCH)
# N/B: This value is managed by Renovate, manual changes are
# possible, as long as they don't disturb the formatting
# (i.e. DO NOT ADD A 'v' prefix!)
GOLANGCI_LINT_VERSION := 1.55.2
ifeq ($(GOBIN),)
GOBIN := $(GOPATH)/bin
endif
@@ -43,6 +38,13 @@ endif
export CONTAINER_RUNTIME ?= $(if $(shell command -v podman ;),podman,docker)
GOMD2MAN ?= $(if $(shell command -v go-md2man ;),go-md2man,$(GOBIN)/go-md2man)
# Go module support: set `-mod=vendor` to use the vendored sources.
# See also hack/make.sh.
ifeq ($(shell go help mod >/dev/null 2>&1 && echo true), true)
GO:=GO111MODULE=on $(GO)
MOD_VENDOR=-mod=vendor
endif
ifeq ($(DEBUG), 1)
override GOGCFLAGS += -N -l
endif
@@ -54,11 +56,17 @@ ifeq ($(GOOS), linux)
endif
# If $TESTFLAGS is set, it is passed as extra arguments to 'go test'.
# You can select certain tests to run, with `-run <regex>` for example:
# You can increase test output verbosity with the option '-test.vv'.
# You can select certain tests to run, with `-test.run <regex>` for example:
#
# make test-unit TESTFLAGS='-run ^TestManifestDigest$'
# make test-integration TESTFLAGS='-run copySuite.TestCopy.*'
export TESTFLAGS ?= -timeout=15m
# make test-unit TESTFLAGS='-test.run ^TestManifestDigest$'
#
# For integration test, we use [gocheck](https://labix.org/gocheck).
# You can increase test output verbosity with the option '-check.vv'.
# You can limit test selection with `-check.f <regex>`, for example:
#
# make test-integration TESTFLAGS='-check.f CopySuite.TestCopy.*'
export TESTFLAGS ?= -v -check.v -test.timeout=15m
# This is assumed to be set non-empty when operating inside a CI/automation environment
CI ?=
@@ -117,7 +125,6 @@ help:
@echo " * 'install' - Install binaries and documents to system locations"
@echo " * 'binary' - Build skopeo with a container"
@echo " * 'bin/skopeo' - Build skopeo locally"
@echo " * 'bin/skopeo.OS.ARCH' - Build skopeo for specific OS and ARCH"
@echo " * 'test-unit' - Execute unit tests"
@echo " * 'test-integration' - Execute integration tests"
@echo " * 'validate' - Verify whether there is no conflict and all Go source files have been formatted, linted and vetted"
@@ -132,9 +139,9 @@ binary: cmd/skopeo
# Build w/o using containers
.PHONY: bin/skopeo
bin/skopeo:
$(GO) build ${GO_DYN_FLAGS} ${SKOPEO_LDFLAGS} -gcflags "$(GOGCFLAGS)" -tags "$(BUILDTAGS)" -o $@ ./cmd/skopeo
$(GO) build $(MOD_VENDOR) ${GO_DYN_FLAGS} ${SKOPEO_LDFLAGS} -gcflags "$(GOGCFLAGS)" -tags "$(BUILDTAGS)" -o $@ ./cmd/skopeo
bin/skopeo.%:
GOOS=$(word 2,$(subst ., ,$@)) GOARCH=$(word 3,$(subst ., ,$@)) $(GO) build ${SKOPEO_LDFLAGS} -tags "containers_image_openpgp $(BUILDTAGS)" -o $@ ./cmd/skopeo
GOOS=$(word 2,$(subst ., ,$@)) GOARCH=$(word 3,$(subst ., ,$@)) $(GO) build $(MOD_VENDOR) ${SKOPEO_LDFLAGS} -tags "containers_image_openpgp $(BUILDTAGS)" -o $@ ./cmd/skopeo
local-cross: bin/skopeo.darwin.amd64 bin/skopeo.linux.arm bin/skopeo.linux.arm64 bin/skopeo.windows.386.exe bin/skopeo.windows.amd64.exe
$(MANPAGES): %: %.md
@@ -187,11 +194,6 @@ install-completions: completions
shell:
$(CONTAINER_RUN) bash
tools:
if [ ! -x "$(GOBIN)/golangci-lint" ]; then \
curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(GOBIN) v$(GOLANGCI_LINT_VERSION) ; \
fi
check: validate test-unit test-integration test-system
test-integration:
@@ -204,8 +206,7 @@ test-integration:
# Intended for CI, assumed to be running in quay.io/libpod/skopeo_cidev container.
test-integration-local: bin/skopeo
hack/warn-destructive-tests.sh
hack/test-integration.sh
hack/make.sh test-integration
# complicated set of options needed to run podman-in-podman
test-system:
@@ -221,8 +222,7 @@ test-system:
# Intended for CI, assumed to already be running in quay.io/libpod/skopeo_cidev container.
test-system-local: bin/skopeo
hack/warn-destructive-tests.sh
hack/test-system.sh
hack/make.sh test-system
test-unit:
# Just call (make test unit-local) here instead of worrying about environment differences
@@ -236,22 +236,19 @@ test-all-local: validate-local validate-docs test-unit-local
.PHONY: validate-local
validate-local:
hack/validate-git-marks.sh
hack/validate-gofmt.sh
GOBIN=$(GOBIN) hack/validate-lint.sh
BUILDTAGS="${BUILDTAGS}" hack/validate-vet.sh
BUILDTAGS="${BUILDTAGS}" hack/make.sh validate-git-marks validate-gofmt validate-lint validate-vet
# This invokes bin/skopeo, hence cannot be run as part of validate-local
.PHONY: validate-docs
validate-docs: bin/skopeo
validate-docs:
hack/man-page-checker
hack/xref-helpmsgs-manpages
test-unit-local:
$(GO) test -tags "$(BUILDTAGS)" $$($(GO) list -tags "$(BUILDTAGS)" -e ./... | grep -v '^github\.com/containers/skopeo/\(integration\|vendor/.*\)$$')
test-unit-local: bin/skopeo
$(GO) test $(MOD_VENDOR) -tags "$(BUILDTAGS)" $$($(GO) list $(MOD_VENDOR) -tags "$(BUILDTAGS)" -e ./... | grep -v '^github\.com/containers/skopeo/\(integration\|vendor/.*\)$$')
vendor:
$(GO) mod tidy
$(GO) mod tidy -compat=1.17
$(GO) mod vendor
$(GO) mod verify

View File

@@ -1,4 +1,8 @@
<img src="https://cdn.rawgit.com/containers/skopeo/main/docs/skopeo.svg" width="250" alt="Skopeo">
<!--- skopeo [![Build Status](https://travis-ci.org/containers/skopeo.svg?branch=main)](https://travis-ci.org/containers/skopeo)
=
--->
<img src="https://cdn.rawgit.com/containers/skopeo/main/docs/skopeo.svg" width="250">
----
@@ -39,12 +43,6 @@ Skopeo works with API V2 container image registries such as [docker.io](https://
* oci:path:tag
An image tag in a directory compliant with "Open Container Image Layout Specification" at path.
[Obtaining skopeo](./install.md)
-
For a detailed description how to install or build skopeo, see
[install.md](./install.md).
## Inspecting a repository
`skopeo` is able to _inspect_ a repository on a container registry and fetch images layers.
The _inspect_ command fetches the repository's manifest and it is able to show you a `docker inspect`-like
@@ -195,6 +193,12 @@ $ skopeo inspect --creds=testuser:testpassword docker://myregistrydomain.com:500
$ skopeo copy --src-creds=testuser:testpassword docker://myregistrydomain.com:5000/private oci:local_oci_image
```
[Obtaining skopeo](./install.md)
-
For a detailed description how to install or build skopeo, see
[install.md](./install.md).
Contributing
-

View File

@@ -62,7 +62,7 @@ func TestGenerateSigstoreKey(t *testing.T) {
// we have to trigger a write failure.
// Success
// Just a smoke-test, usability of the keys is tested in the generate implementation.
// Just a smoke-test, useability of the keys is tested in the generate implementation.
dir := t.TempDir()
prefix := filepath.Join(dir, "prefix")
passphraseFile := filepath.Join(dir, "passphrase")

View File

@@ -6,6 +6,8 @@ import (
"fmt"
"io"
"strings"
"text/tabwriter"
"text/template"
"github.com/containers/common/pkg/report"
"github.com/containers/common/pkg/retry"
@@ -51,8 +53,8 @@ See skopeo(1) section "IMAGE NAMES" for the expected format
`, strings.Join(transports.ListNames(), ", ")),
RunE: commandAction(opts.run),
Example: `skopeo inspect docker://registry.fedoraproject.org/fedora
skopeo inspect --config docker://docker.io/alpine
skopeo inspect --format "Name: {{.Name}} Digest: {{.Digest}}" docker://registry.access.redhat.com/ubi8`,
skopeo inspect --config docker://docker.io/alpine
skopeo inspect --format "Name: {{.Name}} Digest: {{.Digest}}" docker://registry.access.redhat.com/ubi8`,
ValidArgsFunction: autocompleteSupportedTransports,
}
adjustUsage(cmd)
@@ -72,6 +74,7 @@ func (opts *inspectOptions) run(args []string, stdout io.Writer) (retErr error)
rawManifest []byte
src types.ImageSource
imgInspect *types.ImageInspectInfo
data []interface{}
)
ctx, cancel := opts.global.commandTimeoutContext()
defer cancel()
@@ -148,7 +151,18 @@ func (opts *inspectOptions) run(args []string, stdout io.Writer) (retErr error)
}, opts.retryOpts); err != nil {
return fmt.Errorf("Error reading OCI-formatted configuration data: %w", err)
}
if err := opts.writeOutput(stdout, config); err != nil {
if report.IsJSON(opts.format) || opts.format == "" {
var out []byte
out, err = json.MarshalIndent(config, "", " ")
if err == nil {
fmt.Fprintf(stdout, "%s\n", string(out))
}
} else {
row := "{{range . }}" + report.NormalizeFormat(opts.format) + "{{end}}"
data = append(data, config)
err = printTmpl(stdout, row, data)
}
if err != nil {
return fmt.Errorf("Error writing OCI-formatted configuration data to standard output: %w", err)
}
return nil
@@ -214,23 +228,23 @@ func (opts *inspectOptions) run(args []string, stdout io.Writer) (retErr error)
logrus.Warnf("Registry disallows tag list retrieval; skipping")
}
}
return opts.writeOutput(stdout, outputData)
}
// writeOutput writes data depending on opts.format to stdout
func (opts *inspectOptions) writeOutput(stdout io.Writer, data any) error {
if report.IsJSON(opts.format) || opts.format == "" {
out, err := json.MarshalIndent(data, "", " ")
out, err := json.MarshalIndent(outputData, "", " ")
if err == nil {
fmt.Fprintf(stdout, "%s\n", string(out))
}
return err
}
row := "{{range . }}" + report.NormalizeFormat(opts.format) + "{{end}}"
data = append(data, outputData)
return printTmpl(stdout, row, data)
}
rpt, err := report.New(stdout, "skopeo inspect").Parse(report.OriginUser, opts.format)
func printTmpl(stdout io.Writer, row string, data []interface{}) error {
t, err := template.New("skopeo inspect").Parse(row)
if err != nil {
return err
}
defer rpt.Flush()
return rpt.Execute([]any{data})
w := tabwriter.NewWriter(stdout, 8, 2, 2, ' ', 0)
return t.Execute(w, data)
}

View File

@@ -16,7 +16,6 @@ import (
"github.com/containers/image/v5/transports/alltransports"
"github.com/containers/image/v5/types"
"github.com/spf13/cobra"
"golang.org/x/exp/maps"
)
// tagListOutput is the output format of (skopeo list-tags), primarily so that we can format it with a simple json.MarshalIndent.
@@ -38,7 +37,10 @@ var transportHandlers = map[string]func(ctx context.Context, sys *types.SystemCo
// supportedTransports returns all the supported transports
func supportedTransports(joinStr string) string {
res := maps.Keys(transportHandlers)
res := make([]string, 0, len(transportHandlers))
for handlerName := range transportHandlers {
res = append(res, handlerName)
}
sort.Strings(res)
return strings.Join(res, joinStr)
}
@@ -82,12 +84,12 @@ func parseDockerRepositoryReference(refString string) (types.ImageReference, err
return nil, fmt.Errorf("docker: image reference %s does not start with %s://", refString, docker.Transport.Name())
}
_, dockerImageName, hasColon := strings.Cut(refString, ":")
if !hasColon {
parts := strings.SplitN(refString, ":", 2)
if len(parts) != 2 {
return nil, fmt.Errorf(`Invalid image name "%s", expected colon-separated transport:reference`, refString)
}
ref, err := reference.ParseNormalizedNamed(strings.TrimPrefix(dockerImageName, "//"))
ref, err := reference.ParseNormalizedNamed(strings.TrimPrefix(parts[1], "//"))
if err != nil {
return nil, err
}
@@ -128,7 +130,7 @@ func listDockerRepoTags(ctx context.Context, sys *types.SystemContext, opts *tag
}
// return the tagLists from a docker archive file
func listDockerArchiveTags(_ context.Context, sys *types.SystemContext, _ *tagsOptions, userInput string) (repositoryName string, tagListing []string, err error) {
func listDockerArchiveTags(ctx context.Context, sys *types.SystemContext, opts *tagsOptions, userInput string) (repositoryName string, tagListing []string, err error) {
ref, err := alltransports.ParseImageName(userInput)
if err != nil {
return

View File

@@ -16,6 +16,7 @@ func TestDockerRepositoryReferenceParser(t *testing.T) {
{"docker://somehost.com"}, // Valid default expansion
{"docker://nginx"}, // Valid default expansion
} {
ref, err := parseDockerRepositoryReference(test[0])
require.NoError(t, err)
expected, err := alltransports.ParseImageName(test[0])
@@ -46,6 +47,7 @@ func TestDockerRepositoryReferenceParserDrift(t *testing.T) {
{"docker://somehost.com", "docker.io/library/somehost.com"}, // Valid default expansion
{"docker://nginx", "docker.io/library/nginx"}, // Valid default expansion
} {
ref, err := parseDockerRepositoryReference(test[0])
ref2, err2 := alltransports.ParseImageName(test[0])

View File

@@ -1,18 +0,0 @@
package main
import (
"path/filepath"
"testing"
)
func TestLogin(t *testing.T) {
dir := t.TempDir()
authFile := filepath.Join(dir, "auth.json")
compatAuthFile := filepath.Join(dir, "config.json")
// Just a trivial smoke-test exercising one error-handling path.
// We cant test full operation without a registry, unit tests should mostly
// exist in c/common/pkg/auth, not here.
out, err := runSkopeo("login", "--authfile", authFile, "--compat-auth-file", compatAuthFile, "example.com")
assertTestFailed(t, out, err, "options for paths to the credential file and to the Docker-compatible credential file can not be set simultaneously")
}

View File

@@ -1,25 +0,0 @@
package main
import (
"os"
"path/filepath"
"testing"
"github.com/stretchr/testify/require"
)
func TestLogout(t *testing.T) {
dir := t.TempDir()
authFile := filepath.Join(dir, "auth.json")
compatAuthFile := filepath.Join(dir, "config.json")
// Just a trivial smoke-test exercising one error-handling path.
// We cant test full operation without a registry, unit tests should mostly
// exist in c/common/pkg/auth, not here.
err := os.WriteFile(authFile, []byte("{}"), 0o700)
require.NoError(t, err)
err = os.WriteFile(compatAuthFile, []byte("{}"), 0o700)
require.NoError(t, err)
out, err := runSkopeo("logout", "--authfile", authFile, "--compat-auth-file", compatAuthFile, "example.com")
assertTestFailed(t, out, err, "options for paths to the credential file and to the Docker-compatible credential file can not be set simultaneously")
}

View File

@@ -55,12 +55,14 @@ func createApp() (*cobra.Command, *globalOptions) {
opts := globalOptions{}
rootCommand := &cobra.Command{
Use: "skopeo",
Long: "Various operations with container images and container image registries",
RunE: requireSubcommand,
PersistentPreRunE: opts.before,
SilenceUsage: true,
SilenceErrors: true,
Use: "skopeo",
Long: "Various operations with container images and container image registries",
RunE: requireSubcommand,
PersistentPreRunE: func(cmd *cobra.Command, args []string) error {
return opts.before(cmd)
},
SilenceUsage: true,
SilenceErrors: true,
// Hide the completion command which is provided by cobra
CompletionOptions: cobra.CompletionOptions{HiddenDefaultCmd: true},
// This is documented to parse "local" (non-PersistentFlags) flags of parent commands before
@@ -113,7 +115,7 @@ func createApp() (*cobra.Command, *globalOptions) {
}
// before is run by the cli package for any command, before running the command-specific handler.
func (opts *globalOptions) before(cmd *cobra.Command, args []string) error {
func (opts *globalOptions) before(cmd *cobra.Command) error {
if opts.debug {
logrus.SetLevel(logrus.DebugLevel)
}

View File

@@ -95,8 +95,7 @@ import (
// 0.2.3: Added GetFullConfig
// 0.2.4: Added OpenImageOptional
// 0.2.5: Added LayerInfoJSON
// 0.2.6: Policy Verification before pulling OCI
const protocolVersion = "0.2.6"
const protocolVersion = "0.2.5"
// maxMsgSize is the current limit on a packet size.
// Note that all non-metadata (i.e. payload data) is sent over a pipe.
@@ -116,7 +115,7 @@ type request struct {
// Method is the name of the function
Method string `json:"method"`
// Args is the arguments (parsed inside the function)
Args []any `json:"args"`
Args []interface{} `json:"args"`
}
// reply is serialized to JSON as the return value from a function call.
@@ -124,7 +123,7 @@ type reply struct {
// Success is true if and only if the call succeeded.
Success bool `json:"success"`
// Value is an arbitrary value (or values, as array/map) returned from the call.
Value any `json:"value"`
Value interface{} `json:"value"`
// PipeID is an index into open pipes, and should be passed to FinishPipe
PipeID uint32 `json:"pipeid"`
// Error should be non-empty if Success == false
@@ -134,7 +133,7 @@ type reply struct {
// replyBuf is our internal deserialization of reply plus optional fd
type replyBuf struct {
// value will be converted to a reply Value
value any
value interface{}
// fd is the read half of a pipe, passed back to the client
fd *os.File
// pipeid will be provided to the client as PipeID, an index into our open pipes
@@ -155,7 +154,7 @@ type activePipe struct {
// openImage is an opened image reference
type openImage struct {
// id is an opaque integer handle
id uint64
id uint32
src types.ImageSource
cachedimg types.Image
}
@@ -170,9 +169,9 @@ type proxyHandler struct {
cache types.BlobInfoCache
// imageSerial is a counter for open images
imageSerial uint64
imageSerial uint32
// images holds our opened images
images map[uint64]*openImage
images map[uint32]*openImage
// activePipes maps from "pipeid" to a pipe + goroutine pair
activePipes map[uint32]*activePipe
}
@@ -186,7 +185,7 @@ type convertedLayerInfo struct {
}
// Initialize performs one-time initialization, and returns the protocol version
func (h *proxyHandler) Initialize(args []any) (replyBuf, error) {
func (h *proxyHandler) Initialize(args []interface{}) (replyBuf, error) {
h.lock.Lock()
defer h.lock.Unlock()
@@ -215,7 +214,7 @@ func (h *proxyHandler) Initialize(args []any) (replyBuf, error) {
// OpenImage accepts a string image reference i.e. TRANSPORT:REF - like `skopeo copy`.
// The return value is an opaque integer handle.
func (h *proxyHandler) OpenImage(args []any) (replyBuf, error) {
func (h *proxyHandler) OpenImage(args []interface{}) (replyBuf, error) {
return h.openImageImpl(args, false)
}
@@ -238,7 +237,7 @@ func isNotFoundImageError(err error) bool {
errors.Is(err, ocilayout.ImageNotFoundError{})
}
func (h *proxyHandler) openImageImpl(args []any, allowNotFound bool) (retReplyBuf replyBuf, retErr error) {
func (h *proxyHandler) openImageImpl(args []interface{}, allowNotFound bool) (replyBuf, error) {
h.lock.Lock()
defer h.lock.Unlock()
var ret replyBuf
@@ -267,25 +266,6 @@ func (h *proxyHandler) openImageImpl(args []any, allowNotFound bool) (retReplyBu
return ret, err
}
policyContext, err := h.opts.global.getPolicyContext()
if err != nil {
return ret, err
}
defer func() {
if err := policyContext.Destroy(); err != nil {
retErr = noteCloseFailure(retErr, "tearing down policy context", err)
}
}()
unparsedTopLevel := image.UnparsedInstance(imgsrc, nil)
allowed, err := policyContext.IsRunningImageAllowed(context.Background(), unparsedTopLevel)
if err != nil {
return ret, err
}
if !allowed {
return ret, fmt.Errorf("internal inconsistency: policy verification failed without returning an error")
}
// Note that we never return zero as an imageid; this code doesn't yet
// handle overflow though.
h.imageSerial++
@@ -302,11 +282,11 @@ func (h *proxyHandler) openImageImpl(args []any, allowNotFound bool) (retReplyBu
// OpenImage accepts a string image reference i.e. TRANSPORT:REF - like `skopeo copy`.
// The return value is an opaque integer handle. If the image does not exist, zero
// is returned.
func (h *proxyHandler) OpenImageOptional(args []any) (replyBuf, error) {
func (h *proxyHandler) OpenImageOptional(args []interface{}) (replyBuf, error) {
return h.openImageImpl(args, true)
}
func (h *proxyHandler) CloseImage(args []any) (replyBuf, error) {
func (h *proxyHandler) CloseImage(args []interface{}) (replyBuf, error) {
h.lock.Lock()
defer h.lock.Unlock()
var ret replyBuf
@@ -327,8 +307,16 @@ func (h *proxyHandler) CloseImage(args []any) (replyBuf, error) {
return ret, nil
}
func parseImageID(v interface{}) (uint32, error) {
imgidf, ok := v.(float64)
if !ok {
return 0, fmt.Errorf("expecting integer imageid, not %T", v)
}
return uint32(imgidf), nil
}
// parseUint64 validates that a number fits inside a JavaScript safe integer
func parseUint64(v any) (uint64, error) {
func parseUint64(v interface{}) (uint64, error) {
f, ok := v.(float64)
if !ok {
return 0, fmt.Errorf("expecting numeric, not %T", v)
@@ -339,8 +327,8 @@ func parseUint64(v any) (uint64, error) {
return uint64(f), nil
}
func (h *proxyHandler) parseImageFromID(v any) (*openImage, error) {
imgid, err := parseUint64(v)
func (h *proxyHandler) parseImageFromID(v interface{}) (*openImage, error) {
imgid, err := parseImageID(v)
if err != nil {
return nil, err
}
@@ -369,7 +357,7 @@ func (h *proxyHandler) allocPipe() (*os.File, *activePipe, error) {
// returnBytes generates a return pipe() from a byte array
// In the future it might be nicer to return this via memfd_create()
func (h *proxyHandler) returnBytes(retval any, buf []byte) (replyBuf, error) {
func (h *proxyHandler) returnBytes(retval interface{}, buf []byte) (replyBuf, error) {
var ret replyBuf
piper, f, err := h.allocPipe()
if err != nil {
@@ -431,7 +419,7 @@ func (h *proxyHandler) cacheTargetManifest(img *openImage) error {
// GetManifest returns a copy of the manifest, converted to OCI format, along with the original digest.
// Manifest lists are resolved to the current operating system and architecture.
func (h *proxyHandler) GetManifest(args []any) (replyBuf, error) {
func (h *proxyHandler) GetManifest(args []interface{}) (replyBuf, error) {
h.lock.Lock()
defer h.lock.Unlock()
@@ -502,7 +490,7 @@ func (h *proxyHandler) GetManifest(args []any) (replyBuf, error) {
// GetFullConfig returns a copy of the image configuration, converted to OCI format.
// https://github.com/opencontainers/image-spec/blob/main/config.md
func (h *proxyHandler) GetFullConfig(args []any) (replyBuf, error) {
func (h *proxyHandler) GetFullConfig(args []interface{}) (replyBuf, error) {
h.lock.Lock()
defer h.lock.Unlock()
@@ -539,7 +527,7 @@ func (h *proxyHandler) GetFullConfig(args []any) (replyBuf, error) {
// GetConfig returns a copy of the container runtime configuration, converted to OCI format.
// Note that due to a historical mistake, this returns not the full image configuration,
// but just the container runtime configuration. You should use GetFullConfig instead.
func (h *proxyHandler) GetConfig(args []any) (replyBuf, error) {
func (h *proxyHandler) GetConfig(args []interface{}) (replyBuf, error) {
h.lock.Lock()
defer h.lock.Unlock()
@@ -574,7 +562,7 @@ func (h *proxyHandler) GetConfig(args []any) (replyBuf, error) {
}
// GetBlob fetches a blob, performing digest verification.
func (h *proxyHandler) GetBlob(args []any) (replyBuf, error) {
func (h *proxyHandler) GetBlob(args []interface{}) (replyBuf, error) {
h.lock.Lock()
defer h.lock.Unlock()
@@ -644,7 +632,7 @@ func (h *proxyHandler) GetBlob(args []any) (replyBuf, error) {
// This needs to be called since the data returned by GetManifest() does not allow to correctly
// calling GetBlob() for the containers-storage: transport (which doesnt store the original compressed
// representations referenced in the manifest).
func (h *proxyHandler) GetLayerInfo(args []any) (replyBuf, error) {
func (h *proxyHandler) GetLayerInfo(args []interface{}) (replyBuf, error) {
h.lock.Lock()
defer h.lock.Unlock()
@@ -680,7 +668,7 @@ func (h *proxyHandler) GetLayerInfo(args []any) (replyBuf, error) {
layerInfos = img.LayerInfos()
}
layers := make([]convertedLayerInfo, 0, len(layerInfos))
var layers []convertedLayerInfo
for _, layer := range layerInfos {
layers = append(layers, convertedLayerInfo{layer.Digest, layer.Size, layer.MediaType})
}
@@ -690,7 +678,7 @@ func (h *proxyHandler) GetLayerInfo(args []any) (replyBuf, error) {
}
// FinishPipe waits for the worker goroutine to finish, and closes the write side of the pipe.
func (h *proxyHandler) FinishPipe(args []any) (replyBuf, error) {
func (h *proxyHandler) FinishPipe(args []interface{}) (replyBuf, error) {
h.lock.Lock()
defer h.lock.Unlock()
@@ -840,7 +828,7 @@ func (h *proxyHandler) processRequest(readBytes []byte) (rb replyBuf, terminate
func (opts *proxyOptions) run(args []string, stdout io.Writer) error {
handler := &proxyHandler{
opts: opts,
images: make(map[uint64]*openImage),
images: make(map[uint32]*openImage),
activePipes: make(map[uint32]*activePipe),
}
defer handler.close()

View File

@@ -6,7 +6,6 @@ import (
"fmt"
"io"
"os"
"strings"
"github.com/containers/image/v5/pkg/cli"
"github.com/containers/image/v5/signature"
@@ -42,12 +41,12 @@ func (opts *standaloneSignOptions) run(args []string, stdout io.Writer) error {
manifest, err := os.ReadFile(manifestPath)
if err != nil {
return fmt.Errorf("Error reading %s: %w", manifestPath, err)
return fmt.Errorf("Error reading %s: %v", manifestPath, err)
}
mech, err := signature.NewGPGSigningMechanism()
if err != nil {
return fmt.Errorf("Error initializing GPG: %w", err)
return fmt.Errorf("Error initializing GPG: %v", err)
}
defer mech.Close()
@@ -58,31 +57,25 @@ func (opts *standaloneSignOptions) run(args []string, stdout io.Writer) error {
signature, err := signature.SignDockerManifestWithOptions(manifest, dockerReference, mech, fingerprint, &signature.SignOptions{Passphrase: passphrase})
if err != nil {
return fmt.Errorf("Error creating signature: %w", err)
return fmt.Errorf("Error creating signature: %v", err)
}
if err := os.WriteFile(opts.output, signature, 0644); err != nil {
return fmt.Errorf("Error writing signature to %s: %w", opts.output, err)
return fmt.Errorf("Error writing signature to %s: %v", opts.output, err)
}
return nil
}
type standaloneVerifyOptions struct {
publicKeyFile string
}
func standaloneVerifyCmd() *cobra.Command {
opts := standaloneVerifyOptions{}
cmd := &cobra.Command{
Use: "standalone-verify MANIFEST DOCKER-REFERENCE KEY-FINGERPRINTS SIGNATURE",
Use: "standalone-verify MANIFEST DOCKER-REFERENCE KEY-FINGERPRINT SIGNATURE",
Short: "Verify a signature using local files",
Long: `Verify a signature using local files
KEY-FINGERPRINTS can be a comma separated list of fingerprints, or "any" if you trust all the keys in the public key file.`,
RunE: commandAction(opts.run),
RunE: commandAction(opts.run),
}
flags := cmd.Flags()
flags.StringVar(&opts.publicKeyFile, "public-key-file", "", `File containing public keys. If not specified, will use local GPG keys.`)
adjustUsage(cmd)
return cmd
}
@@ -93,51 +86,29 @@ func (opts *standaloneVerifyOptions) run(args []string, stdout io.Writer) error
}
manifestPath := args[0]
expectedDockerReference := args[1]
expectedFingerprints := strings.Split(args[2], ",")
expectedFingerprint := args[2]
signaturePath := args[3]
if opts.publicKeyFile == "" && len(expectedFingerprints) == 1 && expectedFingerprints[0] == "any" {
return fmt.Errorf("Cannot use any fingerprint without a public key file")
}
unverifiedManifest, err := os.ReadFile(manifestPath)
if err != nil {
return fmt.Errorf("Error reading manifest from %s: %w", manifestPath, err)
return fmt.Errorf("Error reading manifest from %s: %v", manifestPath, err)
}
unverifiedSignature, err := os.ReadFile(signaturePath)
if err != nil {
return fmt.Errorf("Error reading signature from %s: %w", signaturePath, err)
return fmt.Errorf("Error reading signature from %s: %v", signaturePath, err)
}
var mech signature.SigningMechanism
var publicKeyfingerprints []string
if opts.publicKeyFile != "" {
publicKeys, err := os.ReadFile(opts.publicKeyFile)
if err != nil {
return fmt.Errorf("Error reading public keys from %s: %w", opts.publicKeyFile, err)
}
mech, publicKeyfingerprints, err = signature.NewEphemeralGPGSigningMechanism(publicKeys)
if err != nil {
return fmt.Errorf("Error initializing GPG: %w", err)
}
} else {
mech, err = signature.NewGPGSigningMechanism()
if err != nil {
return fmt.Errorf("Error initializing GPG: %w", err)
}
mech, err := signature.NewGPGSigningMechanism()
if err != nil {
return fmt.Errorf("Error initializing GPG: %v", err)
}
defer mech.Close()
if len(expectedFingerprints) == 1 && expectedFingerprints[0] == "any" {
expectedFingerprints = publicKeyfingerprints
}
sig, verificationFingerprint, err := signature.VerifyImageManifestSignatureUsingKeyIdentityList(unverifiedSignature, unverifiedManifest, expectedDockerReference, mech, expectedFingerprints)
sig, err := signature.VerifyDockerManifestSignature(unverifiedSignature, unverifiedManifest, expectedDockerReference, mech, expectedFingerprint)
if err != nil {
return fmt.Errorf("Error verifying signature: %w", err)
return fmt.Errorf("Error verifying signature: %v", err)
}
fmt.Fprintf(stdout, "Signature verified using fingerprint %s, digest %s\n", verificationFingerprint, sig.DockerManifestDigest)
fmt.Fprintf(stdout, "Signature verified, digest %s\n", sig.DockerManifestDigest)
return nil
}
@@ -170,7 +141,7 @@ func (opts *untrustedSignatureDumpOptions) run(args []string, stdout io.Writer)
untrustedSignature, err := os.ReadFile(untrustedSignaturePath)
if err != nil {
return fmt.Errorf("Error reading untrusted signature from %s: %w", untrustedSignaturePath, err)
return fmt.Errorf("Error reading untrusted signature from %s: %v", untrustedSignaturePath, err)
}
untrustedInfo, err := signature.GetUntrustedSignatureInformationWithoutVerifying(untrustedSignature)

View File

@@ -127,36 +127,11 @@ func TestStandaloneVerify(t *testing.T) {
dockerReference, fixturesTestKeyFingerprint, "fixtures/corrupt.signature")
assertTestFailed(t, out, err, "Error verifying signature")
// Error using any without a public key file
out, err = runSkopeo("standalone-verify", manifestPath,
dockerReference, "any", signaturePath)
assertTestFailed(t, out, err, "Cannot use any fingerprint without a public key file")
// Success
out, err = runSkopeo("standalone-verify", manifestPath,
dockerReference, fixturesTestKeyFingerprint, signaturePath)
assert.NoError(t, err)
assert.Equal(t, "Signature verified using fingerprint "+fixturesTestKeyFingerprint+", digest "+fixturesTestImageManifestDigest.String()+"\n", out)
// Using multiple fingerprints
out, err = runSkopeo("standalone-verify", manifestPath,
dockerReference, "0123456789ABCDEF0123456789ABCDEF01234567,"+fixturesTestKeyFingerprint+",DEADBEEFDEADBEEFDEADBEEFDEADBEEFDEADBEEF", signaturePath)
assert.NoError(t, err)
assert.Equal(t, "Signature verified using fingerprint "+fixturesTestKeyFingerprint+", digest "+fixturesTestImageManifestDigest.String()+"\n", out)
// Using a public key file
t.Setenv("GNUPGHOME", "")
out, err = runSkopeo("standalone-verify", "--public-key-file", "fixtures/pubring.gpg", manifestPath,
dockerReference, fixturesTestKeyFingerprint, signaturePath)
assert.NoError(t, err)
assert.Equal(t, "Signature verified using fingerprint "+fixturesTestKeyFingerprint+", digest "+fixturesTestImageManifestDigest.String()+"\n", out)
// Using a public key file matching any public key
t.Setenv("GNUPGHOME", "")
out, err = runSkopeo("standalone-verify", "--public-key-file", "fixtures/pubring.gpg", manifestPath,
dockerReference, "any", signaturePath)
assert.NoError(t, err)
assert.Equal(t, "Signature verified using fingerprint "+fixturesTestKeyFingerprint+", digest "+fixturesTestImageManifestDigest.String()+"\n", out)
assert.Equal(t, "Signature verified, digest "+fixturesTestImageManifestDigest.String()+"\n", out)
}
func TestUntrustedSignatureDump(t *testing.T) {

View File

@@ -26,8 +26,7 @@ import (
"github.com/opencontainers/go-digest"
"github.com/sirupsen/logrus"
"github.com/spf13/cobra"
"golang.org/x/exp/slices"
"gopkg.in/yaml.v3"
"gopkg.in/yaml.v2"
)
// syncOptions contains information retrieved from the skopeo sync command line.
@@ -132,12 +131,12 @@ See skopeo-sync(1) for details.
}
// UnmarshalYAML is the implementation of the Unmarshaler interface method
// for the tlsVerifyConfig type.
// method for the tlsVerifyConfig type.
// It unmarshals the 'tls-verify' YAML key so that, when they key is not
// specified, tls verification is enforced.
func (tls *tlsVerifyConfig) UnmarshalYAML(value *yaml.Node) error {
func (tls *tlsVerifyConfig) UnmarshalYAML(unmarshal func(interface{}) error) error {
var verify bool
if err := value.Decode(&verify); err != nil {
if err := unmarshal(&verify); err != nil {
return err
}
@@ -524,17 +523,26 @@ func (opts *syncOptions) run(args []string, stdout io.Writer) (retErr error) {
}()
// validate source and destination options
contains := func(val string, list []string) (_ bool) {
for _, l := range list {
if l == val {
return true
}
}
return
}
if len(opts.source) == 0 {
return errors.New("A source transport must be specified")
}
if !slices.Contains([]string{docker.Transport.Name(), directory.Transport.Name(), "yaml"}, opts.source) {
if !contains(opts.source, []string{docker.Transport.Name(), directory.Transport.Name(), "yaml"}) {
return fmt.Errorf("%q is not a valid source transport", opts.source)
}
if len(opts.destination) == 0 {
return errors.New("A destination transport must be specified")
}
if !slices.Contains([]string{docker.Transport.Name(), directory.Transport.Name()}, opts.destination) {
if !contains(opts.destination, []string{docker.Transport.Name(), directory.Transport.Name()}) {
return fmt.Errorf("%q is not a valid destination transport", opts.destination)
}

View File

@@ -1,46 +0,0 @@
package main
import (
"testing"
"github.com/containers/image/v5/types"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"gopkg.in/yaml.v3"
)
var _ yaml.Unmarshaler = (*tlsVerifyConfig)(nil)
func TestTLSVerifyConfig(t *testing.T) {
type container struct { // An example of a larger config file
TLSVerify tlsVerifyConfig `yaml:"tls-verify"`
}
for _, c := range []struct {
input string
expected tlsVerifyConfig
}{
{
input: `tls-verify: true`,
expected: tlsVerifyConfig{skip: types.OptionalBoolFalse},
},
{
input: `tls-verify: false`,
expected: tlsVerifyConfig{skip: types.OptionalBoolTrue},
},
{
input: ``, // No value
expected: tlsVerifyConfig{skip: types.OptionalBoolUndefined},
},
} {
config := container{}
err := yaml.Unmarshal([]byte(c.input), &config)
require.NoError(t, err, c.input)
assert.Equal(t, c.expected, config.TLSVerify, c.input)
}
// Invalid input
config := container{}
err := yaml.Unmarshal([]byte(`tls-verify: "not a valid bool"`), &config)
assert.Error(t, err)
}

View File

@@ -6,7 +6,6 @@ import (
"github.com/containers/image/v5/transports/alltransports"
"github.com/containers/storage/pkg/unshare"
"github.com/syndtr/gocapability/capability"
"golang.org/x/exp/slices"
)
var neededCapabilities = []capability.Cap{
@@ -22,32 +21,29 @@ func maybeReexec() error {
// With Skopeo we need only the subset of the root capabilities necessary
// for pulling an image to the storage. Do not attempt to create a namespace
// if we already have the capabilities we need.
capabilities, err := capability.NewPid2(0)
capabilities, err := capability.NewPid(0)
if err != nil {
return fmt.Errorf("error reading the current capabilities sets: %w", err)
}
if err := capabilities.Load(); err != nil {
return fmt.Errorf("error loading the current capabilities sets: %w", err)
}
if slices.ContainsFunc(neededCapabilities, func(cap capability.Cap) bool {
return !capabilities.Get(capability.EFFECTIVE, cap)
}) {
// We miss a capability we need, create a user namespaces
unshare.MaybeReexecUsingUserNamespace(true)
return nil
for _, cap := range neededCapabilities {
if !capabilities.Get(capability.EFFECTIVE, cap) {
// We miss a capability we need, create a user namespaces
unshare.MaybeReexecUsingUserNamespace(true)
return nil
}
}
return nil
}
func reexecIfNecessaryForImages(imageNames ...string) error {
// Check if container-storage is used before doing unshare
if slices.ContainsFunc(imageNames, func(imageName string) bool {
for _, imageName := range imageNames {
transport := alltransports.TransportFromImageName(imageName)
// Hard-code the storage name to avoid a reference on c/image/storage.
// See https://github.com/containers/skopeo/issues/771#issuecomment-563125006.
return transport != nil && transport.Name() == "containers-storage"
}) {
return maybeReexec()
if transport != nil && transport.Name() == "containers-storage" {
return maybeReexec()
}
}
return nil
}

View File

@@ -44,7 +44,7 @@ func noteCloseFailure(err error, description string, closeErr error) error {
if err == nil {
return fmt.Errorf("%s: %w", description, closeErr)
}
// In this case we prioritize the primary error for use with %w; closeErr is usually less relevant, or might be a consequence of the primary error.
// In this case we prioritize the primary error for use with %w; closeErr is usually less relevant, or might be a consequence of the primary erorr.
return fmt.Errorf("%w (%s: %v)", err, description, closeErr)
}
@@ -315,11 +315,14 @@ func parseCreds(creds string) (string, string, error) {
if creds == "" {
return "", "", errors.New("credentials can't be empty")
}
username, password, _ := strings.Cut(creds, ":") // Sets password to "" if there is no ":"
if username == "" {
up := strings.SplitN(creds, ":", 2)
if len(up) == 1 {
return up[0], "", nil
}
if up[0] == "" {
return "", "", errors.New("username can't be empty")
}
return username, password, nil
return up[0], up[1], nil
}
func getDockerAuth(creds string) (*types.DockerAuthConfig, error) {

View File

@@ -385,6 +385,7 @@ func TestParseManifestFormat(t *testing.T) {
// since there is a shared authfile image option and a non-shared (prefixed) one, make sure the override logic
// works correctly.
func TestImageOptionsAuthfileOverride(t *testing.T) {
for _, testCase := range []struct {
flagPrefix string
cmdFlags []string

View File

@@ -6,6 +6,17 @@
set -e
_EOL=20270501
if [[ $(date +%Y%m%d) -ge $_EOL ]]; then
die "As of $_EOL this branch is probably
no longer supported in RHEL 9.2/8.8, please
confirm this with RHEL Program Management. If so:
It should be removed from Cirrus-Cron,
the .cirrus.yml file removed, and
the VM images (manually) unmarked
'permanent=true'"
fi
# BEGIN Global export of all variables
set -a

View File

@@ -1,6 +1,6 @@
[comment]: <> (***ATTENTION*** ***WARNING*** ***ALERT*** ***CAUTION*** ***DANGER***)
[comment]: <> ()
[comment]: <> (ANY changes made to this file, once committed/merged must)
[comment]: <> (ANY changes made to this file, once commited/merged must)
[comment]: <> (be manually copy/pasted -in markdown- into the description)
[comment]: <> (field on Quay at the following locations:)
[comment]: <> ()
@@ -25,7 +25,9 @@ the images live are public and can be pulled without credentials. These contain
resulting containers can run safely with privileges within the container.
The container images are built using the latest Fedora and then Skopeo is installed into them.
The ENTRYPOINT of the container is set to execute the `skopeo` binary.
The PATH in the container images is set to the default PATH provided by Fedora. Also, the
ENTRYPOINT and the WORKDIR variables are not set within these container images, as such they
default to `/`.
The container images are:

View File

@@ -20,8 +20,6 @@ automatically inherit any parts of the source name.
## OPTIONS
See also [skopeo(1)](skopeo.1.md) for options placed before the subcommand name.
**--additional-tag**=_strings_
Additional tags (supports docker-archive).
@@ -58,9 +56,7 @@ After copying the image, write the digest of the resulting image to the file.
**--preserve-digests**
Preserve the digests during copying. Fail if the digest cannot be preserved.
This option does not change what will be copied; consider using `--all` at the same time.
Preserve the digests during copying. Fail if the digest cannot be preserved. Consider using `--all` at the same time.
**--encrypt-layer** _ints_
@@ -182,7 +178,7 @@ Existing signatures, if any, are preserved as well.
**--dest-compress-format** _format_
Specifies the compression format to use. Supported values are: `gzip`, `zstd` and `zstd:chunked`.
Specifies the compression format to use. Supported values are: `gzip` and `zstd`.
**--dest-compress-level** _format_

View File

@@ -31,8 +31,6 @@ $ docker exec -it registry /usr/bin/registry garbage-collect /etc/docker-distrib
## OPTIONS
See also [skopeo(1)](skopeo.1.md) for options placed before the subcommand name.
**--authfile** _path_
Path of the authentication file. Default is ${XDG_RUNTIME\_DIR}/containers/auth.json, which is set using `skopeo login`.

View File

@@ -17,8 +17,6 @@ The private key is written to _prefix_**.pub** .
## OPTIONS
See also [skopeo(1)](skopeo.1.md) for options placed before the subcommand name.
**--help**, **-h**
Print usage statement

View File

@@ -17,8 +17,6 @@ To see values for a different architecture/OS, use the **--override-os** / **--o
## OPTIONS
See also [skopeo(1)](skopeo.1.md) for options placed before the subcommand name.
**--authfile** _path_
Path of the authentication file. Default is ${XDG\_RUNTIME\_DIR}/containers/auth.json, which is set using `skopeo login`.
@@ -44,7 +42,6 @@ Use docker daemon host at _host_ (`docker-daemon:` transport only)
Format the output using the given Go template.
The keys of the returned JSON can be used as the values for the --format flag (see examples below).
Supports the Go templating functions available at https://pkg.go.dev/github.com/containers/common/pkg/report#hdr-Template_Functions
**--help**, **-h**

View File

@@ -12,8 +12,6 @@ Return a list of tags from _source-image_ in a registry or a local docker-archiv
## OPTIONS
See also [skopeo(1)](skopeo.1.md) for options placed before the subcommand name.
**--authfile** _path_
Path of the authentication file. Default is ${XDG\_RUNTIME\_DIR}/containers/auth.json, which is set using `skopeo login`.

View File

@@ -15,8 +15,6 @@ flag. The default path used is **${XDG\_RUNTIME\_DIR}/containers/auth.json**.
## OPTIONS
See also [skopeo(1)](skopeo.1.md) for options placed before the subcommand name.
**--password**, **-p**=*password*
Password for registry
@@ -36,10 +34,6 @@ Path of the authentication file. Default is ${XDG\_RUNTIME\_DIR}/containers/auth
Note: You can also override the default path of the authentication file by setting the REGISTRY\_AUTH\_FILE
environment variable. `export REGISTRY_AUTH_FILE=path`
**--compat-auth-file**=*path*
Instead of updating the default credentials file, update the one at *path*, and use a Docker-compatible format.
**--get-login**
Return the logged-in user for the registry. Return error if no login is found.

View File

@@ -14,8 +14,6 @@ All the cached credentials can be removed by setting the **all** flag.
## OPTIONS
See also [skopeo(1)](skopeo.1.md) for options placed before the subcommand name.
**--authfile**=*path*
Path of the authentication file. Default is ${XDG\_RUNTIME\_DIR}/containers/auth.json
@@ -23,10 +21,6 @@ Path of the authentication file. Default is ${XDG\_RUNTIME\_DIR}/containers/auth
Note: You can also override the default path of the authentication file by setting the REGISTRY\_AUTH\_FILE
environment variable. `export REGISTRY_AUTH_FILE=path`
**--compat-auth-file**=*path*
Instead of updating the default credentials file, update the one at *path*, and use a Docker-compatible format.
**--all**, **-a**
Remove the cached credentials for all registries in the auth file

View File

@@ -17,8 +17,6 @@ This is primarily a debugging tool, useful for special cases, and usually should
## OPTIONS
See also [skopeo(1)](skopeo.1.md) for options placed before the subcommand name.
**--help**, **-h**
Print usage statement

View File

@@ -4,7 +4,7 @@
skopeo\-standalone\-verify - Verify an image signature.
## SYNOPSIS
**skopeo standalone-verify** _manifest_ _docker-reference_ _key-fingerprints_ _signature_
**skopeo standalone-verify** _manifest_ _docker-reference_ _key-fingerprint_ _signature_
## DESCRIPTION
@@ -16,7 +16,7 @@ as per containers-policy.json(5).
_docker-reference_ A docker reference expected to identify the image in the signature
_key-fingerprints_ Identities of trusted signing keys (comma separated), or "any" to trust any known key when using a public key file
_key-fingerprint_ Expected identity of the signing key
_signature_ Path to signature file
@@ -24,16 +24,10 @@ as per containers-policy.json(5).
## OPTIONS
See also [skopeo(1)](skopeo.1.md) for options placed before the subcommand name.
**--help**, **-h**
Print usage statement
**--public-key-file** _public key file_
File containing the public keys to use when verifying signatures. If this is not specified, keys from the GPG homedir are used.
## EXAMPLES
```console

View File

@@ -8,7 +8,10 @@ skopeo\-sync - Synchronize images between registry repositories and local direct
**skopeo sync** [*options*] --src _transport_ --dest _transport_ _source_ _destination_
## DESCRIPTION
Synchronize images between registry repositories and local directories. Synchronization is achieved by copying all the images found at _source_ to _destination_ - useful when synchronizing a local container registry mirror or for populating registries running inside of air-gapped environments.
Synchronize images between registry repoositories and local directories.
The synchronization is achieved by copying all the images found at _source_ to _destination_.
Useful to synchronize a local container registry mirror, and to to populate registries running inside of air-gapped environments.
Differently from other skopeo commands, skopeo sync requires both source and destination transports to be specified separately from _source_ and _destination_.
One of the problems of prefixing a destination with its transport is that, the registry `docker://hostname:port` would be wrongly interpreted as an image reference at a non-fully qualified registry, with `hostname` and `port` the image name and tag.
@@ -29,9 +32,6 @@ When the `--scoped` option is specified, images are prefixed with the source ima
name can be stored at _destination_.
## OPTIONS
See also [skopeo(1)](skopeo.1.md) for options placed before the subcommand name.
**--all**, **-a**
If one of the images in __src__ refers to a list of images, instead of copying just the image which matches the current OS and
architecture (subject to the use of the global --override-os, --override-arch and --override-variant options), attempt to copy all of
@@ -68,11 +68,7 @@ Print usage statement.
**--append-suffix** _tag-suffix_ String to append to destination tags.
**--preserve-digests**
Preserve the digests during copying. Fail if the digest cannot be preserved.
This option does not change what will be copied; consider using `--all` at the same time.
**--preserve-digests** Preserve the digests during copying. Fail if the digest cannot be preserved. Consider using `--all` at the same time.
**--remove-signatures** Do not copy signatures, if any, from _source-image_. This is necessary when copying a signed image to a destination which does not support signatures.

View File

@@ -51,9 +51,6 @@ See [containers-transports(5)](https://github.com/containers/image/blob/main/doc
## OPTIONS
These options should be placed before the subcommand name.
Individual subcommands have their own options.
**--command-timeout** _duration_
Timeout for the command execution.
@@ -121,7 +118,7 @@ Print the version number
**/etc/containers/registries.d**
Default directory containing registry configuration, if **--registries.d** is not specified.
The contents of this directory are documented in [containers-registries.d(5)](https://github.com/containers/image/blob/main/docs/containers-registries.d.5.md).
The contents of this directory are documented in [containers-policy.json(5)](https://github.com/containers/image/blob/main/docs/containers-policy.json.5.md).
## SEE ALSO
skopeo-login(1), docker-login(1), containers-auth.json(5), containers-storage.conf(5), containers-policy.json(5), containers-transports(5)

147
go.mod
View File

@@ -1,133 +1,136 @@
module github.com/containers/skopeo
go 1.19
go 1.17
require (
github.com/containers/common v0.57.6
github.com/containers/image/v5 v5.29.4
github.com/containers/common v0.51.4
github.com/containers/image/v5 v5.24.3
github.com/containers/ocicrypt v1.1.10
github.com/containers/storage v1.51.0
github.com/docker/distribution v2.8.3+incompatible
github.com/containers/storage v1.45.3
github.com/docker/distribution v2.8.1+incompatible
github.com/opencontainers/go-digest v1.0.0
github.com/opencontainers/image-spec v1.1.0-rc5
github.com/opencontainers/image-spec v1.1.0-rc2
github.com/opencontainers/image-tools v1.0.0-rc3
github.com/sirupsen/logrus v1.9.3
github.com/spf13/cobra v1.8.0
github.com/sirupsen/logrus v1.9.0
github.com/spf13/cobra v1.6.1
github.com/spf13/pflag v1.0.5
github.com/stretchr/testify v1.8.4
github.com/stretchr/testify v1.8.1
github.com/syndtr/gocapability v0.0.0-20200815063812-42c35b437635
golang.org/x/exp v0.0.0-20231226003508-02704c960a9b
golang.org/x/term v0.17.0
gopkg.in/yaml.v3 v3.0.1
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c
gopkg.in/yaml.v2 v2.4.0
)
require (
dario.cat/mergo v1.0.0 // indirect
github.com/BurntSushi/toml v1.3.2 // indirect
github.com/Microsoft/go-winio v0.6.1 // indirect
github.com/Microsoft/hcsshim v0.12.0-rc.1 // indirect
github.com/BurntSushi/toml v1.2.1 // indirect
github.com/Microsoft/go-winio v0.6.0 // indirect
github.com/Microsoft/hcsshim v0.9.6 // indirect
github.com/VividCortex/ewma v1.2.0 // indirect
github.com/acarl005/stripansi v0.0.0-20180116102854-5a71ef0e047d // indirect
github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2 // indirect
github.com/containerd/cgroups/v3 v3.0.2 // indirect
github.com/containerd/containerd v1.7.9 // indirect
github.com/containerd/stargz-snapshotter/estargz v0.15.1 // indirect
github.com/asaskevich/govalidator v0.0.0-20210307081110-f21760c49a8d // indirect
github.com/containerd/cgroups v1.0.4 // indirect
github.com/containerd/stargz-snapshotter/estargz v0.13.0 // indirect
github.com/containers/libtrust v0.0.0-20230121012942-c1716e8a8d01 // indirect
github.com/coreos/go-oidc/v3 v3.7.0 // indirect
github.com/cyberphone/json-canonicalization v0.0.0-20231011164504-785e29786b46 // indirect
github.com/cyphar/filepath-securejoin v0.2.4 // indirect
github.com/coreos/go-oidc/v3 v3.5.0 // indirect
github.com/cyberphone/json-canonicalization v0.0.0-20220623050100-57a0ce2678a7 // indirect
github.com/cyphar/filepath-securejoin v0.2.3 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/distribution/reference v0.5.0 // indirect
github.com/docker/docker v24.0.7+incompatible // indirect
github.com/docker/docker-credential-helpers v0.8.0 // indirect
github.com/docker/docker v20.10.23+incompatible // indirect
github.com/docker/docker-credential-helpers v0.7.0 // indirect
github.com/docker/go-connections v0.4.0 // indirect
github.com/docker/go-units v0.5.0 // indirect
github.com/dsnet/compress v0.0.2-0.20210315054119-f66993602bf5 // indirect
github.com/ghodss/yaml v1.0.0 // indirect
github.com/go-jose/go-jose/v3 v3.0.3 // indirect
github.com/go-logr/logr v1.3.0 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-openapi/analysis v0.21.4 // indirect
github.com/go-openapi/errors v0.20.4 // indirect
github.com/go-openapi/jsonpointer v0.19.6 // indirect
github.com/go-openapi/jsonreference v0.20.2 // indirect
github.com/go-openapi/errors v0.20.3 // indirect
github.com/go-openapi/jsonpointer v0.19.5 // indirect
github.com/go-openapi/jsonreference v0.20.0 // indirect
github.com/go-openapi/loads v0.21.2 // indirect
github.com/go-openapi/runtime v0.26.0 // indirect
github.com/go-openapi/spec v0.20.9 // indirect
github.com/go-openapi/strfmt v0.21.7 // indirect
github.com/go-openapi/swag v0.22.4 // indirect
github.com/go-openapi/validate v0.22.1 // indirect
github.com/go-openapi/runtime v0.24.1 // indirect
github.com/go-openapi/spec v0.20.7 // indirect
github.com/go-openapi/strfmt v0.21.3 // indirect
github.com/go-openapi/swag v0.22.3 // indirect
github.com/go-openapi/validate v0.22.0 // indirect
github.com/go-playground/locales v0.14.0 // indirect
github.com/go-playground/universal-translator v0.18.0 // indirect
github.com/go-playground/validator/v10 v10.11.1 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/protobuf v1.5.3 // indirect
github.com/google/go-containerregistry v0.16.1 // indirect
github.com/google/go-containerregistry v0.13.0 // indirect
github.com/google/go-intervals v0.0.2 // indirect
github.com/google/uuid v1.3.1 // indirect
github.com/google/trillian v1.5.0 // indirect
github.com/google/uuid v1.3.0 // indirect
github.com/gorilla/mux v1.8.0 // indirect
github.com/hashicorp/errwrap v1.1.0 // indirect
github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
github.com/hashicorp/go-multierror v1.1.1 // indirect
github.com/hashicorp/go-retryablehttp v0.7.5 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/hashicorp/go-retryablehttp v0.7.2 // indirect
github.com/imdario/mergo v0.3.13 // indirect
github.com/inconshreveable/mousetrap v1.0.1 // indirect
github.com/josharian/intern v1.0.0 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/klauspost/compress v1.17.3 // indirect
github.com/klauspost/pgzip v1.2.6 // indirect
github.com/letsencrypt/boulder v0.0.0-20230213213521-fdfea0d469b6 // indirect
github.com/klauspost/compress v1.15.15 // indirect
github.com/klauspost/pgzip v1.2.6-0.20220930104621-17e8dac29df8 // indirect
github.com/kr/pretty v0.3.0 // indirect
github.com/kr/text v0.2.0 // indirect
github.com/leodido/go-urn v1.2.1 // indirect
github.com/letsencrypt/boulder v0.0.0-20230130200452-c091e64aa391 // indirect
github.com/mailru/easyjson v0.7.7 // indirect
github.com/mattn/go-runewidth v0.0.15 // indirect
github.com/mattn/go-runewidth v0.0.14 // indirect
github.com/mattn/go-shellwords v1.0.12 // indirect
github.com/mattn/go-sqlite3 v1.14.18 // indirect
github.com/matttproud/golang_protobuf_extensions v1.0.4 // indirect
github.com/miekg/pkcs11 v1.1.1 // indirect
github.com/mistifyio/go-zfs/v3 v3.0.1 // indirect
github.com/mistifyio/go-zfs/v3 v3.0.0 // indirect
github.com/mitchellh/mapstructure v1.5.0 // indirect
github.com/moby/sys/mountinfo v0.7.1 // indirect
github.com/moby/sys/mountinfo v0.6.2 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/oklog/ulid v1.3.1 // indirect
github.com/opencontainers/runc v1.1.10 // indirect
github.com/opencontainers/runtime-spec v1.1.0 // indirect
github.com/opencontainers/selinux v1.11.0 // indirect
github.com/opencontainers/runc v1.1.4 // indirect
github.com/opencontainers/runtime-spec v1.0.3-0.20220825212826-86290f6a00fb // indirect
github.com/opencontainers/selinux v1.10.2 // indirect
github.com/opentracing/opentracing-go v1.2.0 // indirect
github.com/ostreedev/ostree-go v0.0.0-20210805093236-719684c64e4f // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/proglottis/gpgme v0.1.3 // indirect
github.com/rivo/uniseg v0.4.4 // indirect
github.com/rivo/uniseg v0.4.3 // indirect
github.com/rogpeppe/go-internal v1.8.0 // indirect
github.com/russross/blackfriday v2.0.0+incompatible // indirect
github.com/secure-systems-lab/go-securesystemslib v0.7.0 // indirect
github.com/segmentio/ksuid v1.0.4 // indirect
github.com/sigstore/fulcio v1.4.3 // indirect
github.com/sigstore/rekor v1.2.2 // indirect
github.com/sigstore/sigstore v1.7.5 // indirect
github.com/sigstore/fulcio v1.0.0 // indirect
github.com/sigstore/rekor v1.0.1 // indirect
github.com/sigstore/sigstore v1.5.2 // indirect
github.com/skratchdot/open-golang v0.0.0-20200116055534-eef842397966 // indirect
github.com/stefanberger/go-pkcs11uri v0.0.0-20201008174630-78d3cae3a980 // indirect
github.com/sylabs/sif/v2 v2.15.0 // indirect
github.com/tchap/go-patricia/v2 v2.3.1 // indirect
github.com/sylabs/sif/v2 v2.9.0 // indirect
github.com/tchap/go-patricia v2.3.0+incompatible // indirect
github.com/theupdateframework/go-tuf v0.5.2-0.20221207161717-9cb61d6e65f5 // indirect
github.com/titanous/rocacheck v0.0.0-20171023193734-afe73141d399 // indirect
github.com/ulikunitz/xz v0.5.11 // indirect
github.com/vbatts/tar-split v0.11.5 // indirect
github.com/vbauerster/mpb/v8 v8.6.2 // indirect
github.com/vbatts/tar-split v0.11.2 // indirect
github.com/vbauerster/mpb/v7 v7.5.3 // indirect
github.com/xeipuuv/gojsonpointer v0.0.0-20190905194746-02993c407bfb // indirect
github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 // indirect
github.com/xeipuuv/gojsonschema v1.2.0 // indirect
go.mongodb.org/mongo-driver v1.11.3 // indirect
go.etcd.io/bbolt v1.3.6 // indirect
go.mongodb.org/mongo-driver v1.11.1 // indirect
go.mozilla.org/pkcs7 v0.0.0-20210826202110-33d05740a352 // indirect
go.opencensus.io v0.24.0 // indirect
go.opentelemetry.io/otel v1.19.0 // indirect
go.opentelemetry.io/otel/metric v1.19.0 // indirect
go.opentelemetry.io/otel/trace v1.19.0 // indirect
golang.org/x/crypto v0.19.0 // indirect
golang.org/x/mod v0.14.0 // indirect
golang.org/x/net v0.19.0 // indirect
golang.org/x/oauth2 v0.14.0 // indirect
golang.org/x/sync v0.5.0 // indirect
golang.org/x/mod v0.8.0 // indirect
golang.org/x/net v0.17.0 // indirect
golang.org/x/oauth2 v0.7.0 // indirect
golang.org/x/sync v0.1.0 // indirect
golang.org/x/sys v0.17.0 // indirect
golang.org/x/text v0.14.0 // indirect
golang.org/x/tools v0.16.0 // indirect
google.golang.org/appengine v1.6.8 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20230920204549-e6e6cdab5c13 // indirect
google.golang.org/grpc v1.58.3 // indirect
google.golang.org/protobuf v1.33.0 // indirect
golang.org/x/tools v0.6.0 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 // indirect
google.golang.org/grpc v1.56.3 // indirect
google.golang.org/protobuf v1.30.0 // indirect
gopkg.in/go-jose/go-jose.v2 v2.6.3 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)

1183
go.sum

File diff suppressed because it is too large Load Diff

View File

@@ -5,16 +5,11 @@ fi
tmpdir="$PWD/tmp.$RANDOM"
mkdir -p "$tmpdir"
trap 'rm -fr "$tmpdir"' EXIT
cc -o "$tmpdir"/libsubid_tag -x c - -l subid > /dev/null 2> /dev/null << EOF
cc -o "$tmpdir"/libsubid_tag -l subid -x c - > /dev/null 2> /dev/null << EOF
#include <shadow/subid.h>
#include <stdlib.h>
int main() {
struct subid_range *ranges = NULL;
#if SUBID_ABI_MAJOR >= 4
subid_get_uid_ranges("root", &ranges);
#else
get_subuid_ranges("root", &ranges);
#endif
free(ranges);
return 0;
}

92
hack/make.sh Executable file
View File

@@ -0,0 +1,92 @@
#!/usr/bin/env bash
set -e
# This script builds various binary from a checkout of the skopeo
# source code. DO NOT CALL THIS SCRIPT DIRECTLY.
#
# Requirements:
# - The current directory should be a checkout of the skopeo source code
# (https://github.com/containers/skopeo). Whatever version is checked out
# will be built.
# - The script is intended to be run inside the container specified
# in the output of hack/get_fqin.sh
# - The right way to call this script is to invoke "make" from
# your checkout of the skopeo repository.
# the Makefile will do a "docker build -t skopeo ." and then
# "docker run hack/make.sh" in the resulting image.
#
set -o pipefail
export SKOPEO_PKG='github.com/containers/skopeo'
export SCRIPTDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
export MAKEDIR="$SCRIPTDIR/make"
# Set this to 1 to enable installation/modification of environment/services
export SKOPEO_CONTAINER_TESTS=${SKOPEO_CONTAINER_TESTS:-0}
if [[ "$SKOPEO_CONTAINER_TESTS" == "0" ]] && [[ "$CI" != "true" ]]; then
(
echo "***************************************************************"
echo "WARNING: Executing tests directly on the local development"
echo " host is highly discouraged. Many important items"
echo " will be skipped. For manual execution, please utilize"
echo " the Makefile targets WITHOUT the '-local' suffix."
echo "***************************************************************"
) > /dev/stderr
sleep 5
fi
echo
# List of bundles to create when no argument is passed
# TODO(runcom): these are the one left from Docker...for now
# test-unit
# validate-dco
# cover
DEFAULT_BUNDLES=(
validate-gofmt
validate-lint
validate-vet
validate-git-marks
test-integration
)
# Go module support: set `-mod=vendor` to use the vendored sources
# See also the top-level Makefile.
mod_vendor=
if go help mod >/dev/null 2>&1; then
export GO111MODULE=on
mod_vendor='-mod=vendor'
fi
go_test_dir() {
dir=$1
(
echo '+ go test' $mod_vendor $TESTFLAGS ${BUILDTAGS:+-tags "$BUILDTAGS"} "${SKOPEO_PKG}${dir#.}"
cd "$dir"
export DEST="$ABS_DEST" # we're in a subshell, so this is safe -- our integration-cli tests need DEST, and "cd" screws it up
go test $mod_vendor $TESTFLAGS ${BUILDTAGS:+-tags "$BUILDTAGS"}
)
}
bundle() {
local bundle="$1"; shift
echo "---> Making bundle: $(basename "$bundle")"
source "$SCRIPTDIR/make/$bundle" "$@"
}
main() {
if [ $# -lt 1 ]; then
bundles=(${DEFAULT_BUNDLES[@]})
else
bundles=($@)
fi
for bundle in ${bundles[@]}; do
bundle "$bundle"
echo
done
}
main "$@"

31
hack/make/.validate Normal file
View File

@@ -0,0 +1,31 @@
#!/bin/bash
if [ -z "$VALIDATE_UPSTREAM" ]; then
# this is kind of an expensive check, so let's not do this twice if we
# are running more than one validate bundlescript
VALIDATE_REPO='https://github.com/containers/skopeo.git'
VALIDATE_BRANCH='main'
if [ "$TRAVIS" = 'true' -a "$TRAVIS_PULL_REQUEST" != 'false' ]; then
VALIDATE_REPO="https://github.com/${TRAVIS_REPO_SLUG}.git"
VALIDATE_BRANCH="${TRAVIS_BRANCH}"
fi
VALIDATE_HEAD="$(git rev-parse --verify HEAD)"
git fetch -q "$VALIDATE_REPO" "refs/heads/$VALIDATE_BRANCH"
VALIDATE_UPSTREAM="$(git rev-parse --verify FETCH_HEAD)"
VALIDATE_COMMIT_LOG="$VALIDATE_UPSTREAM..$VALIDATE_HEAD"
VALIDATE_COMMIT_DIFF="$VALIDATE_UPSTREAM...$VALIDATE_HEAD"
validate_diff() {
git diff "$VALIDATE_UPSTREAM" "$@"
}
validate_log() {
if [ "$VALIDATE_UPSTREAM" != "$VALIDATE_HEAD" ]; then
git log "$VALIDATE_COMMIT_LOG" "$@"
fi
}
fi

12
hack/make/test-integration Executable file
View File

@@ -0,0 +1,12 @@
#!/bin/bash
set -e
bundle_test_integration() {
go_test_dir ./integration
}
# subshell so that we can export PATH without breaking other things
(
make PREFIX=/usr install
bundle_test_integration
) 2>&1

44
hack/make/validate-git-marks Executable file
View File

@@ -0,0 +1,44 @@
#!/usr/bin/env bash
source "$(dirname "$BASH_SOURCE")/.validate"
# folders=$(find * -type d | egrep -v '^Godeps|bundles|.git')
IFS=$'\n'
files=( $(validate_diff --diff-filter=ACMR --name-only -- '*' | grep -v '^vendor/' || true) )
unset IFS
badFiles=()
for f in "${files[@]}"; do
if [ $(grep -r "^<<<<<<<" $f) ]; then
badFiles+=( "$f" )
continue
fi
if [ $(grep -r "^>>>>>>>" $f) ]; then
badFiles+=( "$f" )
continue
fi
if [ $(grep -r "^=======$" $f) ]; then
badFiles+=( "$f" )
continue
fi
set -e
done
if [ ${#badFiles[@]} -eq 0 ]; then
echo 'Congratulations! There is no conflict.'
else
{
echo "There is trace of conflict(s) in the following files :"
for f in "${badFiles[@]}"; do
echo " - $f"
done
echo
echo 'Please fix the conflict(s) commit the result.'
echo
} >&2
false
fi

View File

@@ -1,7 +1,9 @@
#!/bin/bash
source "$(dirname "$BASH_SOURCE")/.validate"
IFS=$'\n'
files=( $(find . -name '*.go' | grep -v '^./vendor/' | sort || true) )
files=( $(validate_diff --diff-filter=ACMR --name-only -- '*.go' | grep -v '^vendor/' || true) )
unset IFS
badFiles=()
@@ -23,5 +25,5 @@ else
echo 'Please reformat the above files using "gofmt -s -w" and commit the result.'
echo
} >&2
exit 1
false
fi

33
hack/make/validate-lint Executable file
View File

@@ -0,0 +1,33 @@
#!/bin/bash
source "$(dirname "$BASH_SOURCE")/.validate"
# We will eventually get to the point where packages should be the complete list
# of subpackages, vendoring excluded, as given by:
#
IFS=$'\n'
files=( $(validate_diff --diff-filter=ACMR --name-only -- '*.go' | grep -v '^vendor/\|^integration' || true) )
unset IFS
errors=()
for f in "${files[@]}"; do
failedLint=$(golint "$f")
if [ "$failedLint" ]; then
errors+=( "$failedLint" )
fi
done
if [ ${#errors[@]} -eq 0 ]; then
echo 'Congratulations! All Go source files have been linted.'
else
{
echo "Errors from golint:"
for err in "${errors[@]}"; do
echo "$err"
done
echo
echo 'Please fix the above errors. You can test via "golint" and commit the result.'
echo
} >&2
false
fi

View File

@@ -1,6 +1,6 @@
#!/bin/bash
errors=$(go vet -tags="${BUILDTAGS}" ./... 2>&1)
errors=$(go vet -tags="${BUILDTAGS}" $mod_vendor $(go list $mod_vendor -e ./...))
if [ -z "$errors" ]; then
echo 'Congratulations! All Go source files have been vetted.'
@@ -12,5 +12,5 @@ else
echo 'Please fix the above errors. You can test via "go vet" and commit the result.'
echo
} >&2
exit 1
false
fi

View File

@@ -1,8 +0,0 @@
#!/bin/bash
set -e
make PREFIX=/usr install
echo "cd ./integration;" go test $TESTFLAGS ${BUILDTAGS:+-tags "$BUILDTAGS"}
cd ./integration
go test $TESTFLAGS ${BUILDTAGS:+-tags "$BUILDTAGS"}

View File

@@ -1,30 +0,0 @@
#!/usr/bin/env bash
IFS=$'\n'
files=( $(git ls-tree -r HEAD --name-only | grep -v '^vendor/' || true) )
unset IFS
badFiles=()
for f in "${files[@]}"; do
if [ $(grep -r "^\(<<<<<<<\|>>>>>>>\|^=======$\)" $f) ]; then
badFiles+=( "$f" )
continue
fi
set -e
done
if [ ${#badFiles[@]} -eq 0 ]; then
echo 'Congratulations! There is no conflict.'
else
{
echo "There is trace of conflict(s) in the following files :"
for f in "${badFiles[@]}"; do
echo " - $f"
done
echo
echo 'Please fix the conflict(s) commit the result.'
echo
} >&2
exit 1
fi

View File

@@ -1,16 +0,0 @@
#!/bin/bash
errors=$($GOBIN/golangci-lint run --build-tags "${BUILDTAGS}" 2>&1)
if [ -z "$errors" ]; then
echo 'Congratulations! All Go source files have been linted.'
else
{
echo "Errors from golangci-lint:"
echo "$errors"
echo
echo 'Please fix the above errors. You can test via "golangci-lint" and commit the result.'
echo
} >&2
exit 1
fi

View File

@@ -1,17 +0,0 @@
#!/usr/bin/env bash
set -e
# Set this to 1 to enable installation/modification of environment/services
export SKOPEO_CONTAINER_TESTS=${SKOPEO_CONTAINER_TESTS:-0}
if [[ "$SKOPEO_CONTAINER_TESTS" == "0" ]] && [[ "$CI" != "true" ]]; then
(
echo "***************************************************************"
echo "WARNING: Executing tests directly on the local development"
echo " host is highly discouraged. Many important items"
echo " will be skipped. For manual execution, please utilize"
echo " the Makefile targets WITHOUT the '-local' suffix."
echo "***************************************************************"
) > /dev/stderr
sleep 5
fi

View File

@@ -55,22 +55,6 @@ sudo apk add skopeo
[Package Info](https://pkgs.alpinelinux.org/packages?name=skopeo)
### Gentoo
```sh
sudo emerge app-containers/skopeo
```
[Package Info](https://packages.gentoo.org/packages/app-containers/skopeo)
### Arch Linux
```sh
sudo pacman -S skopeo
```
[Package Info](https://archlinux.org/packages/extra/x86_64/skopeo/)
### macOS
```sh
@@ -139,7 +123,7 @@ podman run docker://quay.io/skopeo/stable:latest copy --help
Otherwise, read on for building and installing it from source:
To build the `skopeo` binary you need at least Go 1.19.
To build the `skopeo` binary you need at least Go 1.12.
There are two ways to build skopeo: in a container, or locally without a
container. Choose the one which better matches your needs and environment.
@@ -175,11 +159,6 @@ brew install gpgme
sudo zypper install libgpgme-devel device-mapper-devel libbtrfs-devel glib2-devel
```
```bash
# Arch Linux:
sudo pacman -S base-devel gpgme device-mapper btrfs-progs
```
Make sure to clone this repository in your `GOPATH` - otherwise compilation fails.
```bash
@@ -195,16 +174,6 @@ document generation can be skipped by passing `DISABLE_DOCS=1`:
DISABLE_DOCS=1 make
```
### Cross-compilation
For cross-building skopeo, use the command `make bin/skopeo.OS.ARCH`, where OS represents
the target operating system and ARCH stands for the desired architecture. For instance,
to build skopeo for RISC-V 64-bit Linux, execute:
```bash
make bin/skopeo.linux.riscv64
```
### Building documentation
To build the manual you will need go-md2man.
@@ -266,8 +235,15 @@ There have been efforts in the past to produce and maintain static builds, but t
That being said, if you would like to build Skopeo statically, you might be able to do it by combining all the following steps.
- Export environment variable `CGO_ENABLED=0` (disabling CGO causes Go to prefer native libraries when possible, instead of dynamically linking against system libraries).
- Set the `BUILDTAGS=containers_image_openpgp` Make variable (this removes the dependency on `libgpgme` and its companion libraries).
- Clear the `GO_DYN_FLAGS` Make variable if even a dependency on the ELF interpreter is undesirable.
- Set the `BUILDTAGS=containers_image_openpgp` Make variable (this remove the dependency on `libgpgme` and its companion libraries).
- Clear the `GO_DYN_FLAGS` Make variable (which otherwise seems to force the creation of a dynamic executable).
The following command implements these steps to produce a static binary in the `bin` subdirectory of the repository:
```bash
docker run -v $PWD:/src -w /src -e CGO_ENABLED=0 golang \
make BUILDTAGS=containers_image_openpgp GO_DYN_FLAGS=
```
Keep in mind that the resulting binary is unsupported and might crash randomly. Only use if you know what you're doing!

View File

@@ -1,34 +1,34 @@
package main
import (
"gopkg.in/check.v1"
)
const blockedRegistriesConf = "./fixtures/blocked-registries.conf"
const blockedErrorRegex = `.*registry registry-blocked.com is blocked in .*`
func (s *skopeoSuite) TestCopyBlockedSource() {
t := s.T()
assertSkopeoFails(t, blockedErrorRegex,
func (s *SkopeoSuite) TestCopyBlockedSource(c *check.C) {
assertSkopeoFails(c, blockedErrorRegex,
"--registries-conf", blockedRegistriesConf, "copy",
"docker://registry-blocked.com/image:test",
"docker://registry-unblocked.com/image:test")
}
func (s *skopeoSuite) TestCopyBlockedDestination() {
t := s.T()
assertSkopeoFails(t, blockedErrorRegex,
func (s *SkopeoSuite) TestCopyBlockedDestination(c *check.C) {
assertSkopeoFails(c, blockedErrorRegex,
"--registries-conf", blockedRegistriesConf, "copy",
"docker://registry-unblocked.com/image:test",
"docker://registry-blocked.com/image:test")
}
func (s *skopeoSuite) TestInspectBlocked() {
t := s.T()
assertSkopeoFails(t, blockedErrorRegex,
func (s *SkopeoSuite) TestInspectBlocked(c *check.C) {
assertSkopeoFails(c, blockedErrorRegex,
"--registries-conf", blockedRegistriesConf, "inspect",
"docker://registry-blocked.com/image:test")
}
func (s *skopeoSuite) TestDeleteBlocked() {
t := s.T()
assertSkopeoFails(t, blockedErrorRegex,
func (s *SkopeoSuite) TestDeleteBlocked(c *check.C) {
assertSkopeoFails(c, blockedErrorRegex,
"--registries-conf", blockedRegistriesConf, "delete",
"docker://registry-blocked.com/image:test")
}

View File

@@ -6,9 +6,7 @@ import (
"testing"
"github.com/containers/skopeo/version"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/stretchr/testify/suite"
"gopkg.in/check.v1"
)
const (
@@ -16,104 +14,98 @@ const (
privateRegistryURL1 = "127.0.0.1:5001"
)
func TestSkopeo(t *testing.T) {
suite.Run(t, &skopeoSuite{})
func Test(t *testing.T) {
check.TestingT(t)
}
type skopeoSuite struct {
suite.Suite
func init() {
check.Suite(&SkopeoSuite{})
}
type SkopeoSuite struct {
regV2 *testRegistryV2
regV2WithAuth *testRegistryV2
}
var _ = suite.SetupAllSuite(&skopeoSuite{})
var _ = suite.TearDownAllSuite(&skopeoSuite{})
func (s *skopeoSuite) SetupSuite() {
t := s.T()
func (s *SkopeoSuite) SetUpSuite(c *check.C) {
_, err := exec.LookPath(skopeoBinary)
require.NoError(t, err)
s.regV2 = setupRegistryV2At(t, privateRegistryURL0, false, false)
s.regV2WithAuth = setupRegistryV2At(t, privateRegistryURL1, true, false)
c.Assert(err, check.IsNil)
s.regV2 = setupRegistryV2At(c, privateRegistryURL0, false, false)
s.regV2WithAuth = setupRegistryV2At(c, privateRegistryURL1, true, false)
}
func (s *skopeoSuite) TearDownSuite() {
func (s *SkopeoSuite) TearDownSuite(c *check.C) {
if s.regV2 != nil {
s.regV2.tearDown()
s.regV2.tearDown(c)
}
if s.regV2WithAuth != nil {
// cmd := exec.Command("docker", "logout", s.regV2WithAuth)
// require.Noerror(t, cmd.Run())
s.regV2WithAuth.tearDown()
//cmd := exec.Command("docker", "logout", s.regV2WithAuth)
//c.Assert(cmd.Run(), check.IsNil)
s.regV2WithAuth.tearDown(c)
}
}
func (s *skopeoSuite) TestVersion() {
t := s.T()
assertSkopeoSucceeds(t, fmt.Sprintf(".*%s version %s.*", skopeoBinary, version.Version),
// TODO like dockerCmd but much easier, just out,err
//func skopeoCmd()
func (s *SkopeoSuite) TestVersion(c *check.C) {
assertSkopeoSucceeds(c, fmt.Sprintf(".*%s version %s.*", skopeoBinary, version.Version),
"--version")
}
func (s *skopeoSuite) TestCanAuthToPrivateRegistryV2WithoutDockerCfg() {
t := s.T()
assertSkopeoFails(t, ".*manifest unknown.*",
func (s *SkopeoSuite) TestCanAuthToPrivateRegistryV2WithoutDockerCfg(c *check.C) {
assertSkopeoFails(c, ".*manifest unknown.*",
"--tls-verify=false", "inspect", "--creds="+s.regV2WithAuth.username+":"+s.regV2WithAuth.password, fmt.Sprintf("docker://%s/busybox:latest", s.regV2WithAuth.url))
}
func (s *skopeoSuite) TestNeedAuthToPrivateRegistryV2WithoutDockerCfg() {
t := s.T()
assertSkopeoFails(t, ".*authentication required.*",
func (s *SkopeoSuite) TestNeedAuthToPrivateRegistryV2WithoutDockerCfg(c *check.C) {
assertSkopeoFails(c, ".*authentication required.*",
"--tls-verify=false", "inspect", fmt.Sprintf("docker://%s/busybox:latest", s.regV2WithAuth.url))
}
func (s *skopeoSuite) TestCertDirInsteadOfCertPath() {
t := s.T()
assertSkopeoFails(t, ".*unknown flag: --cert-path.*",
func (s *SkopeoSuite) TestCertDirInsteadOfCertPath(c *check.C) {
assertSkopeoFails(c, ".*unknown flag: --cert-path.*",
"--tls-verify=false", "inspect", fmt.Sprintf("docker://%s/busybox:latest", s.regV2WithAuth.url), "--cert-path=/")
assertSkopeoFails(t, ".*authentication required.*",
assertSkopeoFails(c, ".*authentication required.*",
"--tls-verify=false", "inspect", fmt.Sprintf("docker://%s/busybox:latest", s.regV2WithAuth.url), "--cert-dir=/etc/docker/certs.d/")
}
// TODO(runcom): as soon as we can push to registries ensure you can inspect here
// not just get image not found :)
func (s *skopeoSuite) TestNoNeedAuthToPrivateRegistryV2ImageNotFound() {
t := s.T()
func (s *SkopeoSuite) TestNoNeedAuthToPrivateRegistryV2ImageNotFound(c *check.C) {
out, err := exec.Command(skopeoBinary, "--tls-verify=false", "inspect", fmt.Sprintf("docker://%s/busybox:latest", s.regV2.url)).CombinedOutput()
assert.Error(t, err, "%s", string(out))
assert.Regexp(t, "(?s).*manifest unknown.*", string(out)) // (?s) : '.' will also match newlines
assert.NotRegexp(t, "(?s).*unauthorized: authentication required.*", string(out)) // (?s) : '.' will also match newlines
c.Assert(err, check.NotNil, check.Commentf(string(out)))
c.Assert(string(out), check.Matches, "(?s).*manifest unknown.*") // (?s) : '.' will also match newlines
c.Assert(string(out), check.Not(check.Matches), "(?s).*unauthorized: authentication required.*") // (?s) : '.' will also match newlines
}
func (s *skopeoSuite) TestInspectFailsWhenReferenceIsInvalid() {
t := s.T()
assertSkopeoFails(t, `.*Invalid image name.*`, "inspect", "unknown")
func (s *SkopeoSuite) TestInspectFailsWhenReferenceIsInvalid(c *check.C) {
assertSkopeoFails(c, `.*Invalid image name.*`, "inspect", "unknown")
}
func (s *skopeoSuite) TestLoginLogout() {
t := s.T()
assertSkopeoSucceeds(t, "^Login Succeeded!\n$",
func (s *SkopeoSuite) TestLoginLogout(c *check.C) {
assertSkopeoSucceeds(c, "^Login Succeeded!\n$",
"login", "--tls-verify=false", "--username="+s.regV2WithAuth.username, "--password="+s.regV2WithAuth.password, s.regV2WithAuth.url)
// test --get-login returns username
assertSkopeoSucceeds(t, fmt.Sprintf("^%s\n$", s.regV2WithAuth.username),
assertSkopeoSucceeds(c, fmt.Sprintf("^%s\n$", s.regV2WithAuth.username),
"login", "--tls-verify=false", "--get-login", s.regV2WithAuth.url)
// test logout
assertSkopeoSucceeds(t, fmt.Sprintf("^Removed login credentials for %s\n$", s.regV2WithAuth.url),
assertSkopeoSucceeds(c, fmt.Sprintf("^Removed login credentials for %s\n$", s.regV2WithAuth.url),
"logout", s.regV2WithAuth.url)
}
func (s *skopeoSuite) TestCopyWithLocalAuth() {
t := s.T()
assertSkopeoSucceeds(t, "^Login Succeeded!\n$",
func (s *SkopeoSuite) TestCopyWithLocalAuth(c *check.C) {
assertSkopeoSucceeds(c, "^Login Succeeded!\n$",
"login", "--tls-verify=false", "--username="+s.regV2WithAuth.username, "--password="+s.regV2WithAuth.password, s.regV2WithAuth.url)
// copy to private registry using local authentication
imageName := fmt.Sprintf("docker://%s/busybox:mine", s.regV2WithAuth.url)
assertSkopeoSucceeds(t, "", "copy", "--dest-tls-verify=false", testFQIN+":latest", imageName)
assertSkopeoSucceeds(c, "", "copy", "--dest-tls-verify=false", testFQIN+":latest", imageName)
// inspect from private registry
assertSkopeoSucceeds(t, "", "inspect", "--tls-verify=false", imageName)
assertSkopeoSucceeds(c, "", "inspect", "--tls-verify=false", imageName)
// logout from the registry
assertSkopeoSucceeds(t, fmt.Sprintf("^Removed login credentials for %s\n$", s.regV2WithAuth.url),
assertSkopeoSucceeds(c, fmt.Sprintf("^Removed login credentials for %s\n$", s.regV2WithAuth.url),
"logout", s.regV2WithAuth.url)
// inspect from private registry should fail after logout
assertSkopeoFails(t, ".*authentication required.*",
assertSkopeoFails(c, ".*authentication required.*",
"inspect", "--tls-verify=false", imageName)
}

File diff suppressed because it is too large Load Diff

View File

@@ -9,11 +9,10 @@ import (
"os/exec"
"path/filepath"
"strings"
"testing"
"time"
"github.com/containers/storage/pkg/homedir"
"github.com/stretchr/testify/require"
"gopkg.in/check.v1"
)
var adminKUBECONFIG = map[string]string{
@@ -31,21 +30,21 @@ type openshiftCluster struct {
// startOpenshiftCluster creates a new openshiftCluster.
// WARNING: This affects state in users' home directory! Only run
// in isolated test environment.
func startOpenshiftCluster(t *testing.T) *openshiftCluster {
func startOpenshiftCluster(c *check.C) *openshiftCluster {
cluster := &openshiftCluster{}
cluster.workingDir = t.TempDir()
cluster.workingDir = c.MkDir()
cluster.startMaster(t)
cluster.prepareRegistryConfig(t)
cluster.startRegistry(t)
cluster.ocLoginToProject(t)
cluster.dockerLogin(t)
cluster.relaxImageSignerPermissions(t)
cluster.startMaster(c)
cluster.prepareRegistryConfig(c)
cluster.startRegistry(c)
cluster.ocLoginToProject(c)
cluster.dockerLogin(c)
cluster.relaxImageSignerPermissions(c)
return cluster
}
// clusterCmd creates an exec.Cmd in cluster.workingDir with current environment modified by environment.
// clusterCmd creates an exec.Cmd in cluster.workingDir with current environment modified by environment
func (cluster *openshiftCluster) clusterCmd(env map[string]string, name string, args ...string) *exec.Cmd {
cmd := exec.Command(name, args...)
cmd.Dir = cluster.workingDir
@@ -57,20 +56,21 @@ func (cluster *openshiftCluster) clusterCmd(env map[string]string, name string,
}
// startMaster starts the OpenShift master (etcd+API server) and waits for it to be ready, or terminates on failure.
func (cluster *openshiftCluster) startMaster(t *testing.T) {
func (cluster *openshiftCluster) startMaster(c *check.C) {
cmd := cluster.clusterCmd(nil, "openshift", "start", "master")
cluster.processes = append(cluster.processes, cmd)
stdout, err := cmd.StdoutPipe()
require.NoError(t, err)
c.Assert(err, check.IsNil)
// Send both to the same pipe. This might cause the two streams to be mixed up,
// but logging actually goes only to stderr - this primarily ensure we log any
// unexpected output to stdout.
cmd.Stderr = cmd.Stdout
err = cmd.Start()
require.NoError(t, err)
c.Assert(err, check.IsNil)
portOpen, terminatePortCheck := newPortChecker(t, 8443)
portOpen, terminatePortCheck := newPortChecker(c, 8443)
defer func() {
t.Logf("Terminating port check")
c.Logf("Terminating port check")
terminatePortCheck <- true
}()
@@ -78,12 +78,12 @@ func (cluster *openshiftCluster) startMaster(t *testing.T) {
logCheckFound := make(chan bool)
go func() {
defer func() {
t.Logf("Log checker exiting")
c.Logf("Log checker exiting")
}()
scanner := bufio.NewScanner(stdout)
for scanner.Scan() {
line := scanner.Text()
t.Logf("Log line: %s", line)
c.Logf("Log line: %s", line)
if strings.Contains(line, "Started Origin Controllers") {
logCheckFound <- true
return
@@ -92,7 +92,7 @@ func (cluster *openshiftCluster) startMaster(t *testing.T) {
// Note: we can block before we get here.
select {
case <-terminateLogCheck:
t.Logf("terminated")
c.Logf("terminated")
return
default:
// Do not block here and read the next line.
@@ -101,7 +101,7 @@ func (cluster *openshiftCluster) startMaster(t *testing.T) {
logCheckFound <- false
}()
defer func() {
t.Logf("Terminating log check")
c.Logf("Terminating log check")
terminateLogCheck <- true
}()
@@ -110,26 +110,26 @@ func (cluster *openshiftCluster) startMaster(t *testing.T) {
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Minute)
defer cancel()
for !gotPortCheck || !gotLogCheck {
t.Logf("Waiting for master")
c.Logf("Waiting for master")
select {
case <-portOpen:
t.Logf("port check done")
c.Logf("port check done")
gotPortCheck = true
case found := <-logCheckFound:
t.Logf("log check done, found: %t", found)
c.Logf("log check done, found: %t", found)
if !found {
t.Fatal("log check done, success message not found")
c.Fatal("log check done, success message not found")
}
gotLogCheck = true
case <-ctx.Done():
t.Fatalf("Timed out waiting for master: %v", ctx.Err())
c.Fatalf("Timed out waiting for master: %v", ctx.Err())
}
}
t.Logf("OK, master started!")
c.Logf("OK, master started!")
}
// prepareRegistryConfig creates a registry service account and a related k8s client configuration in ${cluster.workingDir}/openshift.local.registry.
func (cluster *openshiftCluster) prepareRegistryConfig(t *testing.T) {
func (cluster *openshiftCluster) prepareRegistryConfig(c *check.C) {
// This partially mimics the objects created by (oadm registry), except that we run the
// server directly as an ordinary process instead of a pod with an implicitly attached service account.
saJSON := `{
@@ -140,93 +140,93 @@ func (cluster *openshiftCluster) prepareRegistryConfig(t *testing.T) {
}
}`
cmd := cluster.clusterCmd(adminKUBECONFIG, "oc", "create", "-f", "-")
runExecCmdWithInput(t, cmd, saJSON)
runExecCmdWithInput(c, cmd, saJSON)
cmd = cluster.clusterCmd(adminKUBECONFIG, "oadm", "policy", "add-cluster-role-to-user", "system:registry", "-z", "registry")
out, err := cmd.CombinedOutput()
require.NoError(t, err, "%s", string(out))
require.Equal(t, "cluster role \"system:registry\" added: \"registry\"\n", string(out))
c.Assert(err, check.IsNil, check.Commentf("%s", string(out)))
c.Assert(string(out), check.Equals, "cluster role \"system:registry\" added: \"registry\"\n")
cmd = cluster.clusterCmd(adminKUBECONFIG, "oadm", "create-api-client-config", "--client-dir=openshift.local.registry", "--basename=openshift-registry", "--user=system:serviceaccount:default:registry")
out, err = cmd.CombinedOutput()
require.NoError(t, err, "%s", string(out))
require.Equal(t, "", string(out))
c.Assert(err, check.IsNil, check.Commentf("%s", string(out)))
c.Assert(string(out), check.Equals, "")
}
// startRegistry starts the OpenShift registry with configPart on port, waits for it to be ready, and returns the process object, or terminates on failure.
func (cluster *openshiftCluster) startRegistryProcess(t *testing.T, port uint16, configPath string) *exec.Cmd {
func (cluster *openshiftCluster) startRegistryProcess(c *check.C, port int, configPath string) *exec.Cmd {
cmd := cluster.clusterCmd(map[string]string{
"KUBECONFIG": "openshift.local.registry/openshift-registry.kubeconfig",
"DOCKER_REGISTRY_URL": fmt.Sprintf("127.0.0.1:%d", port),
}, "dockerregistry", configPath)
consumeAndLogOutputs(t, fmt.Sprintf("registry-%d", port), cmd)
consumeAndLogOutputs(c, fmt.Sprintf("registry-%d", port), cmd)
err := cmd.Start()
require.NoError(t, err, "%s")
c.Assert(err, check.IsNil)
portOpen, terminatePortCheck := newPortChecker(t, port)
portOpen, terminatePortCheck := newPortChecker(c, port)
defer func() {
terminatePortCheck <- true
}()
t.Logf("Waiting for registry to start")
c.Logf("Waiting for registry to start")
ctx, cancel := context.WithTimeout(context.Background(), 1*time.Minute)
defer cancel()
select {
case <-portOpen:
t.Logf("OK, Registry port open")
c.Logf("OK, Registry port open")
case <-ctx.Done():
t.Fatalf("Timed out waiting for registry to start: %v", ctx.Err())
c.Fatalf("Timed out waiting for registry to start: %v", ctx.Err())
}
return cmd
}
// startRegistry starts the OpenShift registry and waits for it to be ready, or terminates on failure.
func (cluster *openshiftCluster) startRegistry(t *testing.T) {
func (cluster *openshiftCluster) startRegistry(c *check.C) {
// Our “primary” registry
cluster.processes = append(cluster.processes, cluster.startRegistryProcess(t, 5000, "/atomic-registry-config.yml"))
cluster.processes = append(cluster.processes, cluster.startRegistryProcess(c, 5000, "/atomic-registry-config.yml"))
// A registry configured with acceptschema2:false
schema1Config := fileFromFixture(t, "/atomic-registry-config.yml", map[string]string{
schema1Config := fileFromFixture(c, "/atomic-registry-config.yml", map[string]string{
"addr: :5000": "addr: :5005",
"rootdirectory: /registry": "rootdirectory: /registry-schema1",
// The default configuration currently already contains acceptschema2: false
})
// Make sure the configuration contains "acceptschema2: false", because eventually it will be enabled upstream and this function will need to be updated.
configContents, err := os.ReadFile(schema1Config)
require.NoError(t, err)
require.Regexp(t, "(?s).*acceptschema2: false.*", string(configContents))
cluster.processes = append(cluster.processes, cluster.startRegistryProcess(t, 5005, schema1Config))
c.Assert(err, check.IsNil)
c.Assert(string(configContents), check.Matches, "(?s).*acceptschema2: false.*")
cluster.processes = append(cluster.processes, cluster.startRegistryProcess(c, 5005, schema1Config))
// A registry configured with acceptschema2:true
schema2Config := fileFromFixture(t, "/atomic-registry-config.yml", map[string]string{
schema2Config := fileFromFixture(c, "/atomic-registry-config.yml", map[string]string{
"addr: :5000": "addr: :5006",
"rootdirectory: /registry": "rootdirectory: /registry-schema2",
"acceptschema2: false": "acceptschema2: true",
})
cluster.processes = append(cluster.processes, cluster.startRegistryProcess(t, 5006, schema2Config))
cluster.processes = append(cluster.processes, cluster.startRegistryProcess(c, 5006, schema2Config))
}
// ocLogin runs (oc login) and (oc new-project) on the cluster, or terminates on failure.
func (cluster *openshiftCluster) ocLoginToProject(t *testing.T) {
t.Logf("oc login")
func (cluster *openshiftCluster) ocLoginToProject(c *check.C) {
c.Logf("oc login")
cmd := cluster.clusterCmd(nil, "oc", "login", "--certificate-authority=openshift.local.config/master/ca.crt", "-u", "myuser", "-p", "mypw", "https://localhost:8443")
out, err := cmd.CombinedOutput()
require.NoError(t, err, "%s", out)
require.Regexp(t, "(?s).*Login successful.*", string(out)) // (?s) : '.' will also match newlines
c.Assert(err, check.IsNil, check.Commentf("%s", out))
c.Assert(string(out), check.Matches, "(?s).*Login successful.*") // (?s) : '.' will also match newlines
outString := combinedOutputOfCommand(t, "oc", "new-project", "myns")
require.Regexp(t, `(?s).*Now using project "myns".*`, outString) // (?s) : '.' will also match newlines
outString := combinedOutputOfCommand(c, "oc", "new-project", "myns")
c.Assert(outString, check.Matches, `(?s).*Now using project "myns".*`) // (?s) : '.' will also match newlines
}
// dockerLogin simulates (docker login) to the cluster, or terminates on failure.
// We do not run (docker login) directly, because that requires a running daemon and a docker package.
func (cluster *openshiftCluster) dockerLogin(t *testing.T) {
func (cluster *openshiftCluster) dockerLogin(c *check.C) {
cluster.dockerDir = filepath.Join(homedir.Get(), ".docker")
err := os.Mkdir(cluster.dockerDir, 0700)
require.NoError(t, err)
c.Assert(err, check.IsNil)
out := combinedOutputOfCommand(t, "oc", "config", "view", "-o", "json", "-o", "jsonpath={.users[*].user.token}")
t.Logf("oc config value: %s", out)
out := combinedOutputOfCommand(c, "oc", "config", "view", "-o", "json", "-o", "jsonpath={.users[*].user.token}")
c.Logf("oc config value: %s", out)
authValue := base64.StdEncoding.EncodeToString([]byte("unused:" + out))
auths := []string{}
for _, port := range []int{5000, 5005, 5006} {
@@ -237,22 +237,22 @@ func (cluster *openshiftCluster) dockerLogin(t *testing.T) {
}
configJSON := `{"auths": {` + strings.Join(auths, ",") + `}}`
err = os.WriteFile(filepath.Join(cluster.dockerDir, "config.json"), []byte(configJSON), 0600)
require.NoError(t, err)
c.Assert(err, check.IsNil)
}
// relaxImageSignerPermissions opens up the system:image-signer permissions so that
// anyone can work with signatures
// FIXME: This also allows anyone to DoS anyone else; this design is really not all
// that workable, but it is the best we can do for now.
func (cluster *openshiftCluster) relaxImageSignerPermissions(t *testing.T) {
func (cluster *openshiftCluster) relaxImageSignerPermissions(c *check.C) {
cmd := cluster.clusterCmd(adminKUBECONFIG, "oadm", "policy", "add-cluster-role-to-group", "system:image-signer", "system:authenticated")
out, err := cmd.CombinedOutput()
require.NoError(t, err, "%s", string(out))
require.Equal(t, "cluster role \"system:image-signer\" added: \"system:authenticated\"\n", string(out))
c.Assert(err, check.IsNil, check.Commentf("%s", string(out)))
c.Assert(string(out), check.Equals, "cluster role \"system:image-signer\" added: \"system:authenticated\"\n")
}
// tearDown stops the cluster services and deletes (only some!) of the state.
func (cluster *openshiftCluster) tearDown(t *testing.T) {
func (cluster *openshiftCluster) tearDown(c *check.C) {
for i := len(cluster.processes) - 1; i >= 0; i-- {
// Its undocumented what Kill() returns if the process has terminated,
// so we couldnt check just for that. This is running in a container anyway…
@@ -260,6 +260,6 @@ func (cluster *openshiftCluster) tearDown(t *testing.T) {
}
if cluster.dockerDir != "" {
err := os.RemoveAll(cluster.dockerDir)
require.NoError(t, err)
c.Assert(err, check.IsNil)
}
}

View File

@@ -7,8 +7,7 @@ import (
"os"
"os/exec"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"gopkg.in/check.v1"
)
/*
@@ -21,7 +20,7 @@ To use it, run:
to start a container, then within the container:
SKOPEO_CONTAINER_TESTS=1 PS1='nested> ' go test -tags openshift_shell -timeout=24h ./integration -v -run='copySuite.TestRunShell'
SKOPEO_CONTAINER_TESTS=1 PS1='nested> ' go test -tags openshift_shell -timeout=24h ./integration -v -check.v -check.vv -check.f='CopySuite.TestRunShell'
An example of what can be done within the container:
@@ -34,14 +33,13 @@ An example of what can be done within the container:
curl -L -v 'http://localhost:5000/v2/myns/personal/manifests/personal' --header 'Authorization: Bearer $token_from_oauth'
curl -L -v 'http://localhost:5000/extensions/v2/myns/personal/signatures/$manifest_digest' --header 'Authorization: Bearer $token_from_oauth'
*/
func (s *copySuite) TestRunShell() {
t := s.T()
func (s *CopySuite) TestRunShell(c *check.C) {
cmd := exec.Command("bash", "-i")
tty, err := os.OpenFile("/dev/tty", os.O_RDWR, 0)
require.NoError(t, err)
c.Assert(err, check.IsNil)
cmd.Stdin = tty
cmd.Stdout = tty
cmd.Stderr = tty
err = cmd.Run()
assert.NoError(t, err)
c.Assert(err, check.IsNil)
}

View File

@@ -7,6 +7,6 @@ import (
"os/exec"
)
// cmdLifecycleToParentIfPossible tries to exit if the parent process exits (only works on Linux).
// cmdLifecycleToParentIfPossible tries to exit if the parent process exits (only works on Linux)
func cmdLifecycleToParentIfPossible(c *exec.Cmd) {
}

View File

@@ -9,18 +9,16 @@ import (
"os/exec"
"strings"
"syscall"
"testing"
"time"
"gopkg.in/check.v1"
"github.com/containers/image/v5/manifest"
imgspecv1 "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/stretchr/testify/suite"
)
// This image is known to be x86_64 only right now
const knownNotManifestListedImageX8664 = "docker://quay.io/coreos/11bot"
const knownNotManifestListedImage_x8664 = "docker://quay.io/coreos/11bot"
// knownNotExtantImage would be very surprising if it did exist
const knownNotExtantImage = "docker://quay.io/centos/centos:opensusewindowsubuntu"
@@ -34,7 +32,7 @@ type request struct {
// Method is the name of the function
Method string `json:"method"`
// Args is the arguments (parsed inside the function)
Args []any `json:"args"`
Args []interface{} `json:"args"`
}
// reply is copied from proxy.go
@@ -42,7 +40,7 @@ type reply struct {
// Success is true if and only if the call succeeded.
Success bool `json:"success"`
// Value is an arbitrary value (or values, as array/map) returned from the call.
Value any `json:"value"`
Value interface{} `json:"value"`
// PipeID is an index into open pipes, and should be passed to FinishPipe
PipeID uint32 `json:"pipeid"`
// Error should be non-empty if Success == false
@@ -62,7 +60,7 @@ type pipefd struct {
fd *os.File
}
func (p *proxy) call(method string, args []any) (rval any, fd *pipefd, err error) {
func (p *proxy) call(method string, args []interface{}) (rval interface{}, fd *pipefd, err error) {
req := request{
Method: method,
Args: args,
@@ -83,7 +81,7 @@ func (p *proxy) call(method string, args []any) (rval any, fd *pipefd, err error
replybuf := make([]byte, maxMsgSize)
n, oobn, _, _, err := p.c.ReadMsgUnix(replybuf, oob)
if err != nil {
err = fmt.Errorf("reading reply: %w", err)
err = fmt.Errorf("reading reply: %v", err)
return
}
var reply reply
@@ -101,7 +99,7 @@ func (p *proxy) call(method string, args []any) (rval any, fd *pipefd, err error
var scms []syscall.SocketControlMessage
scms, err = syscall.ParseSocketControlMessage(oob[:oobn])
if err != nil {
err = fmt.Errorf("failed to parse control message: %w", err)
err = fmt.Errorf("failed to parse control message: %v", err)
return
}
if len(scms) != 1 {
@@ -111,7 +109,7 @@ func (p *proxy) call(method string, args []any) (rval any, fd *pipefd, err error
var fds []int
fds, err = syscall.ParseUnixRights(&scms[0])
if err != nil {
err = fmt.Errorf("failed to parse unix rights: %w", err)
err = fmt.Errorf("failed to parse unix rights: %v", err)
return
}
fd = &pipefd{
@@ -124,7 +122,7 @@ func (p *proxy) call(method string, args []any) (rval any, fd *pipefd, err error
return
}
func (p *proxy) callNoFd(method string, args []any) (rval any, err error) {
func (p *proxy) callNoFd(method string, args []interface{}) (rval interface{}, err error) {
var fd *pipefd
rval, fd, err = p.call(method, args)
if err != nil {
@@ -137,7 +135,7 @@ func (p *proxy) callNoFd(method string, args []any) (rval any, err error) {
return rval, nil
}
func (p *proxy) callReadAllBytes(method string, args []any) (rval any, buf []byte, err error) {
func (p *proxy) callReadAllBytes(method string, args []interface{}) (rval interface{}, buf []byte, err error) {
var fd *pipefd
rval, fd, err = p.call(method, args)
if err != nil {
@@ -155,7 +153,7 @@ func (p *proxy) callReadAllBytes(method string, args []any) (rval any, buf []byt
err: err,
}
}()
_, err = p.callNoFd("FinishPipe", []any{fd.id})
_, err = p.callNoFd("FinishPipe", []interface{}{fd.id})
if err != nil {
return
}
@@ -216,12 +214,17 @@ func newProxy() (*proxy, error) {
return p, nil
}
func TestProxy(t *testing.T) {
suite.Run(t, &proxySuite{})
func init() {
check.Suite(&ProxySuite{})
}
type proxySuite struct {
suite.Suite
type ProxySuite struct {
}
func (s *ProxySuite) SetUpSuite(c *check.C) {
}
func (s *ProxySuite) TearDownSuite(c *check.C) {
}
type byteFetch struct {
@@ -230,7 +233,7 @@ type byteFetch struct {
}
func runTestGetManifestAndConfig(p *proxy, img string) error {
v, err := p.callNoFd("OpenImage", []any{img})
v, err := p.callNoFd("OpenImage", []interface{}{knownNotManifestListedImage_x8664})
if err != nil {
return err
}
@@ -239,13 +242,13 @@ func runTestGetManifestAndConfig(p *proxy, img string) error {
if !ok {
return fmt.Errorf("OpenImage return value is %T", v)
}
imgid := uint64(imgidv)
imgid := uint32(imgidv)
if imgid == 0 {
return fmt.Errorf("got zero from expected image")
}
// Also verify the optional path
v, err = p.callNoFd("OpenImageOptional", []any{img})
v, err = p.callNoFd("OpenImageOptional", []interface{}{knownNotManifestListedImage_x8664})
if err != nil {
return err
}
@@ -254,17 +257,17 @@ func runTestGetManifestAndConfig(p *proxy, img string) error {
if !ok {
return fmt.Errorf("OpenImageOptional return value is %T", v)
}
imgid2 := uint64(imgidv)
imgid2 := uint32(imgidv)
if imgid2 == 0 {
return fmt.Errorf("got zero from expected image")
}
_, err = p.callNoFd("CloseImage", []any{imgid2})
_, err = p.callNoFd("CloseImage", []interface{}{imgid2})
if err != nil {
return err
}
_, manifestBytes, err := p.callReadAllBytes("GetManifest", []any{imgid})
_, manifestBytes, err := p.callReadAllBytes("GetManifest", []interface{}{imgid})
if err != nil {
return err
}
@@ -273,7 +276,7 @@ func runTestGetManifestAndConfig(p *proxy, img string) error {
return err
}
_, configBytes, err := p.callReadAllBytes("GetFullConfig", []any{imgid})
_, configBytes, err := p.callReadAllBytes("GetFullConfig", []interface{}{imgid})
if err != nil {
return err
}
@@ -292,7 +295,7 @@ func runTestGetManifestAndConfig(p *proxy, img string) error {
}
// Also test this legacy interface
_, ctrconfigBytes, err := p.callReadAllBytes("GetConfig", []any{imgid})
_, ctrconfigBytes, err := p.callReadAllBytes("GetConfig", []interface{}{imgid})
if err != nil {
return err
}
@@ -307,7 +310,7 @@ func runTestGetManifestAndConfig(p *proxy, img string) error {
return fmt.Errorf("No CMD or ENTRYPOINT set")
}
_, err = p.callNoFd("CloseImage", []any{imgid})
_, err = p.callNoFd("CloseImage", []interface{}{imgid})
if err != nil {
return err
}
@@ -316,7 +319,7 @@ func runTestGetManifestAndConfig(p *proxy, img string) error {
}
func runTestOpenImageOptionalNotFound(p *proxy, img string) error {
v, err := p.callNoFd("OpenImageOptional", []any{img})
v, err := p.callNoFd("OpenImageOptional", []interface{}{img})
if err != nil {
return err
}
@@ -325,33 +328,32 @@ func runTestOpenImageOptionalNotFound(p *proxy, img string) error {
if !ok {
return fmt.Errorf("OpenImageOptional return value is %T", v)
}
imgid := uint64(imgidv)
imgid := uint32(imgidv)
if imgid != 0 {
return fmt.Errorf("Unexpected optional image id %v", imgid)
}
return nil
}
func (s *proxySuite) TestProxy() {
t := s.T()
func (s *ProxySuite) TestProxy(c *check.C) {
p, err := newProxy()
require.NoError(t, err)
c.Assert(err, check.IsNil)
err = runTestGetManifestAndConfig(p, knownNotManifestListedImageX8664)
err = runTestGetManifestAndConfig(p, knownNotManifestListedImage_x8664)
if err != nil {
err = fmt.Errorf("Testing image %s: %v", knownNotManifestListedImageX8664, err)
err = fmt.Errorf("Testing image %s: %v", knownNotManifestListedImage_x8664, err)
}
assert.NoError(t, err)
c.Assert(err, check.IsNil)
err = runTestGetManifestAndConfig(p, knownListImage)
if err != nil {
err = fmt.Errorf("Testing image %s: %v", knownListImage, err)
}
assert.NoError(t, err)
c.Assert(err, check.IsNil)
err = runTestOpenImageOptionalNotFound(p, knownNotExtantImage)
if err != nil {
err = fmt.Errorf("Testing optional image %s: %v", knownNotExtantImage, err)
}
assert.NoError(t, err)
c.Assert(err, check.IsNil)
}

View File

@@ -6,10 +6,9 @@ import (
"os"
"os/exec"
"path/filepath"
"testing"
"time"
"github.com/stretchr/testify/require"
"gopkg.in/check.v1"
)
const (
@@ -25,9 +24,9 @@ type testRegistryV2 struct {
email string
}
func setupRegistryV2At(t *testing.T, url string, auth, schema1 bool) *testRegistryV2 {
reg, err := newTestRegistryV2At(t, url, auth, schema1)
require.NoError(t, err)
func setupRegistryV2At(c *check.C, url string, auth, schema1 bool) *testRegistryV2 {
reg, err := newTestRegistryV2At(c, url, auth, schema1)
c.Assert(err, check.IsNil)
// Wait for registry to be ready to serve requests.
for i := 0; i != 50; i++ {
@@ -38,13 +37,13 @@ func setupRegistryV2At(t *testing.T, url string, auth, schema1 bool) *testRegist
}
if err != nil {
t.Fatal("Timeout waiting for test registry to become available")
c.Fatal("Timeout waiting for test registry to become available")
}
return reg
}
func newTestRegistryV2At(t *testing.T, url string, auth, schema1 bool) (*testRegistryV2, error) {
tmp := t.TempDir()
func newTestRegistryV2At(c *check.C, url string, auth, schema1 bool) (*testRegistryV2, error) {
tmp := c.MkDir()
template := `version: 0.1
loglevel: debug
storage:
@@ -95,10 +94,10 @@ compatibility:
cmd = exec.Command(binaryV2, "serve", confPath)
}
consumeAndLogOutputs(t, fmt.Sprintf("registry-%s", url), cmd)
consumeAndLogOutputs(c, fmt.Sprintf("registry-%s", url), cmd)
if err := cmd.Start(); err != nil {
if os.IsNotExist(err) {
t.Skip(err.Error())
c.Skip(err.Error())
}
return nil, err
}
@@ -111,9 +110,9 @@ compatibility:
}, nil
}
func (r *testRegistryV2) Ping() error {
func (t *testRegistryV2) Ping() error {
// We always ping through HTTP for our test registry.
resp, err := http.Get(fmt.Sprintf("http://%s/v2/", r.url))
resp, err := http.Get(fmt.Sprintf("http://%s/v2/", t.url))
if err != nil {
return err
}
@@ -124,8 +123,8 @@ func (r *testRegistryV2) Ping() error {
return nil
}
func (r *testRegistryV2) tearDown() {
func (t *testRegistryV2) tearDown(c *check.C) {
// Its undocumented what Kill() returns if the process has terminated,
// so we couldnt check just for that. This is running in a container anyway…
_ = r.cmd.Process.Kill()
_ = t.cmd.Process.Kill()
}

View File

@@ -6,28 +6,23 @@ import (
"os"
"os/exec"
"strings"
"testing"
"github.com/containers/image/v5/signature"
"github.com/stretchr/testify/require"
"github.com/stretchr/testify/suite"
"gopkg.in/check.v1"
)
const (
gpgBinary = "gpg"
)
func TestSigning(t *testing.T) {
suite.Run(t, &signingSuite{})
func init() {
check.Suite(&SigningSuite{})
}
type signingSuite struct {
suite.Suite
type SigningSuite struct {
fingerprint string
}
var _ = suite.SetupAllSuite(&signingSuite{})
func findFingerprint(lineBytes []byte) (string, error) {
lines := string(lineBytes)
for _, line := range strings.Split(lines, "\n") {
@@ -39,41 +34,43 @@ func findFingerprint(lineBytes []byte) (string, error) {
return "", errors.New("No fingerprint found")
}
func (s *signingSuite) SetupSuite() {
t := s.T()
func (s *SigningSuite) SetUpSuite(c *check.C) {
_, err := exec.LookPath(skopeoBinary)
require.NoError(t, err)
c.Assert(err, check.IsNil)
gpgHome := t.TempDir()
t.Setenv("GNUPGHOME", gpgHome)
gpgHome := c.MkDir()
os.Setenv("GNUPGHOME", gpgHome)
runCommandWithInput(t, "Key-Type: RSA\nName-Real: Testing user\n%no-protection\n%commit\n", gpgBinary, "--homedir", gpgHome, "--batch", "--gen-key")
runCommandWithInput(c, "Key-Type: RSA\nName-Real: Testing user\n%no-protection\n%commit\n", gpgBinary, "--homedir", gpgHome, "--batch", "--gen-key")
lines, err := exec.Command(gpgBinary, "--homedir", gpgHome, "--with-colons", "--no-permission-warning", "--fingerprint").Output()
require.NoError(t, err)
c.Assert(err, check.IsNil)
s.fingerprint, err = findFingerprint(lines)
require.NoError(t, err)
c.Assert(err, check.IsNil)
}
func (s *signingSuite) TestSignVerifySmoke() {
t := s.T()
func (s *SigningSuite) TearDownSuite(c *check.C) {
os.Unsetenv("GNUPGHOME")
}
func (s *SigningSuite) TestSignVerifySmoke(c *check.C) {
mech, _, err := signature.NewEphemeralGPGSigningMechanism([]byte{})
require.NoError(t, err)
c.Assert(err, check.IsNil)
defer mech.Close()
if err := mech.SupportsSigning(); err != nil { // FIXME? Test that verification and policy enforcement works, using signatures from fixtures
t.Skipf("Signing not supported: %v", err)
c.Skip(fmt.Sprintf("Signing not supported: %v", err))
}
manifestPath := "fixtures/image.manifest.json"
dockerReference := "testing/smoketest"
sigOutput, err := os.CreateTemp("", "sig")
require.NoError(t, err)
c.Assert(err, check.IsNil)
defer os.Remove(sigOutput.Name())
assertSkopeoSucceeds(t, "^$", "standalone-sign", "-o", sigOutput.Name(),
assertSkopeoSucceeds(c, "^$", "standalone-sign", "-o", sigOutput.Name(),
manifestPath, dockerReference, s.fingerprint)
expected := fmt.Sprintf("^Signature verified using fingerprint %s, digest %s\n$", s.fingerprint, TestImageManifestDigest)
assertSkopeoSucceeds(t, expected, "standalone-verify", manifestPath,
expected := fmt.Sprintf("^Signature verified, digest %s\n$", TestImageManifestDigest)
assertSkopeoSucceeds(c, expected, "standalone-verify", manifestPath,
dockerReference, s.fingerprint, sigOutput.Name())
}

View File

@@ -9,16 +9,13 @@ import (
"path/filepath"
"regexp"
"strings"
"testing"
"github.com/containers/image/v5/docker"
"github.com/containers/image/v5/docker/reference"
"github.com/containers/image/v5/manifest"
"github.com/containers/image/v5/types"
imgspecv1 "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"github.com/stretchr/testify/suite"
"gopkg.in/check.v1"
)
const (
@@ -36,36 +33,30 @@ const (
pullableRepoWithLatestTag = "registry.k8s.io/pause"
)
func TestSync(t *testing.T) {
suite.Run(t, &syncSuite{})
func init() {
check.Suite(&SyncSuite{})
}
type syncSuite struct {
suite.Suite
type SyncSuite struct {
cluster *openshiftCluster
registry *testRegistryV2
}
var _ = suite.SetupAllSuite(&syncSuite{})
var _ = suite.TearDownAllSuite(&syncSuite{})
func (s *syncSuite) SetupSuite() {
t := s.T()
func (s *SyncSuite) SetUpSuite(c *check.C) {
const registryAuth = false
const registrySchema1 = false
if os.Getenv("SKOPEO_LOCAL_TESTS") == "1" {
t.Log("Running tests without a container")
c.Log("Running tests without a container")
fmt.Printf("NOTE: tests requires a V2 registry at url=%s, with auth=%t, schema1=%t \n", v2DockerRegistryURL, registryAuth, registrySchema1)
return
}
if os.Getenv("SKOPEO_CONTAINER_TESTS") != "1" {
t.Skip("Not running in a container, refusing to affect user state")
c.Skip("Not running in a container, refusing to affect user state")
}
s.cluster = startOpenshiftCluster(t) // FIXME: Set up TLS for the docker registry port instead of using "--tls-verify=false" all over the place.
s.cluster = startOpenshiftCluster(c) // FIXME: Set up TLS for the docker registry port instead of using "--tls-verify=false" all over the place.
for _, stream := range []string{"unsigned", "personal", "official", "naming", "cosigned", "compression", "schema1", "schema2"} {
isJSON := fmt.Sprintf(`{
@@ -76,42 +67,41 @@ func (s *syncSuite) SetupSuite() {
},
"spec": {}
}`, stream)
runCommandWithInput(t, isJSON, "oc", "create", "-f", "-")
runCommandWithInput(c, isJSON, "oc", "create", "-f", "-")
}
// FIXME: Set up TLS for the docker registry port instead of using "--tls-verify=false" all over the place.
s.registry = setupRegistryV2At(t, v2DockerRegistryURL, registryAuth, registrySchema1)
s.registry = setupRegistryV2At(c, v2DockerRegistryURL, registryAuth, registrySchema1)
gpgHome := t.TempDir()
t.Setenv("GNUPGHOME", gpgHome)
gpgHome := c.MkDir()
os.Setenv("GNUPGHOME", gpgHome)
for _, key := range []string{"personal", "official"} {
batchInput := fmt.Sprintf("Key-Type: RSA\nName-Real: Test key - %s\nName-email: %s@example.com\n%%no-protection\n%%commit\n",
key, key)
runCommandWithInput(t, batchInput, gpgBinary, "--batch", "--gen-key")
runCommandWithInput(c, batchInput, gpgBinary, "--batch", "--gen-key")
out := combinedOutputOfCommand(t, gpgBinary, "--armor", "--export", fmt.Sprintf("%s@example.com", key))
out := combinedOutputOfCommand(c, gpgBinary, "--armor", "--export", fmt.Sprintf("%s@example.com", key))
err := os.WriteFile(filepath.Join(gpgHome, fmt.Sprintf("%s-pubkey.gpg", key)),
[]byte(out), 0600)
require.NoError(t, err)
c.Assert(err, check.IsNil)
}
}
func (s *syncSuite) TearDownSuite() {
t := s.T()
func (s *SyncSuite) TearDownSuite(c *check.C) {
if os.Getenv("SKOPEO_LOCAL_TESTS") == "1" {
return
}
if s.registry != nil {
s.registry.tearDown()
s.registry.tearDown(c)
}
if s.cluster != nil {
s.cluster.tearDown(t)
s.cluster.tearDown(c)
}
}
func assertNumberOfManifestsInSubdirs(t *testing.T, dir string, expectedCount int) {
func assertNumberOfManifestsInSubdirs(c *check.C, dir string, expectedCount int) {
nManifests := 0
err := filepath.WalkDir(dir, func(path string, d fs.DirEntry, err error) error {
if err != nil {
@@ -123,163 +113,156 @@ func assertNumberOfManifestsInSubdirs(t *testing.T, dir string, expectedCount in
}
return nil
})
require.NoError(t, err)
assert.Equal(t, expectedCount, nManifests)
c.Assert(err, check.IsNil)
c.Assert(nManifests, check.Equals, expectedCount)
}
func (s *syncSuite) TestDocker2DirTagged() {
t := s.T()
tmpDir := t.TempDir()
func (s *SyncSuite) TestDocker2DirTagged(c *check.C) {
tmpDir := c.MkDir()
// FIXME: It would be nice to use one of the local Docker registries instead of needing an Internet connection.
image := pullableTaggedImage
imageRef, err := docker.ParseReference(fmt.Sprintf("//%s", image))
require.NoError(t, err)
c.Assert(err, check.IsNil)
imagePath := imageRef.DockerReference().String()
dir1 := path.Join(tmpDir, "dir1")
dir2 := path.Join(tmpDir, "dir2")
// sync docker => dir
assertSkopeoSucceeds(t, "", "sync", "--scoped", "--src", "docker", "--dest", "dir", image, dir1)
assertSkopeoSucceeds(c, "", "sync", "--scoped", "--src", "docker", "--dest", "dir", image, dir1)
_, err = os.Stat(path.Join(dir1, imagePath, "manifest.json"))
require.NoError(t, err)
c.Assert(err, check.IsNil)
// copy docker => dir
assertSkopeoSucceeds(t, "", "copy", "docker://"+image, "dir:"+dir2)
assertSkopeoSucceeds(c, "", "copy", "docker://"+image, "dir:"+dir2)
_, err = os.Stat(path.Join(dir2, "manifest.json"))
require.NoError(t, err)
c.Assert(err, check.IsNil)
out := combinedOutputOfCommand(t, "diff", "-urN", path.Join(dir1, imagePath), dir2)
assert.Equal(t, "", out)
out := combinedOutputOfCommand(c, "diff", "-urN", path.Join(dir1, imagePath), dir2)
c.Assert(out, check.Equals, "")
}
func (s *syncSuite) TestDocker2DirTaggedAll() {
t := s.T()
tmpDir := t.TempDir()
func (s *SyncSuite) TestDocker2DirTaggedAll(c *check.C) {
tmpDir := c.MkDir()
// FIXME: It would be nice to use one of the local Docker registries instead of needing an Internet connection.
image := pullableTaggedManifestList
imageRef, err := docker.ParseReference(fmt.Sprintf("//%s", image))
require.NoError(t, err)
c.Assert(err, check.IsNil)
imagePath := imageRef.DockerReference().String()
dir1 := path.Join(tmpDir, "dir1")
dir2 := path.Join(tmpDir, "dir2")
// sync docker => dir
assertSkopeoSucceeds(t, "", "sync", "--all", "--scoped", "--src", "docker", "--dest", "dir", image, dir1)
assertSkopeoSucceeds(c, "", "sync", "--all", "--scoped", "--src", "docker", "--dest", "dir", image, dir1)
_, err = os.Stat(path.Join(dir1, imagePath, "manifest.json"))
require.NoError(t, err)
c.Assert(err, check.IsNil)
// copy docker => dir
assertSkopeoSucceeds(t, "", "copy", "--all", "docker://"+image, "dir:"+dir2)
assertSkopeoSucceeds(c, "", "copy", "--all", "docker://"+image, "dir:"+dir2)
_, err = os.Stat(path.Join(dir2, "manifest.json"))
require.NoError(t, err)
c.Assert(err, check.IsNil)
out := combinedOutputOfCommand(t, "diff", "-urN", path.Join(dir1, imagePath), dir2)
assert.Equal(t, "", out)
out := combinedOutputOfCommand(c, "diff", "-urN", path.Join(dir1, imagePath), dir2)
c.Assert(out, check.Equals, "")
}
func (s *syncSuite) TestPreserveDigests() {
t := s.T()
tmpDir := t.TempDir()
func (s *SyncSuite) TestPreserveDigests(c *check.C) {
tmpDir := c.MkDir()
// FIXME: It would be nice to use one of the local Docker registries instead of needing an Internet connection.
image := pullableTaggedManifestList
// copy docker => dir
assertSkopeoSucceeds(t, "", "copy", "--all", "--preserve-digests", "docker://"+image, "dir:"+tmpDir)
assertSkopeoSucceeds(c, "", "copy", "--all", "--preserve-digests", "docker://"+image, "dir:"+tmpDir)
_, err := os.Stat(path.Join(tmpDir, "manifest.json"))
require.NoError(t, err)
c.Assert(err, check.IsNil)
assertSkopeoFails(t, ".*Instructed to preserve digests.*", "copy", "--all", "--preserve-digests", "--format=oci", "docker://"+image, "dir:"+tmpDir)
assertSkopeoFails(c, ".*Instructed to preserve digests.*", "copy", "--all", "--preserve-digests", "--format=oci", "docker://"+image, "dir:"+tmpDir)
}
func (s *syncSuite) TestScoped() {
t := s.T()
func (s *SyncSuite) TestScoped(c *check.C) {
// FIXME: It would be nice to use one of the local Docker registries instead of needing an Internet connection.
image := pullableTaggedImage
imageRef, err := docker.ParseReference(fmt.Sprintf("//%s", image))
require.NoError(t, err)
c.Assert(err, check.IsNil)
imagePath := imageRef.DockerReference().String()
dir1 := t.TempDir()
assertSkopeoSucceeds(t, "", "sync", "--src", "docker", "--dest", "dir", image, dir1)
dir1 := c.MkDir()
assertSkopeoSucceeds(c, "", "sync", "--src", "docker", "--dest", "dir", image, dir1)
_, err = os.Stat(path.Join(dir1, path.Base(imagePath), "manifest.json"))
require.NoError(t, err)
c.Assert(err, check.IsNil)
assertSkopeoSucceeds(t, "", "sync", "--scoped", "--src", "docker", "--dest", "dir", image, dir1)
assertSkopeoSucceeds(c, "", "sync", "--scoped", "--src", "docker", "--dest", "dir", image, dir1)
_, err = os.Stat(path.Join(dir1, imagePath, "manifest.json"))
require.NoError(t, err)
c.Assert(err, check.IsNil)
}
func (s *syncSuite) TestDirIsNotOverwritten() {
t := s.T()
func (s *SyncSuite) TestDirIsNotOverwritten(c *check.C) {
// FIXME: It would be nice to use one of the local Docker registries instead of needing an Internet connection.
image := pullableRepoWithLatestTag
imageRef, err := docker.ParseReference(fmt.Sprintf("//%s", image))
require.NoError(t, err)
c.Assert(err, check.IsNil)
imagePath := imageRef.DockerReference().String()
// make a copy of the image in the local registry
assertSkopeoSucceeds(t, "", "copy", "--dest-tls-verify=false", "docker://"+image, "docker://"+path.Join(v2DockerRegistryURL, reference.Path(imageRef.DockerReference())))
assertSkopeoSucceeds(c, "", "copy", "--dest-tls-verify=false", "docker://"+image, "docker://"+path.Join(v2DockerRegistryURL, reference.Path(imageRef.DockerReference())))
//sync upstream image to dir, not scoped
dir1 := t.TempDir()
assertSkopeoSucceeds(t, "", "sync", "--src", "docker", "--dest", "dir", image, dir1)
dir1 := c.MkDir()
assertSkopeoSucceeds(c, "", "sync", "--src", "docker", "--dest", "dir", image, dir1)
_, err = os.Stat(path.Join(dir1, path.Base(imagePath), "manifest.json"))
require.NoError(t, err)
c.Assert(err, check.IsNil)
//sync local registry image to dir, not scoped
assertSkopeoFails(t, ".*Refusing to overwrite destination directory.*", "sync", "--src-tls-verify=false", "--src", "docker", "--dest", "dir", path.Join(v2DockerRegistryURL, reference.Path(imageRef.DockerReference())), dir1)
assertSkopeoFails(c, ".*Refusing to overwrite destination directory.*", "sync", "--src-tls-verify=false", "--src", "docker", "--dest", "dir", path.Join(v2DockerRegistryURL, reference.Path(imageRef.DockerReference())), dir1)
//sync local registry image to dir, scoped
imageRef, err = docker.ParseReference(fmt.Sprintf("//%s", path.Join(v2DockerRegistryURL, reference.Path(imageRef.DockerReference()))))
require.NoError(t, err)
c.Assert(err, check.IsNil)
imagePath = imageRef.DockerReference().String()
assertSkopeoSucceeds(t, "", "sync", "--scoped", "--src-tls-verify=false", "--src", "docker", "--dest", "dir", path.Join(v2DockerRegistryURL, reference.Path(imageRef.DockerReference())), dir1)
assertSkopeoSucceeds(c, "", "sync", "--scoped", "--src-tls-verify=false", "--src", "docker", "--dest", "dir", path.Join(v2DockerRegistryURL, reference.Path(imageRef.DockerReference())), dir1)
_, err = os.Stat(path.Join(dir1, imagePath, "manifest.json"))
require.NoError(t, err)
c.Assert(err, check.IsNil)
}
func (s *syncSuite) TestDocker2DirUntagged() {
t := s.T()
tmpDir := t.TempDir()
func (s *SyncSuite) TestDocker2DirUntagged(c *check.C) {
tmpDir := c.MkDir()
// FIXME: It would be nice to use one of the local Docker registries instead of needing an Internet connection.
image := pullableRepo
imageRef, err := docker.ParseReference(fmt.Sprintf("//%s", image))
require.NoError(t, err)
c.Assert(err, check.IsNil)
imagePath := imageRef.DockerReference().String()
dir1 := path.Join(tmpDir, "dir1")
assertSkopeoSucceeds(t, "", "sync", "--scoped", "--src", "docker", "--dest", "dir", image, dir1)
assertSkopeoSucceeds(c, "", "sync", "--scoped", "--src", "docker", "--dest", "dir", image, dir1)
sysCtx := types.SystemContext{}
tags, err := docker.GetRepositoryTags(context.Background(), &sysCtx, imageRef)
require.NoError(t, err)
assert.NotZero(t, len(tags))
c.Assert(err, check.IsNil)
c.Check(len(tags), check.Not(check.Equals), 0)
nManifests, err := filepath.Glob(path.Join(dir1, path.Dir(imagePath), "*", "manifest.json"))
require.NoError(t, err)
assert.Len(t, nManifests, len(tags))
c.Assert(err, check.IsNil)
c.Assert(len(nManifests), check.Equals, len(tags))
}
func (s *syncSuite) TestYamlUntagged() {
t := s.T()
tmpDir := t.TempDir()
func (s *SyncSuite) TestYamlUntagged(c *check.C) {
tmpDir := c.MkDir()
dir1 := path.Join(tmpDir, "dir1")
image := pullableRepo
imageRef, err := docker.ParseReference(fmt.Sprintf("//%s", image))
require.NoError(t, err)
c.Assert(err, check.IsNil)
imagePath := imageRef.DockerReference().Name()
sysCtx := types.SystemContext{}
tags, err := docker.GetRepositoryTags(context.Background(), &sysCtx, imageRef)
require.NoError(t, err)
assert.NotZero(t, len(tags))
c.Assert(err, check.IsNil)
c.Check(len(tags), check.Not(check.Equals), 0)
yamlConfig := fmt.Sprintf(`
%s:
@@ -290,8 +273,8 @@ func (s *syncSuite) TestYamlUntagged() {
// sync to the local registry
yamlFile := path.Join(tmpDir, "registries.yaml")
err = os.WriteFile(yamlFile, []byte(yamlConfig), 0644)
require.NoError(t, err)
assertSkopeoSucceeds(t, "", "sync", "--scoped", "--src", "yaml", "--dest", "docker", "--dest-tls-verify=false", yamlFile, v2DockerRegistryURL)
c.Assert(err, check.IsNil)
assertSkopeoSucceeds(c, "", "sync", "--scoped", "--src", "yaml", "--dest", "docker", "--dest-tls-verify=false", yamlFile, v2DockerRegistryURL)
// sync back from local registry to a folder
os.Remove(yamlFile)
yamlConfig = fmt.Sprintf(`
@@ -302,24 +285,23 @@ func (s *syncSuite) TestYamlUntagged() {
`, v2DockerRegistryURL, imagePath)
err = os.WriteFile(yamlFile, []byte(yamlConfig), 0644)
require.NoError(t, err)
assertSkopeoSucceeds(t, "", "sync", "--scoped", "--src", "yaml", "--dest", "dir", yamlFile, dir1)
c.Assert(err, check.IsNil)
assertSkopeoSucceeds(c, "", "sync", "--scoped", "--src", "yaml", "--dest", "dir", yamlFile, dir1)
sysCtx = types.SystemContext{
DockerInsecureSkipTLSVerify: types.NewOptionalBool(true),
}
localImageRef, err := docker.ParseReference(fmt.Sprintf("//%s/%s", v2DockerRegistryURL, imagePath))
require.NoError(t, err)
c.Assert(err, check.IsNil)
localTags, err := docker.GetRepositoryTags(context.Background(), &sysCtx, localImageRef)
require.NoError(t, err)
assert.NotZero(t, len(localTags))
assert.Len(t, localTags, len(tags))
assertNumberOfManifestsInSubdirs(t, dir1, len(tags))
c.Assert(err, check.IsNil)
c.Check(len(localTags), check.Not(check.Equals), 0)
c.Assert(len(localTags), check.Equals, len(tags))
assertNumberOfManifestsInSubdirs(c, dir1, len(tags))
}
func (s *syncSuite) TestYamlRegex2Dir() {
t := s.T()
tmpDir := t.TempDir()
func (s *SyncSuite) TestYamlRegex2Dir(c *check.C) {
tmpDir := c.MkDir()
dir1 := path.Join(tmpDir, "dir1")
yamlConfig := `
@@ -329,18 +311,17 @@ registry.k8s.io:
`
// the ↑ regex strings always matches only 2 images
var nTags = 2
assert.NotZero(t, nTags)
c.Assert(nTags, check.Not(check.Equals), 0)
yamlFile := path.Join(tmpDir, "registries.yaml")
err := os.WriteFile(yamlFile, []byte(yamlConfig), 0644)
require.NoError(t, err)
assertSkopeoSucceeds(t, "", "sync", "--scoped", "--src", "yaml", "--dest", "dir", yamlFile, dir1)
assertNumberOfManifestsInSubdirs(t, dir1, nTags)
c.Assert(err, check.IsNil)
assertSkopeoSucceeds(c, "", "sync", "--scoped", "--src", "yaml", "--dest", "dir", yamlFile, dir1)
assertNumberOfManifestsInSubdirs(c, dir1, nTags)
}
func (s *syncSuite) TestYamlDigest2Dir() {
t := s.T()
tmpDir := t.TempDir()
func (s *SyncSuite) TestYamlDigest2Dir(c *check.C) {
tmpDir := c.MkDir()
dir1 := path.Join(tmpDir, "dir1")
yamlConfig := `
@@ -351,14 +332,13 @@ registry.k8s.io:
`
yamlFile := path.Join(tmpDir, "registries.yaml")
err := os.WriteFile(yamlFile, []byte(yamlConfig), 0644)
require.NoError(t, err)
assertSkopeoSucceeds(t, "", "sync", "--scoped", "--src", "yaml", "--dest", "dir", yamlFile, dir1)
assertNumberOfManifestsInSubdirs(t, dir1, 1)
c.Assert(err, check.IsNil)
assertSkopeoSucceeds(c, "", "sync", "--scoped", "--src", "yaml", "--dest", "dir", yamlFile, dir1)
assertNumberOfManifestsInSubdirs(c, dir1, 1)
}
func (s *syncSuite) TestYaml2Dir() {
t := s.T()
tmpDir := t.TempDir()
func (s *SyncSuite) TestYaml2Dir(c *check.C) {
tmpDir := c.MkDir()
dir1 := path.Join(tmpDir, "dir1")
yamlConfig := `
@@ -386,26 +366,25 @@ quay.io:
nTags++
}
}
assert.NotZero(t, nTags)
c.Assert(nTags, check.Not(check.Equals), 0)
yamlFile := path.Join(tmpDir, "registries.yaml")
err := os.WriteFile(yamlFile, []byte(yamlConfig), 0644)
require.NoError(t, err)
assertSkopeoSucceeds(t, "", "sync", "--scoped", "--src", "yaml", "--dest", "dir", yamlFile, dir1)
assertNumberOfManifestsInSubdirs(t, dir1, nTags)
c.Assert(err, check.IsNil)
assertSkopeoSucceeds(c, "", "sync", "--scoped", "--src", "yaml", "--dest", "dir", yamlFile, dir1)
assertNumberOfManifestsInSubdirs(c, dir1, nTags)
}
func (s *syncSuite) TestYamlTLSVerify() {
t := s.T()
func (s *SyncSuite) TestYamlTLSVerify(c *check.C) {
const localRegURL = "docker://" + v2DockerRegistryURL + "/"
tmpDir := t.TempDir()
tmpDir := c.MkDir()
dir1 := path.Join(tmpDir, "dir1")
image := pullableRepoWithLatestTag
tag := "latest"
// FIXME: It would be nice to use one of the local Docker registries instead of needing an Internet connection.
// copy docker => docker
assertSkopeoSucceeds(t, "", "copy", "--dest-tls-verify=false", "docker://"+image+":"+tag, localRegURL+image+":"+tag)
assertSkopeoSucceeds(c, "", "copy", "--dest-tls-verify=false", "docker://"+image+":"+tag, localRegURL+image+":"+tag)
yamlTemplate := `
%s:
@@ -417,7 +396,7 @@ func (s *syncSuite) TestYamlTLSVerify() {
testCfg := []struct {
tlsVerify string
msg string
checker func(t *testing.T, regexp string, args ...string)
checker func(c *check.C, regexp string, args ...string)
}{
{
tlsVerify: "tls-verify: false",
@@ -441,18 +420,17 @@ func (s *syncSuite) TestYamlTLSVerify() {
yamlConfig := fmt.Sprintf(yamlTemplate, v2DockerRegistryURL, cfg.tlsVerify, image, tag)
yamlFile := path.Join(tmpDir, "registries.yaml")
err := os.WriteFile(yamlFile, []byte(yamlConfig), 0644)
require.NoError(t, err)
c.Assert(err, check.IsNil)
cfg.checker(t, cfg.msg, "sync", "--scoped", "--src", "yaml", "--dest", "dir", yamlFile, dir1)
cfg.checker(c, cfg.msg, "sync", "--scoped", "--src", "yaml", "--dest", "dir", yamlFile, dir1)
os.Remove(yamlFile)
os.RemoveAll(dir1)
}
}
func (s *syncSuite) TestSyncManifestOutput() {
t := s.T()
tmpDir := t.TempDir()
func (s *SyncSuite) TestSyncManifestOutput(c *check.C) {
tmpDir := c.MkDir()
destDir1 := filepath.Join(tmpDir, "dest1")
destDir2 := filepath.Join(tmpDir, "dest2")
@@ -461,162 +439,154 @@ func (s *syncSuite) TestSyncManifestOutput() {
//Split image:tag path from image URI for manifest comparison
imageDir := pullableTaggedImage[strings.LastIndex(pullableTaggedImage, "/")+1:]
assertSkopeoSucceeds(t, "", "sync", "--format=oci", "--all", "--src", "docker", "--dest", "dir", pullableTaggedImage, destDir1)
verifyManifestMIMEType(t, filepath.Join(destDir1, imageDir), imgspecv1.MediaTypeImageManifest)
assertSkopeoSucceeds(t, "", "sync", "--format=v2s2", "--all", "--src", "docker", "--dest", "dir", pullableTaggedImage, destDir2)
verifyManifestMIMEType(t, filepath.Join(destDir2, imageDir), manifest.DockerV2Schema2MediaType)
assertSkopeoSucceeds(t, "", "sync", "--format=v2s1", "--all", "--src", "docker", "--dest", "dir", pullableTaggedImage, destDir3)
verifyManifestMIMEType(t, filepath.Join(destDir3, imageDir), manifest.DockerV2Schema1SignedMediaType)
assertSkopeoSucceeds(c, "", "sync", "--format=oci", "--all", "--src", "docker", "--dest", "dir", pullableTaggedImage, destDir1)
verifyManifestMIMEType(c, filepath.Join(destDir1, imageDir), imgspecv1.MediaTypeImageManifest)
assertSkopeoSucceeds(c, "", "sync", "--format=v2s2", "--all", "--src", "docker", "--dest", "dir", pullableTaggedImage, destDir2)
verifyManifestMIMEType(c, filepath.Join(destDir2, imageDir), manifest.DockerV2Schema2MediaType)
assertSkopeoSucceeds(c, "", "sync", "--format=v2s1", "--all", "--src", "docker", "--dest", "dir", pullableTaggedImage, destDir3)
verifyManifestMIMEType(c, filepath.Join(destDir3, imageDir), manifest.DockerV2Schema1SignedMediaType)
}
func (s *syncSuite) TestDocker2DockerTagged() {
t := s.T()
func (s *SyncSuite) TestDocker2DockerTagged(c *check.C) {
const localRegURL = "docker://" + v2DockerRegistryURL + "/"
tmpDir := t.TempDir()
tmpDir := c.MkDir()
// FIXME: It would be nice to use one of the local Docker registries instead of needing an Internet connection.
image := pullableTaggedImage
imageRef, err := docker.ParseReference(fmt.Sprintf("//%s", image))
require.NoError(t, err)
c.Assert(err, check.IsNil)
imagePath := imageRef.DockerReference().String()
dir1 := path.Join(tmpDir, "dir1")
dir2 := path.Join(tmpDir, "dir2")
// sync docker => docker
assertSkopeoSucceeds(t, "", "sync", "--scoped", "--dest-tls-verify=false", "--src", "docker", "--dest", "docker", image, v2DockerRegistryURL)
assertSkopeoSucceeds(c, "", "sync", "--scoped", "--dest-tls-verify=false", "--src", "docker", "--dest", "docker", image, v2DockerRegistryURL)
// copy docker => dir
assertSkopeoSucceeds(t, "", "copy", "docker://"+image, "dir:"+dir1)
assertSkopeoSucceeds(c, "", "copy", "docker://"+image, "dir:"+dir1)
_, err = os.Stat(path.Join(dir1, "manifest.json"))
require.NoError(t, err)
c.Assert(err, check.IsNil)
// copy docker => dir
assertSkopeoSucceeds(t, "", "copy", "--src-tls-verify=false", localRegURL+imagePath, "dir:"+dir2)
assertSkopeoSucceeds(c, "", "copy", "--src-tls-verify=false", localRegURL+imagePath, "dir:"+dir2)
_, err = os.Stat(path.Join(dir2, "manifest.json"))
require.NoError(t, err)
c.Assert(err, check.IsNil)
out := combinedOutputOfCommand(t, "diff", "-urN", dir1, dir2)
assert.Equal(t, "", out)
out := combinedOutputOfCommand(c, "diff", "-urN", dir1, dir2)
c.Assert(out, check.Equals, "")
}
func (s *syncSuite) TestDir2DockerTagged() {
t := s.T()
func (s *SyncSuite) TestDir2DockerTagged(c *check.C) {
const localRegURL = "docker://" + v2DockerRegistryURL + "/"
tmpDir := t.TempDir()
tmpDir := c.MkDir()
// FIXME: It would be nice to use one of the local Docker registries instead of needing an Internet connection.
image := pullableRepoWithLatestTag
dir1 := path.Join(tmpDir, "dir1")
err := os.Mkdir(dir1, 0755)
require.NoError(t, err)
c.Assert(err, check.IsNil)
dir2 := path.Join(tmpDir, "dir2")
err = os.Mkdir(dir2, 0755)
require.NoError(t, err)
c.Assert(err, check.IsNil)
// create leading dirs
err = os.MkdirAll(path.Dir(path.Join(dir1, image)), 0755)
require.NoError(t, err)
c.Assert(err, check.IsNil)
// copy docker => dir
assertSkopeoSucceeds(t, "", "copy", "docker://"+image, "dir:"+path.Join(dir1, image))
assertSkopeoSucceeds(c, "", "copy", "docker://"+image, "dir:"+path.Join(dir1, image))
_, err = os.Stat(path.Join(dir1, image, "manifest.json"))
require.NoError(t, err)
c.Assert(err, check.IsNil)
// sync dir => docker
assertSkopeoSucceeds(t, "", "sync", "--scoped", "--dest-tls-verify=false", "--src", "dir", "--dest", "docker", dir1, v2DockerRegistryURL)
assertSkopeoSucceeds(c, "", "sync", "--scoped", "--dest-tls-verify=false", "--src", "dir", "--dest", "docker", dir1, v2DockerRegistryURL)
// create leading dirs
err = os.MkdirAll(path.Dir(path.Join(dir2, image)), 0755)
require.NoError(t, err)
c.Assert(err, check.IsNil)
// copy docker => dir
assertSkopeoSucceeds(t, "", "copy", "--src-tls-verify=false", localRegURL+image, "dir:"+path.Join(dir2, image))
assertSkopeoSucceeds(c, "", "copy", "--src-tls-verify=false", localRegURL+image, "dir:"+path.Join(dir2, image))
_, err = os.Stat(path.Join(dir2, image, "manifest.json"))
require.NoError(t, err)
c.Assert(err, check.IsNil)
out := combinedOutputOfCommand(t, "diff", "-urN", dir1, dir2)
assert.Equal(t, "", out)
out := combinedOutputOfCommand(c, "diff", "-urN", dir1, dir2)
c.Assert(out, check.Equals, "")
}
func (s *syncSuite) TestFailsWithDir2Dir() {
t := s.T()
tmpDir := t.TempDir()
func (s *SyncSuite) TestFailsWithDir2Dir(c *check.C) {
tmpDir := c.MkDir()
dir1 := path.Join(tmpDir, "dir1")
dir2 := path.Join(tmpDir, "dir2")
// sync dir => dir is not allowed
assertSkopeoFails(t, ".*sync from 'dir' to 'dir' not implemented.*", "sync", "--scoped", "--src", "dir", "--dest", "dir", dir1, dir2)
assertSkopeoFails(c, ".*sync from 'dir' to 'dir' not implemented.*", "sync", "--scoped", "--src", "dir", "--dest", "dir", dir1, dir2)
}
func (s *syncSuite) TestFailsNoSourceImages() {
t := s.T()
tmpDir := t.TempDir()
func (s *SyncSuite) TestFailsNoSourceImages(c *check.C) {
tmpDir := c.MkDir()
assertSkopeoFails(t, ".*No images to sync found in .*",
assertSkopeoFails(c, ".*No images to sync found in .*",
"sync", "--scoped", "--dest-tls-verify=false", "--src", "dir", "--dest", "docker", tmpDir, v2DockerRegistryURL)
assertSkopeoFails(t, ".*Error determining repository tags for repo docker.io/library/hopefully_no_images_will_ever_be_called_like_this: fetching tags list: requested access to the resource is denied.*",
assertSkopeoFails(c, ".*Error determining repository tags for repo docker.io/library/hopefully_no_images_will_ever_be_called_like_this: fetching tags list: requested access to the resource is denied.*",
"sync", "--scoped", "--dest-tls-verify=false", "--src", "docker", "--dest", "docker", "hopefully_no_images_will_ever_be_called_like_this", v2DockerRegistryURL)
}
func (s *syncSuite) TestFailsWithDockerSourceNoRegistry() {
t := s.T()
func (s *SyncSuite) TestFailsWithDockerSourceNoRegistry(c *check.C) {
const regURL = "google.com/namespace/imagename"
tmpDir := t.TempDir()
tmpDir := c.MkDir()
//untagged
assertSkopeoFails(t, ".*StatusCode: 404.*",
assertSkopeoFails(c, ".*StatusCode: 404.*",
"sync", "--scoped", "--src", "docker", "--dest", "dir", regURL, tmpDir)
//tagged
assertSkopeoFails(t, ".*StatusCode: 404.*",
assertSkopeoFails(c, ".*StatusCode: 404.*",
"sync", "--scoped", "--src", "docker", "--dest", "dir", regURL+":thetag", tmpDir)
}
func (s *syncSuite) TestFailsWithDockerSourceUnauthorized() {
t := s.T()
func (s *SyncSuite) TestFailsWithDockerSourceUnauthorized(c *check.C) {
const repo = "privateimagenamethatshouldnotbepublic"
tmpDir := t.TempDir()
tmpDir := c.MkDir()
//untagged
assertSkopeoFails(t, ".*requested access to the resource is denied.*",
assertSkopeoFails(c, ".*requested access to the resource is denied.*",
"sync", "--scoped", "--src", "docker", "--dest", "dir", repo, tmpDir)
//tagged
assertSkopeoFails(t, ".*requested access to the resource is denied.*",
assertSkopeoFails(c, ".*requested access to the resource is denied.*",
"sync", "--scoped", "--src", "docker", "--dest", "dir", repo+":thetag", tmpDir)
}
func (s *syncSuite) TestFailsWithDockerSourceNotExisting() {
t := s.T()
func (s *SyncSuite) TestFailsWithDockerSourceNotExisting(c *check.C) {
repo := path.Join(v2DockerRegistryURL, "imagedoesnotexist")
tmpDir := t.TempDir()
tmpDir := c.MkDir()
//untagged
assertSkopeoFails(t, ".*repository name not known to registry.*",
assertSkopeoFails(c, ".*repository name not known to registry.*",
"sync", "--scoped", "--src-tls-verify=false", "--src", "docker", "--dest", "dir", repo, tmpDir)
//tagged
assertSkopeoFails(t, ".*reading manifest.*",
assertSkopeoFails(c, ".*reading manifest.*",
"sync", "--scoped", "--src-tls-verify=false", "--src", "docker", "--dest", "dir", repo+":thetag", tmpDir)
}
func (s *syncSuite) TestFailsWithDirSourceNotExisting() {
t := s.T()
func (s *SyncSuite) TestFailsWithDirSourceNotExisting(c *check.C) {
// Make sure the dir does not exist!
tmpDir := t.TempDir()
tmpDir := c.MkDir()
tmpDir = filepath.Join(tmpDir, "this-does-not-exist")
err := os.RemoveAll(tmpDir)
require.NoError(t, err)
c.Assert(err, check.IsNil)
_, err = os.Stat(path.Join(tmpDir))
assert.True(t, os.IsNotExist(err))
c.Check(os.IsNotExist(err), check.Equals, true)
assertSkopeoFails(t, ".*no such file or directory.*",
assertSkopeoFails(c, ".*no such file or directory.*",
"sync", "--scoped", "--dest-tls-verify=false", "--src", "dir", "--dest", "docker", tmpDir, v2DockerRegistryURL)
}

View File

@@ -4,17 +4,14 @@ import (
"bytes"
"io"
"net"
"net/netip"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
"time"
"github.com/containers/image/v5/manifest"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"gopkg.in/check.v1"
)
const skopeoBinary = "skopeo"
@@ -24,19 +21,19 @@ const testFQIN = "docker://quay.io/libpod/busybox" // tag left off on purpose, s
const testFQIN64 = "docker://quay.io/libpod/busybox:amd64"
const testFQINMultiLayer = "docker://quay.io/libpod/alpine_nginx:latest" // multi-layer
// consumeAndLogOutputStream takes (f, err) from an exec.*Pipe(), and causes all output to it to be logged to t.
func consumeAndLogOutputStream(t *testing.T, id string, f io.ReadCloser, err error) {
require.NoError(t, err)
// consumeAndLogOutputStream takes (f, err) from an exec.*Pipe(), and causes all output to it to be logged to c.
func consumeAndLogOutputStream(c *check.C, id string, f io.ReadCloser, err error) {
c.Assert(err, check.IsNil)
go func() {
defer func() {
f.Close()
t.Logf("Output %s: Closed", id)
c.Logf("Output %s: Closed", id)
}()
buf := make([]byte, 1024)
for {
t.Logf("Output %s: waiting", id)
c.Logf("Output %s: waiting", id)
n, err := f.Read(buf)
t.Logf("Output %s: got %d,%#v: %s", id, n, err, strings.TrimSuffix(string(buf[:n]), "\n"))
c.Logf("Output %s: got %d,%#v: %s", id, n, err, strings.TrimSuffix(string(buf[:n]), "\n"))
if n <= 0 {
break
}
@@ -44,73 +41,72 @@ func consumeAndLogOutputStream(t *testing.T, id string, f io.ReadCloser, err err
}()
}
// consumeAndLogOutputs causes all output to stdout and stderr from an *exec.Cmd to be logged to c.
func consumeAndLogOutputs(t *testing.T, id string, cmd *exec.Cmd) {
// consumeAndLogOutputs causes all output to stdout and stderr from an *exec.Cmd to be logged to c
func consumeAndLogOutputs(c *check.C, id string, cmd *exec.Cmd) {
stdout, err := cmd.StdoutPipe()
consumeAndLogOutputStream(t, id+" stdout", stdout, err)
consumeAndLogOutputStream(c, id+" stdout", stdout, err)
stderr, err := cmd.StderrPipe()
consumeAndLogOutputStream(t, id+" stderr", stderr, err)
consumeAndLogOutputStream(c, id+" stderr", stderr, err)
}
// combinedOutputOfCommand runs a command as if exec.Command().CombinedOutput(), verifies that the exit status is 0, and returns the output,
// or terminates c on failure.
func combinedOutputOfCommand(t *testing.T, name string, args ...string) string {
t.Logf("Running %s %s", name, strings.Join(args, " "))
func combinedOutputOfCommand(c *check.C, name string, args ...string) string {
c.Logf("Running %s %s", name, strings.Join(args, " "))
out, err := exec.Command(name, args...).CombinedOutput()
require.NoError(t, err, "%s", out)
c.Assert(err, check.IsNil, check.Commentf("%s", out))
return string(out)
}
// assertSkopeoSucceeds runs a skopeo command as if exec.Command().CombinedOutput, verifies that the exit status is 0,
// and optionally that the output matches a multi-line regexp if it is nonempty;
// or terminates c on failure
func assertSkopeoSucceeds(t *testing.T, regexp string, args ...string) {
t.Logf("Running %s %s", skopeoBinary, strings.Join(args, " "))
func assertSkopeoSucceeds(c *check.C, regexp string, args ...string) {
c.Logf("Running %s %s", skopeoBinary, strings.Join(args, " "))
out, err := exec.Command(skopeoBinary, args...).CombinedOutput()
assert.NoError(t, err, "%s", out)
c.Assert(err, check.IsNil, check.Commentf("%s", out))
if regexp != "" {
assert.Regexp(t, "(?s)"+regexp, string(out)) // (?s) : '.' will also match newlines
c.Assert(string(out), check.Matches, "(?s)"+regexp) // (?s) : '.' will also match newlines
}
}
// assertSkopeoFails runs a skopeo command as if exec.Command().CombinedOutput, verifies that the exit status is 0,
// and that the output matches a multi-line regexp;
// or terminates c on failure
func assertSkopeoFails(t *testing.T, regexp string, args ...string) {
t.Logf("Running %s %s", skopeoBinary, strings.Join(args, " "))
func assertSkopeoFails(c *check.C, regexp string, args ...string) {
c.Logf("Running %s %s", skopeoBinary, strings.Join(args, " "))
out, err := exec.Command(skopeoBinary, args...).CombinedOutput()
assert.Error(t, err, "%s", out)
assert.Regexp(t, "(?s)"+regexp, string(out)) // (?s) : '.' will also match newlines
c.Assert(err, check.NotNil, check.Commentf("%s", out))
c.Assert(string(out), check.Matches, "(?s)"+regexp) // (?s) : '.' will also match newlines
}
// runCommandWithInput runs a command as if exec.Command(), sending it the input to stdin,
// and verifies that the exit status is 0, or terminates c on failure.
func runCommandWithInput(t *testing.T, input string, name string, args ...string) {
func runCommandWithInput(c *check.C, input string, name string, args ...string) {
cmd := exec.Command(name, args...)
runExecCmdWithInput(t, cmd, input)
runExecCmdWithInput(c, cmd, input)
}
// runExecCmdWithInput runs an exec.Cmd, sending it the input to stdin,
// and verifies that the exit status is 0, or terminates c on failure.
func runExecCmdWithInput(t *testing.T, cmd *exec.Cmd, input string) {
t.Logf("Running %s %s", cmd.Path, strings.Join(cmd.Args, " "))
consumeAndLogOutputs(t, cmd.Path+" "+strings.Join(cmd.Args, " "), cmd)
func runExecCmdWithInput(c *check.C, cmd *exec.Cmd, input string) {
c.Logf("Running %s %s", cmd.Path, strings.Join(cmd.Args, " "))
consumeAndLogOutputs(c, cmd.Path+" "+strings.Join(cmd.Args, " "), cmd)
stdin, err := cmd.StdinPipe()
require.NoError(t, err)
c.Assert(err, check.IsNil)
err = cmd.Start()
require.NoError(t, err)
_, err = io.WriteString(stdin, input)
require.NoError(t, err)
c.Assert(err, check.IsNil)
_, err = stdin.Write([]byte(input))
c.Assert(err, check.IsNil)
err = stdin.Close()
require.NoError(t, err)
c.Assert(err, check.IsNil)
err = cmd.Wait()
assert.NoError(t, err)
c.Assert(err, check.IsNil)
}
// isPortOpen returns true iff the specified port on localhost is open.
func isPortOpen(port uint16) bool {
ap := netip.AddrPortFrom(netip.AddrFrom4([4]byte{127, 0, 0, 1}), port)
conn, err := net.DialTCP("tcp", nil, net.TCPAddrFromAddrPort(ap))
func isPortOpen(port int) bool {
conn, err := net.DialTCP("tcp", nil, &net.TCPAddr{IP: net.IPv4(127, 0, 0, 1), Port: port})
if err != nil {
return false
}
@@ -122,29 +118,29 @@ func isPortOpen(port uint16) bool {
// The checking can be aborted by sending a value to the terminate channel, which the caller should
// always do using
// defer func() {terminate <- true}()
func newPortChecker(t *testing.T, port uint16) (portOpen <-chan bool, terminate chan<- bool) {
func newPortChecker(c *check.C, port int) (portOpen <-chan bool, terminate chan<- bool) {
portOpenBidi := make(chan bool)
// Buffered, so that sending a terminate request after the goroutine has exited does not block.
terminateBidi := make(chan bool, 1)
go func() {
defer func() {
t.Logf("Port checker for port %d exiting", port)
c.Logf("Port checker for port %d exiting", port)
}()
for {
t.Logf("Checking for port %d...", port)
c.Logf("Checking for port %d...", port)
if isPortOpen(port) {
t.Logf("Port %d open", port)
c.Logf("Port %d open", port)
portOpenBidi <- true
return
}
t.Logf("Sleeping for port %d", port)
c.Logf("Sleeping for port %d", port)
sleepChan := time.After(100 * time.Millisecond)
select {
case <-sleepChan: // Try again
t.Logf("Sleeping for port %d done, will retry", port)
c.Logf("Sleeping for port %d done, will retry", port)
case <-terminateBidi:
t.Logf("Check for port %d terminated", port)
c.Logf("Check for port %d terminated", port)
return
}
}
@@ -166,51 +162,54 @@ func modifyEnviron(env []string, name, value string) []string {
// fileFromFixtureFixture applies edits to inputPath and returns a path to the temporary file.
// Callers should defer os.Remove(the_returned_path)
func fileFromFixture(t *testing.T, inputPath string, edits map[string]string) string {
func fileFromFixture(c *check.C, inputPath string, edits map[string]string) string {
contents, err := os.ReadFile(inputPath)
require.NoError(t, err)
c.Assert(err, check.IsNil)
for template, value := range edits {
updated := bytes.ReplaceAll(contents, []byte(template), []byte(value))
require.NotEqual(t, contents, updated, "Replacing %s in %#v failed", template, string(contents)) // Verify that the template has matched something and we are not silently ignoring it.
c.Assert(bytes.Equal(updated, contents), check.Equals, false, check.Commentf("Replacing %s in %#v failed", template, string(contents))) // Verify that the template has matched something and we are not silently ignoring it.
contents = updated
}
file, err := os.CreateTemp("", "policy.json")
require.NoError(t, err)
c.Assert(err, check.IsNil)
path := file.Name()
_, err = file.Write(contents)
require.NoError(t, err)
c.Assert(err, check.IsNil)
err = file.Close()
require.NoError(t, err)
c.Assert(err, check.IsNil)
return path
}
// runDecompressDirs runs decompress-dirs.sh using exec.Command().CombinedOutput, verifies that the exit status is 0,
// and optionally that the output matches a multi-line regexp if it is nonempty; or terminates c on failure
func runDecompressDirs(t *testing.T, args ...string) {
t.Logf("Running %s %s", decompressDirsBinary, strings.Join(args, " "))
func runDecompressDirs(c *check.C, regexp string, args ...string) {
c.Logf("Running %s %s", decompressDirsBinary, strings.Join(args, " "))
for i, dir := range args {
m, err := os.ReadFile(filepath.Join(dir, "manifest.json"))
require.NoError(t, err)
t.Logf("manifest %d before: %s", i+1, string(m))
c.Assert(err, check.IsNil)
c.Logf("manifest %d before: %s", i+1, string(m))
}
out, err := exec.Command(decompressDirsBinary, args...).CombinedOutput()
assert.NoError(t, err, "%s", out)
c.Assert(err, check.IsNil, check.Commentf("%s", out))
for i, dir := range args {
if len(out) > 0 {
t.Logf("output: %s", out)
c.Logf("output: %s", out)
}
m, err := os.ReadFile(filepath.Join(dir, "manifest.json"))
require.NoError(t, err)
t.Logf("manifest %d after: %s", i+1, string(m))
c.Assert(err, check.IsNil)
c.Logf("manifest %d after: %s", i+1, string(m))
}
if regexp != "" {
c.Assert(string(out), check.Matches, "(?s)"+regexp) // (?s) : '.' will also match newlines
}
}
// Verify manifest in a dir: image at dir is expectedMIMEType.
func verifyManifestMIMEType(t *testing.T, dir string, expectedMIMEType string) {
func verifyManifestMIMEType(c *check.C, dir string, expectedMIMEType string) {
manifestBlob, err := os.ReadFile(filepath.Join(dir, "manifest.json"))
require.NoError(t, err)
c.Assert(err, check.IsNil)
mimeType := manifest.GuessMIMEType(manifestBlob)
assert.Equal(t, expectedMIMEType, mimeType)
c.Assert(mimeType, check.Equals, expectedMIMEType)
}

View File

@@ -1,174 +0,0 @@
%global with_debug 1
%if 0%{?with_debug}
%global _find_debuginfo_dwz_opts %{nil}
%global _dwz_low_mem_die_limit 0
%else
%global debug_package %{nil}
%endif
# RHEL's default %%gobuild macro doesn't account for the BUILDTAGS variable, so we
# set it separately here and do not depend on RHEL's go-[s]rpm-macros package
# until that's fixed.
# c9s bz: https://bugzilla.redhat.com/show_bug.cgi?id=2227328
# c8s bz: https://bugzilla.redhat.com/show_bug.cgi?id=2227331
%if %{defined rhel}
%define gobuild(o:) go build -buildmode pie -compiler gc -tags="rpm_crashtraceback libtrust_openssl ${BUILDTAGS:-}" -ldflags "-linkmode=external -compressdwarf=false ${LDFLAGS:-} -B 0x$(head -c20 /dev/urandom|od -An -tx1|tr -d ' \\n') -extldflags '%__global_ldflags'" -a -v -x %{?**};
%endif
%global gomodulesmode GO111MODULE=on
# No btrfs on RHEL
%if %{defined fedora}
%define build_with_btrfs 1
%endif
# Only used in official koji builds
# Copr builds set a separate epoch for all environments
%if %{defined fedora}
%define conditional_epoch 1
%else
%define conditional_epoch 2
%endif
Name: skopeo
%if %{defined copr_username}
Epoch: 102
%else
Epoch: %{conditional_epoch}
%endif
# DO NOT TOUCH the Version string!
# The TRUE source of this specfile is:
# https://github.com/containers/skopeo/blob/main/rpm/skopeo.spec
# If that's what you're reading, Version must be 0, and will be updated by Packit for
# copr and koji builds.
# If you're reading this on dist-git, the version is automatically filled in by Packit.
Version: 0
# The `AND` needs to be uppercase in the License for SPDX compatibility
License: Apache-2.0 AND BSD-2-Clause AND BSD-3-Clause AND ISC AND MIT AND MPL-2.0
Release: %autorelease
%if %{defined golang_arches_future}
ExclusiveArch: %{golang_arches_future}
%else
ExclusiveArch: aarch64 ppc64le s390x x86_64
%endif
Summary: Inspect container images and repositories on registries
URL: https://github.com/containers/%{name}
# Tarball fetched from upstream
Source0: %{url}/archive/v%{version}.tar.gz
BuildRequires: %{_bindir}/go-md2man
%if %{defined build_with_btrfs}
BuildRequires: btrfs-progs-devel
%endif
BuildRequires: git-core
BuildRequires: golang
%if !%{defined gobuild}
BuildRequires: go-rpm-macros
%endif
BuildRequires: gpgme-devel
BuildRequires: libassuan-devel
BuildRequires: pkgconfig(devmapper)
BuildRequires: ostree-devel
BuildRequires: glib2-devel
BuildRequires: make
BuildRequires: shadow-utils-subid-devel
Requires: containers-common >= 4:1-21
%description
Command line utility to inspect images and repositories directly on Docker
registries without the need to pull them
%package tests
Summary: Tests for %{name}
Requires: %{name} = %{epoch}:%{version}-%{release}
Requires: bats
Requires: gnupg
Requires: jq
Requires: golang
Requires: podman
Requires: crun
Requires: httpd-tools
Requires: openssl
Requires: fakeroot
Requires: squashfs-tools
%description tests
%{summary}
This package contains system tests for %{name}
%prep
%autosetup -Sgit %{name}-%{version}
# The %%install stage should not rebuild anything but only install what's
# built in the %%build stage. So, remove any dependency on build targets.
sed -i 's/^install-binary: bin\/%{name}.*/install-binary:/' Makefile
sed -i 's/^completions: bin\/%{name}.*/completions:/' Makefile
sed -i 's/^install-docs: docs.*/install-docs:/' Makefile
%build
%set_build_flags
export CGO_CFLAGS=$CFLAGS
# These extra flags present in $CFLAGS have been skipped for now as they break the build
CGO_CFLAGS=$(echo $CGO_CFLAGS | sed 's/-flto=auto//g')
CGO_CFLAGS=$(echo $CGO_CFLAGS | sed 's/-Wp,D_GLIBCXX_ASSERTIONS//g')
CGO_CFLAGS=$(echo $CGO_CFLAGS | sed 's/-specs=\/usr\/lib\/rpm\/redhat\/redhat-annobin-cc1//g')
%ifarch x86_64
export CGO_CFLAGS="$CGO_CFLAGS -m64 -mtune=generic -fcf-protection=full"
%endif
BASEBUILDTAGS="$(hack/libdm_tag.sh) $(hack/libsubid_tag.sh)"
%if %{defined build_with_btrfs}
export BUILDTAGS="$BASEBUILDTAGS $(hack/btrfs_tag.sh) $(hack/btrfs_installed_tag.sh)"
%else
export BUILDTAGS="$BASEBUILDTAGS btrfs_noversion exclude_graphdriver_btrfs"
%endif
# unset LDFLAGS earlier set from set_build_flags
LDFLAGS=''
%gobuild -o bin/%{name} ./cmd/%{name}
%{__make} docs
%install
make \
DESTDIR=%{buildroot} \
PREFIX=%{_prefix} \
install-binary install-docs install-completions
# system tests
install -d -p %{buildroot}/%{_datadir}/%{name}/test/system
cp -pav systemtest/* %{buildroot}/%{_datadir}/%{name}/test/system/
#define license tag if not already defined
%{!?_licensedir:%global license %doc}
%files
%license LICENSE
%doc README.md
%{_bindir}/%{name}
%{_mandir}/man1/%{name}*
%dir %{_datadir}/bash-completion
%dir %{_datadir}/bash-completion/completions
%{_datadir}/bash-completion/completions/%{name}
%dir %{_datadir}/fish/vendor_completions.d
%{_datadir}/fish/vendor_completions.d/%{name}.fish
%dir %{_datadir}/zsh/site-functions
%{_datadir}/zsh/site-functions/_%{name}
%files tests
%license LICENSE
%{_datadir}/%{name}/test
%changelog
%if %{defined autochangelog}
%autochangelog
%else
# NOTE: This changelog will be visible on CentOS 8 Stream builds
# Other envs are capable of handling autochangelog
* Tue Jun 13 2023 RH Container Bot <rhcontainerbot@fedoraproject.org>
- Placeholder changelog for envs that are not autochangelog-ready.
- Contact upstream if you need to report an issue with the build.
%endif

132
skopeo.spec.rpkg Normal file
View File

@@ -0,0 +1,132 @@
# For automatic rebuilds in COPR
# The following tag is to get correct syntax highlighting for this file in vim text editor
# vim: syntax=spec
# Any additinoal comments should go below this line or else syntax highlighting
# may not work.
# CAUTION: This is not a replacement for RPMs provided by your distro.
# Only intended to build and test the latest unreleased changes.
%global gomodulesmode GO111MODULE=on
%global with_debug 1
%if 0%{?with_debug}
%global _find_debuginfo_dwz_opts %{nil}
%global _dwz_low_mem_die_limit 0
%else
%global debug_package %{nil}
%endif
%if ! 0%{?gobuild:1}
%define gobuild(o:) go build -buildmode pie -compiler gc -tags="rpm_crashtraceback ${BUILDTAGS:-}" -ldflags "${LDFLAGS:-} -B 0x$(head -c20 /dev/urandom|od -An -tx1|tr -d ' \\n') -extldflags '-Wl,-z,relro -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld '" -a -v -x %{?**};
%endif
Name: {{{ git_dir_name }}}
Epoch: 101
Version: {{{ git_dir_version }}}
Release: 1%{?dist}
Summary: Inspect container images and repositories on registries
License: ASL 2.0
URL: https://github.com/containers/skopeo
VCS: {{{ git_dir_vcs }}}
Source: {{{ git_dir_pack }}}
%if 0%{?fedora} && ! 0%{?rhel}
BuildRequires: btrfs-progs-devel
%endif
BuildRequires: golang >= 1.16.6
BuildRequires: glib2-devel
BuildRequires: git-core
BuildRequires: go-md2man
%if 0%{?fedora} || 0%{?rhel} >= 9
BuildRequires: go-rpm-macros
%endif
BuildRequires: pkgconfig(devmapper)
BuildRequires: gpgme-devel
BuildRequires: libassuan-devel
BuildRequires: pkgconfig
BuildRequires: make
BuildRequires: ostree-devel
%if 0%{?fedora} <= 35
Requires: containers-common >= 4:1-39
%else
Requires: containers-common >= 4:1-46
%endif
%description
Command line utility to inspect images and repositories directly on Docker
registries without the need to pull them.
%package tests
Summary: Tests for %{name}
Requires: %{name} = %{epoch}:%{version}-%{release}
Requires: bats
Requires: gnupg
Requires: jq
Requires: podman
Requires: httpd-tools
Requires: openssl
Requires: fakeroot
Requires: squashfs-tools
%description tests
%{summary}
This package contains system tests for %{name}
%prep
{{{ git_dir_setup_macro }}}
sed -i 's/install-binary: bin\/skopeo/install-binary:/' Makefile
# This will invoke `make` command in the directory with the extracted sources.
%build
%set_build_flags
export CGO_CFLAGS=$CFLAGS
# These extra flags present in $CFLAGS have been skipped for now as they break the build
CGO_CFLAGS=$(echo $CGO_CFLAGS | sed 's/-flto=auto//g')
CGO_CFLAGS=$(echo $CGO_CFLAGS | sed 's/-Wp,D_GLIBCXX_ASSERTIONS//g')
CGO_CFLAGS=$(echo $CGO_CFLAGS | sed 's/-specs=\/usr\/lib\/rpm\/redhat\/redhat-annobin-cc1//g')
%ifarch x86_64
export CGO_CFLAGS+=" -m64 -mtune=generic -fcf-protection=full"
%endif
LDFLAGS=""
export BUILDTAGS="$(hack/libdm_tag.sh)"
%if 0%{?rhel}
export BUILDTAGS="$BUILDTAGS exclude_graphdriver_btrfs btrfs_noversion"
%endif
%gobuild -o bin/%{name} ./cmd/%{name}
%install
%{__make} PREFIX=%{buildroot}%{_prefix} install-binary install-docs install-completions
# system tests
install -d -p %{buildroot}/%{_datadir}/%{name}/test/system
cp -pav systemtest/* %{buildroot}/%{_datadir}/%{name}/test/system/
%files
%license LICENSE
%doc README.md
%{_bindir}/%{name}
%{_mandir}/man1/%%{name}*
%dir %{_datadir}/bash-completion
%dir %{_datadir}/bash-completion/completions
%{_datadir}/bash-completion/completions/%{name}
%dir %{_datadir}/fish
%dir %{_datadir}/fish/vendor_completions.d
%{_datadir}/fish/vendor_completions.d/%{name}.fish
%dir %{_datadir}/zsh
%dir %{_datadir}/zsh/site-functions
%{_datadir}/zsh/site-functions/_%{name}
%files tests
%license LICENSE
%{_datadir}/%{name}/test
%changelog
{{{ git_dir_changelog }}}

View File

@@ -16,8 +16,7 @@ function setup() {
_cred_dir=$TESTDIR/credentials
# It is important to change XDG_RUNTIME_DIR only after we start the registry, otherwise it affects the path of $XDG_RUNTIME_DIR/netns maintained by Podman,
# making it impossible to clean up after ourselves.
export XDG_RUNTIME_DIR_OLD=$XDG_RUNTIME_DIR
# making it imposible to clean up after ourselves.
export XDG_RUNTIME_DIR=$_cred_dir
mkdir -p $_cred_dir/containers
# Remove old/stale cred file
@@ -112,9 +111,6 @@ function setup() {
}
teardown() {
# Need to restore XDG_RUNTIME_DIR.
XDG_RUNTIME_DIR=$XDG_RUNTIME_DIR_OLD
podman rm -f reg
if [[ -n $_cred_dir ]]; then

View File

@@ -242,7 +242,7 @@ END_TESTS
$fingerprint \
$TESTDIR/busybox.signature
# manifest digest
digest=$(echo "$output" | awk '{print $NF;}')
digest=$(echo "$output" | awk '{print $4;}')
run_skopeo manifest-digest $TESTDIR/busybox/manifest.json
expect_output $digest
}

View File

@@ -1,112 +0,0 @@
<!-- omit in toc -->
# Contributing to mergo
First off, thanks for taking the time to contribute! ❤️
All types of contributions are encouraged and valued. See the [Table of Contents](#table-of-contents) for different ways to help and details about how this project handles them. Please make sure to read the relevant section before making your contribution. It will make it a lot easier for us maintainers and smooth out the experience for all involved. The community looks forward to your contributions. 🎉
> And if you like the project, but just don't have time to contribute, that's fine. There are other easy ways to support the project and show your appreciation, which we would also be very happy about:
> - Star the project
> - Tweet about it
> - Refer this project in your project's readme
> - Mention the project at local meetups and tell your friends/colleagues
<!-- omit in toc -->
## Table of Contents
- [Code of Conduct](#code-of-conduct)
- [I Have a Question](#i-have-a-question)
- [I Want To Contribute](#i-want-to-contribute)
- [Reporting Bugs](#reporting-bugs)
- [Suggesting Enhancements](#suggesting-enhancements)
## Code of Conduct
This project and everyone participating in it is governed by the
[mergo Code of Conduct](https://github.com/imdario/mergoblob/master/CODE_OF_CONDUCT.md).
By participating, you are expected to uphold this code. Please report unacceptable behavior
to <>.
## I Have a Question
> If you want to ask a question, we assume that you have read the available [Documentation](https://pkg.go.dev/github.com/imdario/mergo).
Before you ask a question, it is best to search for existing [Issues](https://github.com/imdario/mergo/issues) that might help you. In case you have found a suitable issue and still need clarification, you can write your question in this issue. It is also advisable to search the internet for answers first.
If you then still feel the need to ask a question and need clarification, we recommend the following:
- Open an [Issue](https://github.com/imdario/mergo/issues/new).
- Provide as much context as you can about what you're running into.
- Provide project and platform versions (nodejs, npm, etc), depending on what seems relevant.
We will then take care of the issue as soon as possible.
## I Want To Contribute
> ### Legal Notice <!-- omit in toc -->
> When contributing to this project, you must agree that you have authored 100% of the content, that you have the necessary rights to the content and that the content you contribute may be provided under the project license.
### Reporting Bugs
<!-- omit in toc -->
#### Before Submitting a Bug Report
A good bug report shouldn't leave others needing to chase you up for more information. Therefore, we ask you to investigate carefully, collect information and describe the issue in detail in your report. Please complete the following steps in advance to help us fix any potential bug as fast as possible.
- Make sure that you are using the latest version.
- Determine if your bug is really a bug and not an error on your side e.g. using incompatible environment components/versions (Make sure that you have read the [documentation](). If you are looking for support, you might want to check [this section](#i-have-a-question)).
- To see if other users have experienced (and potentially already solved) the same issue you are having, check if there is not already a bug report existing for your bug or error in the [bug tracker](https://github.com/imdario/mergoissues?q=label%3Abug).
- Also make sure to search the internet (including Stack Overflow) to see if users outside of the GitHub community have discussed the issue.
- Collect information about the bug:
- Stack trace (Traceback)
- OS, Platform and Version (Windows, Linux, macOS, x86, ARM)
- Version of the interpreter, compiler, SDK, runtime environment, package manager, depending on what seems relevant.
- Possibly your input and the output
- Can you reliably reproduce the issue? And can you also reproduce it with older versions?
<!-- omit in toc -->
#### How Do I Submit a Good Bug Report?
> You must never report security related issues, vulnerabilities or bugs including sensitive information to the issue tracker, or elsewhere in public. Instead sensitive bugs must be sent by email to .
<!-- You may add a PGP key to allow the messages to be sent encrypted as well. -->
We use GitHub issues to track bugs and errors. If you run into an issue with the project:
- Open an [Issue](https://github.com/imdario/mergo/issues/new). (Since we can't be sure at this point whether it is a bug or not, we ask you not to talk about a bug yet and not to label the issue.)
- Explain the behavior you would expect and the actual behavior.
- Please provide as much context as possible and describe the *reproduction steps* that someone else can follow to recreate the issue on their own. This usually includes your code. For good bug reports you should isolate the problem and create a reduced test case.
- Provide the information you collected in the previous section.
Once it's filed:
- The project team will label the issue accordingly.
- A team member will try to reproduce the issue with your provided steps. If there are no reproduction steps or no obvious way to reproduce the issue, the team will ask you for those steps and mark the issue as `needs-repro`. Bugs with the `needs-repro` tag will not be addressed until they are reproduced.
- If the team is able to reproduce the issue, it will be marked `needs-fix`, as well as possibly other tags (such as `critical`), and the issue will be left to be implemented by someone.
### Suggesting Enhancements
This section guides you through submitting an enhancement suggestion for mergo, **including completely new features and minor improvements to existing functionality**. Following these guidelines will help maintainers and the community to understand your suggestion and find related suggestions.
<!-- omit in toc -->
#### Before Submitting an Enhancement
- Make sure that you are using the latest version.
- Read the [documentation]() carefully and find out if the functionality is already covered, maybe by an individual configuration.
- Perform a [search](https://github.com/imdario/mergo/issues) to see if the enhancement has already been suggested. If it has, add a comment to the existing issue instead of opening a new one.
- Find out whether your idea fits with the scope and aims of the project. It's up to you to make a strong case to convince the project's developers of the merits of this feature. Keep in mind that we want features that will be useful to the majority of our users and not just a small subset. If you're just targeting a minority of users, consider writing an add-on/plugin library.
<!-- omit in toc -->
#### How Do I Submit a Good Enhancement Suggestion?
Enhancement suggestions are tracked as [GitHub issues](https://github.com/imdario/mergo/issues).
- Use a **clear and descriptive title** for the issue to identify the suggestion.
- Provide a **step-by-step description of the suggested enhancement** in as many details as possible.
- **Describe the current behavior** and **explain which behavior you expected to see instead** and why. At this point you can also tell which alternatives do not work for you.
- You may want to **include screenshots and animated GIFs** which help you demonstrate the steps or point out the part which the suggestion is related to. You can use [this tool](https://www.cockos.com/licecap/) to record GIFs on macOS and Windows, and [this tool](https://github.com/colinkeenan/silentcast) or [this tool](https://github.com/GNOME/byzanz) on Linux. <!-- this should only be included if the project has a GUI -->
- **Explain why this enhancement would be useful** to most mergo users. You may also want to point out the other projects that solved it better and which could serve as inspiration.
<!-- omit in toc -->
## Attribution
This guide is based on the **contributing-gen**. [Make your own](https://github.com/bttger/contributing-gen)!

View File

@@ -1,14 +0,0 @@
# Security Policy
## Supported Versions
| Version | Supported |
| ------- | ------------------ |
| 0.3.x | :white_check_mark: |
| < 0.3 | :x: |
## Security contact information
To report a security vulnerability, please use the
[Tidelift security contact](https://tidelift.com/security).
Tidelift will coordinate the fix and disclosure.

View File

@@ -91,7 +91,7 @@ const (
// UnmarshalText method. See the Unmarshaler example for a demonstration with
// email addresses.
//
// # Key mapping
// ### Key mapping
//
// TOML keys can map to either keys in a Go map or field names in a Go struct.
// The special `toml` struct tag can be used to map TOML keys to struct fields
@@ -248,7 +248,7 @@ func (md *MetaData) unify(data interface{}, rv reflect.Value) error {
case reflect.Bool:
return md.unifyBool(data, rv)
case reflect.Interface:
if rv.NumMethod() > 0 { /// Only empty interfaces are supported.
if rv.NumMethod() > 0 { // Only support empty interfaces are supported.
return md.e("unsupported type %s", rv.Type())
}
return md.unifyAnything(data, rv)

View File

@@ -5,25 +5,17 @@ import (
"io"
)
// TextMarshaler is an alias for encoding.TextMarshaler.
//
// Deprecated: use encoding.TextMarshaler
type TextMarshaler encoding.TextMarshaler
// TextUnmarshaler is an alias for encoding.TextUnmarshaler.
//
// Deprecated: use encoding.TextUnmarshaler
type TextUnmarshaler encoding.TextUnmarshaler
// PrimitiveDecode is an alias for MetaData.PrimitiveDecode().
//
// Deprecated: use MetaData.PrimitiveDecode.
func PrimitiveDecode(primValue Primitive, v interface{}) error {
md := MetaData{decoded: make(map[string]struct{})}
return md.unify(primValue.undecoded, rvalue(v))
}
// DecodeReader is an alias for NewDecoder(r).Decode(v).
//
// Deprecated: use NewDecoder(reader).Decode(&value).
func DecodeReader(r io.Reader, v interface{}) (MetaData, error) { return NewDecoder(r).Decode(v) }

View File

@@ -136,8 +136,7 @@ func NewEncoder(w io.Writer) *Encoder {
// document.
func (enc *Encoder) Encode(v interface{}) error {
rv := eindirect(reflect.ValueOf(v))
err := enc.safeEncode(Key([]string{}), rv)
if err != nil {
if err := enc.safeEncode(Key([]string{}), rv); err != nil {
return err
}
return enc.w.Flush()
@@ -458,16 +457,6 @@ func (enc *Encoder) eStruct(key Key, rv reflect.Value, inline bool) {
frv := eindirect(rv.Field(i))
if is32Bit {
// Copy so it works correct on 32bit archs; not clear why this
// is needed. See #314, and https://www.reddit.com/r/golang/comments/pnx8v4
// This also works fine on 64bit, but 32bit archs are somewhat
// rare and this is a wee bit faster.
copyStart := make([]int, len(start))
copy(copyStart, start)
start = copyStart
}
// Treat anonymous struct fields with tag names as though they are
// not anonymous, like encoding/json does.
//
@@ -482,7 +471,17 @@ func (enc *Encoder) eStruct(key Key, rv reflect.Value, inline bool) {
if typeIsTable(tomlTypeOfGo(frv)) {
fieldsSub = append(fieldsSub, append(start, f.Index...))
} else {
fieldsDirect = append(fieldsDirect, append(start, f.Index...))
// Copy so it works correct on 32bit archs; not clear why this
// is needed. See #314, and https://www.reddit.com/r/golang/comments/pnx8v4
// This also works fine on 64bit, but 32bit archs are somewhat
// rare and this is a wee bit faster.
if is32Bit {
copyStart := make([]int, len(start))
copy(copyStart, start)
fieldsDirect = append(fieldsDirect, append(copyStart, f.Index...))
} else {
fieldsDirect = append(fieldsDirect, append(start, f.Index...))
}
}
}
}
@@ -491,27 +490,24 @@ func (enc *Encoder) eStruct(key Key, rv reflect.Value, inline bool) {
writeFields := func(fields [][]int) {
for _, fieldIndex := range fields {
fieldType := rt.FieldByIndex(fieldIndex)
fieldVal := rv.FieldByIndex(fieldIndex)
opts := getOptions(fieldType.Tag)
if opts.skip {
continue
}
if opts.omitempty && isEmpty(fieldVal) {
continue
}
fieldVal = eindirect(fieldVal)
fieldVal := eindirect(rv.FieldByIndex(fieldIndex))
if isNil(fieldVal) { /// Don't write anything for nil fields.
continue
}
opts := getOptions(fieldType.Tag)
if opts.skip {
continue
}
keyName := fieldType.Name
if opts.name != "" {
keyName = opts.name
}
if opts.omitempty && enc.isEmpty(fieldVal) {
continue
}
if opts.omitzero && isZero(fieldVal) {
continue
}
@@ -653,7 +649,7 @@ func isZero(rv reflect.Value) bool {
return false
}
func isEmpty(rv reflect.Value) bool {
func (enc *Encoder) isEmpty(rv reflect.Value) bool {
switch rv.Kind() {
case reflect.Array, reflect.Slice, reflect.Map, reflect.String:
return rv.Len() == 0
@@ -668,15 +664,13 @@ func isEmpty(rv reflect.Value) bool {
// type b struct{ s []string }
// s := a{field: b{s: []string{"AAA"}}}
for i := 0; i < rv.NumField(); i++ {
if !isEmpty(rv.Field(i)) {
if !enc.isEmpty(rv.Field(i)) {
return false
}
}
return true
case reflect.Bool:
return !rv.Bool()
case reflect.Ptr:
return rv.IsNil()
}
return false
}
@@ -699,11 +693,8 @@ func (enc *Encoder) newline() {
// v v v v vv
// key = {k = 1, k2 = 2}
func (enc *Encoder) writeKeyValue(key Key, val reflect.Value, inline bool) {
/// Marshaler used on top-level document; call eElement() to just call
/// Marshal{TOML,Text}.
if len(key) == 0 {
enc.eElement(val)
return
encPanic(errNoKey)
}
enc.wf("%s%s = ", enc.indentStr(key), key.maybeQuoted(len(key)-1))
enc.eElement(val)

View File

@@ -84,7 +84,7 @@ func (pe ParseError) Error() string {
pe.Position.Line, pe.LastKey, msg)
}
// ErrorWithPosition returns the error with detailed location context.
// ErrorWithUsage() returns the error with detailed location context.
//
// See the documentation on [ParseError].
func (pe ParseError) ErrorWithPosition() string {
@@ -124,7 +124,7 @@ func (pe ParseError) ErrorWithPosition() string {
return b.String()
}
// ErrorWithUsage returns the error with detailed location context and usage
// ErrorWithUsage() returns the error with detailed location context and usage
// guidance.
//
// See the documentation on [ParseError].

View File

@@ -46,13 +46,12 @@ func (p Position) String() string {
}
type lexer struct {
input string
start int
pos int
line int
state stateFn
items chan item
tomlNext bool
input string
start int
pos int
line int
state stateFn
items chan item
// Allow for backing up up to 4 runes. This is necessary because TOML
// contains 3-rune tokens (""" and ''').
@@ -88,14 +87,13 @@ func (lx *lexer) nextItem() item {
}
}
func lex(input string, tomlNext bool) *lexer {
func lex(input string) *lexer {
lx := &lexer{
input: input,
state: lexTop,
items: make(chan item, 10),
stack: make([]stateFn, 0, 10),
line: 1,
tomlNext: tomlNext,
input: input,
state: lexTop,
items: make(chan item, 10),
stack: make([]stateFn, 0, 10),
line: 1,
}
return lx
}
@@ -410,7 +408,7 @@ func lexTableNameEnd(lx *lexer) stateFn {
// Lexes only one part, e.g. only 'a' inside 'a.b'.
func lexBareName(lx *lexer) stateFn {
r := lx.next()
if isBareKeyChar(r, lx.tomlNext) {
if isBareKeyChar(r) {
return lexBareName
}
lx.backup()
@@ -620,9 +618,6 @@ func lexInlineTableValue(lx *lexer) stateFn {
case isWhitespace(r):
return lexSkip(lx, lexInlineTableValue)
case isNL(r):
if lx.tomlNext {
return lexSkip(lx, lexInlineTableValue)
}
return lx.errorPrevLine(errLexInlineTableNL{})
case r == '#':
lx.push(lexInlineTableValue)
@@ -645,9 +640,6 @@ func lexInlineTableValueEnd(lx *lexer) stateFn {
case isWhitespace(r):
return lexSkip(lx, lexInlineTableValueEnd)
case isNL(r):
if lx.tomlNext {
return lexSkip(lx, lexInlineTableValueEnd)
}
return lx.errorPrevLine(errLexInlineTableNL{})
case r == '#':
lx.push(lexInlineTableValueEnd)
@@ -656,9 +648,6 @@ func lexInlineTableValueEnd(lx *lexer) stateFn {
lx.ignore()
lx.skip(isWhitespace)
if lx.peek() == '}' {
if lx.tomlNext {
return lexInlineTableValueEnd
}
return lx.errorf("trailing comma not allowed in inline tables")
}
return lexInlineTableValue
@@ -781,8 +770,8 @@ func lexRawString(lx *lexer) stateFn {
}
}
// lexMultilineRawString consumes a raw string. Nothing can be escaped in such a
// string. It assumes that the beginning triple-' has already been consumed and
// lexMultilineRawString consumes a raw string. Nothing can be escaped in such
// a string. It assumes that the beginning ''' has already been consumed and
// ignored.
func lexMultilineRawString(lx *lexer) stateFn {
r := lx.next()
@@ -839,11 +828,6 @@ func lexMultilineStringEscape(lx *lexer) stateFn {
func lexStringEscape(lx *lexer) stateFn {
r := lx.next()
switch r {
case 'e':
if !lx.tomlNext {
return lx.error(errLexEscape{r})
}
fallthrough
case 'b':
fallthrough
case 't':
@@ -862,11 +846,6 @@ func lexStringEscape(lx *lexer) stateFn {
fallthrough
case '\\':
return lx.pop()
case 'x':
if !lx.tomlNext {
return lx.error(errLexEscape{r})
}
return lexHexEscape
case 'u':
return lexShortUnicodeEscape
case 'U':
@@ -875,19 +854,6 @@ func lexStringEscape(lx *lexer) stateFn {
return lx.error(errLexEscape{r})
}
func lexHexEscape(lx *lexer) stateFn {
var r rune
for i := 0; i < 2; i++ {
r = lx.next()
if !isHexadecimal(r) {
return lx.errorf(
`expected two hexadecimal digits after '\x', but got %q instead`,
lx.current())
}
}
return lx.pop()
}
func lexShortUnicodeEscape(lx *lexer) stateFn {
var r rune
for i := 0; i < 4; i++ {
@@ -1259,23 +1225,7 @@ func isOctal(r rune) bool { return r >= '0' && r <= '7' }
func isHexadecimal(r rune) bool {
return (r >= '0' && r <= '9') || (r >= 'a' && r <= 'f') || (r >= 'A' && r <= 'F')
}
func isBareKeyChar(r rune, tomlNext bool) bool {
if tomlNext {
return (r >= 'A' && r <= 'Z') ||
(r >= 'a' && r <= 'z') ||
(r >= '0' && r <= '9') ||
r == '_' || r == '-' ||
r == 0xb2 || r == 0xb3 || r == 0xb9 || (r >= 0xbc && r <= 0xbe) ||
(r >= 0xc0 && r <= 0xd6) || (r >= 0xd8 && r <= 0xf6) || (r >= 0xf8 && r <= 0x037d) ||
(r >= 0x037f && r <= 0x1fff) ||
(r >= 0x200c && r <= 0x200d) || (r >= 0x203f && r <= 0x2040) ||
(r >= 0x2070 && r <= 0x218f) || (r >= 0x2460 && r <= 0x24ff) ||
(r >= 0x2c00 && r <= 0x2fef) || (r >= 0x3001 && r <= 0xd7ff) ||
(r >= 0xf900 && r <= 0xfdcf) || (r >= 0xfdf0 && r <= 0xfffd) ||
(r >= 0x10000 && r <= 0xeffff)
}
func isBareKeyChar(r rune) bool {
return (r >= 'A' && r <= 'Z') ||
(r >= 'a' && r <= 'z') ||
(r >= '0' && r <= '9') ||

View File

@@ -106,7 +106,7 @@ func (k Key) maybeQuoted(i int) string {
return `""`
}
for _, c := range k[i] {
if !isBareKeyChar(c, false) {
if !isBareKeyChar(c) {
return `"` + dblQuotedReplacer.Replace(k[i]) + `"`
}
}

View File

@@ -2,7 +2,6 @@ package toml
import (
"fmt"
"os"
"strconv"
"strings"
"time"
@@ -16,7 +15,6 @@ type parser struct {
context Key // Full key for the current hash in scope.
currentKey string // Base key name for everything except hashes.
pos Position // Current position in the TOML file.
tomlNext bool
ordered []Key // List of keys in the order that they appear in the TOML data.
@@ -31,8 +29,6 @@ type keyInfo struct {
}
func parse(data string) (p *parser, err error) {
_, tomlNext := os.LookupEnv("BURNTSUSHI_TOML_110")
defer func() {
if r := recover(); r != nil {
if pErr, ok := r.(ParseError); ok {
@@ -45,12 +41,9 @@ func parse(data string) (p *parser, err error) {
}()
// Read over BOM; do this here as the lexer calls utf8.DecodeRuneInString()
// which mangles stuff. UTF-16 BOM isn't strictly valid, but some tools add
// it anyway.
if strings.HasPrefix(data, "\xff\xfe") || strings.HasPrefix(data, "\xfe\xff") { // UTF-16
// which mangles stuff.
if strings.HasPrefix(data, "\xff\xfe") || strings.HasPrefix(data, "\xfe\xff") {
data = data[2:]
} else if strings.HasPrefix(data, "\xef\xbb\xbf") { // UTF-8
data = data[3:]
}
// Examine first few bytes for NULL bytes; this probably means it's a UTF-16
@@ -72,10 +65,9 @@ func parse(data string) (p *parser, err error) {
p = &parser{
keyInfo: make(map[string]keyInfo),
mapping: make(map[string]interface{}),
lx: lex(data, tomlNext),
lx: lex(data),
ordered: make([]Key, 0),
implicits: make(map[string]struct{}),
tomlNext: tomlNext,
}
for {
item := p.next()
@@ -202,12 +194,12 @@ func (p *parser) topLevel(item item) {
for i := range context {
p.addImplicitContext(append(p.context, context[i:i+1]...))
}
p.ordered = append(p.ordered, p.context.add(p.currentKey))
/// Set value.
vItem := p.next()
val, typ := p.value(vItem, false)
p.set(p.currentKey, val, typ, vItem.pos)
p.ordered = append(p.ordered, p.context.add(p.currentKey))
/// Remove the context we added (preserving any context from [tbl] lines).
p.context = outerContext
@@ -244,7 +236,7 @@ func (p *parser) value(it item, parentIsArray bool) (interface{}, tomlType) {
case itemString:
return p.replaceEscapes(it, it.val), p.typeOfPrimitive(it)
case itemMultilineString:
return p.replaceEscapes(it, p.stripEscapedNewlines(stripFirstNewline(it.val))), p.typeOfPrimitive(it)
return p.replaceEscapes(it, stripFirstNewline(p.stripEscapedNewlines(it.val))), p.typeOfPrimitive(it)
case itemRawString:
return it.val, p.typeOfPrimitive(it)
case itemRawMultilineString:
@@ -339,17 +331,11 @@ func (p *parser) valueFloat(it item) (interface{}, tomlType) {
var dtTypes = []struct {
fmt string
zone *time.Location
next bool
}{
{time.RFC3339Nano, time.Local, false},
{"2006-01-02T15:04:05.999999999", internal.LocalDatetime, false},
{"2006-01-02", internal.LocalDate, false},
{"15:04:05.999999999", internal.LocalTime, false},
// tomlNext
{"2006-01-02T15:04Z07:00", time.Local, true},
{"2006-01-02T15:04", internal.LocalDatetime, true},
{"15:04", internal.LocalTime, true},
{time.RFC3339Nano, time.Local},
{"2006-01-02T15:04:05.999999999", internal.LocalDatetime},
{"2006-01-02", internal.LocalDate},
{"15:04:05.999999999", internal.LocalTime},
}
func (p *parser) valueDatetime(it item) (interface{}, tomlType) {
@@ -360,9 +346,6 @@ func (p *parser) valueDatetime(it item) (interface{}, tomlType) {
err error
)
for _, dt := range dtTypes {
if dt.next && !p.tomlNext {
continue
}
t, err = time.ParseInLocation(dt.fmt, it.val, dt.zone)
if err == nil {
ok = true
@@ -401,7 +384,6 @@ func (p *parser) valueArray(it item) (interface{}, tomlType) {
//
// Not entirely sure how to best store this; could use "key[0]",
// "key[1]" notation, or maybe store it on the Array type?
_ = types
}
return array, tomlArray
}
@@ -444,11 +426,11 @@ func (p *parser) valueInlineTable(it item, parentIsArray bool) (interface{}, tom
for i := range context {
p.addImplicitContext(append(p.context, context[i:i+1]...))
}
p.ordered = append(p.ordered, p.context.add(p.currentKey))
/// Set the value.
val, typ := p.value(p.next(), false)
p.set(p.currentKey, val, typ, it.pos)
p.ordered = append(p.ordered, p.context.add(p.currentKey))
hash[p.currentKey] = val
/// Restore context.
@@ -569,6 +551,7 @@ func (p *parser) addContext(key Key, array bool) {
func (p *parser) set(key string, val interface{}, typ tomlType, pos Position) {
p.setValue(key, val)
p.setType(key, typ, pos)
}
// setValue sets the given key to the given value in the current context.
@@ -649,11 +632,14 @@ func (p *parser) setType(key string, typ tomlType, pos Position) {
// Implicit keys need to be created when tables are implied in "a.b.c.d = 1" and
// "[a.b.c]" (the "a", "b", and "c" hashes are never created explicitly).
func (p *parser) addImplicit(key Key) { p.implicits[key.String()] = struct{}{} }
func (p *parser) removeImplicit(key Key) { delete(p.implicits, key.String()) }
func (p *parser) isImplicit(key Key) bool { _, ok := p.implicits[key.String()]; return ok }
func (p *parser) isArray(key Key) bool { return p.keyInfo[key.String()].tomlType == tomlArray }
func (p *parser) addImplicitContext(key Key) { p.addImplicit(key); p.addContext(key, false) }
func (p *parser) addImplicit(key Key) { p.implicits[key.String()] = struct{}{} }
func (p *parser) removeImplicit(key Key) { delete(p.implicits, key.String()) }
func (p *parser) isImplicit(key Key) bool { _, ok := p.implicits[key.String()]; return ok }
func (p *parser) isArray(key Key) bool { return p.keyInfo[key.String()].tomlType == tomlArray }
func (p *parser) addImplicitContext(key Key) {
p.addImplicit(key)
p.addContext(key, false)
}
// current returns the full key name of the current context.
func (p *parser) current() string {
@@ -676,54 +662,49 @@ func stripFirstNewline(s string) string {
return s
}
// stripEscapedNewlines removes whitespace after line-ending backslashes in
// multiline strings.
//
// A line-ending backslash is an unescaped \ followed only by whitespace until
// the next newline. After a line-ending backslash, all whitespace is removed
// until the next non-whitespace character.
// Remove newlines inside triple-quoted strings if a line ends with "\".
func (p *parser) stripEscapedNewlines(s string) string {
var b strings.Builder
var i int
for {
ix := strings.Index(s[i:], `\`)
if ix < 0 {
b.WriteString(s)
return b.String()
}
i += ix
if len(s) > i+1 && s[i+1] == '\\' {
// Escaped backslash.
i += 2
continue
}
// Scan until the next non-whitespace.
j := i + 1
whitespaceLoop:
for ; j < len(s); j++ {
switch s[j] {
case ' ', '\t', '\r', '\n':
default:
break whitespaceLoop
}
}
if j == i+1 {
// Not a whitespace escape.
i++
continue
}
if !strings.Contains(s[i:j], "\n") {
// This is not a line-ending backslash.
// (It's a bad escape sequence, but we can let
// replaceEscapes catch it.)
i++
continue
}
b.WriteString(s[:i])
s = s[j:]
i = 0
split := strings.Split(s, "\n")
if len(split) < 1 {
return s
}
escNL := false // Keep track of the last non-blank line was escaped.
for i, line := range split {
line = strings.TrimRight(line, " \t\r")
if len(line) == 0 || line[len(line)-1] != '\\' {
split[i] = strings.TrimRight(split[i], "\r")
if !escNL && i != len(split)-1 {
split[i] += "\n"
}
continue
}
escBS := true
for j := len(line) - 1; j >= 0 && line[j] == '\\'; j-- {
escBS = !escBS
}
if escNL {
line = strings.TrimLeft(line, " \t\r")
}
escNL = !escBS
if escBS {
split[i] += "\n"
continue
}
if i == len(split)-1 {
p.panicf("invalid escape: '\\ '")
}
split[i] = line[:len(line)-1] // Remove \
if len(split)-1 > i {
split[i+1] = strings.TrimLeft(split[i+1], " \t\r")
}
}
return strings.Join(split, "")
}
func (p *parser) replaceEscapes(it item, str string) string {
@@ -762,23 +743,12 @@ func (p *parser) replaceEscapes(it item, str string) string {
case 'r':
replaced = append(replaced, rune(0x000D))
r += 1
case 'e':
if p.tomlNext {
replaced = append(replaced, rune(0x001B))
r += 1
}
case '"':
replaced = append(replaced, rune(0x0022))
r += 1
case '\\':
replaced = append(replaced, rune(0x005C))
r += 1
case 'x':
if p.tomlNext {
escaped := p.asciiEscapeToUnicode(it, s[r+1:r+3])
replaced = append(replaced, escaped)
r += 3
}
case 'u':
// At this point, we know we have a Unicode escape of the form
// `uXXXX` at [r, r+5). (Because the lexer guarantees this

View File

@@ -8,8 +8,12 @@ linters:
- containedctx # struct contains a context
- dupl # duplicate code
- errname # erorrs are named correctly
- goconst # strings that should be constants
- godot # comments end in a period
- misspell
- nolintlint # "//nolint" directives are properly explained
- revive # golint replacement
- stylecheck # golint replacement, less configurable than revive
- unconvert # unnecessary conversions
- wastedassign
@@ -19,7 +23,10 @@ linters:
- exhaustive # check exhaustiveness of enum switch statements
- gofmt # files are gofmt'ed
- gosec # security
- nestif # deeply nested ifs
- nilerr # returns nil even with non-nil error
- prealloc # slices that can be pre-allocated
- structcheck # unused struct fields
- unparam # unused function params
issues:
@@ -35,18 +42,6 @@ issues:
text: "^line-length-limit: "
source: "^//(go:generate|sys) "
#TODO: remove after upgrading to go1.18
# ignore comment spacing for nolint and sys directives
- linters:
- revive
text: "^comment-spacings: no space between comment delimiter and comment text"
source: "//(cspell:|nolint:|sys |todo)"
# not on go 1.18 yet, so no any
- linters:
- revive
text: "^use-any: since GO 1.18 'interface{}' can be replaced by 'any'"
# allow unjustified ignores of error checks in defer statements
- linters:
- nolintlint
@@ -61,8 +56,6 @@ issues:
linters-settings:
exhaustive:
default-signifies-exhaustive: true
govet:
enable-all: true
disable:
@@ -105,8 +98,6 @@ linters-settings:
disabled: true
- name: flag-parameter # excessive, and a common idiom we use
disabled: true
- name: unhandled-error # warns over common fmt.Print* and io.Close; rely on errcheck instead
disabled: true
# general config
- name: line-length-limit
arguments:
@@ -147,3 +138,7 @@ linters-settings:
- VPCI
- WCOW
- WIM
stylecheck:
checks:
- "all"
- "-ST1003" # use revive's var naming

View File

@@ -23,7 +23,7 @@ import (
const afHVSock = 34 // AF_HYPERV
// Well known Service and VM IDs
// https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/user-guide/make-integration-service#vmid-wildcards
//https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/user-guide/make-integration-service#vmid-wildcards
// HvsockGUIDWildcard is the wildcard VmId for accepting connections from all partitions.
func HvsockGUIDWildcard() guid.GUID { // 00000000-0000-0000-0000-000000000000
@@ -31,7 +31,7 @@ func HvsockGUIDWildcard() guid.GUID { // 00000000-0000-0000-0000-000000000000
}
// HvsockGUIDBroadcast is the wildcard VmId for broadcasting sends to all partitions.
func HvsockGUIDBroadcast() guid.GUID { // ffffffff-ffff-ffff-ffff-ffffffffffff
func HvsockGUIDBroadcast() guid.GUID { //ffffffff-ffff-ffff-ffff-ffffffffffff
return guid.GUID{
Data1: 0xffffffff,
Data2: 0xffff,
@@ -246,7 +246,7 @@ func (l *HvsockListener) Accept() (_ net.Conn, err error) {
var addrbuf [addrlen * 2]byte
var bytes uint32
err = syscall.AcceptEx(l.sock.handle, sock.handle, &addrbuf[0], 0 /* rxdatalen */, addrlen, addrlen, &bytes, &c.o)
err = syscall.AcceptEx(l.sock.handle, sock.handle, &addrbuf[0], 0 /*rxdatalen*/, addrlen, addrlen, &bytes, &c.o)
if _, err = l.sock.asyncIO(c, nil, bytes, err); err != nil {
return nil, l.opErr("accept", os.NewSyscallError("acceptex", err))
}

View File

@@ -1,2 +0,0 @@
// This package contains Win32 filesystem functionality.
package fs

View File

@@ -1,202 +0,0 @@
//go:build windows
package fs
import (
"golang.org/x/sys/windows"
"github.com/Microsoft/go-winio/internal/stringbuffer"
)
//go:generate go run github.com/Microsoft/go-winio/tools/mkwinsyscall -output zsyscall_windows.go fs.go
// https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-createfilew
//sys CreateFile(name string, access AccessMask, mode FileShareMode, sa *syscall.SecurityAttributes, createmode FileCreationDisposition, attrs FileFlagOrAttribute, templatefile windows.Handle) (handle windows.Handle, err error) [failretval==windows.InvalidHandle] = CreateFileW
const NullHandle windows.Handle = 0
// AccessMask defines standard, specific, and generic rights.
//
// Bitmask:
// 3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1
// 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0
// +---------------+---------------+-------------------------------+
// |G|G|G|G|Resvd|A| StandardRights| SpecificRights |
// |R|W|E|A| |S| | |
// +-+-------------+---------------+-------------------------------+
//
// GR Generic Read
// GW Generic Write
// GE Generic Exectue
// GA Generic All
// Resvd Reserved
// AS Access Security System
//
// https://learn.microsoft.com/en-us/windows/win32/secauthz/access-mask
//
// https://learn.microsoft.com/en-us/windows/win32/secauthz/generic-access-rights
//
// https://learn.microsoft.com/en-us/windows/win32/fileio/file-access-rights-constants
type AccessMask = windows.ACCESS_MASK
//nolint:revive // SNAKE_CASE is not idiomatic in Go, but aligned with Win32 API.
const (
// Not actually any.
//
// For CreateFile: "query certain metadata such as file, directory, or device attributes without accessing that file or device"
// https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-createfilew#parameters
FILE_ANY_ACCESS AccessMask = 0
// Specific Object Access
// from ntioapi.h
FILE_READ_DATA AccessMask = (0x0001) // file & pipe
FILE_LIST_DIRECTORY AccessMask = (0x0001) // directory
FILE_WRITE_DATA AccessMask = (0x0002) // file & pipe
FILE_ADD_FILE AccessMask = (0x0002) // directory
FILE_APPEND_DATA AccessMask = (0x0004) // file
FILE_ADD_SUBDIRECTORY AccessMask = (0x0004) // directory
FILE_CREATE_PIPE_INSTANCE AccessMask = (0x0004) // named pipe
FILE_READ_EA AccessMask = (0x0008) // file & directory
FILE_READ_PROPERTIES AccessMask = FILE_READ_EA
FILE_WRITE_EA AccessMask = (0x0010) // file & directory
FILE_WRITE_PROPERTIES AccessMask = FILE_WRITE_EA
FILE_EXECUTE AccessMask = (0x0020) // file
FILE_TRAVERSE AccessMask = (0x0020) // directory
FILE_DELETE_CHILD AccessMask = (0x0040) // directory
FILE_READ_ATTRIBUTES AccessMask = (0x0080) // all
FILE_WRITE_ATTRIBUTES AccessMask = (0x0100) // all
FILE_ALL_ACCESS AccessMask = (STANDARD_RIGHTS_REQUIRED | SYNCHRONIZE | 0x1FF)
FILE_GENERIC_READ AccessMask = (STANDARD_RIGHTS_READ | FILE_READ_DATA | FILE_READ_ATTRIBUTES | FILE_READ_EA | SYNCHRONIZE)
FILE_GENERIC_WRITE AccessMask = (STANDARD_RIGHTS_WRITE | FILE_WRITE_DATA | FILE_WRITE_ATTRIBUTES | FILE_WRITE_EA | FILE_APPEND_DATA | SYNCHRONIZE)
FILE_GENERIC_EXECUTE AccessMask = (STANDARD_RIGHTS_EXECUTE | FILE_READ_ATTRIBUTES | FILE_EXECUTE | SYNCHRONIZE)
SPECIFIC_RIGHTS_ALL AccessMask = 0x0000FFFF
// Standard Access
// from ntseapi.h
DELETE AccessMask = 0x0001_0000
READ_CONTROL AccessMask = 0x0002_0000
WRITE_DAC AccessMask = 0x0004_0000
WRITE_OWNER AccessMask = 0x0008_0000
SYNCHRONIZE AccessMask = 0x0010_0000
STANDARD_RIGHTS_REQUIRED AccessMask = 0x000F_0000
STANDARD_RIGHTS_READ AccessMask = READ_CONTROL
STANDARD_RIGHTS_WRITE AccessMask = READ_CONTROL
STANDARD_RIGHTS_EXECUTE AccessMask = READ_CONTROL
STANDARD_RIGHTS_ALL AccessMask = 0x001F_0000
)
type FileShareMode uint32
//nolint:revive // SNAKE_CASE is not idiomatic in Go, but aligned with Win32 API.
const (
FILE_SHARE_NONE FileShareMode = 0x00
FILE_SHARE_READ FileShareMode = 0x01
FILE_SHARE_WRITE FileShareMode = 0x02
FILE_SHARE_DELETE FileShareMode = 0x04
FILE_SHARE_VALID_FLAGS FileShareMode = 0x07
)
type FileCreationDisposition uint32
//nolint:revive // SNAKE_CASE is not idiomatic in Go, but aligned with Win32 API.
const (
// from winbase.h
CREATE_NEW FileCreationDisposition = 0x01
CREATE_ALWAYS FileCreationDisposition = 0x02
OPEN_EXISTING FileCreationDisposition = 0x03
OPEN_ALWAYS FileCreationDisposition = 0x04
TRUNCATE_EXISTING FileCreationDisposition = 0x05
)
// CreateFile and co. take flags or attributes together as one parameter.
// Define alias until we can use generics to allow both
// https://learn.microsoft.com/en-us/windows/win32/fileio/file-attribute-constants
type FileFlagOrAttribute uint32
//nolint:revive // SNAKE_CASE is not idiomatic in Go, but aligned with Win32 API.
const ( // from winnt.h
FILE_FLAG_WRITE_THROUGH FileFlagOrAttribute = 0x8000_0000
FILE_FLAG_OVERLAPPED FileFlagOrAttribute = 0x4000_0000
FILE_FLAG_NO_BUFFERING FileFlagOrAttribute = 0x2000_0000
FILE_FLAG_RANDOM_ACCESS FileFlagOrAttribute = 0x1000_0000
FILE_FLAG_SEQUENTIAL_SCAN FileFlagOrAttribute = 0x0800_0000
FILE_FLAG_DELETE_ON_CLOSE FileFlagOrAttribute = 0x0400_0000
FILE_FLAG_BACKUP_SEMANTICS FileFlagOrAttribute = 0x0200_0000
FILE_FLAG_POSIX_SEMANTICS FileFlagOrAttribute = 0x0100_0000
FILE_FLAG_OPEN_REPARSE_POINT FileFlagOrAttribute = 0x0020_0000
FILE_FLAG_OPEN_NO_RECALL FileFlagOrAttribute = 0x0010_0000
FILE_FLAG_FIRST_PIPE_INSTANCE FileFlagOrAttribute = 0x0008_0000
)
type FileSQSFlag = FileFlagOrAttribute
//nolint:revive // SNAKE_CASE is not idiomatic in Go, but aligned with Win32 API.
const ( // from winbase.h
SECURITY_ANONYMOUS FileSQSFlag = FileSQSFlag(SecurityAnonymous << 16)
SECURITY_IDENTIFICATION FileSQSFlag = FileSQSFlag(SecurityIdentification << 16)
SECURITY_IMPERSONATION FileSQSFlag = FileSQSFlag(SecurityImpersonation << 16)
SECURITY_DELEGATION FileSQSFlag = FileSQSFlag(SecurityDelegation << 16)
SECURITY_SQOS_PRESENT FileSQSFlag = 0x00100000
SECURITY_VALID_SQOS_FLAGS FileSQSFlag = 0x001F0000
)
// GetFinalPathNameByHandle flags
//
// https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-getfinalpathnamebyhandlew#parameters
type GetFinalPathFlag uint32
//nolint:revive // SNAKE_CASE is not idiomatic in Go, but aligned with Win32 API.
const (
GetFinalPathDefaultFlag GetFinalPathFlag = 0x0
FILE_NAME_NORMALIZED GetFinalPathFlag = 0x0
FILE_NAME_OPENED GetFinalPathFlag = 0x8
VOLUME_NAME_DOS GetFinalPathFlag = 0x0
VOLUME_NAME_GUID GetFinalPathFlag = 0x1
VOLUME_NAME_NT GetFinalPathFlag = 0x2
VOLUME_NAME_NONE GetFinalPathFlag = 0x4
)
// getFinalPathNameByHandle facilitates calling the Windows API GetFinalPathNameByHandle
// with the given handle and flags. It transparently takes care of creating a buffer of the
// correct size for the call.
//
// https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-getfinalpathnamebyhandlew
func GetFinalPathNameByHandle(h windows.Handle, flags GetFinalPathFlag) (string, error) {
b := stringbuffer.NewWString()
//TODO: can loop infinitely if Win32 keeps returning the same (or a larger) n?
for {
n, err := windows.GetFinalPathNameByHandle(h, b.Pointer(), b.Cap(), uint32(flags))
if err != nil {
return "", err
}
// If the buffer wasn't large enough, n will be the total size needed (including null terminator).
// Resize and try again.
if n > b.Cap() {
b.ResizeTo(n)
continue
}
// If the buffer is large enough, n will be the size not including the null terminator.
// Convert to a Go string and return.
return b.String(), nil
}
}

View File

@@ -1,12 +0,0 @@
package fs
// https://learn.microsoft.com/en-us/windows/win32/api/winnt/ne-winnt-security_impersonation_level
type SecurityImpersonationLevel int32 // C default enums underlying type is `int`, which is Go `int32`
// Impersonation levels
const (
SecurityAnonymous SecurityImpersonationLevel = 0
SecurityIdentification SecurityImpersonationLevel = 1
SecurityImpersonation SecurityImpersonationLevel = 2
SecurityDelegation SecurityImpersonationLevel = 3
)

View File

@@ -1,64 +0,0 @@
//go:build windows
// Code generated by 'go generate' using "github.com/Microsoft/go-winio/tools/mkwinsyscall"; DO NOT EDIT.
package fs
import (
"syscall"
"unsafe"
"golang.org/x/sys/windows"
)
var _ unsafe.Pointer
// Do the interface allocations only once for common
// Errno values.
const (
errnoERROR_IO_PENDING = 997
)
var (
errERROR_IO_PENDING error = syscall.Errno(errnoERROR_IO_PENDING)
errERROR_EINVAL error = syscall.EINVAL
)
// errnoErr returns common boxed Errno values, to prevent
// allocations at runtime.
func errnoErr(e syscall.Errno) error {
switch e {
case 0:
return errERROR_EINVAL
case errnoERROR_IO_PENDING:
return errERROR_IO_PENDING
}
// TODO: add more here, after collecting data on the common
// error values see on Windows. (perhaps when running
// all.bat?)
return e
}
var (
modkernel32 = windows.NewLazySystemDLL("kernel32.dll")
procCreateFileW = modkernel32.NewProc("CreateFileW")
)
func CreateFile(name string, access AccessMask, mode FileShareMode, sa *syscall.SecurityAttributes, createmode FileCreationDisposition, attrs FileFlagOrAttribute, templatefile windows.Handle) (handle windows.Handle, err error) {
var _p0 *uint16
_p0, err = syscall.UTF16PtrFromString(name)
if err != nil {
return
}
return _CreateFile(_p0, access, mode, sa, createmode, attrs, templatefile)
}
func _CreateFile(name *uint16, access AccessMask, mode FileShareMode, sa *syscall.SecurityAttributes, createmode FileCreationDisposition, attrs FileFlagOrAttribute, templatefile windows.Handle) (handle windows.Handle, err error) {
r0, _, e1 := syscall.Syscall9(procCreateFileW.Addr(), 7, uintptr(unsafe.Pointer(name)), uintptr(access), uintptr(mode), uintptr(unsafe.Pointer(sa)), uintptr(createmode), uintptr(attrs), uintptr(templatefile), 0, 0)
handle = windows.Handle(r0)
if handle == windows.InvalidHandle {
err = errnoErr(e1)
}
return
}

View File

@@ -100,8 +100,8 @@ func (f *runtimeFunc) Load() error {
(*byte)(unsafe.Pointer(&f.addr)),
uint32(unsafe.Sizeof(f.addr)),
&n,
nil, // overlapped
0, // completionRoutine
nil, //overlapped
0, //completionRoutine
)
})
return f.err

View File

@@ -1,132 +0,0 @@
package stringbuffer
import (
"sync"
"unicode/utf16"
)
// TODO: worth exporting and using in mkwinsyscall?
// Uint16BufferSize is the buffer size in the pool, chosen somewhat arbitrarily to accommodate
// large path strings:
// MAX_PATH (260) + size of volume GUID prefix (49) + null terminator = 310.
const MinWStringCap = 310
// use *[]uint16 since []uint16 creates an extra allocation where the slice header
// is copied to heap and then referenced via pointer in the interface header that sync.Pool
// stores.
var pathPool = sync.Pool{ // if go1.18+ adds Pool[T], use that to store []uint16 directly
New: func() interface{} {
b := make([]uint16, MinWStringCap)
return &b
},
}
func newBuffer() []uint16 { return *(pathPool.Get().(*[]uint16)) }
// freeBuffer copies the slice header data, and puts a pointer to that in the pool.
// This avoids taking a pointer to the slice header in WString, which can be set to nil.
func freeBuffer(b []uint16) { pathPool.Put(&b) }
// WString is a wide string buffer ([]uint16) meant for storing UTF-16 encoded strings
// for interacting with Win32 APIs.
// Sizes are specified as uint32 and not int.
//
// It is not thread safe.
type WString struct {
// type-def allows casting to []uint16 directly, use struct to prevent that and allow adding fields in the future.
// raw buffer
b []uint16
}
// NewWString returns a [WString] allocated from a shared pool with an
// initial capacity of at least [MinWStringCap].
// Since the buffer may have been previously used, its contents are not guaranteed to be empty.
//
// The buffer should be freed via [WString.Free]
func NewWString() *WString {
return &WString{
b: newBuffer(),
}
}
func (b *WString) Free() {
if b.empty() {
return
}
freeBuffer(b.b)
b.b = nil
}
// ResizeTo grows the buffer to at least c and returns the new capacity, freeing the
// previous buffer back into pool.
func (b *WString) ResizeTo(c uint32) uint32 {
// allready sufficient (or n is 0)
if c <= b.Cap() {
return b.Cap()
}
if c <= MinWStringCap {
c = MinWStringCap
}
// allocate at-least double buffer size, as is done in [bytes.Buffer] and other places
if c <= 2*b.Cap() {
c = 2 * b.Cap()
}
b2 := make([]uint16, c)
if !b.empty() {
copy(b2, b.b)
freeBuffer(b.b)
}
b.b = b2
return c
}
// Buffer returns the underlying []uint16 buffer.
func (b *WString) Buffer() []uint16 {
if b.empty() {
return nil
}
return b.b
}
// Pointer returns a pointer to the first uint16 in the buffer.
// If the [WString.Free] has already been called, the pointer will be nil.
func (b *WString) Pointer() *uint16 {
if b.empty() {
return nil
}
return &b.b[0]
}
// String returns the returns the UTF-8 encoding of the UTF-16 string in the buffer.
//
// It assumes that the data is null-terminated.
func (b *WString) String() string {
// Using [windows.UTF16ToString] would require importing "golang.org/x/sys/windows"
// and would make this code Windows-only, which makes no sense.
// So copy UTF16ToString code into here.
// If other windows-specific code is added, switch to [windows.UTF16ToString]
s := b.b
for i, v := range s {
if v == 0 {
s = s[:i]
break
}
}
return string(utf16.Decode(s))
}
// Cap returns the underlying buffer capacity.
func (b *WString) Cap() uint32 {
if b.empty() {
return 0
}
return b.cap()
}
func (b *WString) cap() uint32 { return uint32(cap(b.b)) }
func (b *WString) empty() bool { return b == nil || b.cap() == 0 }

View File

@@ -16,12 +16,11 @@ import (
"unsafe"
"golang.org/x/sys/windows"
"github.com/Microsoft/go-winio/internal/fs"
)
//sys connectNamedPipe(pipe syscall.Handle, o *syscall.Overlapped) (err error) = ConnectNamedPipe
//sys createNamedPipe(name string, flags uint32, pipeMode uint32, maxInstances uint32, outSize uint32, inSize uint32, defaultTimeout uint32, sa *syscall.SecurityAttributes) (handle syscall.Handle, err error) [failretval==syscall.InvalidHandle] = CreateNamedPipeW
//sys createFile(name string, access uint32, mode uint32, sa *syscall.SecurityAttributes, createmode uint32, attrs uint32, templatefile syscall.Handle) (handle syscall.Handle, err error) [failretval==syscall.InvalidHandle] = CreateFileW
//sys getNamedPipeInfo(pipe syscall.Handle, flags *uint32, outSize *uint32, inSize *uint32, maxInstances *uint32) (err error) = GetNamedPipeInfo
//sys getNamedPipeHandleState(pipe syscall.Handle, state *uint32, curInstances *uint32, maxCollectionCount *uint32, collectDataTimeout *uint32, userName *uint16, maxUserNameSize uint32) (err error) = GetNamedPipeHandleStateW
//sys localAlloc(uFlags uint32, length uint32) (ptr uintptr) = LocalAlloc
@@ -164,21 +163,19 @@ func (s pipeAddress) String() string {
}
// tryDialPipe attempts to dial the pipe at `path` until `ctx` cancellation or timeout.
func tryDialPipe(ctx context.Context, path *string, access fs.AccessMask) (syscall.Handle, error) {
func tryDialPipe(ctx context.Context, path *string, access uint32) (syscall.Handle, error) {
for {
select {
case <-ctx.Done():
return syscall.Handle(0), ctx.Err()
default:
wh, err := fs.CreateFile(*path,
h, err := createFile(*path,
access,
0, // mode
nil, // security attributes
fs.OPEN_EXISTING,
fs.FILE_FLAG_OVERLAPPED|fs.SECURITY_SQOS_PRESENT|fs.SECURITY_ANONYMOUS,
0, // template file handle
)
h := syscall.Handle(wh)
0,
nil,
syscall.OPEN_EXISTING,
windows.FILE_FLAG_OVERLAPPED|windows.SECURITY_SQOS_PRESENT|windows.SECURITY_ANONYMOUS,
0)
if err == nil {
return h, nil
}
@@ -222,7 +219,7 @@ func DialPipeContext(ctx context.Context, path string) (net.Conn, error) {
func DialPipeAccess(ctx context.Context, path string, access uint32) (net.Conn, error) {
var err error
var h syscall.Handle
h, err = tryDialPipe(ctx, &path, fs.AccessMask(access))
h, err = tryDialPipe(ctx, &path, access)
if err != nil {
return nil, err
}
@@ -282,7 +279,6 @@ func makeServerPipeHandle(path string, sd []byte, c *PipeConfig, first bool) (sy
}
defer localFree(ntPath.Buffer)
oa.ObjectName = &ntPath
oa.Attributes = windows.OBJ_CASE_INSENSITIVE
// The security descriptor is only needed for the first pipe.
if first {

View File

@@ -20,37 +20,34 @@ type (
securityInformation uint32
trusteeForm uint32
trusteeType uint32
//nolint:structcheck // structcheck thinks fields are unused, but the are used to pass data to OS
explicitAccess struct {
accessPermissions accessMask
accessMode accessMode
inheritance inheritMode
trustee trustee
}
//nolint:structcheck,unused // structcheck thinks fields are unused, but the are used to pass data to OS
trustee struct {
multipleTrustee *trustee
multipleTrusteeOperation int32
trusteeForm trusteeForm
trusteeType trusteeType
name uintptr
}
)
type explicitAccess struct {
accessPermissions accessMask
accessMode accessMode
inheritance inheritMode
trustee trustee
}
type trustee struct {
multipleTrustee *trustee
multipleTrusteeOperation int32
trusteeForm trusteeForm
trusteeType trusteeType
name uintptr
}
const (
AccessMaskNone accessMask = 0
AccessMaskRead accessMask = 1 << 31 // GENERIC_READ
AccessMaskWrite accessMask = 1 << 30 // GENERIC_WRITE
AccessMaskExecute accessMask = 1 << 29 // GENERIC_EXECUTE
AccessMaskAll accessMask = 1 << 28 // GENERIC_ALL
accessMaskDesiredPermission = AccessMaskRead
accessMaskDesiredPermission accessMask = 1 << 31 // GENERIC_READ
accessModeGrant accessMode = 1
desiredAccessReadControl desiredAccess = 0x20000
desiredAccessWriteDac desiredAccess = 0x40000
//cspell:disable-next-line
gvmga = "GrantVmGroupAccess:"
inheritModeNoInheritance inheritMode = 0x0
@@ -63,28 +60,20 @@ const (
shareModeRead shareMode = 0x1
shareModeWrite shareMode = 0x2
//nolint:stylecheck // ST1003
sidVmGroup = "S-1-5-83-0"
sidVMGroup = "S-1-5-83-0"
trusteeFormIsSid trusteeForm = 0
trusteeFormIsSID trusteeForm = 0
trusteeTypeWellKnownGroup trusteeType = 5
)
// GrantVmGroupAccess sets the DACL for a specified file or directory to
// GrantVMGroupAccess sets the DACL for a specified file or directory to
// include Grant ACE entries for the VM Group SID. This is a golang re-
// implementation of the same function in vmcompute, just not exported in
// RS5. Which kind of sucks. Sucks a lot :/
func GrantVmGroupAccess(name string) error { //nolint:stylecheck // ST1003
return GrantVmGroupAccessWithMask(name, accessMaskDesiredPermission)
}
// GrantVmGroupAccessWithMask sets the desired DACL for a specified file or
// directory.
func GrantVmGroupAccessWithMask(name string, access accessMask) error { //nolint:stylecheck // ST1003
if access == 0 || access<<4 != 0 {
return fmt.Errorf("invalid access mask: 0x%08x", access)
}
//
//revive:disable-next-line:var-naming VM, not Vm
func GrantVmGroupAccess(name string) error {
// Stat (to determine if `name` is a directory).
s, err := os.Stat(name)
if err != nil {
@@ -96,9 +85,7 @@ func GrantVmGroupAccessWithMask(name string, access accessMask) error { //nolint
if err != nil {
return err // Already wrapped
}
defer func() {
_ = syscall.CloseHandle(fd)
}()
defer syscall.CloseHandle(fd) //nolint:errcheck
// Get the current DACL and Security Descriptor. Must defer LocalFree on success.
ot := objectTypeFileObject
@@ -108,19 +95,15 @@ func GrantVmGroupAccessWithMask(name string, access accessMask) error { //nolint
if err := getSecurityInfo(fd, uint32(ot), uint32(si), nil, nil, &origDACL, nil, &sd); err != nil {
return fmt.Errorf("%s GetSecurityInfo %s: %w", gvmga, name, err)
}
defer func() {
_, _ = syscall.LocalFree((syscall.Handle)(unsafe.Pointer(sd)))
}()
defer syscall.LocalFree((syscall.Handle)(unsafe.Pointer(sd))) //nolint:errcheck
// Generate a new DACL which is the current DACL with the required ACEs added.
// Must defer LocalFree on success.
newDACL, err := generateDACLWithAcesAdded(name, s.IsDir(), access, origDACL)
newDACL, err := generateDACLWithAcesAdded(name, s.IsDir(), origDACL)
if err != nil {
return err // Already wrapped
}
defer func() {
_, _ = syscall.LocalFree((syscall.Handle)(unsafe.Pointer(newDACL)))
}()
defer syscall.LocalFree((syscall.Handle)(unsafe.Pointer(newDACL))) //nolint:errcheck
// And finally use SetSecurityInfo to apply the updated DACL.
if err := setSecurityInfo(fd, uint32(ot), uint32(si), uintptr(0), uintptr(0), newDACL, uintptr(0)); err != nil {
@@ -135,28 +118,28 @@ func GrantVmGroupAccessWithMask(name string, access accessMask) error { //nolint
func createFile(name string, isDir bool) (syscall.Handle, error) {
namep, err := syscall.UTF16FromString(name)
if err != nil {
return 0, fmt.Errorf("syscall.UTF16FromString %s: %w", name, err)
return syscall.InvalidHandle, fmt.Errorf("could not convernt name to UTF-16: %w", err)
}
da := uint32(desiredAccessReadControl | desiredAccessWriteDac)
sm := uint32(shareModeRead | shareModeWrite)
fa := uint32(syscall.FILE_ATTRIBUTE_NORMAL)
if isDir {
fa = uint32(fa | syscall.FILE_FLAG_BACKUP_SEMANTICS)
fa |= syscall.FILE_FLAG_BACKUP_SEMANTICS
}
fd, err := syscall.CreateFile(&namep[0], da, sm, nil, syscall.OPEN_EXISTING, fa, 0)
if err != nil {
return 0, fmt.Errorf("%s syscall.CreateFile %s: %w", gvmga, name, err)
return syscall.InvalidHandle, fmt.Errorf("%s syscall.CreateFile %s: %w", gvmga, name, err)
}
return fd, nil
}
// generateDACLWithAcesAdded generates a new DACL with the two needed ACEs added.
// The caller is responsible for LocalFree of the returned DACL on success.
func generateDACLWithAcesAdded(name string, isDir bool, desiredAccess accessMask, origDACL uintptr) (uintptr, error) {
func generateDACLWithAcesAdded(name string, isDir bool, origDACL uintptr) (uintptr, error) {
// Generate pointers to the SIDs based on the string SIDs
sid, err := syscall.StringToSid(sidVmGroup)
sid, err := syscall.StringToSid(sidVMGroup)
if err != nil {
return 0, fmt.Errorf("%s syscall.StringToSid %s %s: %w", gvmga, name, sidVmGroup, err)
return 0, fmt.Errorf("%s syscall.StringToSid %s %s: %w", gvmga, name, sidVMGroup, err)
}
inheritance := inheritModeNoInheritance
@@ -166,11 +149,11 @@ func generateDACLWithAcesAdded(name string, isDir bool, desiredAccess accessMask
eaArray := []explicitAccess{
{
accessPermissions: desiredAccess,
accessPermissions: accessMaskDesiredPermission,
accessMode: accessModeGrant,
inheritance: inheritance,
trustee: trustee{
trusteeForm: trusteeFormIsSid,
trusteeForm: trusteeFormIsSID,
trusteeType: trusteeTypeWellKnownGroup,
name: uintptr(unsafe.Pointer(sid)),
},

View File

@@ -63,6 +63,7 @@ var (
procBackupWrite = modkernel32.NewProc("BackupWrite")
procCancelIoEx = modkernel32.NewProc("CancelIoEx")
procConnectNamedPipe = modkernel32.NewProc("ConnectNamedPipe")
procCreateFileW = modkernel32.NewProc("CreateFileW")
procCreateIoCompletionPort = modkernel32.NewProc("CreateIoCompletionPort")
procCreateNamedPipeW = modkernel32.NewProc("CreateNamedPipeW")
procGetCurrentThread = modkernel32.NewProc("GetCurrentThread")
@@ -304,6 +305,24 @@ func connectNamedPipe(pipe syscall.Handle, o *syscall.Overlapped) (err error) {
return
}
func createFile(name string, access uint32, mode uint32, sa *syscall.SecurityAttributes, createmode uint32, attrs uint32, templatefile syscall.Handle) (handle syscall.Handle, err error) {
var _p0 *uint16
_p0, err = syscall.UTF16PtrFromString(name)
if err != nil {
return
}
return _createFile(_p0, access, mode, sa, createmode, attrs, templatefile)
}
func _createFile(name *uint16, access uint32, mode uint32, sa *syscall.SecurityAttributes, createmode uint32, attrs uint32, templatefile syscall.Handle) (handle syscall.Handle, err error) {
r0, _, e1 := syscall.Syscall9(procCreateFileW.Addr(), 7, uintptr(unsafe.Pointer(name)), uintptr(access), uintptr(mode), uintptr(unsafe.Pointer(sa)), uintptr(createmode), uintptr(attrs), uintptr(templatefile), 0, 0)
handle = syscall.Handle(r0)
if handle == syscall.InvalidHandle {
err = errnoErr(e1)
}
return
}
func createIoCompletionPort(file syscall.Handle, port syscall.Handle, key uintptr, threadCount uint32) (newport syscall.Handle, err error) {
r0, _, e1 := syscall.Syscall6(procCreateIoCompletionPort.Addr(), 4, uintptr(file), uintptr(port), uintptr(key), uintptr(threadCount), 0, 0)
newport = syscall.Handle(r0)

View File

@@ -1,3 +1 @@
* text=auto eol=lf
vendor/** -text
test/vendor/** -text
* text=auto eol=lf

View File

@@ -6,7 +6,6 @@
# Ignore vscode setting files
.vscode/
.idea/
# Test binary, build with `go test -c`
*.test
@@ -24,30 +23,16 @@ service/pkg/
*.img
*.vhd
*.tar.gz
*.tar
# Make stuff
.rootfs-done
bin/*
rootfs/*
rootfs-conv/*
*.o
/build/
deps/*
out/*
# protobuf files
# only files at root of the repo, otherwise this will cause issues with vendoring
/protobuf/*
# test results
test/results
# go workspace files
go.work
go.work.sum
# keys and related artifacts
*.pem
*.cose
.idea/
.vscode/

View File

@@ -1,65 +1,23 @@
run:
timeout: 8m
tests: true
build-tags:
- admin
- functional
- integration
skip-dirs:
# paths are relative to module root
- cri-containerd/test-images
linters:
enable:
# defaults:
# - errcheck
# - gosimple
# - govet
# - ineffassign
# - staticcheck
# - typecheck
# - unused
- gofmt # whether code was gofmt-ed
- govet # enabled by default, but just to be sure
- nolintlint # ill-formed or insufficient nolint directives
- stylecheck # golint replacement
- thelper # test helpers without t.Helper()
- stylecheck
linters-settings:
govet:
enable-all: true
disable:
# struct order is often for Win32 compat
# also, ignore pointer bytes/GC issues for now until performance becomes an issue
- fieldalignment
check-shadowing: true
stylecheck:
# https://staticcheck.io/docs/checks
checks: ["all"]
issues:
# This repo has a LOT of generated schema files, operating system bindings, and other things that ST1003 from stylecheck won't like
# (screaming case Windows api constants for example). There's also some structs that we *could* change the initialisms to be Go
# friendly (Id -> ID) but they're exported and it would be a breaking change. This makes it so that most new code, code that isn't
# supposed to be a pretty faithful mapping to an OS call/constants, or non-generated code still checks if we're following idioms,
# while ignoring the things that are just noise or would be more of a hassle than it'd be worth to change.
exclude-rules:
# err is very often shadowed in nested scopes
- linters:
- govet
text: '^shadow: declaration of "err" shadows declaration'
# path is relative to module root, which is ./test/
- path: cri-containerd
linters:
- stylecheck
text: "^ST1003: should not use underscores in package names$"
source: "^package cri_containerd$"
# This repo has a LOT of generated schema files, operating system bindings, and other
# things that ST1003 from stylecheck won't like (screaming case Windows api constants for example).
# There's also some structs that we *could* change the initialisms to be Go friendly
# (Id -> ID) but they're exported and it would be a breaking change.
# This makes it so that most new code, code that isn't supposed to be a pretty faithful
# mapping to an OS call/constants, or non-generated code still checks if we're following idioms,
# while ignoring the things that are just noise or would be more of a hassle than it'd be worth to change.
- path: layer.go
linters:
- stylecheck
@@ -70,20 +28,10 @@ issues:
- stylecheck
Text: "ST1003:"
- path: cmd\\ncproxy\\nodenetsvc\\
linters:
- stylecheck
Text: "ST1003:"
- path: cmd\\ncproxy_mock\\
linters:
- stylecheck
Text: "ST1003:"
- path: internal\\hcs\\schema2\\
linters:
- stylecheck
- gofmt
Text: "ST1003:"
- path: internal\\wclayer\\
linters:
@@ -148,20 +96,4 @@ issues:
- path: internal\\hcserror\\
linters:
- stylecheck
Text: "ST1003:"
# v0 APIs are deprecated, but still retained for backwards compatability
- path: cmd\\ncproxy\\
linters:
- staticcheck
text: "^SA1019: .*(ncproxygrpc|nodenetsvc)[/]?v0"
- path: internal\\tools\\networkagent
linters:
- staticcheck
text: "^SA1019: .*nodenetsvc[/]?v0"
- path: internal\\vhdx\\info
linters:
- stylecheck
Text: "ST1003:"
Text: "ST1003:"

View File

@@ -1,5 +1,4 @@
BASE:=base.tar.gz
DEV_BUILD:=0
GO:=go
GO_FLAGS:=-ldflags "-s -w" # strip Go binaries
@@ -13,31 +12,16 @@ GO_FLAGS_EXTRA:=
ifeq "$(GOMODVENDOR)" "1"
GO_FLAGS_EXTRA += -mod=vendor
endif
GO_BUILD_TAGS:=
ifneq ($(strip $(GO_BUILD_TAGS)),)
GO_FLAGS_EXTRA += -tags="$(GO_BUILD_TAGS)"
endif
GO_BUILD:=CGO_ENABLED=$(CGO_ENABLED) $(GO) build $(GO_FLAGS) $(GO_FLAGS_EXTRA)
SRCROOT=$(dir $(abspath $(firstword $(MAKEFILE_LIST))))
# additional directories to search for rule prerequisites and targets
VPATH=$(SRCROOT)
DELTA_TARGET=out/delta.tar.gz
ifeq "$(DEV_BUILD)" "1"
DELTA_TARGET=out/delta-dev.tar.gz
endif
# The link aliases for gcstools
GCS_TOOLS=\
generichook \
install-drivers
generichook
.PHONY: all always rootfs test
.DEFAULT_GOAL := all
all: out/initrd.img out/rootfs.tar.gz
clean:
@@ -45,13 +29,21 @@ clean:
rm -rf bin deps rootfs out
test:
cd $(SRCROOT) && $(GO) test -v ./internal/guest/...
cd $(SRCROOT) && go test -v ./internal/guest/...
rootfs: out/rootfs.vhd
out/rootfs.vhd: out/rootfs.tar.gz bin/cmd/tar2ext4
gzip -f -d ./out/rootfs.tar.gz
bin/cmd/tar2ext4 -vhd -i ./out/rootfs.tar -o $@
out/delta.tar.gz: bin/init bin/vsockexec bin/cmd/gcs bin/cmd/gcstools Makefile
@mkdir -p out
rm -rf rootfs
mkdir -p rootfs/bin/
cp bin/init rootfs/
cp bin/vsockexec rootfs/bin/
cp bin/cmd/gcs rootfs/bin/
cp bin/cmd/gcstools rootfs/bin/
for tool in $(GCS_TOOLS); do ln -s gcstools rootfs/bin/$$tool; done
git -C $(SRCROOT) rev-parse HEAD > rootfs/gcs.commit && \
git -C $(SRCROOT) rev-parse --abbrev-ref HEAD > rootfs/gcs.branch
tar -zcf $@ -C rootfs .
rm -rf rootfs
out/rootfs.tar.gz: out/initrd.img
rm -rf rootfs-conv
@@ -60,43 +52,27 @@ out/rootfs.tar.gz: out/initrd.img
tar -zcf $@ -C rootfs-conv .
rm -rf rootfs-conv
out/initrd.img: $(BASE) $(DELTA_TARGET) $(SRCROOT)/hack/catcpio.sh
$(SRCROOT)/hack/catcpio.sh "$(BASE)" $(DELTA_TARGET) > out/initrd.img.uncompressed
out/initrd.img: $(BASE) out/delta.tar.gz $(SRCROOT)/hack/catcpio.sh
$(SRCROOT)/hack/catcpio.sh "$(BASE)" out/delta.tar.gz > out/initrd.img.uncompressed
gzip -c out/initrd.img.uncompressed > $@
rm out/initrd.img.uncompressed
# This target includes utilities which may be useful for testing purposes.
out/delta-dev.tar.gz: out/delta.tar.gz bin/internal/tools/snp-report
rm -rf rootfs-dev
mkdir rootfs-dev
tar -xzf out/delta.tar.gz -C rootfs-dev
cp bin/internal/tools/snp-report rootfs-dev/bin/
tar -zcf $@ -C rootfs-dev .
rm -rf rootfs-dev
-include deps/cmd/gcs.gomake
-include deps/cmd/gcstools.gomake
out/delta.tar.gz: bin/init bin/vsockexec bin/cmd/gcs bin/cmd/gcstools bin/cmd/hooks/wait-paths Makefile
@mkdir -p out
rm -rf rootfs
mkdir -p rootfs/bin/
mkdir -p rootfs/info/
cp bin/init rootfs/
cp bin/vsockexec rootfs/bin/
cp bin/cmd/gcs rootfs/bin/
cp bin/cmd/gcstools rootfs/bin/
cp bin/cmd/hooks/wait-paths rootfs/bin/
for tool in $(GCS_TOOLS); do ln -s gcstools rootfs/bin/$$tool; done
git -C $(SRCROOT) rev-parse HEAD > rootfs/info/gcs.commit && \
git -C $(SRCROOT) rev-parse --abbrev-ref HEAD > rootfs/info/gcs.branch && \
date --iso-8601=minute --utc > rootfs/info/tar.date
$(if $(and $(realpath $(subst .tar,.testdata.json,$(BASE))), $(shell which jq)), \
jq -r '.IMAGE_NAME' $(subst .tar,.testdata.json,$(BASE)) 2>/dev/null > rootfs/info/image.name && \
jq -r '.DATETIME' $(subst .tar,.testdata.json,$(BASE)) 2>/dev/null > rootfs/info/build.date)
tar -zcf $@ -C rootfs .
rm -rf rootfs
bin/cmd/gcs bin/cmd/gcstools bin/cmd/hooks/wait-paths bin/cmd/tar2ext4 bin/internal/tools/snp-report:
# Implicit rule for includes that define Go targets.
%.gomake: $(SRCROOT)/Makefile
@mkdir -p $(dir $@)
GOOS=linux $(GO_BUILD) -o $@ $(SRCROOT)/$(@:bin/%=%)
@/bin/echo $(@:deps/%.gomake=bin/%): $(SRCROOT)/hack/gomakedeps.sh > $@.new
@/bin/echo -e '\t@mkdir -p $$(dir $$@) $(dir $@)' >> $@.new
@/bin/echo -e '\t$$(GO_BUILD) -o $$@.new $$(SRCROOT)/$$(@:bin/%=%)' >> $@.new
@/bin/echo -e '\tGO="$(GO)" $$(SRCROOT)/hack/gomakedeps.sh $$@ $$(SRCROOT)/$$(@:bin/%=%) $$(GO_FLAGS) $$(GO_FLAGS_EXTRA) > $(@:%.gomake=%.godeps).new' >> $@.new
@/bin/echo -e '\tmv $(@:%.gomake=%.godeps).new $(@:%.gomake=%.godeps)' >> $@.new
@/bin/echo -e '\tmv $$@.new $$@' >> $@.new
@/bin/echo -e '-include $(@:%.gomake=%.godeps)' >> $@.new
mv $@.new $@
VPATH=$(SRCROOT)
bin/vsockexec: vsockexec/vsockexec.o vsockexec/vsock.o
@mkdir -p bin

View File

@@ -1,25 +1,49 @@
version = "2"
generators = ["go", "go-grpc"]
version = "unstable"
generator = "gogoctrd"
plugins = ["grpc", "fieldpath"]
# Control protoc include paths.
# Control protoc include paths. Below are usually some good defaults, but feel
# free to try it without them if it works for your project.
[includes]
# Include paths that will be added before all others. Typically, you want to
# treat the root of the project as an include, but this may not be necessary.
before = ["./protobuf"]
# defaults are "/usr/local/include" and "/usr/include", which don't exist on Windows.
# override defaults to supress errors about non-existant directories.
after = []
# Paths that should be treated as include roots in relation to the vendor
# directory. These will be calculated with the vendor directory nearest the
# target package.
packages = ["github.com/gogo/protobuf"]
# This section maps protobuf imports to Go packages.
# Paths that will be added untouched to the end of the includes. We use
# `/usr/local/include` to pickup the common install location of protobuf.
# This is the default.
after = ["/usr/local/include"]
# This section maps protobuf imports to Go packages. These will become
# `-M` directives in the call to the go protobuf generator.
[packages]
# github.com/containerd/cgroups protofiles still list their go path as "github.com/containerd/cgroups/cgroup1/stats"
"github.com/containerd/cgroups/v3/cgroup1/stats/metrics.proto" = "github.com/containerd/cgroups/v3/cgroup1/stats"
"gogoproto/gogo.proto" = "github.com/gogo/protobuf/gogoproto"
"google/protobuf/any.proto" = "github.com/gogo/protobuf/types"
"google/protobuf/empty.proto" = "github.com/gogo/protobuf/types"
"google/protobuf/struct.proto" = "github.com/gogo/protobuf/types"
"google/protobuf/descriptor.proto" = "github.com/gogo/protobuf/protoc-gen-gogo/descriptor"
"google/protobuf/field_mask.proto" = "github.com/gogo/protobuf/types"
"google/protobuf/timestamp.proto" = "github.com/gogo/protobuf/types"
"google/protobuf/duration.proto" = "github.com/gogo/protobuf/types"
"github/containerd/cgroups/stats/v1/metrics.proto" = "github.com/containerd/cgroups/stats/v1"
[[overrides]]
prefixes = [
"github.com/Microsoft/hcsshim/internal/shimdiag",
"github.com/Microsoft/hcsshim/internal/extendedtask",
"github.com/Microsoft/hcsshim/internal/computeagent",
"github.com/Microsoft/hcsshim/internal/ncproxyttrpc",
"github.com/Microsoft/hcsshim/internal/vmservice",
]
generators = ["go", "go-ttrpc"]
prefixes = ["github.com/Microsoft/hcsshim/internal/shimdiag"]
plugins = ["ttrpc"]
[[overrides]]
prefixes = ["github.com/Microsoft/hcsshim/internal/computeagent"]
plugins = ["ttrpc"]
[[overrides]]
prefixes = ["github.com/Microsoft/hcsshim/internal/ncproxyttrpc"]
plugins = ["ttrpc"]
[[overrides]]
prefixes = ["github.com/Microsoft/hcsshim/internal/vmservice"]
plugins = ["ttrpc"]

View File

@@ -75,6 +75,24 @@ certify they either authored the work themselves or otherwise have permission to
more info, as well as to make sure that you can attest to the rules listed. Our CI uses the [DCO Github app](https://github.com/apps/dco) to ensure
that all commits in a given PR are signed-off.
### Test Directory (Important to note)
This project has tried to trim some dependencies from the root Go modules file that would be cumbersome to get transitively included if this
project is being vendored/used as a library. Some of these dependencies were only being used for tests, so the /test directory in this project also has
its own go.mod file where these are now included to get around this issue. Our tests rely on the code in this project to run, so the test Go modules file
has a relative path replace directive to pull in the latest hcsshim code that the tests actually touch from this project
(which is the repo itself on your disk).
```
replace (
github.com/Microsoft/hcsshim => ../
)
```
Because of this, for most code changes you may need to run `go mod vendor` + `go mod tidy` in the /test directory in this repository, as the
CI in this project will check if the files are out of date and will fail if this is true.
## Code of Conduct
This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/).
@@ -83,7 +101,7 @@ contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additio
## Dependencies
This project requires Golang 1.17 or newer to build.
This project requires Golang 1.9 or newer to build.
For system requirements to run this project, see the Microsoft docs on [Windows Container requirements](https://docs.microsoft.com/en-us/virtualization/windowscontainers/deploy-containers/system-requirements).

View File

@@ -1,41 +0,0 @@
<!-- BEGIN MICROSOFT SECURITY.MD V0.0.7 BLOCK -->
## Security
Microsoft takes the security of our software products and services seriously, which includes all source code repositories managed through our GitHub organizations, which include [Microsoft](https://github.com/Microsoft), [Azure](https://github.com/Azure), [DotNet](https://github.com/dotnet), [AspNet](https://github.com/aspnet), [Xamarin](https://github.com/xamarin), and [our GitHub organizations](https://opensource.microsoft.com/).
If you believe you have found a security vulnerability in any Microsoft-owned repository that meets [Microsoft's definition of a security vulnerability](https://aka.ms/opensource/security/definition), please report it to us as described below.
## Reporting Security Issues
**Please do not report security vulnerabilities through public GitHub issues.**
Instead, please report them to the Microsoft Security Response Center (MSRC) at [https://msrc.microsoft.com/create-report](https://aka.ms/opensource/security/create-report).
If you prefer to submit without logging in, send email to [secure@microsoft.com](mailto:secure@microsoft.com). If possible, encrypt your message with our PGP key; please download it from the [Microsoft Security Response Center PGP Key page](https://aka.ms/opensource/security/pgpkey).
You should receive a response within 24 hours. If for some reason you do not, please follow up via email to ensure we received your original message. Additional information can be found at [microsoft.com/msrc](https://aka.ms/opensource/security/msrc).
Please include the requested information listed below (as much as you can provide) to help us better understand the nature and scope of the possible issue:
* Type of issue (e.g. buffer overflow, SQL injection, cross-site scripting, etc.)
* Full paths of source file(s) related to the manifestation of the issue
* The location of the affected source code (tag/branch/commit or direct URL)
* Any special configuration required to reproduce the issue
* Step-by-step instructions to reproduce the issue
* Proof-of-concept or exploit code (if possible)
* Impact of the issue, including how an attacker might exploit the issue
This information will help us triage your report more quickly.
If you are reporting for a bug bounty, more complete reports can contribute to a higher bounty award. Please visit our [Microsoft Bug Bounty Program](https://aka.ms/opensource/security/bounty) page for more details about our active programs.
## Preferred Languages
We prefer all communications to be in English.
## Policy
Microsoft follows the principle of [Coordinated Vulnerability Disclosure](https://aka.ms/opensource/security/cvd).
<!-- END MICROSOFT SECURITY.MD BLOCK -->

View File

@@ -1,5 +1,3 @@
//go:build windows
package computestorage
import (
@@ -19,8 +17,8 @@ import (
//
// `layerData` is the parent read-only layer data.
func AttachLayerStorageFilter(ctx context.Context, layerPath string, layerData LayerData) (err error) {
title := "hcsshim::AttachLayerStorageFilter"
ctx, span := oc.StartSpan(ctx, title) //nolint:ineffassign,staticcheck
title := "hcsshim.AttachLayerStorageFilter"
ctx, span := trace.StartSpan(ctx, title) //nolint:ineffassign,staticcheck
defer span.End()
defer func() { oc.SetSpanStatus(span, err) }()
span.AddAttributes(

View File

@@ -1,5 +1,3 @@
//go:build windows
package computestorage
import (
@@ -14,8 +12,8 @@ import (
//
// `layerPath` is a path to a directory containing the layer to export.
func DestroyLayer(ctx context.Context, layerPath string) (err error) {
title := "hcsshim::DestroyLayer"
ctx, span := oc.StartSpan(ctx, title) //nolint:ineffassign,staticcheck
title := "hcsshim.DestroyLayer"
ctx, span := trace.StartSpan(ctx, title) //nolint:ineffassign,staticcheck
defer span.End()
defer func() { oc.SetSpanStatus(span, err) }()
span.AddAttributes(trace.StringAttribute("layerPath", layerPath))

View File

@@ -1,5 +1,3 @@
//go:build windows
package computestorage
import (
@@ -14,8 +12,8 @@ import (
//
// `layerPath` is a path to a directory containing the layer to export.
func DetachLayerStorageFilter(ctx context.Context, layerPath string) (err error) {
title := "hcsshim::DetachLayerStorageFilter"
ctx, span := oc.StartSpan(ctx, title) //nolint:ineffassign,staticcheck
title := "hcsshim.DetachLayerStorageFilter"
ctx, span := trace.StartSpan(ctx, title) //nolint:ineffassign,staticcheck
defer span.End()
defer func() { oc.SetSpanStatus(span, err) }()
span.AddAttributes(trace.StringAttribute("layerPath", layerPath))

Some files were not shown because too many files have changed in this diff Show More