Compare commits

...

25 Commits

Author SHA1 Message Date
Miloslav Trmač
89966c513c Release 1.9.3
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-19 19:11:14 +02:00
Miloslav Trmač
2382f751cb Update to c/image v5.22.1
> go get github.com/containers/image/v5@v5.22.1
> make vendor

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-19 19:11:14 +02:00
Miloslav Trmač
f24433e290 Pin Go to 1.18
1.19 has changed the expected gofmt format, and we don't want
to follow such changes on the stable branch.

go@1.18 is "keg-only", i.e. not installed by Brew to /usr/local/bin,
so we need to change PATH to point at it (as the installation instructs
us to).

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-10-19 19:11:14 +02:00
Daniel J Walsh
14b05e8064 Merge pull request #1724 from mtrmac/release-1.9.2
Release 1.9.2
2022-08-02 14:12:59 -04:00
Miloslav Trmač
e95123a2d4 Bump to v1.9.3-dev
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-08-02 18:59:24 +02:00
Miloslav Trmač
ca1b0f34d1 Release v1.9.2
- [CI:DOCS] Cirrus: Use the latest imgts container
- Cirrus: Update CI VM images to match podman CI
- Bump github.com/containers/common from 0.49.0 to 0.49.1

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-08-02 18:58:52 +02:00
Daniel J Walsh
28a5365945 Merge pull request #1722 from containers/dependabot/go_modules/github.com/containers/common-0.49.1
Bump github.com/containers/common from 0.49.0 to 0.49.1
2022-08-01 07:21:40 -04:00
dependabot[bot]
73a668e99d Bump github.com/containers/common from 0.49.0 to 0.49.1
Bumps [github.com/containers/common](https://github.com/containers/common) from 0.49.0 to 0.49.1.
- [Release notes](https://github.com/containers/common/releases)
- [Commits](https://github.com/containers/common/compare/v0.49.0...v0.49.1)

---
updated-dependencies:
- dependency-name: github.com/containers/common
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-08-01 08:20:56 +00:00
Miloslav Trmač
61c28f5d47 Merge pull request #1711 from cevich/updated_f36
Cirrus: Update CI VM images to match podman CI
2022-07-29 00:24:59 +02:00
Chris Evich
eafd7e5518 Cirrus: Update CI VM images to match podman CI
Note: Removed disused `PRIOR_FEDORA*` and `UBUNTU_*` references since
they're not actually used in this CI.  Further, F35 VM images were not
built as part of `c6013173500215296` due to a missing golang 1.18
requirement for podman.

Signed-off-by: Chris Evich <cevich@redhat.com>
2022-07-27 14:22:05 -04:00
Chris Evich
2cfbeb2db8 Merge pull request #1714 from cevich/latest_imgts
[CI:DOCS] Cirrus: Use the latest imgts container
2022-07-26 15:59:58 -04:00
Chris Evich
b9cf626ea3 [CI:DOCS] Cirrus: Use the latest imgts container
Contains important updates re: preserving release-branch CI VM images.
Ref: https://github.com/containers/automation_images/pull/157

Signed-off-by: Chris Evich <cevich@redhat.com>
2022-07-26 15:47:39 -04:00
Daniel J Walsh
263d3264ba Merge pull request #1713 from mtrmac/release
Release v1.9.1
2022-07-25 15:54:20 -04:00
Miloslav Trmač
63dabfcf8b Bump to v1.9.2-dev
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-07-25 21:15:41 +02:00
Miloslav Trmač
2eac0f463a Release v1.9.1
- Bump github.com/sirupsen/logrus from 1.8.1 to 1.9.0
- Bump github.com/containers/storage from 1.41.0 to 1.42.0
- Update to github.com/containers/image/v5 v5.22.0
- Update to github.com/containers/common v0.49.0
- Stop using deprecated names from c/common/pkg/retry

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-07-25 21:14:44 +02:00
Miloslav Trmač
c10b63dc71 Merge pull request #1712 from mtrmac/release
Update c/image and c/common
2022-07-25 21:13:40 +02:00
Miloslav Trmač
b7e7374e71 Stop using deprecated names from c/common/pkg/retry
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-07-25 18:16:13 +02:00
Miloslav Trmač
08846d18cc Update to github.com/containers/common v0.49.0
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-07-25 18:13:11 +02:00
Miloslav Trmač
049163fcec Update to github.com/containers/image/v5 v5.22.0
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-07-25 18:05:03 +02:00
Miloslav Trmač
3039cd5a77 Merge pull request #1710 from containers/dependabot/go_modules/github.com/containers/storage-1.42.0
Bump github.com/containers/storage from 1.41.0 to 1.42.0
2022-07-22 16:28:11 +02:00
dependabot[bot]
b42e664854 Bump github.com/containers/storage from 1.41.0 to 1.42.0
Bumps [github.com/containers/storage](https://github.com/containers/storage) from 1.41.0 to 1.42.0.
- [Release notes](https://github.com/containers/storage/releases)
- [Changelog](https://github.com/containers/storage/blob/main/docs/containers-storage-changes.md)
- [Commits](https://github.com/containers/storage/compare/v1.41.0...v1.42.0)

---
updated-dependencies:
- dependency-name: github.com/containers/storage
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-07-22 08:17:06 +00:00
Miloslav Trmač
ad12a292a3 Merge pull request #1709 from containers/dependabot/go_modules/github.com/sirupsen/logrus-1.9.0
Bump github.com/sirupsen/logrus from 1.8.1 to 1.9.0
2022-07-19 12:56:15 +02:00
dependabot[bot]
ee477d8877 Bump github.com/sirupsen/logrus from 1.8.1 to 1.9.0
Bumps [github.com/sirupsen/logrus](https://github.com/sirupsen/logrus) from 1.8.1 to 1.9.0.
- [Release notes](https://github.com/sirupsen/logrus/releases)
- [Changelog](https://github.com/sirupsen/logrus/blob/master/CHANGELOG.md)
- [Commits](https://github.com/sirupsen/logrus/compare/v1.8.1...v1.9.0)

---
updated-dependencies:
- dependency-name: github.com/sirupsen/logrus
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-07-19 08:11:51 +00:00
Daniel J Walsh
dbe47d765a Merge pull request #1705 from mtrmac/release-1.9.0
Release v1.9.0
2022-07-13 09:52:15 -04:00
Miloslav Trmač
f1485781be Bump to v1.9.1-dev
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2022-07-13 10:34:41 +02:00
247 changed files with 12255 additions and 5293 deletions

View File

@@ -24,19 +24,13 @@ env:
#### Cache-image names to test with (double-quotes around names are critical)
####
FEDORA_NAME: "fedora-36"
PRIOR_FEDORA_NAME: "fedora-35"
UBUNTU_NAME: "ubuntu-2204"
# Google-cloud VM Images
IMAGE_SUFFIX: "c6340043416535040"
IMAGE_SUFFIX: "c5495735033528320"
FEDORA_CACHE_IMAGE_NAME: "fedora-${IMAGE_SUFFIX}"
PRIOR_FEDORA_CACHE_IMAGE_NAME: "prior-fedora-${IMAGE_SUFFIX}"
UBUNTU_CACHE_IMAGE_NAME: "ubuntu-${IMAGE_SUFFIX}"
# Container FQIN's
FEDORA_CONTAINER_FQIN: "quay.io/libpod/fedora_podman:${IMAGE_SUFFIX}"
PRIOR_FEDORA_CONTAINER_FQIN: "quay.io/libpod/prior-fedora_podman:${IMAGE_SUFFIX}"
UBUNTU_CONTAINER_FQIN: "quay.io/libpod/ubuntu_podman:${IMAGE_SUFFIX}"
# Built along with the standard PR-based workflow in c/automation_images
SKOPEO_CIDEV_CONTAINER_FQIN: "quay.io/libpod/skopeo_cidev:${IMAGE_SUFFIX}"
@@ -90,12 +84,13 @@ osx_task:
macos_instance:
image: catalina-xcode
setup_script: |
export PATH=$GOPATH/bin:$PATH
# /usr/local/opt/go@1.18 will be populated by (brew install go@1.18) below
export PATH=$GOPATH/bin:/usr/local/opt/go@1.18/bin:$PATH
brew update
brew install gpgme go go-md2man
brew install gpgme go@1.18 go-md2man
go install golang.org/x/lint/golint@latest
test_script: |
export PATH=$GOPATH/bin:$PATH
export PATH=$GOPATH/bin:/usr/local/opt/go@1.18/bin:$PATH
go version
go env
make validate-local test-unit-local bin/skopeo
@@ -221,13 +216,12 @@ meta_task:
container: &smallcontainer
cpu: 2
memory: 2
image: quay.io/libpod/imgts:$IMAGE_SUFFIX
image: quay.io/libpod/imgts:latest
env:
# Space-separated list of images used by this repository state
IMGNAMES: >-
IMGNAMES: |
${FEDORA_CACHE_IMAGE_NAME}
${PRIOR_FEDORA_CACHE_IMAGE_NAME}
${UBUNTU_CACHE_IMAGE_NAME}
build-push-${IMAGE_SUFFIX}
BUILDID: "${CIRRUS_BUILD_ID}"
REPOREF: "${CIRRUS_REPO_NAME}"
GCPJSON: ENCRYPTED[6867b5a83e960e7c159a98fe6c8360064567a071c6f4b5e7d532283ecd870aa65c94ccd74bdaa9bf7aadac9d42e20a67]

View File

@@ -25,7 +25,7 @@ type copyOptions struct {
deprecatedTLSVerify *deprecatedTLSVerifyOption
srcImage *imageOptions
destImage *imageDestOptions
retryOpts *retry.RetryOptions
retryOpts *retry.Options
additionalTags []string // For docker-archive: destinations, in addition to the name:tag specified as destination, also add these
removeSignatures bool // Do not copy signatures from the source image
signByFingerprint string // Sign the image using a GPG key with the specified fingerprint
@@ -260,7 +260,7 @@ func (opts *copyOptions) run(args []string, stdout io.Writer) (retErr error) {
}
}
return retry.RetryIfNecessary(ctx, func() error {
return retry.IfNecessary(ctx, func() error {
manifestBytes, err := copy.Image(ctx, policyContext, destRef, srcRef, &copy.Options{
RemoveSignatures: opts.removeSignatures,
SignBy: opts.signByFingerprint,

View File

@@ -15,7 +15,7 @@ import (
type deleteOptions struct {
global *globalOptions
image *imageOptions
retryOpts *retry.RetryOptions
retryOpts *retry.Options
}
func deleteCmd(global *globalOptions) *cobra.Command {
@@ -70,7 +70,7 @@ func (opts *deleteOptions) run(args []string, stdout io.Writer) error {
ctx, cancel := opts.global.commandTimeoutContext()
defer cancel()
return retry.RetryIfNecessary(ctx, func() error {
return retry.IfNecessary(ctx, func() error {
return ref.DeleteImage(ctx, sys)
}, opts.retryOpts)
}

View File

@@ -26,7 +26,7 @@ import (
type inspectOptions struct {
global *globalOptions
image *imageOptions
retryOpts *retry.RetryOptions
retryOpts *retry.Options
format string
raw bool // Output the raw manifest instead of parsing information about the image
config bool // Output the raw config blob instead of parsing information about the image
@@ -96,7 +96,7 @@ func (opts *inspectOptions) run(args []string, stdout io.Writer) (retErr error)
return err
}
if err := retry.RetryIfNecessary(ctx, func() error {
if err := retry.IfNecessary(ctx, func() error {
src, err = parseImageSource(ctx, opts.image, imageName)
return err
}, opts.retryOpts); err != nil {
@@ -109,7 +109,7 @@ func (opts *inspectOptions) run(args []string, stdout io.Writer) (retErr error)
}
}()
if err := retry.RetryIfNecessary(ctx, func() error {
if err := retry.IfNecessary(ctx, func() error {
rawManifest, _, err = src.GetManifest(ctx, nil)
return err
}, opts.retryOpts); err != nil {
@@ -132,7 +132,7 @@ func (opts *inspectOptions) run(args []string, stdout io.Writer) (retErr error)
if opts.config && opts.raw {
var configBlob []byte
if err := retry.RetryIfNecessary(ctx, func() error {
if err := retry.IfNecessary(ctx, func() error {
configBlob, err = img.ConfigBlob(ctx)
return err
}, opts.retryOpts); err != nil {
@@ -145,7 +145,7 @@ func (opts *inspectOptions) run(args []string, stdout io.Writer) (retErr error)
return nil
} else if opts.config {
var config *v1.Image
if err := retry.RetryIfNecessary(ctx, func() error {
if err := retry.IfNecessary(ctx, func() error {
config, err = img.OCIConfig(ctx)
return err
}, opts.retryOpts); err != nil {
@@ -168,7 +168,7 @@ func (opts *inspectOptions) run(args []string, stdout io.Writer) (retErr error)
return nil
}
if err := retry.RetryIfNecessary(ctx, func() error {
if err := retry.IfNecessary(ctx, func() error {
imgInspect, err = img.Inspect(ctx)
return err
}, opts.retryOpts); err != nil {

View File

@@ -19,7 +19,7 @@ import (
type layersOptions struct {
global *globalOptions
image *imageOptions
retryOpts *retry.RetryOptions
retryOpts *retry.Options
}
func layersCmd(global *globalOptions) *cobra.Command {
@@ -68,13 +68,13 @@ func (opts *layersOptions) run(args []string, stdout io.Writer) (retErr error) {
rawSource types.ImageSource
src types.ImageCloser
)
if err = retry.RetryIfNecessary(ctx, func() error {
if err = retry.IfNecessary(ctx, func() error {
rawSource, err = parseImageSource(ctx, opts.image, imageName)
return err
}, opts.retryOpts); err != nil {
return err
}
if err = retry.RetryIfNecessary(ctx, func() error {
if err = retry.IfNecessary(ctx, func() error {
src, err = image.FromSource(ctx, sys, rawSource)
return err
}, opts.retryOpts); err != nil {
@@ -145,7 +145,7 @@ func (opts *layersOptions) run(args []string, stdout io.Writer) (retErr error) {
r io.ReadCloser
blobSize int64
)
if err = retry.RetryIfNecessary(ctx, func() error {
if err = retry.IfNecessary(ctx, func() error {
r, blobSize, err = rawSource.GetBlob(ctx, types.BlobInfo{Digest: bd.digest, Size: -1}, cache)
return err
}, opts.retryOpts); err != nil {
@@ -160,7 +160,7 @@ func (opts *layersOptions) run(args []string, stdout io.Writer) (retErr error) {
}
var manifest []byte
if err = retry.RetryIfNecessary(ctx, func() error {
if err = retry.IfNecessary(ctx, func() error {
manifest, _, err = src.Manifest(ctx)
return err
}, opts.retryOpts); err != nil {

View File

@@ -27,7 +27,7 @@ type tagListOutput struct {
type tagsOptions struct {
global *globalOptions
image *imageOptions
retryOpts *retry.RetryOptions
retryOpts *retry.Options
}
var transportHandlers = map[string]func(ctx context.Context, sys *types.SystemContext, opts *tagsOptions, userInput string) (repositoryName string, tagListing []string, err error){
@@ -120,7 +120,7 @@ func listDockerRepoTags(ctx context.Context, sys *types.SystemContext, opts *tag
if err != nil {
return
}
if err = retry.RetryIfNecessary(ctx, func() error {
if err = retry.IfNecessary(ctx, func() error {
repositoryName, tagListing, err = listDockerTags(ctx, sys, imgRef)
return err
}, opts.retryOpts); err != nil {

View File

@@ -33,7 +33,7 @@ type syncOptions struct {
deprecatedTLSVerify *deprecatedTLSVerifyOption
srcImage *imageOptions // Source image options
destImage *imageDestOptions // Destination image options
retryOpts *retry.RetryOptions
retryOpts *retry.Options
removeSignatures bool // Do not copy signatures from the source image
signByFingerprint string // Sign the image using a GPG key with the specified fingerprint
signBySigstorePrivateKey string // Sign the image using a sigstore private key
@@ -571,7 +571,7 @@ func (opts *syncOptions) run(args []string, stdout io.Writer) (retErr error) {
sourceArg := args[0]
var srcRepoList []repoDescriptor
if err = retry.RetryIfNecessary(ctx, func() error {
if err = retry.IfNecessary(ctx, func() error {
srcRepoList, err = imagesToCopy(sourceArg, opts.source, sourceCtx)
return err
}, opts.retryOpts); err != nil {
@@ -657,7 +657,7 @@ func (opts *syncOptions) run(args []string, stdout io.Writer) (retErr error) {
logrus.WithFields(fromToFields).Infof("Would have copied image ref %d/%d", counter+1, len(srcRepo.ImageRefs))
} else {
logrus.WithFields(fromToFields).Infof("Copying image ref %d/%d", counter+1, len(srcRepo.ImageRefs))
if err = retry.RetryIfNecessary(ctx, func() error {
if err = retry.IfNecessary(ctx, func() error {
_, err = copy.Image(ctx, policyContext, destRef, ref, &options)
return err
}, opts.retryOpts); err != nil {

View File

@@ -174,8 +174,8 @@ func imageFlags(global *globalOptions, shared *sharedImageOptions, deprecatedTLS
return fs, opts
}
func retryFlags() (pflag.FlagSet, *retry.RetryOptions) {
opts := retry.RetryOptions{}
func retryFlags() (pflag.FlagSet, *retry.Options) {
opts := retry.Options{}
fs := pflag.FlagSet{}
fs.IntVar(&opts.MaxRetry, "retry-times", 0, "the number of times to possibly retry")
return fs, &opts

25
go.mod
View File

@@ -3,15 +3,15 @@ module github.com/containers/skopeo
go 1.17
require (
github.com/containers/common v0.48.0
github.com/containers/image/v5 v5.21.2-0.20220712113758-29aec5f7bbbf
github.com/containers/common v0.49.1
github.com/containers/image/v5 v5.22.1
github.com/containers/ocicrypt v1.1.5
github.com/containers/storage v1.41.0
github.com/containers/storage v1.42.0
github.com/docker/docker v20.10.17+incompatible
github.com/opencontainers/go-digest v1.0.0
github.com/opencontainers/image-spec v1.0.3-0.20220114050600-8b9d41f48198
github.com/opencontainers/image-tools v1.0.0-rc3
github.com/sirupsen/logrus v1.8.1
github.com/sirupsen/logrus v1.9.0
github.com/spf13/cobra v1.5.0
github.com/spf13/pflag v1.0.5
github.com/stretchr/testify v1.8.0
@@ -22,15 +22,15 @@ require (
)
require (
github.com/BurntSushi/toml v1.1.0 // indirect
github.com/BurntSushi/toml v1.2.0 // indirect
github.com/Microsoft/go-winio v0.5.2 // indirect
github.com/Microsoft/hcsshim v0.9.2 // indirect
github.com/Microsoft/hcsshim v0.9.3 // indirect
github.com/VividCortex/ewma v1.2.0 // indirect
github.com/acarl005/stripansi v0.0.0-20180116102854-5a71ef0e047d // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/cespare/xxhash/v2 v2.1.2 // indirect
github.com/containerd/cgroups v1.0.3 // indirect
github.com/containerd/stargz-snapshotter/estargz v0.11.4 // indirect
github.com/containerd/stargz-snapshotter/estargz v0.12.0 // indirect
github.com/containers/libtrust v0.0.0-20200511145503-9c3a6c22cd9a // indirect
github.com/cyphar/filepath-securejoin v0.2.3 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
@@ -53,7 +53,7 @@ require (
github.com/imdario/mergo v0.3.13 // indirect
github.com/inconshreveable/mousetrap v1.0.0 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/klauspost/compress v1.15.7 // indirect
github.com/klauspost/compress v1.15.9 // indirect
github.com/klauspost/pgzip v1.2.5 // indirect
github.com/kr/pretty v0.2.1 // indirect
github.com/kr/text v0.2.0 // indirect
@@ -63,10 +63,10 @@ require (
github.com/matttproud/golang_protobuf_extensions v1.0.2-0.20181231171920-c182affec369 // indirect
github.com/miekg/pkcs11 v1.1.1 // indirect
github.com/mistifyio/go-zfs v2.1.2-0.20190413222219-f784269be439+incompatible // indirect
github.com/moby/sys/mountinfo v0.6.1 // indirect
github.com/moby/sys/mountinfo v0.6.2 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/opencontainers/runc v1.1.2 // indirect
github.com/opencontainers/runc v1.1.3 // indirect
github.com/opencontainers/runtime-spec v1.0.3-0.20210326190908-1c3f411f0417 // indirect
github.com/opencontainers/selinux v1.10.1 // indirect
github.com/ostreedev/ostree-go v0.0.0-20210805093236-719684c64e4f // indirect
@@ -83,7 +83,7 @@ require (
github.com/stefanberger/go-pkcs11uri v0.0.0-20201008174630-78d3cae3a980 // indirect
github.com/sylabs/sif/v2 v2.7.1 // indirect
github.com/tchap/go-patricia v2.3.0+incompatible // indirect
github.com/theupdateframework/go-tuf v0.3.0 // indirect
github.com/theupdateframework/go-tuf v0.3.1 // indirect
github.com/titanous/rocacheck v0.0.0-20171023193734-afe73141d399 // indirect
github.com/ulikunitz/xz v0.5.10 // indirect
github.com/vbatts/tar-split v0.11.2 // indirect
@@ -94,10 +94,11 @@ require (
go.etcd.io/bbolt v1.3.6 // indirect
go.mozilla.org/pkcs7 v0.0.0-20200128120323-432b2356ecb1 // indirect
go.opencensus.io v0.23.0 // indirect
go.opentelemetry.io/otel/trace v1.3.0 // indirect
golang.org/x/crypto v0.0.0-20220131195533-30dcbda58838 // indirect
golang.org/x/net v0.0.0-20220624214902-1bab6f366d9e // indirect
golang.org/x/sync v0.0.0-20220601150217-0de741cfad7f // indirect
golang.org/x/sys v0.0.0-20220624220833-87e55d714810 // indirect
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8 // indirect
golang.org/x/text v0.3.7 // indirect
google.golang.org/genproto v0.0.0-20220624142145-8cd45d7dbd1f // indirect
google.golang.org/grpc v1.47.0 // indirect

74
go.sum
View File

@@ -101,8 +101,8 @@ github.com/Azure/go-autorest/logger v0.2.1/go.mod h1:T9E3cAhj2VqvPOtCYAvby9aBXkZ
github.com/Azure/go-autorest/tracing v0.6.0/go.mod h1:+vhtPC754Xsa23ID7GlGsrdKBpUA79WCAKPPZVC2DeU=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/BurntSushi/toml v0.4.1/go.mod h1:CxXYINrC8qIiEnFrOxCa7Jy5BFHlXnUU2pbicEuybxQ=
github.com/BurntSushi/toml v1.1.0 h1:ksErzDEI1khOiGPgpwuI7x2ebx/uXQNw7xJpn9Eq1+I=
github.com/BurntSushi/toml v1.1.0/go.mod h1:CxXYINrC8qIiEnFrOxCa7Jy5BFHlXnUU2pbicEuybxQ=
github.com/BurntSushi/toml v1.2.0 h1:Rt8g24XnyGTyglgET/PRUNlrUeu9F5L+7FilkXfZgs0=
github.com/BurntSushi/toml v1.2.0/go.mod h1:CxXYINrC8qIiEnFrOxCa7Jy5BFHlXnUU2pbicEuybxQ=
github.com/BurntSushi/xgb v0.0.0-20160522181843-27f122750802/go.mod h1:IVnqGOEym/WlBOVXweHU+Q+/VP0lqqI8lqeDx9IjBqo=
github.com/DATA-DOG/go-sqlmock v1.5.0/go.mod h1:f/Ixk793poVmq4qj/V1dPUg2JEAKC73Q5eFN3EC/SaM=
github.com/DataDog/datadog-go v3.2.0+incompatible/go.mod h1:LButxg5PwREeZtORoXG3tL4fMGNddJ+vMq1mwgfaqoQ=
@@ -135,15 +135,15 @@ github.com/Microsoft/hcsshim v0.8.16/go.mod h1:o5/SZqmR7x9JNKsW3pu+nqHm0MF8vbA+V
github.com/Microsoft/hcsshim v0.8.20/go.mod h1:+w2gRZ5ReXQhFOrvSQeNfhrYB/dg3oDwTOcER2fw4I4=
github.com/Microsoft/hcsshim v0.8.21/go.mod h1:+w2gRZ5ReXQhFOrvSQeNfhrYB/dg3oDwTOcER2fw4I4=
github.com/Microsoft/hcsshim v0.8.23/go.mod h1:4zegtUJth7lAvFyc6cH2gGQ5B3OFQim01nnU2M8jKDg=
github.com/Microsoft/hcsshim v0.9.2 h1:wB06W5aYFfUB3IvootYAY2WnOmIdgPGfqSI6tufQNnY=
github.com/Microsoft/hcsshim v0.9.2/go.mod h1:7pLA8lDk46WKDWlVsENo92gC0XFa8rbKfyFRBqxEbCc=
github.com/Microsoft/hcsshim v0.9.3 h1:k371PzBuRrz2b+ebGuI2nVgVhgsVX60jMfSw80NECxo=
github.com/Microsoft/hcsshim v0.9.3/go.mod h1:7pLA8lDk46WKDWlVsENo92gC0XFa8rbKfyFRBqxEbCc=
github.com/Microsoft/hcsshim/test v0.0.0-20201218223536-d3e5debf77da/go.mod h1:5hlzMzRKMLyo42nCZ9oml8AdTlq/0cvIaBv6tK1RehU=
github.com/Microsoft/hcsshim/test v0.0.0-20210227013316-43a75bb4edd3/go.mod h1:mw7qgWloBUl75W/gVH3cQszUg1+gUITj7D6NY7ywVnY=
github.com/NYTimes/gziphandler v0.0.0-20170623195520-56545f4a5d46/go.mod h1:3wb06e3pkSAbeQ52E9H9iFoQsEEwGN64994WTCIhntQ=
github.com/NYTimes/gziphandler v1.1.1/go.mod h1:n/CVRwUEOgIxrgPvAQhUUr9oeUtvrhMomdKFjzJNB0c=
github.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU=
github.com/OpenPeeDeeP/depguard v1.0.1/go.mod h1:xsIw86fROiiwelg+jB2uM9PiKihMMmUx/1V+TNhjQvM=
github.com/ProtonMail/go-crypto v0.0.0-20220407094043-a94812496cf5/go.mod h1:z4/9nQmJSSwwds7ejkxaJwO37dru3geImFUdJlaLzQo=
github.com/ProtonMail/go-crypto v0.0.0-20220517143526-88bb52951d5b/go.mod h1:z4/9nQmJSSwwds7ejkxaJwO37dru3geImFUdJlaLzQo=
github.com/PuerkitoBio/purell v1.0.0/go.mod h1:c11w/QuzBsJSee3cPx9rAFu61PvFxuPbtSwDGJws/X0=
github.com/PuerkitoBio/purell v1.1.1/go.mod h1:c11w/QuzBsJSee3cPx9rAFu61PvFxuPbtSwDGJws/X0=
@@ -210,6 +210,7 @@ github.com/bketelsen/crypt v0.0.3-0.20200106085610-5cbc8cc4026c/go.mod h1:MKsuJm
github.com/bketelsen/crypt v0.0.4/go.mod h1:aI6NrJ0pMGgvZKL1iVgXLnfIFJtfV+bKCoqOes/6LfM=
github.com/bkielbasa/cyclop v1.2.0/go.mod h1:qOI0yy6A7dYC4Zgsa72Ppm9kONl0RoIlPbzot9mhmeI=
github.com/blang/semver v3.1.0+incompatible/go.mod h1:kRBLl5iJ+tD4TcOOxsy/0fnwebNt5EWlYSAyrTnjyyk=
github.com/blang/semver v3.5.0+incompatible/go.mod h1:kRBLl5iJ+tD4TcOOxsy/0fnwebNt5EWlYSAyrTnjyyk=
github.com/blang/semver v3.5.1+incompatible/go.mod h1:kRBLl5iJ+tD4TcOOxsy/0fnwebNt5EWlYSAyrTnjyyk=
github.com/blizzy78/varnamelen v0.3.0/go.mod h1:hbwRdBvoBqxk34XyQ6HA0UH3G0/1TKuv5AC4eaBT0Ec=
github.com/bmizerany/assert v0.0.0-20160611221934-b7ed37b82869/go.mod h1:Ekp36dRnpXw/yCqJaO+ZrUyxD+3VXMFFr56k5XYrpB4=
@@ -305,7 +306,7 @@ github.com/containerd/containerd v1.5.1/go.mod h1:0DOxVqwDy2iZvrZp2JUx/E+hS0UNTV
github.com/containerd/containerd v1.5.7/go.mod h1:gyvv6+ugqY25TiXxcZC3L5yOeYgEw0QMhscqVp1AR9c=
github.com/containerd/containerd v1.5.8/go.mod h1:YdFSv5bTFLpG2HIYmfqDpSYYTDX+mc5qtSuYx1YUb/s=
github.com/containerd/containerd v1.6.1/go.mod h1:1nJz5xCZPusx6jJU8Frfct988y0NpumIq9ODB0kLtoE=
github.com/containerd/containerd v1.6.3/go.mod h1:gCVGrYRYFm2E8GmuUIbj/NGD7DLZQLzSJQazjVKDOig=
github.com/containerd/containerd v1.6.6/go.mod h1:ZoP1geJldzCVY3Tonoz7b1IXk8rIX0Nltt5QE4OMNk0=
github.com/containerd/continuity v0.0.0-20190426062206-aaeac12a7ffc/go.mod h1:GL3xCUCBDV3CZiTSEKksMWbLE66hEyuu9qyDOOqM47Y=
github.com/containerd/continuity v0.0.0-20190815185530-f2a389ac0a02/go.mod h1:GL3xCUCBDV3CZiTSEKksMWbLE66hEyuu9qyDOOqM47Y=
github.com/containerd/continuity v0.0.0-20191127005431-f65d91d395eb/go.mod h1:GL3xCUCBDV3CZiTSEKksMWbLE66hEyuu9qyDOOqM47Y=
@@ -324,7 +325,7 @@ github.com/containerd/go-cni v1.0.1/go.mod h1:+vUpYxKvAF72G9i1WoDOiPGRtQpqsNW/ZH
github.com/containerd/go-cni v1.0.2/go.mod h1:nrNABBHzu0ZwCug9Ije8hL2xBCYh/pjfMb1aZGrrohk=
github.com/containerd/go-cni v1.1.0/go.mod h1:Rflh2EJ/++BA2/vY5ao3K6WJRR/bZKsX123aPk+kUtA=
github.com/containerd/go-cni v1.1.3/go.mod h1:Rflh2EJ/++BA2/vY5ao3K6WJRR/bZKsX123aPk+kUtA=
github.com/containerd/go-cni v1.1.4/go.mod h1:Rflh2EJ/++BA2/vY5ao3K6WJRR/bZKsX123aPk+kUtA=
github.com/containerd/go-cni v1.1.6/go.mod h1:BWtoWl5ghVymxu6MBjg79W9NZrCRyHIdUtk4cauMe34=
github.com/containerd/go-runc v0.0.0-20180907222934-5a6d9f37cfa3/go.mod h1:IV7qH3hrUgRmyYrtgEeGWJfWbgcHL9CSRruz2Vqcph0=
github.com/containerd/go-runc v0.0.0-20190911050354-e029b79d8cda/go.mod h1:IV7qH3hrUgRmyYrtgEeGWJfWbgcHL9CSRruz2Vqcph0=
github.com/containerd/go-runc v0.0.0-20200220073739-7016d3ce2328/go.mod h1:PpyHrqVs8FTi9vpyHwPwiNEGaACDxT/N/pLcvMSRA9g=
@@ -340,8 +341,9 @@ github.com/containerd/nri v0.0.0-20201007170849-eb1350a75164/go.mod h1:+2wGSDGFY
github.com/containerd/nri v0.0.0-20210316161719-dbaa18c31c14/go.mod h1:lmxnXF6oMkbqs39FiCt1s0R2HSMhcLel9vNL3m4AaeY=
github.com/containerd/nri v0.1.0/go.mod h1:lmxnXF6oMkbqs39FiCt1s0R2HSMhcLel9vNL3m4AaeY=
github.com/containerd/stargz-snapshotter/estargz v0.4.1/go.mod h1:x7Q9dg9QYb4+ELgxmo4gBUeJB0tl5dqH1Sdz0nJU1QM=
github.com/containerd/stargz-snapshotter/estargz v0.11.4 h1:LjrYUZpyOhiSaU7hHrdR82/RBoxfGWSaC0VeSSMXqnk=
github.com/containerd/stargz-snapshotter/estargz v0.11.4/go.mod h1:7vRJIcImfY8bpifnMjt+HTJoQxASq7T28MYbP15/Nf0=
github.com/containerd/stargz-snapshotter/estargz v0.12.0 h1:idtwRTLjk2erqiYhPWy2L844By8NRFYEwYHcXhoIWPM=
github.com/containerd/stargz-snapshotter/estargz v0.12.0/go.mod h1:AIQ59TewBFJ4GOPEQXujcrJ/EKxh5xXZegW1rkR1P/M=
github.com/containerd/ttrpc v0.0.0-20190828154514-0e0f228740de/go.mod h1:PvCDdDGpgqzQIzDW1TphrGLssLDZp2GuS+X5DkEJB8o=
github.com/containerd/ttrpc v0.0.0-20190828172938-92c8520ef9f8/go.mod h1:PvCDdDGpgqzQIzDW1TphrGLssLDZp2GuS+X5DkEJB8o=
github.com/containerd/ttrpc v0.0.0-20191028202541-4f1b8fe65a5c/go.mod h1:LPm1u0xBw8r8NOKoOdNMeVHSawSsltak+Ihv+etqsE8=
@@ -361,16 +363,17 @@ github.com/containernetworking/cni v0.7.1/go.mod h1:LGwApLUm2FpoOfxTDEeq8T9ipbpZ
github.com/containernetworking/cni v0.8.0/go.mod h1:LGwApLUm2FpoOfxTDEeq8T9ipbpZ61X79hmU3w8FmsY=
github.com/containernetworking/cni v0.8.1/go.mod h1:LGwApLUm2FpoOfxTDEeq8T9ipbpZ61X79hmU3w8FmsY=
github.com/containernetworking/cni v1.0.1/go.mod h1:AKuhXbN5EzmD4yTNtfSsX3tPcmtrBI6QcRV0NiNt15Y=
github.com/containernetworking/cni v1.1.0/go.mod h1:sDpYKmGVENF3s6uvMvGgldDWeG8dMxakj/u+i9ht9vw=
github.com/containernetworking/cni v1.1.1/go.mod h1:sDpYKmGVENF3s6uvMvGgldDWeG8dMxakj/u+i9ht9vw=
github.com/containernetworking/cni v1.1.2/go.mod h1:sDpYKmGVENF3s6uvMvGgldDWeG8dMxakj/u+i9ht9vw=
github.com/containernetworking/plugins v0.8.6/go.mod h1:qnw5mN19D8fIwkqW7oHHYDHVlzhJpcY6TQxn/fUyDDM=
github.com/containernetworking/plugins v0.9.1/go.mod h1:xP/idU2ldlzN6m4p5LmGiwRDjeJr6FLK6vuiUwoH7P8=
github.com/containernetworking/plugins v1.0.1/go.mod h1:QHCfGpaTwYTbbH+nZXKVTxNBDZcxSOplJT5ico8/FLE=
github.com/containernetworking/plugins v1.1.1/go.mod h1:Sr5TH/eBsGLXK/h71HeLfX19sZPp3ry5uHSkI4LPxV8=
github.com/containers/common v0.48.0 h1:997nnXBZ+eNpfSM7L4SxhhZubQrfEyw3jRyNMTSsNlw=
github.com/containers/common v0.48.0/go.mod h1:zPLZCfLXfnd1jI0QRsD4By54fP4k1+ifQs+tulIe3o0=
github.com/containers/image/v5 v5.21.1/go.mod h1:zl35egpcDQa79IEXIuoUe1bW+D1pdxRxYjNlyb3YiXw=
github.com/containers/image/v5 v5.21.2-0.20220712113758-29aec5f7bbbf h1:EAVt3Tr8n2Iv3L2L4J66vcFB+jHGCSHQlD15+Z2U1Io=
github.com/containers/image/v5 v5.21.2-0.20220712113758-29aec5f7bbbf/go.mod h1:0+N0ZM9mgMmoZZc6uNcgnEsbX85Ne7b29cIW5lqWwVU=
github.com/containers/common v0.49.1 h1:6y4/s2WwYxrv+Cox7fotOo316wuZI+iKKPUQweCYv50=
github.com/containers/common v0.49.1/go.mod h1:ueM5hT0itKqCQvVJDs+EtjornAQtrHYxQJzP2gxeGIg=
github.com/containers/image/v5 v5.22.0/go.mod h1:D8Ksv2RNB8qLJ7xe1P3rgJJOSQpahA6amv2Ax++/YO4=
github.com/containers/image/v5 v5.22.1 h1:Oj8C/ACrNn18f8MNTqOeBOHAN51GF1rEhoVSCA/ylvI=
github.com/containers/image/v5 v5.22.1/go.mod h1:D8Ksv2RNB8qLJ7xe1P3rgJJOSQpahA6amv2Ax++/YO4=
github.com/containers/libtrust v0.0.0-20200511145503-9c3a6c22cd9a h1:spAGlqziZjCJL25C6F1zsQY05tfCKE9F5YwtEWWe6hU=
github.com/containers/libtrust v0.0.0-20200511145503-9c3a6c22cd9a/go.mod h1:9rfv8iPl1ZP7aqh9YA68wnZv2NUDbXdcdPHVz0pFbPY=
github.com/containers/ocicrypt v1.0.1/go.mod h1:MeJDzk1RJHv89LjsH0Sp5KTY3ZYkjXO/C+bKAeWFIrc=
@@ -378,12 +381,10 @@ github.com/containers/ocicrypt v1.1.0/go.mod h1:b8AOe0YR67uU8OqfVNcznfFpAzu3rdgU
github.com/containers/ocicrypt v1.1.1/go.mod h1:Dm55fwWm1YZAjYRaJ94z2mfZikIyIN4B0oB3dj3jFxY=
github.com/containers/ocicrypt v1.1.2/go.mod h1:Dm55fwWm1YZAjYRaJ94z2mfZikIyIN4B0oB3dj3jFxY=
github.com/containers/ocicrypt v1.1.3/go.mod h1:xpdkbVAuaH3WzbEabUd5yDsl9SwJA5pABH85425Es2g=
github.com/containers/ocicrypt v1.1.4-0.20220428134531-566b808bdf6f/go.mod h1:xpdkbVAuaH3WzbEabUd5yDsl9SwJA5pABH85425Es2g=
github.com/containers/ocicrypt v1.1.5 h1:UO+gBnBXvMvC7HTXLh0bPgLslfW8HlY+oxYcoSHBcZQ=
github.com/containers/ocicrypt v1.1.5/go.mod h1:WgjxPWdTJMqYMjf3M6cuIFFA1/MpyyhIM99YInA+Rvc=
github.com/containers/storage v1.40.0/go.mod h1:zUyPC3CFIGR1OhY1CKkffxgw9+LuH76PGvVcFj38dgs=
github.com/containers/storage v1.41.0 h1:IsoAJ1q3s/jfHB7eoiRhvTRTcuV+ywY3CkbbgKtT5Bo=
github.com/containers/storage v1.41.0/go.mod h1:Pb0l5Sm/89kolX3o2KolKQ5cCHk5vPNpJrhNaLcdS5s=
github.com/containers/storage v1.42.0 h1:zm2AQD4NDeTB3JQ8X+Wo5+VRqNB+b4ocEd7Qj6ylPJA=
github.com/containers/storage v1.42.0/go.mod h1:JiUJwOgOo1dr2DdOUc1MRe2GCAXABYoYmOdPF8yvH78=
github.com/coreos/bbolt v1.3.2/go.mod h1:iRUV2dpdMOn7Bo10OQBFzIJO9kkE559Wcmn+qkEiiKk=
github.com/coreos/etcd v3.3.10+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE=
github.com/coreos/etcd v3.3.13+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE=
@@ -443,7 +444,6 @@ github.com/docker/distribution v2.7.1+incompatible/go.mod h1:J2gT2udsDAN96Uj4Kfc
github.com/docker/distribution v2.8.1+incompatible h1:Q50tZOPR6T/hjNsyc9g8/syEs6bk8XXApsHjKukMl68=
github.com/docker/distribution v2.8.1+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w=
github.com/docker/docker v1.4.2-0.20190924003213-a8608b5b67c7/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/docker v20.10.14+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/docker v20.10.16+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/docker v20.10.17+incompatible h1:JYCuMrWaVNophQTOrMMoSwudOVEfcegoZZrleKc1xwE=
github.com/docker/docker v20.10.17+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
@@ -514,6 +514,7 @@ github.com/frankban/quicktest v1.13.0/go.mod h1:qLE0fzW0VuyUAJgPU19zByoIr0HtCHN/
github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ=
github.com/fsnotify/fsnotify v1.5.1/go.mod h1:T3375wBYaZdLLcVNkcVbzGHY7f1l/uK5T5Ai1i3InKU=
github.com/fsnotify/fsnotify v1.5.4/go.mod h1:OVB6XrOHzAwXMpEM7uPOzcehqUV2UqJxmVXmkdnm1bU=
github.com/fullsailor/pkcs7 v0.0.0-20190404230743-d7302db945fa/go.mod h1:KnogPXtdwXqoenmZCw6S+25EAm2MkxbG0deNDu4cbSA=
github.com/fullstorydev/grpcurl v1.6.0/go.mod h1:ZQ+ayqbKMJNhzLmbpCiurTVlaK2M/3nqZCxaQ2Ze/sM=
github.com/fzipp/gocyclo v0.3.1/go.mod h1:DJHO6AUmbdqj2ET4Z9iArSuwWgYDRryYt2wASxc7x3E=
@@ -761,7 +762,6 @@ github.com/gorhill/cronexpr v0.0.0-20180427100037-88b0669f7d75/go.mod h1:g2644b0
github.com/gorilla/handlers v0.0.0-20150720190736-60c7bfde3e33/go.mod h1:Qkdc/uu4tH4g6mTK6auzZ766c4CA0Ng8+o/OAirnOIQ=
github.com/gorilla/mux v1.7.2/go.mod h1:1lud6UwP+6orDFRuTfBEV8e9/aOM/c4fVVCaMa2zaAs=
github.com/gorilla/mux v1.7.3/go.mod h1:1lud6UwP+6orDFRuTfBEV8e9/aOM/c4fVVCaMa2zaAs=
github.com/gorilla/mux v1.7.4/go.mod h1:DVbg23sWSpFRCP0SfiEN6jmj59UnW/n46BH5rLB71So=
github.com/gorilla/mux v1.8.0 h1:i40aqfkR1h2SlN9hojwV5ZA91wcXFOvkdNIeFDP5koI=
github.com/gorilla/mux v1.8.0/go.mod h1:DVbg23sWSpFRCP0SfiEN6jmj59UnW/n46BH5rLB71So=
github.com/gorilla/websocket v0.0.0-20170926233335-4201258b820c/go.mod h1:E7qHFY5m1UJ88s3WnNqhKjPHQ0heANvMoAMk2YaljkQ=
@@ -963,10 +963,10 @@ github.com/klauspost/compress v1.11.13/go.mod h1:aoV0uJVorq1K+umq18yTdKaF57EivdY
github.com/klauspost/compress v1.13.4/go.mod h1:8dP1Hq4DHOhN9w426knH3Rhby4rFm6D8eO+e+Dq5Gzg=
github.com/klauspost/compress v1.13.5/go.mod h1:/3/Vjq9QcHkK5uEr5lBEmyoZ1iFhe47etQ6QUkpK6sk=
github.com/klauspost/compress v1.15.1/go.mod h1:/3/Vjq9QcHkK5uEr5lBEmyoZ1iFhe47etQ6QUkpK6sk=
github.com/klauspost/compress v1.15.2/go.mod h1:PhcZ0MbTNciWF3rruxRgKxI5NkcHHrHUDtV4Yw2GlzU=
github.com/klauspost/compress v1.15.4/go.mod h1:PhcZ0MbTNciWF3rruxRgKxI5NkcHHrHUDtV4Yw2GlzU=
github.com/klauspost/compress v1.15.7 h1:7cgTQxJCU/vy+oP/E3B9RGbQTgbiVzIJWIKOLoAsPok=
github.com/klauspost/compress v1.15.7/go.mod h1:PhcZ0MbTNciWF3rruxRgKxI5NkcHHrHUDtV4Yw2GlzU=
github.com/klauspost/compress v1.15.9 h1:wKRjX6JRtDdrE9qwa4b/Cip7ACOshUI4smpCQanqjSY=
github.com/klauspost/compress v1.15.9/go.mod h1:PhcZ0MbTNciWF3rruxRgKxI5NkcHHrHUDtV4Yw2GlzU=
github.com/klauspost/cpuid v1.2.0/go.mod h1:Pj4uuM528wm8OyEC2QMXAi2YiTZ96dNQPGgoMS4s3ek=
github.com/klauspost/pgzip v1.2.5 h1:qnWYvvKqedOF2ulHpMG72XQol4ILEJ8k2wwRl/Km8oE=
github.com/klauspost/pgzip v1.2.5/go.mod h1:Ch1tH69qFZu15pkjo5kYi6mth2Zzwzt50oCQKQE9RUs=
@@ -1100,13 +1100,14 @@ github.com/mitchellh/osext v0.0.0-20151018003038-5e2d6d41470f/go.mod h1:OkQIRizQ
github.com/mitchellh/reflectwalk v1.0.0/go.mod h1:mSTlrgnPZtwu0c4WaC2kGObEpuNDbx0jmZXqmk4esnw=
github.com/mitchellh/reflectwalk v1.0.1/go.mod h1:mSTlrgnPZtwu0c4WaC2kGObEpuNDbx0jmZXqmk4esnw=
github.com/mitchellh/reflectwalk v1.0.2/go.mod h1:mSTlrgnPZtwu0c4WaC2kGObEpuNDbx0jmZXqmk4esnw=
github.com/mndrix/tap-go v0.0.0-20171203230836-629fa407e90b/go.mod h1:pzzDgJWZ34fGzaAZGFW22KVZDfyrYW+QABMrWnJBnSs=
github.com/moby/locker v1.0.1/go.mod h1:S7SDdo5zpBK84bzzVlKr2V0hz+7x9hWbYC/kq7oQppc=
github.com/moby/spdystream v0.2.0/go.mod h1:f7i0iNDQJ059oMTcWxx8MA/zKFIuD/lY+0GqbN2Wy8c=
github.com/moby/sys/mountinfo v0.4.0/go.mod h1:rEr8tzG/lsIZHBtN/JjGG+LMYx9eXgW2JI+6q0qou+A=
github.com/moby/sys/mountinfo v0.4.1/go.mod h1:rEr8tzG/lsIZHBtN/JjGG+LMYx9eXgW2JI+6q0qou+A=
github.com/moby/sys/mountinfo v0.5.0/go.mod h1:3bMD3Rg+zkqx8MRYPi7Pyb0Ie97QEBmdxbhnCLlSvSU=
github.com/moby/sys/mountinfo v0.6.1 h1:+H/KnGEAGRpTrEAqNVQ2AM3SiwMgJUt/TXj+Z8cmCIc=
github.com/moby/sys/mountinfo v0.6.1/go.mod h1:3bMD3Rg+zkqx8MRYPi7Pyb0Ie97QEBmdxbhnCLlSvSU=
github.com/moby/sys/mountinfo v0.6.2 h1:BzJjoreD5BMFNmD9Rus6gdd1pLuecOFPt8wC+Vygl78=
github.com/moby/sys/mountinfo v0.6.2/go.mod h1:IJb6JQeOklcdMU9F5xQ8ZALD+CUr5VlGpwtX+VE0rpI=
github.com/moby/sys/signal v0.6.0/go.mod h1:GQ6ObYZfqacOwTtlXvcmh9A26dVRul/hbOZn88Kg8Tg=
github.com/moby/sys/symlink v0.1.0/go.mod h1:GGDODQmbFOjFsXvfLVn3+ZRxkch54RkSiGqsZeMYowQ=
github.com/moby/sys/symlink v0.2.0/go.mod h1:7uZVF2dqJjG/NsClqul95CqKOBRQyYSNnJ6BMgR/gFs=
@@ -1183,8 +1184,9 @@ github.com/onsi/gomega v1.15.0/go.mod h1:cIuvLEne0aoVhAgh/O6ac0Op8WWw9H6eYCriF+t
github.com/onsi/gomega v1.16.0/go.mod h1:HnhC7FXeEQY45zxNK3PPoIUhzk/80Xly9PcubAlGdZY=
github.com/onsi/gomega v1.17.0/go.mod h1:HnhC7FXeEQY45zxNK3PPoIUhzk/80Xly9PcubAlGdZY=
github.com/onsi/gomega v1.18.1/go.mod h1:0q+aL8jAiMXy9hbwj2mr5GziHiwhAIQpFmmtT5hitRs=
github.com/onsi/gomega v1.19.0 h1:4ieX6qQjPP/BfC3mpsAtIGGlxTWPeA3Inl/7DtXw1tw=
github.com/onsi/gomega v1.19.0/go.mod h1:LY+I3pBVzYsTBU1AnDwOSxaYi9WoWiqgwooUqq9yPro=
github.com/onsi/gomega v1.20.0 h1:8W0cWlwFkflGPLltQvLRB7ZVD5HuP6ng320w2IS245Q=
github.com/onsi/gomega v1.20.0/go.mod h1:DtrZpjmvpn2mPm4YWQa0/ALMDj9v4YxLgojwPeREyVo=
github.com/op/go-logging v0.0.0-20160315200505-970db520ece7/go.mod h1:HzydrMdWErDVzsI23lYNej1Htcns9BCg93Dk0bBINWk=
github.com/opencontainers/go-digest v0.0.0-20170106003457-a6d0ee40d420/go.mod h1:cMLVZDEM3+U2I4VmLI6N8jQYUd2OVphdqWwCJHrFt2s=
github.com/opencontainers/go-digest v0.0.0-20180430190053-c9281466c8b2/go.mod h1:cMLVZDEM3+U2I4VmLI6N8jQYUd2OVphdqWwCJHrFt2s=
@@ -1197,7 +1199,6 @@ github.com/opencontainers/image-spec v1.0.1/go.mod h1:BtxoFyWECRxE4U/7sNtV5W15zM
github.com/opencontainers/image-spec v1.0.2-0.20211117181255-693428a734f5/go.mod h1:BtxoFyWECRxE4U/7sNtV5W15zMzWCbyJoFRP3s7yZA0=
github.com/opencontainers/image-spec v1.0.2/go.mod h1:BtxoFyWECRxE4U/7sNtV5W15zMzWCbyJoFRP3s7yZA0=
github.com/opencontainers/image-spec v1.0.3-0.20211202183452-c5a74bcca799/go.mod h1:BtxoFyWECRxE4U/7sNtV5W15zMzWCbyJoFRP3s7yZA0=
github.com/opencontainers/image-spec v1.0.3-0.20211202193544-a5463b7f9c84/go.mod h1:Qnt1q4cjDNQI9bT832ziho5Iw2BhK8o1KwLOwW56VP4=
github.com/opencontainers/image-spec v1.0.3-0.20220114050600-8b9d41f48198 h1:+czc/J8SlhPKLOtVLMQc+xDCFBT73ZStMsRhSsUhsSg=
github.com/opencontainers/image-spec v1.0.3-0.20220114050600-8b9d41f48198/go.mod h1:j4h1pJW6ZcJTgMZWP3+7RlG3zTaP02aDZ/Qw0sppK7Q=
github.com/opencontainers/image-tools v1.0.0-rc3 h1:ZR837lBIxq6mmwEqfYrbLMuf75eBSHhccVHy6lsBeM4=
@@ -1209,21 +1210,23 @@ github.com/opencontainers/runc v1.0.0-rc9/go.mod h1:qT5XzbpPznkRYVz/mWwUaVBUv2rm
github.com/opencontainers/runc v1.0.0-rc93/go.mod h1:3NOsor4w32B2tC0Zbl8Knk4Wg84SM2ImC1fxBuqJ/H0=
github.com/opencontainers/runc v1.0.2/go.mod h1:aTaHFFwQXuA71CiyxOdFFIorAoemI04suvGRQFzWTD0=
github.com/opencontainers/runc v1.1.0/go.mod h1:Tj1hFw6eFWp/o33uxGf5yF2BX5yz2Z6iptFpuvbbKqc=
github.com/opencontainers/runc v1.1.1/go.mod h1:Tj1hFw6eFWp/o33uxGf5yF2BX5yz2Z6iptFpuvbbKqc=
github.com/opencontainers/runc v1.1.2 h1:2VSZwLx5k/BfsBxMMipG/LYUnmqOD/BPkIVgQUcTlLw=
github.com/opencontainers/runc v1.1.2/go.mod h1:Tj1hFw6eFWp/o33uxGf5yF2BX5yz2Z6iptFpuvbbKqc=
github.com/opencontainers/runc v1.1.3 h1:vIXrkId+0/J2Ymu2m7VjGvbSlAId9XNRPhn2p4b+d8w=
github.com/opencontainers/runc v1.1.3/go.mod h1:1J5XiS+vdZ3wCyZybsuxXZWGrgSr8fFJHLXuG2PsnNg=
github.com/opencontainers/runtime-spec v0.1.2-0.20190507144316-5b71a03e2700/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/C5hL+LL3ko9hs6T5lQ0=
github.com/opencontainers/runtime-spec v1.0.1/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/C5hL+LL3ko9hs6T5lQ0=
github.com/opencontainers/runtime-spec v1.0.2-0.20190207185410-29686dbc5559/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/C5hL+LL3ko9hs6T5lQ0=
github.com/opencontainers/runtime-spec v1.0.2/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/C5hL+LL3ko9hs6T5lQ0=
github.com/opencontainers/runtime-spec v1.0.3-0.20200929063507-e6143ca7d51d/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/C5hL+LL3ko9hs6T5lQ0=
github.com/opencontainers/runtime-spec v1.0.3-0.20201121164853-7413a7f753e1/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/C5hL+LL3ko9hs6T5lQ0=
github.com/opencontainers/runtime-spec v1.0.3-0.20210326190908-1c3f411f0417 h1:3snG66yBm59tKhhSPQrQ/0bCrv1LQbKt40LnUPiUxdc=
github.com/opencontainers/runtime-spec v1.0.3-0.20210326190908-1c3f411f0417/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/C5hL+LL3ko9hs6T5lQ0=
github.com/opencontainers/runtime-tools v0.0.0-20181011054405-1d69bd0f9c39/go.mod h1:r3f7wjNzSs2extwzU3Y+6pKfobzPh+kKFJ3ofN+3nfs=
github.com/opencontainers/runtime-tools v0.9.0/go.mod h1:r3f7wjNzSs2extwzU3Y+6pKfobzPh+kKFJ3ofN+3nfs=
github.com/opencontainers/runtime-tools v0.9.1-0.20220714195903-17b3287fafb7/go.mod h1:/tgP02fPXGHkU3/qKK1Y0Db4yqNyGm03vLq/mzHzcS4=
github.com/opencontainers/selinux v1.6.0/go.mod h1:VVGKuOLlE7v4PJyT6h7mNWvq1rzqiriPsEqVhc+svHE=
github.com/opencontainers/selinux v1.8.0/go.mod h1:RScLhm78qiWa2gbVCcGkC7tCGdgk3ogry1nUQF8Evvo=
github.com/opencontainers/selinux v1.8.2/go.mod h1:MUIHuUEvKB1wtJjQdOyYRgOnLD2xAPP8dBsCoU0KuF8=
github.com/opencontainers/selinux v1.9.1/go.mod h1:2i0OySw99QjzBBQByd1Gr9gSjvuho1lHsJxIJ3gGbJI=
github.com/opencontainers/selinux v1.10.0/go.mod h1:2i0OySw99QjzBBQByd1Gr9gSjvuho1lHsJxIJ3gGbJI=
github.com/opencontainers/selinux v1.10.1 h1:09LIPVRP3uuZGQvgR+SgMSNBd1Eb3vlRbGqQpoHsF8w=
github.com/opencontainers/selinux v1.10.1/go.mod h1:2i0OySw99QjzBBQByd1Gr9gSjvuho1lHsJxIJ3gGbJI=
@@ -1261,7 +1264,6 @@ github.com/posener/complete v1.1.1/go.mod h1:em0nMJCgc9GFtwrmVmEMR/ZL6WyhyjMBndr
github.com/posener/complete v1.2.3/go.mod h1:WZIdtGGp+qx0sLrYKtIRAruyNpv6hFCicSgv7Sy7s/s=
github.com/poy/onpar v0.0.0-20190519213022-ee068f8ea4d1/go.mod h1:nSbFQvMj97ZyhFRSJYtut+msi4sOY6zJDGCdSc+/rZU=
github.com/pquerna/cachecontrol v0.0.0-20171018203845-0dec1b30a021/go.mod h1:prYjPmNq4d1NPVmpShWobRqXY3q7Vp+80DqgxxUrUIA=
github.com/proglottis/gpgme v0.1.1/go.mod h1:fPbW/EZ0LvwQtH8Hy7eixhp1eF3G39dtx7GUN+0Gmy0=
github.com/proglottis/gpgme v0.1.3 h1:Crxx0oz4LKB3QXc5Ea0J19K/3ICfy3ftr5exgUK1AU0=
github.com/proglottis/gpgme v0.1.3/go.mod h1:fPbW/EZ0LvwQtH8Hy7eixhp1eF3G39dtx7GUN+0Gmy0=
github.com/prometheus/client_golang v0.0.0-20180209125602-c332b6f63c06/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw=
@@ -1351,6 +1353,8 @@ github.com/sebdah/goldie/v2 v2.5.3 h1:9ES/mNN+HNUbNWpVAlrzuZ7jE+Nrczbj8uFRjM7624
github.com/sebdah/goldie/v2 v2.5.3/go.mod h1:oZ9fp0+se1eapSRjfYbsV/0Hqhbuu3bJVvKI/NNtssI=
github.com/seccomp/libseccomp-golang v0.9.1/go.mod h1:GbW5+tmTXfcxTToHLXlScSlAvWlF4P2Ca7zGrPiEpWo=
github.com/seccomp/libseccomp-golang v0.9.2-0.20210429002308-3879420cc921/go.mod h1:JA8cRccbGaA1s33RQf7Y1+q9gHmZX1yB/z9WDN1C6fg=
github.com/seccomp/libseccomp-golang v0.9.2-0.20220502022130-f33da4d89646/go.mod h1:JA8cRccbGaA1s33RQf7Y1+q9gHmZX1yB/z9WDN1C6fg=
github.com/seccomp/libseccomp-golang v0.10.0/go.mod h1:JA8cRccbGaA1s33RQf7Y1+q9gHmZX1yB/z9WDN1C6fg=
github.com/secure-systems-lab/go-securesystemslib v0.3.1/go.mod h1:o8hhjkbNl2gOamKUA/eNW3xUrntHT9L4W89W1nfj43U=
github.com/secure-systems-lab/go-securesystemslib v0.4.0/go.mod h1:FGBZgq2tXWICsxWQW1msNf49F0Pf2Op5Htayx335Qbs=
github.com/securego/gosec/v2 v2.9.1/go.mod h1:oDcDLcatOJxkCGaCaq8lua1jTnYf6Sou4wdiJ1n4iHc=
@@ -1376,8 +1380,9 @@ github.com/sirupsen/logrus v1.4.1/go.mod h1:ni0Sbl8bgC9z8RoU9G6nDWqqs/fq4eDPysMB
github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE=
github.com/sirupsen/logrus v1.6.0/go.mod h1:7uNnSEd1DgxDLC74fIahvMZmmYsHGZGEOFrfsX/uA88=
github.com/sirupsen/logrus v1.7.0/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0=
github.com/sirupsen/logrus v1.8.1 h1:dJKuHgqk1NNQlqoA6BTlM1Wf9DOH3NBjQyu0h9+AZZE=
github.com/sirupsen/logrus v1.8.1/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0=
github.com/sirupsen/logrus v1.9.0 h1:trlNQbNUG3OdDrDil03MCb1H2o9nJ1x4/5LYw7byDE0=
github.com/sirupsen/logrus v1.9.0/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
github.com/sivchari/tenv v1.4.7/go.mod h1:5nF+bITvkebQVanjU6IuMbvIot/7ReNsUV7I5NbprB0=
github.com/skratchdot/open-golang v0.0.0-20200116055534-eef842397966/go.mod h1:sUM3LWHvSMaG192sy56D9F7CNvL7jUJVXoqM1QKLnog=
github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1:OnSkiWE9lh6wB0YB77sQom3nweQdgAjqCqsofrRNTgc=
@@ -1442,7 +1447,6 @@ github.com/stretchr/testify v1.7.5/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO
github.com/stretchr/testify v1.8.0 h1:pSgiaMZlXftHpm5L7V1+rVB+AZJydKsMxsQBIJw4PKk=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/subosito/gotenv v1.2.0/go.mod h1:N0PQaV/YGNqwC0u51sEeR/aUtSLEXKX9iv69rRypqCw=
github.com/sylabs/sif/v2 v2.7.0/go.mod h1:TiyBWsgWeh5yBeQFNuQnvROwswqK7YJT8JA1L53bsXQ=
github.com/sylabs/sif/v2 v2.7.1 h1:XXt9AP39sQfsMCGOGQ/XP9H47yqZOvAonalkaCaNIYM=
github.com/sylabs/sif/v2 v2.7.1/go.mod h1:bBse2nEFd3yHkmq6KmAOFEWQg5LdFYiQUdVcgamxlc8=
github.com/sylvia7788/contextcheck v1.0.4/go.mod h1:vuPKJMQ7MQ91ZTqfdyreNKwZjyUg6KO+IebVyQDedZQ=
@@ -1459,8 +1463,9 @@ github.com/tdakkota/asciicheck v0.0.0-20200416200610-e657995f937b/go.mod h1:yHp0
github.com/tenntenn/modver v1.0.1/go.mod h1:bePIyQPb7UeioSRkw3Q0XeMhYZSMx9B8ePqg6SAMGH0=
github.com/tenntenn/text/transform v0.0.0-20200319021203-7eef512accb3/go.mod h1:ON8b8w4BN/kE1EOhwT0o+d62W65a6aPw1nouo9LMgyY=
github.com/tetafro/godot v1.4.11/go.mod h1:LR3CJpxDVGlYOWn3ZZg1PgNZdTUvzsZWu8xaEohUpn8=
github.com/theupdateframework/go-tuf v0.3.0 h1:od2sc5+BSkKZhmUG2o2rmruy0BGSmhrbDhCnpxh87X8=
github.com/theupdateframework/go-tuf v0.3.0/go.mod h1:E5XP0wXitrFUHe4b8cUcAAdxBW4LbfnqF4WXXGLgWNo=
github.com/theupdateframework/go-tuf v0.3.1 h1:NkjMlCuLcDpHNtsWXY4lTmbbQQ5nOM7JSBbOKEEiI1c=
github.com/theupdateframework/go-tuf v0.3.1/go.mod h1:lhHZ3Vt2pdAh15h0Cc6gWdlI+Okn2ZznD3q/cNjd5jw=
github.com/timakin/bodyclose v0.0.0-20200424151742-cb6215831a94/go.mod h1:Qimiffbc6q9tBWlVV6x0P9sat/ao1xEkREYPPj9hphk=
github.com/titanous/rocacheck v0.0.0-20171023193734-afe73141d399 h1:e/5i7d4oYZ+C1wj2THlRK+oAhjeS/TRQwMfkIuet3w0=
github.com/titanous/rocacheck v0.0.0-20171023193734-afe73141d399/go.mod h1:LdwHTNJT99C5fTAzDz0ud328OgXz+gierycbcIx2fRs=
@@ -1485,6 +1490,7 @@ github.com/ulikunitz/xz v0.5.10/go.mod h1:nbz6k7qbPmH4IRqmfOplQw/tblSgqTqBwxkY0o
github.com/ultraware/funlen v0.0.3/go.mod h1:Dp4UiAus7Wdb9KUZsYWZEWiRzGuM2kXM1lPbfaF6xhA=
github.com/ultraware/whitespace v0.0.4/go.mod h1:aVMh/gQve5Maj9hQ/hg+F75lr/X5A89uZnzAmWSineA=
github.com/urfave/cli v0.0.0-20171014202726-7bc6a0acffa5/go.mod h1:70zkFmudgCuE/ngEzBv17Jvp/497gISqfk5gWijbERA=
github.com/urfave/cli v1.19.1/go.mod h1:70zkFmudgCuE/ngEzBv17Jvp/497gISqfk5gWijbERA=
github.com/urfave/cli v1.20.0/go.mod h1:70zkFmudgCuE/ngEzBv17Jvp/497gISqfk5gWijbERA=
github.com/urfave/cli v1.22.1/go.mod h1:Gos4lmkARVdJ6EkW0WaNv/tZAAMe9V7XWyB60NtXRu0=
github.com/urfave/cli v1.22.2/go.mod h1:Gos4lmkARVdJ6EkW0WaNv/tZAAMe9V7XWyB60NtXRu0=
@@ -1498,7 +1504,6 @@ github.com/valyala/quicktemplate v1.7.0/go.mod h1:sqKJnoaOF88V07vkO+9FL8fb9uZg/V
github.com/valyala/tcplisten v1.0.0/go.mod h1:T0xQ8SeCZGxckz9qRXTfG43PvQ/mcWh7FwZEA7Ioqkc=
github.com/vbatts/tar-split v0.11.2 h1:Via6XqJr0hceW4wff3QRzD5gAk/tatMw/4ZA7cTlIME=
github.com/vbatts/tar-split v0.11.2/go.mod h1:vV3ZuO2yWSVsz+pfFzDG/upWH1JhjOiEaWq6kXyQ3VI=
github.com/vbauerster/mpb/v7 v7.4.1/go.mod h1:Ygg2mV9Vj9sQBWqsK2m2pidcf9H3s6bNKtqd3/M4gBo=
github.com/vbauerster/mpb/v7 v7.4.2 h1:n917F4d8EWdUKc9c81wFkksyG6P6Mg7IETfKCE1Xqng=
github.com/vbauerster/mpb/v7 v7.4.2/go.mod h1:UmOiIUI8aPqWXIps0ciik3RKMdzx7+ooQpq+fBcXwBA=
github.com/viki-org/dnscache v0.0.0-20130720023526-c70c1f23c5d8/go.mod h1:dniwbG03GafCjFohMDmz6Zc6oCuiqgH6tGNyXTkHzXE=
@@ -1949,8 +1954,9 @@ golang.org/x/sys v0.0.0-20220502124256-b6088ccd6cba/go.mod h1:oPkhp1MJrh7nUepCBc
golang.org/x/sys v0.0.0-20220503163025-988cb79eb6c6/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220610221304-9f5ed59c137d/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220624220833-87e55d714810 h1:rHZQSjJdAI4Xf5Qzeh2bBc5YJIkPFVM6oDtMFYmgws0=
golang.org/x/sys v0.0.0-20220624220833-87e55d714810/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8 h1:0A+M6Uqn+Eje4kHMK80dtF3JCXC4ykBgQG4Fe06QRhQ=
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210220032956-6a3ed077a48d/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=

View File

@@ -1,2 +1,2 @@
toml.test
/toml.test
/toml-test

View File

@@ -1 +0,0 @@
Compatible with TOML version [v1.0.0](https://toml.io/en/v1.0.0).

View File

@@ -1,6 +1,5 @@
TOML stands for Tom's Obvious, Minimal Language. This Go package provides a
reflection interface similar to Go's standard library `json` and `xml`
packages.
reflection interface similar to Go's standard library `json` and `xml` packages.
Compatible with TOML version [v1.0.0](https://toml.io/en/v1.0.0).
@@ -10,7 +9,7 @@ See the [releases page](https://github.com/BurntSushi/toml/releases) for a
changelog; this information is also in the git tag annotations (e.g. `git show
v0.4.0`).
This library requires Go 1.13 or newer; install it with:
This library requires Go 1.13 or newer; add it to your go.mod with:
% go get github.com/BurntSushi/toml@latest
@@ -19,16 +18,7 @@ It also comes with a TOML validator CLI tool:
% go install github.com/BurntSushi/toml/cmd/tomlv@latest
% tomlv some-toml-file.toml
### Testing
This package passes all tests in [toml-test] for both the decoder and the
encoder.
[toml-test]: https://github.com/BurntSushi/toml-test
### Examples
This package works similar to how the Go standard library handles XML and JSON.
Namely, data is loaded into Go values via reflection.
For the simplest example, consider some TOML file as just a list of keys and
values:
@@ -40,7 +30,7 @@ Perfection = [ 6, 28, 496, 8128 ]
DOB = 1987-07-05T05:45:00Z
```
Which could be defined in Go as:
Which can be decoded with:
```go
type Config struct {
@@ -48,20 +38,15 @@ type Config struct {
Cats []string
Pi float64
Perfection []int
DOB time.Time // requires `import time`
DOB time.Time
}
```
And then decoded with:
```go
var conf Config
_, err := toml.Decode(tomlData, &conf)
// handle error
```
You can also use struct tags if your struct field name doesn't map to a TOML
key value directly:
You can also use struct tags if your struct field name doesn't map to a TOML key
value directly:
```toml
some_key_NAME = "wat"
@@ -73,139 +58,63 @@ type TOML struct {
}
```
Beware that like other most other decoders **only exported fields** are
considered when encoding and decoding; private fields are silently ignored.
Beware that like other decoders **only exported fields** are considered when
encoding and decoding; private fields are silently ignored.
### Using the `Marshaler` and `encoding.TextUnmarshaler` interfaces
Here's an example that automatically parses duration strings into
`time.Duration` values:
Here's an example that automatically parses values in a `mail.Address`:
```toml
[[song]]
name = "Thunder Road"
duration = "4m49s"
[[song]]
name = "Stairway to Heaven"
duration = "8m03s"
contacts = [
"Donald Duck <donald@duckburg.com>",
"Scrooge McDuck <scrooge@duckburg.com>",
]
```
Which can be decoded with:
Can be decoded with:
```go
type song struct {
Name string
Duration duration
}
type songs struct {
Song []song
}
var favorites songs
if _, err := toml.Decode(blob, &favorites); err != nil {
log.Fatal(err)
// Create address type which satisfies the encoding.TextUnmarshaler interface.
type address struct {
*mail.Address
}
for _, s := range favorites.Song {
fmt.Printf("%s (%s)\n", s.Name, s.Duration)
}
```
And you'll also need a `duration` type that satisfies the
`encoding.TextUnmarshaler` interface:
```go
type duration struct {
time.Duration
}
func (d *duration) UnmarshalText(text []byte) error {
func (a *address) UnmarshalText(text []byte) error {
var err error
d.Duration, err = time.ParseDuration(string(text))
a.Address, err = mail.ParseAddress(string(text))
return err
}
// Decode it.
func decode() {
blob := `
contacts = [
"Donald Duck <donald@duckburg.com>",
"Scrooge McDuck <scrooge@duckburg.com>",
]
`
var contacts struct {
Contacts []address
}
_, err := toml.Decode(blob, &contacts)
if err != nil {
log.Fatal(err)
}
for _, c := range contacts.Contacts {
fmt.Printf("%#v\n", c.Address)
}
// Output:
// &mail.Address{Name:"Donald Duck", Address:"donald@duckburg.com"}
// &mail.Address{Name:"Scrooge McDuck", Address:"scrooge@duckburg.com"}
}
```
To target TOML specifically you can implement `UnmarshalTOML` TOML interface in
a similar way.
### More complex usage
Here's an example of how to load the example from the official spec page:
```toml
# This is a TOML document. Boom.
title = "TOML Example"
[owner]
name = "Tom Preston-Werner"
organization = "GitHub"
bio = "GitHub Cofounder & CEO\nLikes tater tots and beer."
dob = 1979-05-27T07:32:00Z # First class dates? Why not?
[database]
server = "192.168.1.1"
ports = [ 8001, 8001, 8002 ]
connection_max = 5000
enabled = true
[servers]
# You can indent as you please. Tabs or spaces. TOML don't care.
[servers.alpha]
ip = "10.0.0.1"
dc = "eqdc10"
[servers.beta]
ip = "10.0.0.2"
dc = "eqdc10"
[clients]
data = [ ["gamma", "delta"], [1, 2] ] # just an update to make sure parsers support it
# Line breaks are OK when inside arrays
hosts = [
"alpha",
"omega"
]
```
And the corresponding Go types are:
```go
type tomlConfig struct {
Title string
Owner ownerInfo
DB database `toml:"database"`
Servers map[string]server
Clients clients
}
type ownerInfo struct {
Name string
Org string `toml:"organization"`
Bio string
DOB time.Time
}
type database struct {
Server string
Ports []int
ConnMax int `toml:"connection_max"`
Enabled bool
}
type server struct {
IP string
DC string
}
type clients struct {
Data [][]interface{}
Hosts []string
}
```
Note that a case insensitive match will be tried if an exact match can't be
found.
A working example of the above can be found in `_example/example.{go,toml}`.
See the [`_example/`](/_example) directory for a more complex example.

View File

@@ -3,13 +3,16 @@ package toml
import (
"bytes"
"encoding"
"encoding/json"
"fmt"
"io"
"io/ioutil"
"math"
"os"
"reflect"
"strconv"
"strings"
"time"
)
// Unmarshaler is the interface implemented by objects that can unmarshal a
@@ -18,7 +21,7 @@ type Unmarshaler interface {
UnmarshalTOML(interface{}) error
}
// Unmarshal decodes the contents of `p` in TOML format into a pointer `v`.
// Unmarshal decodes the contents of `data` in TOML format into a pointer `v`.
func Unmarshal(data []byte, v interface{}) error {
_, err := NewDecoder(bytes.NewReader(data)).Decode(v)
return err
@@ -75,6 +78,9 @@ const (
// TOML datetimes correspond to Go time.Time values. Local datetimes are parsed
// in the local timezone.
//
// time.Duration types are treated as nanoseconds if the TOML value is an
// integer, or they're parsed with time.ParseDuration() if they're strings.
//
// All other TOML types (float, string, int, bool and array) correspond to the
// obvious Go types.
//
@@ -82,7 +88,7 @@ const (
// interface, in which case any primitive TOML value (floats, strings, integers,
// booleans, datetimes) will be converted to a []byte and given to the value's
// UnmarshalText method. See the Unmarshaler example for a demonstration with
// time duration strings.
// email addresses.
//
// Key mapping
//
@@ -111,6 +117,7 @@ func NewDecoder(r io.Reader) *Decoder {
var (
unmarshalToml = reflect.TypeOf((*Unmarshaler)(nil)).Elem()
unmarshalText = reflect.TypeOf((*encoding.TextUnmarshaler)(nil)).Elem()
primitiveType = reflect.TypeOf((*Primitive)(nil)).Elem()
)
// Decode TOML data in to the pointer `v`.
@@ -122,10 +129,10 @@ func (dec *Decoder) Decode(v interface{}) (MetaData, error) {
s = "%v"
}
return MetaData{}, e("cannot decode to non-pointer "+s, reflect.TypeOf(v))
return MetaData{}, fmt.Errorf("toml: cannot decode to non-pointer "+s, reflect.TypeOf(v))
}
if rv.IsNil() {
return MetaData{}, e("cannot decode to nil value of %q", reflect.TypeOf(v))
return MetaData{}, fmt.Errorf("toml: cannot decode to nil value of %q", reflect.TypeOf(v))
}
// Check if this is a supported type: struct, map, interface{}, or something
@@ -135,7 +142,7 @@ func (dec *Decoder) Decode(v interface{}) (MetaData, error) {
if rv.Kind() != reflect.Struct && rv.Kind() != reflect.Map &&
!(rv.Kind() == reflect.Interface && rv.NumMethod() == 0) &&
!rt.Implements(unmarshalToml) && !rt.Implements(unmarshalText) {
return MetaData{}, e("cannot decode to type %s", rt)
return MetaData{}, fmt.Errorf("toml: cannot decode to type %s", rt)
}
// TODO: parser should read from io.Reader? Or at the very least, make it
@@ -152,10 +159,11 @@ func (dec *Decoder) Decode(v interface{}) (MetaData, error) {
md := MetaData{
mapping: p.mapping,
types: p.types,
keyInfo: p.keyInfo,
keys: p.ordered,
decoded: make(map[string]struct{}, len(p.ordered)),
context: nil,
data: data,
}
return md, md.unify(p.mapping, rv)
}
@@ -185,7 +193,7 @@ func (md *MetaData) PrimitiveDecode(primValue Primitive, v interface{}) error {
func (md *MetaData) unify(data interface{}, rv reflect.Value) error {
// Special case. Look for a `Primitive` value.
// TODO: #76 would make this superfluous after implemented.
if rv.Type() == reflect.TypeOf((*Primitive)(nil)).Elem() {
if rv.Type() == primitiveType {
// Save the undecoded data and the key context into the primitive
// value.
context := make(Key, len(md.context))
@@ -197,17 +205,14 @@ func (md *MetaData) unify(data interface{}, rv reflect.Value) error {
return nil
}
// Special case. Unmarshaler Interface support.
if rv.CanAddr() {
if v, ok := rv.Addr().Interface().(Unmarshaler); ok {
return v.UnmarshalTOML(data)
}
rvi := rv.Interface()
if v, ok := rvi.(Unmarshaler); ok {
return v.UnmarshalTOML(data)
}
// Special case. Look for a value satisfying the TextUnmarshaler interface.
if v, ok := rv.Interface().(encoding.TextUnmarshaler); ok {
if v, ok := rvi.(encoding.TextUnmarshaler); ok {
return md.unifyText(data, v)
}
// TODO:
// The behavior here is incorrect whenever a Go type satisfies the
// encoding.TextUnmarshaler interface but also corresponds to a TOML hash or
@@ -218,7 +223,6 @@ func (md *MetaData) unify(data interface{}, rv reflect.Value) error {
k := rv.Kind()
// laziness
if k >= reflect.Int && k <= reflect.Uint64 {
return md.unifyInt(data, rv)
}
@@ -244,15 +248,14 @@ func (md *MetaData) unify(data interface{}, rv reflect.Value) error {
case reflect.Bool:
return md.unifyBool(data, rv)
case reflect.Interface:
// we only support empty interfaces.
if rv.NumMethod() > 0 {
return e("unsupported type %s", rv.Type())
if rv.NumMethod() > 0 { // Only support empty interfaces are supported.
return md.e("unsupported type %s", rv.Type())
}
return md.unifyAnything(data, rv)
case reflect.Float32, reflect.Float64:
return md.unifyFloat64(data, rv)
}
return e("unsupported type %s", rv.Kind())
return md.e("unsupported type %s", rv.Kind())
}
func (md *MetaData) unifyStruct(mapping interface{}, rv reflect.Value) error {
@@ -261,7 +264,7 @@ func (md *MetaData) unifyStruct(mapping interface{}, rv reflect.Value) error {
if mapping == nil {
return nil
}
return e("type mismatch for %s: expected table but found %T",
return md.e("type mismatch for %s: expected table but found %T",
rv.Type().String(), mapping)
}
@@ -287,13 +290,14 @@ func (md *MetaData) unifyStruct(mapping interface{}, rv reflect.Value) error {
if isUnifiable(subv) {
md.decoded[md.context.add(key).String()] = struct{}{}
md.context = append(md.context, key)
err := md.unify(datum, subv)
if err != nil {
return err
}
md.context = md.context[0 : len(md.context)-1]
} else if f.name != "" {
return e("cannot write unexported field %s.%s", rv.Type().String(), f.name)
return md.e("cannot write unexported field %s.%s", rv.Type().String(), f.name)
}
}
}
@@ -301,10 +305,10 @@ func (md *MetaData) unifyStruct(mapping interface{}, rv reflect.Value) error {
}
func (md *MetaData) unifyMap(mapping interface{}, rv reflect.Value) error {
if k := rv.Type().Key().Kind(); k != reflect.String {
return fmt.Errorf(
"toml: cannot decode to a map with non-string key type (%s in %q)",
k, rv.Type())
keyType := rv.Type().Key().Kind()
if keyType != reflect.String && keyType != reflect.Interface {
return fmt.Errorf("toml: cannot decode to a map with non-string key type (%s in %q)",
keyType, rv.Type())
}
tmap, ok := mapping.(map[string]interface{})
@@ -322,13 +326,22 @@ func (md *MetaData) unifyMap(mapping interface{}, rv reflect.Value) error {
md.context = append(md.context, k)
rvval := reflect.Indirect(reflect.New(rv.Type().Elem()))
if err := md.unify(v, rvval); err != nil {
err := md.unify(v, indirect(rvval))
if err != nil {
return err
}
md.context = md.context[0 : len(md.context)-1]
rvkey := indirect(reflect.New(rv.Type().Key()))
rvkey.SetString(k)
switch keyType {
case reflect.Interface:
rvkey.Set(reflect.ValueOf(k))
case reflect.String:
rvkey.SetString(k)
}
rv.SetMapIndex(rvkey, rvval)
}
return nil
@@ -343,7 +356,7 @@ func (md *MetaData) unifyArray(data interface{}, rv reflect.Value) error {
return md.badtype("slice", data)
}
if l := datav.Len(); l != rv.Len() {
return e("expected array length %d; got TOML array of length %d", rv.Len(), l)
return md.e("expected array length %d; got TOML array of length %d", rv.Len(), l)
}
return md.unifySliceArray(datav, rv)
}
@@ -376,6 +389,18 @@ func (md *MetaData) unifySliceArray(data, rv reflect.Value) error {
}
func (md *MetaData) unifyString(data interface{}, rv reflect.Value) error {
_, ok := rv.Interface().(json.Number)
if ok {
if i, ok := data.(int64); ok {
rv.SetString(strconv.FormatInt(i, 10))
} else if f, ok := data.(float64); ok {
rv.SetString(strconv.FormatFloat(f, 'f', -1, 64))
} else {
return md.badtype("string", data)
}
return nil
}
if s, ok := data.(string); ok {
rv.SetString(s)
return nil
@@ -384,11 +409,13 @@ func (md *MetaData) unifyString(data interface{}, rv reflect.Value) error {
}
func (md *MetaData) unifyFloat64(data interface{}, rv reflect.Value) error {
rvk := rv.Kind()
if num, ok := data.(float64); ok {
switch rv.Kind() {
switch rvk {
case reflect.Float32:
if num < -math.MaxFloat32 || num > math.MaxFloat32 {
return e("value %f is out of range for float32", num)
return md.parseErr(errParseRange{i: num, size: rvk.String()})
}
fallthrough
case reflect.Float64:
@@ -400,20 +427,11 @@ func (md *MetaData) unifyFloat64(data interface{}, rv reflect.Value) error {
}
if num, ok := data.(int64); ok {
switch rv.Kind() {
case reflect.Float32:
if num < -maxSafeFloat32Int || num > maxSafeFloat32Int {
return e("value %d is out of range for float32", num)
}
fallthrough
case reflect.Float64:
if num < -maxSafeFloat64Int || num > maxSafeFloat64Int {
return e("value %d is out of range for float64", num)
}
rv.SetFloat(float64(num))
default:
panic("bug")
if (rvk == reflect.Float32 && (num < -maxSafeFloat32Int || num > maxSafeFloat32Int)) ||
(rvk == reflect.Float64 && (num < -maxSafeFloat64Int || num > maxSafeFloat64Int)) {
return md.parseErr(errParseRange{i: num, size: rvk.String()})
}
rv.SetFloat(float64(num))
return nil
}
@@ -421,50 +439,46 @@ func (md *MetaData) unifyFloat64(data interface{}, rv reflect.Value) error {
}
func (md *MetaData) unifyInt(data interface{}, rv reflect.Value) error {
if num, ok := data.(int64); ok {
if rv.Kind() >= reflect.Int && rv.Kind() <= reflect.Int64 {
switch rv.Kind() {
case reflect.Int, reflect.Int64:
// No bounds checking necessary.
case reflect.Int8:
if num < math.MinInt8 || num > math.MaxInt8 {
return e("value %d is out of range for int8", num)
}
case reflect.Int16:
if num < math.MinInt16 || num > math.MaxInt16 {
return e("value %d is out of range for int16", num)
}
case reflect.Int32:
if num < math.MinInt32 || num > math.MaxInt32 {
return e("value %d is out of range for int32", num)
}
_, ok := rv.Interface().(time.Duration)
if ok {
// Parse as string duration, and fall back to regular integer parsing
// (as nanosecond) if this is not a string.
if s, ok := data.(string); ok {
dur, err := time.ParseDuration(s)
if err != nil {
return md.parseErr(errParseDuration{s})
}
rv.SetInt(num)
} else if rv.Kind() >= reflect.Uint && rv.Kind() <= reflect.Uint64 {
unum := uint64(num)
switch rv.Kind() {
case reflect.Uint, reflect.Uint64:
// No bounds checking necessary.
case reflect.Uint8:
if num < 0 || unum > math.MaxUint8 {
return e("value %d is out of range for uint8", num)
}
case reflect.Uint16:
if num < 0 || unum > math.MaxUint16 {
return e("value %d is out of range for uint16", num)
}
case reflect.Uint32:
if num < 0 || unum > math.MaxUint32 {
return e("value %d is out of range for uint32", num)
}
}
rv.SetUint(unum)
} else {
panic("unreachable")
rv.SetInt(int64(dur))
return nil
}
return nil
}
return md.badtype("integer", data)
num, ok := data.(int64)
if !ok {
return md.badtype("integer", data)
}
rvk := rv.Kind()
switch {
case rvk >= reflect.Int && rvk <= reflect.Int64:
if (rvk == reflect.Int8 && (num < math.MinInt8 || num > math.MaxInt8)) ||
(rvk == reflect.Int16 && (num < math.MinInt16 || num > math.MaxInt16)) ||
(rvk == reflect.Int32 && (num < math.MinInt32 || num > math.MaxInt32)) {
return md.parseErr(errParseRange{i: num, size: rvk.String()})
}
rv.SetInt(num)
case rvk >= reflect.Uint && rvk <= reflect.Uint64:
unum := uint64(num)
if rvk == reflect.Uint8 && (num < 0 || unum > math.MaxUint8) ||
rvk == reflect.Uint16 && (num < 0 || unum > math.MaxUint16) ||
rvk == reflect.Uint32 && (num < 0 || unum > math.MaxUint32) {
return md.parseErr(errParseRange{i: num, size: rvk.String()})
}
rv.SetUint(unum)
default:
panic("unreachable")
}
return nil
}
func (md *MetaData) unifyBool(data interface{}, rv reflect.Value) error {
@@ -489,7 +503,7 @@ func (md *MetaData) unifyText(data interface{}, v encoding.TextUnmarshaler) erro
return err
}
s = string(text)
case TextMarshaler:
case encoding.TextMarshaler:
text, err := sdata.MarshalText()
if err != nil {
return err
@@ -515,7 +529,30 @@ func (md *MetaData) unifyText(data interface{}, v encoding.TextUnmarshaler) erro
}
func (md *MetaData) badtype(dst string, data interface{}) error {
return e("incompatible types: TOML key %q has type %T; destination has type %s", md.context, data, dst)
return md.e("incompatible types: TOML value has type %T; destination has type %s", data, dst)
}
func (md *MetaData) parseErr(err error) error {
k := md.context.String()
return ParseError{
LastKey: k,
Position: md.keyInfo[k].pos,
Line: md.keyInfo[k].pos.Line,
err: err,
input: string(md.data),
}
}
func (md *MetaData) e(format string, args ...interface{}) error {
f := "toml: "
if len(md.context) > 0 {
f = fmt.Sprintf("toml: (last key %q): ", md.context)
p := md.keyInfo[md.context.String()].pos
if p.Line > 0 {
f = fmt.Sprintf("toml: line %d (last key %q): ", p.Line, md.context)
}
}
return fmt.Errorf(f+format, args...)
}
// rvalue returns a reflect.Value of `v`. All pointers are resolved.
@@ -534,7 +571,11 @@ func indirect(v reflect.Value) reflect.Value {
if v.Kind() != reflect.Ptr {
if v.CanSet() {
pv := v.Addr()
if _, ok := pv.Interface().(encoding.TextUnmarshaler); ok {
pvi := pv.Interface()
if _, ok := pvi.(encoding.TextUnmarshaler); ok {
return pv
}
if _, ok := pvi.(Unmarshaler); ok {
return pv
}
}
@@ -550,12 +591,12 @@ func isUnifiable(rv reflect.Value) bool {
if rv.CanSet() {
return true
}
if _, ok := rv.Interface().(encoding.TextUnmarshaler); ok {
rvi := rv.Interface()
if _, ok := rvi.(encoding.TextUnmarshaler); ok {
return true
}
if _, ok := rvi.(Unmarshaler); ok {
return true
}
return false
}
func e(format string, args ...interface{}) error {
return fmt.Errorf("toml: "+format, args...)
}

View File

@@ -3,6 +3,7 @@ package toml
import (
"bufio"
"encoding"
"encoding/json"
"errors"
"fmt"
"io"
@@ -63,6 +64,12 @@ var dblQuotedReplacer = strings.NewReplacer(
"\x7f", `\u007f`,
)
var (
marshalToml = reflect.TypeOf((*Marshaler)(nil)).Elem()
marshalText = reflect.TypeOf((*encoding.TextMarshaler)(nil)).Elem()
timeType = reflect.TypeOf((*time.Time)(nil)).Elem()
)
// Marshaler is the interface implemented by types that can marshal themselves
// into valid TOML.
type Marshaler interface {
@@ -74,6 +81,9 @@ type Marshaler interface {
// The mapping between Go values and TOML values should be precisely the same as
// for the Decode* functions.
//
// time.Time is encoded as a RFC 3339 string, and time.Duration as its string
// representation.
//
// The toml.Marshaler and encoder.TextMarshaler interfaces are supported to
// encoding the value as custom TOML.
//
@@ -85,6 +95,17 @@ type Marshaler interface {
//
// Go maps will be sorted alphabetically by key for deterministic output.
//
// The toml struct tag can be used to provide the key name; if omitted the
// struct field name will be used. If the "omitempty" option is present the
// following value will be skipped:
//
// - arrays, slices, maps, and string with len of 0
// - struct with all zero values
// - bool false
//
// If omitzero is given all int and float types with a value of 0 will be
// skipped.
//
// Encoding Go values without a corresponding TOML representation will return an
// error. Examples of this includes maps with non-string keys, slices with nil
// elements, embedded non-struct types, and nested slices containing maps or
@@ -136,18 +157,15 @@ func (enc *Encoder) safeEncode(key Key, rv reflect.Value) (err error) {
}
func (enc *Encoder) encode(key Key, rv reflect.Value) {
// Special case: time needs to be in ISO8601 format.
//
// Special case: if we can marshal the type to text, then we used that. This
// prevents the encoder for handling these types as generic structs (or
// whatever the underlying type of a TextMarshaler is).
switch t := rv.Interface().(type) {
case time.Time, encoding.TextMarshaler, Marshaler:
// If we can marshal the type to text, then we use that. This prevents the
// encoder for handling these types as generic structs (or whatever the
// underlying type of a TextMarshaler is).
switch {
case isMarshaler(rv):
enc.writeKeyValue(key, rv, false)
return
// TODO: #76 would make this superfluous after implemented.
case Primitive:
enc.encode(key, reflect.ValueOf(t.undecoded))
case rv.Type() == primitiveType: // TODO: #76 would make this superfluous after implemented.
enc.encode(key, reflect.ValueOf(rv.Interface().(Primitive).undecoded))
return
}
@@ -212,6 +230,9 @@ func (enc *Encoder) eElement(rv reflect.Value) {
if err != nil {
encPanic(err)
}
if s == nil {
encPanic(errors.New("MarshalTOML returned nil and no error"))
}
enc.w.Write(s)
return
case encoding.TextMarshaler:
@@ -219,11 +240,34 @@ func (enc *Encoder) eElement(rv reflect.Value) {
if err != nil {
encPanic(err)
}
if s == nil {
encPanic(errors.New("MarshalText returned nil and no error"))
}
enc.writeQuoted(string(s))
return
case time.Duration:
enc.writeQuoted(v.String())
return
case json.Number:
n, _ := rv.Interface().(json.Number)
if n == "" { /// Useful zero value.
enc.w.WriteByte('0')
return
} else if v, err := n.Int64(); err == nil {
enc.eElement(reflect.ValueOf(v))
return
} else if v, err := n.Float64(); err == nil {
enc.eElement(reflect.ValueOf(v))
return
}
encPanic(errors.New(fmt.Sprintf("Unable to convert \"%s\" to neither int64 nor float64", n)))
}
switch rv.Kind() {
case reflect.Ptr:
enc.eElement(rv.Elem())
return
case reflect.String:
enc.writeQuoted(rv.String())
case reflect.Bool:
@@ -259,7 +303,7 @@ func (enc *Encoder) eElement(rv reflect.Value) {
case reflect.Interface:
enc.eElement(rv.Elem())
default:
encPanic(fmt.Errorf("unexpected primitive type: %T", rv.Interface()))
encPanic(fmt.Errorf("unexpected type: %T", rv.Interface()))
}
}
@@ -280,7 +324,7 @@ func (enc *Encoder) eArrayOrSliceElement(rv reflect.Value) {
length := rv.Len()
enc.wf("[")
for i := 0; i < length; i++ {
elem := rv.Index(i)
elem := eindirect(rv.Index(i))
enc.eElement(elem)
if i != length-1 {
enc.wf(", ")
@@ -294,7 +338,7 @@ func (enc *Encoder) eArrayOfTables(key Key, rv reflect.Value) {
encPanic(errNoKey)
}
for i := 0; i < rv.Len(); i++ {
trv := rv.Index(i)
trv := eindirect(rv.Index(i))
if isNil(trv) {
continue
}
@@ -319,7 +363,7 @@ func (enc *Encoder) eTable(key Key, rv reflect.Value) {
}
func (enc *Encoder) eMapOrStruct(key Key, rv reflect.Value, inline bool) {
switch rv := eindirect(rv); rv.Kind() {
switch rv.Kind() {
case reflect.Map:
enc.eMap(key, rv, inline)
case reflect.Struct:
@@ -341,7 +385,7 @@ func (enc *Encoder) eMap(key Key, rv reflect.Value, inline bool) {
var mapKeysDirect, mapKeysSub []string
for _, mapKey := range rv.MapKeys() {
k := mapKey.String()
if typeIsTable(tomlTypeOfGo(rv.MapIndex(mapKey))) {
if typeIsTable(tomlTypeOfGo(eindirect(rv.MapIndex(mapKey)))) {
mapKeysSub = append(mapKeysSub, k)
} else {
mapKeysDirect = append(mapKeysDirect, k)
@@ -351,7 +395,7 @@ func (enc *Encoder) eMap(key Key, rv reflect.Value, inline bool) {
var writeMapKeys = func(mapKeys []string, trailC bool) {
sort.Strings(mapKeys)
for i, mapKey := range mapKeys {
val := rv.MapIndex(reflect.ValueOf(mapKey))
val := eindirect(rv.MapIndex(reflect.ValueOf(mapKey)))
if isNil(val) {
continue
}
@@ -379,6 +423,13 @@ func (enc *Encoder) eMap(key Key, rv reflect.Value, inline bool) {
const is32Bit = (32 << (^uint(0) >> 63)) == 32
func pointerTo(t reflect.Type) reflect.Type {
if t.Kind() == reflect.Ptr {
return pointerTo(t.Elem())
}
return t
}
func (enc *Encoder) eStruct(key Key, rv reflect.Value, inline bool) {
// Write keys for fields directly under this key first, because if we write
// a field that creates a new table then all keys under it will be in that
@@ -395,7 +446,8 @@ func (enc *Encoder) eStruct(key Key, rv reflect.Value, inline bool) {
addFields = func(rt reflect.Type, rv reflect.Value, start []int) {
for i := 0; i < rt.NumField(); i++ {
f := rt.Field(i)
if f.PkgPath != "" && !f.Anonymous { /// Skip unexported fields.
isEmbed := f.Anonymous && pointerTo(f.Type).Kind() == reflect.Struct
if f.PkgPath != "" && !isEmbed { /// Skip unexported fields.
continue
}
opts := getOptions(f.Tag)
@@ -403,27 +455,16 @@ func (enc *Encoder) eStruct(key Key, rv reflect.Value, inline bool) {
continue
}
frv := rv.Field(i)
frv := eindirect(rv.Field(i))
// Treat anonymous struct fields with tag names as though they are
// not anonymous, like encoding/json does.
//
// Non-struct anonymous fields use the normal encoding logic.
if f.Anonymous {
t := f.Type
switch t.Kind() {
case reflect.Struct:
if getOptions(f.Tag).name == "" {
addFields(t, frv, append(start, f.Index...))
continue
}
case reflect.Ptr:
if t.Elem().Kind() == reflect.Struct && getOptions(f.Tag).name == "" {
if !frv.IsNil() {
addFields(t.Elem(), frv.Elem(), append(start, f.Index...))
}
continue
}
if isEmbed {
if getOptions(f.Tag).name == "" && frv.Kind() == reflect.Struct {
addFields(frv.Type(), frv, append(start, f.Index...))
continue
}
}
@@ -449,7 +490,7 @@ func (enc *Encoder) eStruct(key Key, rv reflect.Value, inline bool) {
writeFields := func(fields [][]int) {
for _, fieldIndex := range fields {
fieldType := rt.FieldByIndex(fieldIndex)
fieldVal := rv.FieldByIndex(fieldIndex)
fieldVal := eindirect(rv.FieldByIndex(fieldIndex))
if isNil(fieldVal) { /// Don't write anything for nil fields.
continue
@@ -502,6 +543,21 @@ func tomlTypeOfGo(rv reflect.Value) tomlType {
if isNil(rv) || !rv.IsValid() {
return nil
}
if rv.Kind() == reflect.Struct {
if rv.Type() == timeType {
return tomlDatetime
}
if isMarshaler(rv) {
return tomlString
}
return tomlHash
}
if isMarshaler(rv) {
return tomlString
}
switch rv.Kind() {
case reflect.Bool:
return tomlBool
@@ -513,7 +569,7 @@ func tomlTypeOfGo(rv reflect.Value) tomlType {
case reflect.Float32, reflect.Float64:
return tomlFloat
case reflect.Array, reflect.Slice:
if typeEqual(tomlHash, tomlArrayType(rv)) {
if isTableArray(rv) {
return tomlArrayHash
}
return tomlArray
@@ -523,67 +579,35 @@ func tomlTypeOfGo(rv reflect.Value) tomlType {
return tomlString
case reflect.Map:
return tomlHash
case reflect.Struct:
if _, ok := rv.Interface().(time.Time); ok {
return tomlDatetime
}
if isMarshaler(rv) {
return tomlString
}
return tomlHash
default:
if isMarshaler(rv) {
return tomlString
}
encPanic(errors.New("unsupported type: " + rv.Kind().String()))
panic("unreachable")
}
}
func isMarshaler(rv reflect.Value) bool {
switch rv.Interface().(type) {
case encoding.TextMarshaler:
return true
case Marshaler:
return true
}
// Someone used a pointer receiver: we can make it work for pointer values.
if rv.CanAddr() {
if _, ok := rv.Addr().Interface().(encoding.TextMarshaler); ok {
return true
}
if _, ok := rv.Addr().Interface().(Marshaler); ok {
return true
}
}
return false
return rv.Type().Implements(marshalText) || rv.Type().Implements(marshalToml)
}
// tomlArrayType returns the element type of a TOML array. The type returned
// may be nil if it cannot be determined (e.g., a nil slice or a zero length
// slize). This function may also panic if it finds a type that cannot be
// expressed in TOML (such as nil elements, heterogeneous arrays or directly
// nested arrays of tables).
func tomlArrayType(rv reflect.Value) tomlType {
if isNil(rv) || !rv.IsValid() || rv.Len() == 0 {
return nil
// isTableArray reports if all entries in the array or slice are a table.
func isTableArray(arr reflect.Value) bool {
if isNil(arr) || !arr.IsValid() || arr.Len() == 0 {
return false
}
/// Don't allow nil.
rvlen := rv.Len()
for i := 1; i < rvlen; i++ {
if tomlTypeOfGo(rv.Index(i)) == nil {
ret := true
for i := 0; i < arr.Len(); i++ {
tt := tomlTypeOfGo(eindirect(arr.Index(i)))
// Don't allow nil.
if tt == nil {
encPanic(errArrayNilElement)
}
}
firstType := tomlTypeOfGo(rv.Index(0))
if firstType == nil {
encPanic(errArrayNilElement)
if ret && !typeEqual(tomlHash, tt) {
ret = false
}
}
return firstType
return ret
}
type tagOptions struct {
@@ -628,6 +652,8 @@ func isEmpty(rv reflect.Value) bool {
switch rv.Kind() {
case reflect.Array, reflect.Slice, reflect.Map, reflect.String:
return rv.Len() == 0
case reflect.Struct:
return reflect.Zero(rv.Type()).Interface() == rv.Interface()
case reflect.Bool:
return !rv.Bool()
}
@@ -679,13 +705,25 @@ func encPanic(err error) {
panic(tomlEncodeError{err})
}
// Resolve any level of pointers to the actual value (e.g. **string → string).
func eindirect(v reflect.Value) reflect.Value {
switch v.Kind() {
case reflect.Ptr, reflect.Interface:
return eindirect(v.Elem())
default:
if v.Kind() != reflect.Ptr && v.Kind() != reflect.Interface {
if isMarshaler(v) {
return v
}
if v.CanAddr() { /// Special case for marshalers; see #358.
if pv := v.Addr(); isMarshaler(pv) {
return pv
}
}
return v
}
if v.IsNil() {
return v
}
return eindirect(v.Elem())
}
func isNil(rv reflect.Value) bool {

View File

@@ -128,9 +128,13 @@ func (pe ParseError) ErrorWithPosition() string {
func (pe ParseError) ErrorWithUsage() string {
m := pe.ErrorWithPosition()
if u, ok := pe.err.(interface{ Usage() string }); ok && u.Usage() != "" {
return m + "Error help:\n\n " +
strings.ReplaceAll(strings.TrimSpace(u.Usage()), "\n", "\n ") +
"\n"
lines := strings.Split(strings.TrimSpace(u.Usage()), "\n")
for i := range lines {
if lines[i] != "" {
lines[i] = " " + lines[i]
}
}
return m + "Error help:\n\n" + strings.Join(lines, "\n") + "\n"
}
return m
}
@@ -160,6 +164,11 @@ type (
errLexInvalidDate struct{ v string }
errLexInlineTableNL struct{}
errLexStringNL struct{}
errParseRange struct {
i interface{} // int or float
size string // "int64", "uint16", etc.
}
errParseDuration struct{ d string }
)
func (e errLexControl) Error() string {
@@ -179,6 +188,10 @@ func (e errLexInlineTableNL) Error() string { return "newlines not allowed withi
func (e errLexInlineTableNL) Usage() string { return usageInlineNewline }
func (e errLexStringNL) Error() string { return "strings cannot contain newlines" }
func (e errLexStringNL) Usage() string { return usageStringNewline }
func (e errParseRange) Error() string { return fmt.Sprintf("%v is out of range for %s", e.i, e.size) }
func (e errParseRange) Usage() string { return usageIntOverflow }
func (e errParseDuration) Error() string { return fmt.Sprintf("invalid duration: %q", e.d) }
func (e errParseDuration) Usage() string { return usageDuration }
const usageEscape = `
A '\' inside a "-delimited string is interpreted as an escape character.
@@ -227,3 +240,37 @@ Instead use """ or ''' to split strings over multiple lines:
string = """Hello,
world!"""
`
const usageIntOverflow = `
This number is too large; this may be an error in the TOML, but it can also be a
bug in the program that uses too small of an integer.
The maximum and minimum values are:
size │ lowest │ highest
───────┼────────────────┼──────────
int8 │ -128 │ 127
int16 │ -32,768 │ 32,767
int32 │ -2,147,483,648 │ 2,147,483,647
int64 │ -9.2 × 10¹⁷ │ 9.2 × 10¹⁷
uint8 │ 0 │ 255
uint16 │ 0 │ 65535
uint32 │ 0 │ 4294967295
uint64 │ 0 │ 1.8 × 10¹⁸
int refers to int32 on 32-bit systems and int64 on 64-bit systems.
`
const usageDuration = `
A duration must be as "number<unit>", without any spaces. Valid units are:
ns nanoseconds (billionth of a second)
us, µs microseconds (millionth of a second)
ms milliseconds (thousands of a second)
s seconds
m minutes
h hours
You can combine multiple units; for example "5m10s" for 5 minutes and 10
seconds.
`

View File

@@ -82,7 +82,7 @@ func (lx *lexer) nextItem() item {
return item
default:
lx.state = lx.state(lx)
//fmt.Printf(" STATE %-24s current: %-10q stack: %s\n", lx.state, lx.current(), lx.stack)
//fmt.Printf(" STATE %-24s current: %-10s stack: %s\n", lx.state, lx.current(), lx.stack)
}
}
}
@@ -716,7 +716,17 @@ func lexMultilineString(lx *lexer) stateFn {
if lx.peek() == '"' {
/// Check if we already lexed 5 's; if so we have 6 now, and
/// that's just too many man!
if strings.HasSuffix(lx.current(), `"""""`) {
///
/// Second check is for the edge case:
///
/// two quotes allowed.
/// vv
/// """lol \""""""
/// ^^ ^^^---- closing three
/// escaped
///
/// But ugly, but it works
if strings.HasSuffix(lx.current(), `"""""`) && !strings.HasSuffix(lx.current(), `\"""""`) {
return lx.errorf(`unexpected '""""""'`)
}
lx.backup()
@@ -807,8 +817,7 @@ func lexMultilineRawString(lx *lexer) stateFn {
// lexMultilineStringEscape consumes an escaped character. It assumes that the
// preceding '\\' has already been consumed.
func lexMultilineStringEscape(lx *lexer) stateFn {
// Handle the special case first:
if isNL(lx.next()) {
if isNL(lx.next()) { /// \ escaping newline.
return lexMultilineString
}
lx.backup()

View File

@@ -12,10 +12,11 @@ import (
type MetaData struct {
context Key // Used only during decoding.
keyInfo map[string]keyInfo
mapping map[string]interface{}
types map[string]tomlType
keys []Key
decoded map[string]struct{}
data []byte // Input file; for errors.
}
// IsDefined reports if the key exists in the TOML data.
@@ -50,8 +51,8 @@ func (md *MetaData) IsDefined(key ...string) bool {
// Type will return the empty string if given an empty key or a key that does
// not exist. Keys are case sensitive.
func (md *MetaData) Type(key ...string) string {
if typ, ok := md.types[Key(key).String()]; ok {
return typ.typeString()
if ki, ok := md.keyInfo[Key(key).String()]; ok {
return ki.tomlType.typeString()
}
return ""
}

View File

@@ -16,12 +16,18 @@ type parser struct {
currentKey string // Base key name for everything except hashes.
pos Position // Current position in the TOML file.
ordered []Key // List of keys in the order that they appear in the TOML data.
ordered []Key // List of keys in the order that they appear in the TOML data.
keyInfo map[string]keyInfo // Map keyname → info about the TOML key.
mapping map[string]interface{} // Map keyname → key value.
types map[string]tomlType // Map keyname → TOML type.
implicits map[string]struct{} // Record implicit keys (e.g. "key.group.names").
}
type keyInfo struct {
pos Position
tomlType tomlType
}
func parse(data string) (p *parser, err error) {
defer func() {
if r := recover(); r != nil {
@@ -57,8 +63,8 @@ func parse(data string) (p *parser, err error) {
}
p = &parser{
keyInfo: make(map[string]keyInfo),
mapping: make(map[string]interface{}),
types: make(map[string]tomlType),
lx: lex(data),
ordered: make([]Key, 0),
implicits: make(map[string]struct{}),
@@ -74,6 +80,15 @@ func parse(data string) (p *parser, err error) {
return p, nil
}
func (p *parser) panicErr(it item, err error) {
panic(ParseError{
err: err,
Position: it.pos,
Line: it.pos.Len,
LastKey: p.current(),
})
}
func (p *parser) panicItemf(it item, format string, v ...interface{}) {
panic(ParseError{
Message: fmt.Sprintf(format, v...),
@@ -94,7 +109,7 @@ func (p *parser) panicf(format string, v ...interface{}) {
func (p *parser) next() item {
it := p.lx.nextItem()
//fmt.Printf("ITEM %-18s line %-3d │ %q\n", it.typ, it.line, it.val)
//fmt.Printf("ITEM %-18s line %-3d │ %q\n", it.typ, it.pos.Line, it.val)
if it.typ == itemError {
if it.err != nil {
panic(ParseError{
@@ -146,7 +161,7 @@ func (p *parser) topLevel(item item) {
p.assertEqual(itemTableEnd, name.typ)
p.addContext(key, false)
p.setType("", tomlHash)
p.setType("", tomlHash, item.pos)
p.ordered = append(p.ordered, key)
case itemArrayTableStart: // [[ .. ]]
name := p.nextPos()
@@ -158,7 +173,7 @@ func (p *parser) topLevel(item item) {
p.assertEqual(itemArrayTableEnd, name.typ)
p.addContext(key, true)
p.setType("", tomlArrayHash)
p.setType("", tomlArrayHash, item.pos)
p.ordered = append(p.ordered, key)
case itemKeyStart: // key = ..
outerContext := p.context
@@ -181,8 +196,9 @@ func (p *parser) topLevel(item item) {
}
/// Set value.
val, typ := p.value(p.next(), false)
p.set(p.currentKey, val, typ)
vItem := p.next()
val, typ := p.value(vItem, false)
p.set(p.currentKey, val, typ, vItem.pos)
p.ordered = append(p.ordered, p.context.add(p.currentKey))
/// Remove the context we added (preserving any context from [tbl] lines).
@@ -266,7 +282,7 @@ func (p *parser) valueInteger(it item) (interface{}, tomlType) {
// So mark the former as a bug but the latter as a legitimate user
// error.
if e, ok := err.(*strconv.NumError); ok && e.Err == strconv.ErrRange {
p.panicItemf(it, "Integer '%s' is out of the range of 64-bit signed integers.", it.val)
p.panicErr(it, errParseRange{i: it.val, size: "int64"})
} else {
p.bug("Expected integer value, but got '%s'.", it.val)
}
@@ -304,7 +320,7 @@ func (p *parser) valueFloat(it item) (interface{}, tomlType) {
num, err := strconv.ParseFloat(val, 64)
if err != nil {
if e, ok := err.(*strconv.NumError); ok && e.Err == strconv.ErrRange {
p.panicItemf(it, "Float '%s' is out of the range of 64-bit IEEE-754 floating-point numbers.", it.val)
p.panicErr(it, errParseRange{i: it.val, size: "float64"})
} else {
p.panicItemf(it, "Invalid float value: %q", it.val)
}
@@ -343,9 +359,8 @@ func (p *parser) valueDatetime(it item) (interface{}, tomlType) {
}
func (p *parser) valueArray(it item) (interface{}, tomlType) {
p.setType(p.currentKey, tomlArray)
p.setType(p.currentKey, tomlArray, it.pos)
// p.setType(p.currentKey, typ)
var (
types []tomlType
@@ -414,7 +429,7 @@ func (p *parser) valueInlineTable(it item, parentIsArray bool) (interface{}, tom
/// Set the value.
val, typ := p.value(p.next(), false)
p.set(p.currentKey, val, typ)
p.set(p.currentKey, val, typ, it.pos)
p.ordered = append(p.ordered, p.context.add(p.currentKey))
hash[p.currentKey] = val
@@ -533,9 +548,10 @@ func (p *parser) addContext(key Key, array bool) {
}
// set calls setValue and setType.
func (p *parser) set(key string, val interface{}, typ tomlType) {
func (p *parser) set(key string, val interface{}, typ tomlType, pos Position) {
p.setValue(key, val)
p.setType(key, typ)
p.setType(key, typ, pos)
}
// setValue sets the given key to the given value in the current context.
@@ -599,7 +615,7 @@ func (p *parser) setValue(key string, value interface{}) {
//
// Note that if `key` is empty, then the type given will be applied to the
// current context (which is either a table or an array of tables).
func (p *parser) setType(key string, typ tomlType) {
func (p *parser) setType(key string, typ tomlType, pos Position) {
keyContext := make(Key, 0, len(p.context)+1)
keyContext = append(keyContext, p.context...)
if len(key) > 0 { // allow type setting for hashes
@@ -611,7 +627,7 @@ func (p *parser) setType(key string, typ tomlType) {
if len(keyContext) == 0 {
keyContext = Key{""}
}
p.types[keyContext.String()] = typ
p.keyInfo[keyContext.String()] = keyInfo{tomlType: typ, pos: pos}
}
// Implicit keys need to be created when tables are implied in "a.b.c.d = 1" and
@@ -619,7 +635,7 @@ func (p *parser) setType(key string, typ tomlType) {
func (p *parser) addImplicit(key Key) { p.implicits[key.String()] = struct{}{} }
func (p *parser) removeImplicit(key Key) { delete(p.implicits, key.String()) }
func (p *parser) isImplicit(key Key) bool { _, ok := p.implicits[key.String()]; return ok }
func (p *parser) isArray(key Key) bool { return p.types[key.String()] == tomlArray }
func (p *parser) isArray(key Key) bool { return p.keyInfo[key.String()].tomlType == tomlArray }
func (p *parser) addImplicitContext(key Key) {
p.addImplicit(key)
p.addContext(key, false)
@@ -710,10 +726,8 @@ func (p *parser) replaceEscapes(it item, str string) string {
switch s[r] {
default:
p.bug("Expected valid escape code after \\, but got %q.", s[r])
return ""
case ' ', '\t':
p.panicItemf(it, "invalid escape: '\\%c'", s[r])
return ""
case 'b':
replaced = append(replaced, rune(0x0008))
r += 1

View File

@@ -4,17 +4,22 @@ import (
"context"
"encoding/json"
"errors"
"fmt"
"strings"
"sync"
"syscall"
"time"
"github.com/Microsoft/hcsshim/internal/cow"
"github.com/Microsoft/hcsshim/internal/hcs/schema1"
hcsschema "github.com/Microsoft/hcsshim/internal/hcs/schema2"
"github.com/Microsoft/hcsshim/internal/jobobject"
"github.com/Microsoft/hcsshim/internal/log"
"github.com/Microsoft/hcsshim/internal/logfields"
"github.com/Microsoft/hcsshim/internal/oc"
"github.com/Microsoft/hcsshim/internal/timeout"
"github.com/Microsoft/hcsshim/internal/vmcompute"
"github.com/sirupsen/logrus"
"go.opencensus.io/trace"
)
@@ -28,7 +33,8 @@ type System struct {
waitBlock chan struct{}
waitError error
exitError error
os, typ string
os, typ, owner string
startTime time.Time
}
func newSystem(id string) *System {
@@ -38,6 +44,11 @@ func newSystem(id string) *System {
}
}
// Implementation detail for silo naming, this should NOT be relied upon very heavily.
func siloNameFmt(containerID string) string {
return fmt.Sprintf(`\Container_%s`, containerID)
}
// CreateComputeSystem creates a new compute system with the given configuration but does not start it.
func CreateComputeSystem(ctx context.Context, id string, hcsDocumentInterface interface{}) (_ *System, err error) {
operation := "hcs::CreateComputeSystem"
@@ -127,6 +138,7 @@ func (computeSystem *System) getCachedProperties(ctx context.Context) error {
}
computeSystem.typ = strings.ToLower(props.SystemType)
computeSystem.os = strings.ToLower(props.RuntimeOSType)
computeSystem.owner = strings.ToLower(props.Owner)
if computeSystem.os == "" && computeSystem.typ == "container" {
// Pre-RS5 HCS did not return the OS, but it only supported containers
// that ran Windows.
@@ -195,7 +207,7 @@ func (computeSystem *System) Start(ctx context.Context) (err error) {
if err != nil {
return makeSystemError(computeSystem, operation, err, events)
}
computeSystem.startTime = time.Now()
return nil
}
@@ -324,11 +336,115 @@ func (computeSystem *System) Properties(ctx context.Context, types ...schema1.Pr
return properties, nil
}
// PropertiesV2 returns the requested container properties targeting a V2 schema container.
func (computeSystem *System) PropertiesV2(ctx context.Context, types ...hcsschema.PropertyType) (*hcsschema.Properties, error) {
computeSystem.handleLock.RLock()
defer computeSystem.handleLock.RUnlock()
// queryInProc handles querying for container properties without reaching out to HCS. `props`
// will be updated to contain any data returned from the queries present in `types`. If any properties
// failed to be queried they will be tallied up and returned in as the first return value. Failures on
// query are NOT considered errors; the only failure case for this method is if the containers job object
// cannot be opened.
func (computeSystem *System) queryInProc(ctx context.Context, props *hcsschema.Properties, types []hcsschema.PropertyType) ([]hcsschema.PropertyType, error) {
// In the future we can make use of some new functionality in the HCS that allows you
// to pass a job object for HCS to use for the container. Currently, the only way we'll
// be able to open the job/silo is if we're running as SYSTEM.
jobOptions := &jobobject.Options{
UseNTVariant: true,
Name: siloNameFmt(computeSystem.id),
}
job, err := jobobject.Open(ctx, jobOptions)
if err != nil {
return nil, err
}
defer job.Close()
var fallbackQueryTypes []hcsschema.PropertyType
for _, propType := range types {
switch propType {
case hcsschema.PTStatistics:
// Handle a bad caller asking for the same type twice. No use in re-querying if this is
// filled in already.
if props.Statistics == nil {
props.Statistics, err = computeSystem.statisticsInProc(job)
if err != nil {
log.G(ctx).WithError(err).Warn("failed to get statistics in-proc")
fallbackQueryTypes = append(fallbackQueryTypes, propType)
}
}
default:
fallbackQueryTypes = append(fallbackQueryTypes, propType)
}
}
return fallbackQueryTypes, nil
}
// statisticsInProc emulates what HCS does to grab statistics for a given container with a small
// change to make grabbing the private working set total much more efficient.
func (computeSystem *System) statisticsInProc(job *jobobject.JobObject) (*hcsschema.Statistics, error) {
// Start timestamp for these stats before we grab them to match HCS
timestamp := time.Now()
memInfo, err := job.QueryMemoryStats()
if err != nil {
return nil, err
}
processorInfo, err := job.QueryProcessorStats()
if err != nil {
return nil, err
}
storageInfo, err := job.QueryStorageStats()
if err != nil {
return nil, err
}
// This calculates the private working set more efficiently than HCS does. HCS calls NtQuerySystemInformation
// with the class SystemProcessInformation which returns an array containing system information for *every*
// process running on the machine. They then grab the pids that are running in the container and filter down
// the entries in the array to only what's running in that silo and start tallying up the total. This doesn't
// work well as performance should get worse if more processess are running on the machine in general and not
// just in the container. All of the additional information besides the WorkingSetPrivateSize field is ignored
// as well which isn't great and is wasted work to fetch.
//
// HCS only let's you grab statistics in an all or nothing fashion, so we can't just grab the private
// working set ourselves and ask for everything else seperately. The optimization we can make here is
// to open the silo ourselves and do the same queries for the rest of the info, as well as calculating
// the private working set in a more efficient manner by:
//
// 1. Find the pids running in the silo
// 2. Get a process handle for every process (only need PROCESS_QUERY_LIMITED_INFORMATION access)
// 3. Call NtQueryInformationProcess on each process with the class ProcessVmCounters
// 4. Tally up the total using the field PrivateWorkingSetSize in VM_COUNTERS_EX2.
privateWorkingSet, err := job.QueryPrivateWorkingSet()
if err != nil {
return nil, err
}
return &hcsschema.Statistics{
Timestamp: timestamp,
ContainerStartTime: computeSystem.startTime,
Uptime100ns: uint64(time.Since(computeSystem.startTime).Nanoseconds()) / 100,
Memory: &hcsschema.MemoryStats{
MemoryUsageCommitBytes: memInfo.JobMemory,
MemoryUsageCommitPeakBytes: memInfo.PeakJobMemoryUsed,
MemoryUsagePrivateWorkingSetBytes: privateWorkingSet,
},
Processor: &hcsschema.ProcessorStats{
RuntimeKernel100ns: uint64(processorInfo.TotalKernelTime),
RuntimeUser100ns: uint64(processorInfo.TotalUserTime),
TotalRuntime100ns: uint64(processorInfo.TotalKernelTime + processorInfo.TotalUserTime),
},
Storage: &hcsschema.StorageStats{
ReadCountNormalized: uint64(storageInfo.ReadStats.IoCount),
ReadSizeBytes: storageInfo.ReadStats.TotalSize,
WriteCountNormalized: uint64(storageInfo.WriteStats.IoCount),
WriteSizeBytes: storageInfo.WriteStats.TotalSize,
},
}, nil
}
// hcsPropertiesV2Query is a helper to make a HcsGetComputeSystemProperties call using the V2 schema property types.
func (computeSystem *System) hcsPropertiesV2Query(ctx context.Context, types []hcsschema.PropertyType) (*hcsschema.Properties, error) {
operation := "hcs::System::PropertiesV2"
queryBytes, err := json.Marshal(hcsschema.PropertyQuery{PropertyTypes: types})
@@ -345,12 +461,66 @@ func (computeSystem *System) PropertiesV2(ctx context.Context, types ...hcsschem
if propertiesJSON == "" {
return nil, ErrUnexpectedValue
}
properties := &hcsschema.Properties{}
if err := json.Unmarshal([]byte(propertiesJSON), properties); err != nil {
props := &hcsschema.Properties{}
if err := json.Unmarshal([]byte(propertiesJSON), props); err != nil {
return nil, makeSystemError(computeSystem, operation, err, nil)
}
return properties, nil
return props, nil
}
// PropertiesV2 returns the requested compute systems properties targeting a V2 schema compute system.
func (computeSystem *System) PropertiesV2(ctx context.Context, types ...hcsschema.PropertyType) (_ *hcsschema.Properties, err error) {
computeSystem.handleLock.RLock()
defer computeSystem.handleLock.RUnlock()
// Let HCS tally up the total for VM based queries instead of querying ourselves.
if computeSystem.typ != "container" {
return computeSystem.hcsPropertiesV2Query(ctx, types)
}
// Define a starter Properties struct with the default fields returned from every
// query. Owner is only returned from Statistics but it's harmless to include.
properties := &hcsschema.Properties{
Id: computeSystem.id,
SystemType: computeSystem.typ,
RuntimeOsType: computeSystem.os,
Owner: computeSystem.owner,
}
logEntry := log.G(ctx)
// First lets try and query ourselves without reaching to HCS. If any of the queries fail
// we'll take note and fallback to querying HCS for any of the failed types.
fallbackTypes, err := computeSystem.queryInProc(ctx, properties, types)
if err == nil && len(fallbackTypes) == 0 {
return properties, nil
} else if err != nil {
logEntry.WithError(fmt.Errorf("failed to query compute system properties in-proc: %w", err))
fallbackTypes = types
}
logEntry.WithFields(logrus.Fields{
logfields.ContainerID: computeSystem.id,
"propertyTypes": fallbackTypes,
}).Info("falling back to HCS for property type queries")
hcsProperties, err := computeSystem.hcsPropertiesV2Query(ctx, fallbackTypes)
if err != nil {
return nil, err
}
// Now add in anything that we might have successfully queried in process.
if properties.Statistics != nil {
hcsProperties.Statistics = properties.Statistics
hcsProperties.Owner = properties.Owner
}
// For future support for querying processlist in-proc as well.
if properties.ProcessList != nil {
hcsProperties.ProcessList = properties.ProcessList
}
return hcsProperties, nil
}
// Pause pauses the execution of the computeSystem. This feature is not enabled in TP5.

View File

@@ -21,10 +21,11 @@ const (
)
type NatPolicy struct {
Type PolicyType `json:"Type"`
Protocol string `json:",omitempty"`
InternalPort uint16 `json:",omitempty"`
ExternalPort uint16 `json:",omitempty"`
Type PolicyType `json:"Type"`
Protocol string `json:",omitempty"`
InternalPort uint16 `json:",omitempty"`
ExternalPort uint16 `json:",omitempty"`
ExternalPortReserved bool `json:",omitempty"`
}
type QosPolicy struct {

View File

@@ -0,0 +1,111 @@
package jobobject
import (
"context"
"fmt"
"sync"
"unsafe"
"github.com/Microsoft/hcsshim/internal/log"
"github.com/Microsoft/hcsshim/internal/queue"
"github.com/Microsoft/hcsshim/internal/winapi"
"github.com/sirupsen/logrus"
"golang.org/x/sys/windows"
)
var (
ioInitOnce sync.Once
initIOErr error
// Global iocp handle that will be re-used for every job object
ioCompletionPort windows.Handle
// Mapping of job handle to queue to place notifications in.
jobMap sync.Map
)
// MsgAllProcessesExited is a type representing a message that every process in a job has exited.
type MsgAllProcessesExited struct{}
// MsgUnimplemented represents a message that we are aware of, but that isn't implemented currently.
// This should not be treated as an error.
type MsgUnimplemented struct{}
// pollIOCP polls the io completion port forever.
func pollIOCP(ctx context.Context, iocpHandle windows.Handle) {
var (
overlapped uintptr
code uint32
key uintptr
)
for {
err := windows.GetQueuedCompletionStatus(iocpHandle, &code, &key, (**windows.Overlapped)(unsafe.Pointer(&overlapped)), windows.INFINITE)
if err != nil {
log.G(ctx).WithError(err).Error("failed to poll for job object message")
continue
}
if val, ok := jobMap.Load(key); ok {
msq, ok := val.(*queue.MessageQueue)
if !ok {
log.G(ctx).WithField("value", msq).Warn("encountered non queue type in job map")
continue
}
notification, err := parseMessage(code, overlapped)
if err != nil {
log.G(ctx).WithFields(logrus.Fields{
"code": code,
"overlapped": overlapped,
}).Warn("failed to parse job object message")
continue
}
if err := msq.Write(notification); err == queue.ErrQueueClosed {
// Write will only return an error when the queue is closed.
// The only time a queue would ever be closed is when we call `Close` on
// the job it belongs to which also removes it from the jobMap, so something
// went wrong here. We can't return as this is reading messages for all jobs
// so just log it and move on.
log.G(ctx).WithFields(logrus.Fields{
"code": code,
"overlapped": overlapped,
}).Warn("tried to write to a closed queue")
continue
}
} else {
log.G(ctx).Warn("received a message for a job not present in the mapping")
}
}
}
func parseMessage(code uint32, overlapped uintptr) (interface{}, error) {
// Check code and parse out relevant information related to that notification
// that we care about. For now all we handle is the message that all processes
// in the job have exited.
switch code {
case winapi.JOB_OBJECT_MSG_ACTIVE_PROCESS_ZERO:
return MsgAllProcessesExited{}, nil
// Other messages for completeness and a check to make sure that if we fall
// into the default case that this is a code we don't know how to handle.
case winapi.JOB_OBJECT_MSG_END_OF_JOB_TIME:
case winapi.JOB_OBJECT_MSG_END_OF_PROCESS_TIME:
case winapi.JOB_OBJECT_MSG_ACTIVE_PROCESS_LIMIT:
case winapi.JOB_OBJECT_MSG_NEW_PROCESS:
case winapi.JOB_OBJECT_MSG_EXIT_PROCESS:
case winapi.JOB_OBJECT_MSG_ABNORMAL_EXIT_PROCESS:
case winapi.JOB_OBJECT_MSG_PROCESS_MEMORY_LIMIT:
case winapi.JOB_OBJECT_MSG_JOB_MEMORY_LIMIT:
case winapi.JOB_OBJECT_MSG_NOTIFICATION_LIMIT:
default:
return nil, fmt.Errorf("unknown job notification type: %d", code)
}
return MsgUnimplemented{}, nil
}
// Assigns an IO completion port to get notified of events for the registered job
// object.
func attachIOCP(job windows.Handle, iocp windows.Handle) error {
info := winapi.JOBOBJECT_ASSOCIATE_COMPLETION_PORT{
CompletionKey: job,
CompletionPort: iocp,
}
_, err := windows.SetInformationJobObject(job, windows.JobObjectAssociateCompletionPortInformation, uintptr(unsafe.Pointer(&info)), uint32(unsafe.Sizeof(info)))
return err
}

View File

@@ -0,0 +1,499 @@
package jobobject
import (
"context"
"errors"
"fmt"
"sync"
"unsafe"
"github.com/Microsoft/hcsshim/internal/queue"
"github.com/Microsoft/hcsshim/internal/winapi"
"golang.org/x/sys/windows"
)
// This file provides higher level constructs for the win32 job object API.
// Most of the core creation and management functions are already present in "golang.org/x/sys/windows"
// (CreateJobObject, AssignProcessToJobObject, etc.) as well as most of the limit information
// structs and associated limit flags. Whatever is not present from the job object API
// in golang.org/x/sys/windows is located in /internal/winapi.
//
// https://docs.microsoft.com/en-us/windows/win32/procthread/job-objects
// JobObject is a high level wrapper around a Windows job object. Holds a handle to
// the job, a queue to receive iocp notifications about the lifecycle
// of the job and a mutex for synchronized handle access.
type JobObject struct {
handle windows.Handle
mq *queue.MessageQueue
handleLock sync.RWMutex
}
// JobLimits represents the resource constraints that can be applied to a job object.
type JobLimits struct {
CPULimit uint32
CPUWeight uint32
MemoryLimitInBytes uint64
MaxIOPS int64
MaxBandwidth int64
}
type CPURateControlType uint32
const (
WeightBased CPURateControlType = iota
RateBased
)
// Processor resource controls
const (
cpuLimitMin = 1
cpuLimitMax = 10000
cpuWeightMin = 1
cpuWeightMax = 9
)
var (
ErrAlreadyClosed = errors.New("the handle has already been closed")
ErrNotRegistered = errors.New("job is not registered to receive notifications")
)
// Options represents the set of configurable options when making or opening a job object.
type Options struct {
// `Name` specifies the name of the job object if a named job object is desired.
Name string
// `Notifications` specifies if the job will be registered to receive notifications.
// Defaults to false.
Notifications bool
// `UseNTVariant` specifies if we should use the `Nt` variant of Open/CreateJobObject.
// Defaults to false.
UseNTVariant bool
}
// Create creates a job object.
//
// If options.Name is an empty string, the job will not be assigned a name.
//
// If options.Notifications are not enabled `PollNotifications` will return immediately with error `errNotRegistered`.
//
// If `options` is nil, use default option values.
//
// Returns a JobObject structure and an error if there is one.
func Create(ctx context.Context, options *Options) (_ *JobObject, err error) {
if options == nil {
options = &Options{}
}
var jobName *winapi.UnicodeString
if options.Name != "" {
jobName, err = winapi.NewUnicodeString(options.Name)
if err != nil {
return nil, err
}
}
var jobHandle windows.Handle
if options.UseNTVariant {
oa := winapi.ObjectAttributes{
Length: unsafe.Sizeof(winapi.ObjectAttributes{}),
ObjectName: jobName,
Attributes: 0,
}
status := winapi.NtCreateJobObject(&jobHandle, winapi.JOB_OBJECT_ALL_ACCESS, &oa)
if status != 0 {
return nil, winapi.RtlNtStatusToDosError(status)
}
} else {
var jobNameBuf *uint16
if jobName != nil && jobName.Buffer != nil {
jobNameBuf = jobName.Buffer
}
jobHandle, err = windows.CreateJobObject(nil, jobNameBuf)
if err != nil {
return nil, err
}
}
defer func() {
if err != nil {
windows.Close(jobHandle)
}
}()
job := &JobObject{
handle: jobHandle,
}
// If the IOCP we'll be using to receive messages for all jobs hasn't been
// created, create it and start polling.
if options.Notifications {
mq, err := setupNotifications(ctx, job)
if err != nil {
return nil, err
}
job.mq = mq
}
return job, nil
}
// Open opens an existing job object with name provided in `options`. If no name is provided
// return an error since we need to know what job object to open.
//
// If options.Notifications is false `PollNotifications` will return immediately with error `errNotRegistered`.
//
// Returns a JobObject structure and an error if there is one.
func Open(ctx context.Context, options *Options) (_ *JobObject, err error) {
if options == nil || (options != nil && options.Name == "") {
return nil, errors.New("no job object name specified to open")
}
unicodeJobName, err := winapi.NewUnicodeString(options.Name)
if err != nil {
return nil, err
}
var jobHandle windows.Handle
if options != nil && options.UseNTVariant {
oa := winapi.ObjectAttributes{
Length: unsafe.Sizeof(winapi.ObjectAttributes{}),
ObjectName: unicodeJobName,
Attributes: 0,
}
status := winapi.NtOpenJobObject(&jobHandle, winapi.JOB_OBJECT_ALL_ACCESS, &oa)
if status != 0 {
return nil, winapi.RtlNtStatusToDosError(status)
}
} else {
jobHandle, err = winapi.OpenJobObject(winapi.JOB_OBJECT_ALL_ACCESS, false, unicodeJobName.Buffer)
if err != nil {
return nil, err
}
}
defer func() {
if err != nil {
windows.Close(jobHandle)
}
}()
job := &JobObject{
handle: jobHandle,
}
// If the IOCP we'll be using to receive messages for all jobs hasn't been
// created, create it and start polling.
if options != nil && options.Notifications {
mq, err := setupNotifications(ctx, job)
if err != nil {
return nil, err
}
job.mq = mq
}
return job, nil
}
// helper function to setup notifications for creating/opening a job object
func setupNotifications(ctx context.Context, job *JobObject) (*queue.MessageQueue, error) {
job.handleLock.RLock()
defer job.handleLock.RUnlock()
if job.handle == 0 {
return nil, ErrAlreadyClosed
}
ioInitOnce.Do(func() {
h, err := windows.CreateIoCompletionPort(windows.InvalidHandle, 0, 0, 0xffffffff)
if err != nil {
initIOErr = err
return
}
ioCompletionPort = h
go pollIOCP(ctx, h)
})
if initIOErr != nil {
return nil, initIOErr
}
mq := queue.NewMessageQueue()
jobMap.Store(uintptr(job.handle), mq)
if err := attachIOCP(job.handle, ioCompletionPort); err != nil {
jobMap.Delete(uintptr(job.handle))
return nil, fmt.Errorf("failed to attach job to IO completion port: %w", err)
}
return mq, nil
}
// PollNotification will poll for a job object notification. This call should only be called once
// per job (ideally in a goroutine loop) and will block if there is not a notification ready.
// This call will return immediately with error `ErrNotRegistered` if the job was not registered
// to receive notifications during `Create`. Internally, messages will be queued and there
// is no worry of messages being dropped.
func (job *JobObject) PollNotification() (interface{}, error) {
if job.mq == nil {
return nil, ErrNotRegistered
}
return job.mq.ReadOrWait()
}
// UpdateProcThreadAttribute updates the passed in ProcThreadAttributeList to contain what is necessary to
// launch a process in a job at creation time. This can be used to avoid having to call Assign() after a process
// has already started running.
func (job *JobObject) UpdateProcThreadAttribute(attrList *windows.ProcThreadAttributeListContainer) error {
job.handleLock.RLock()
defer job.handleLock.RUnlock()
if job.handle == 0 {
return ErrAlreadyClosed
}
if err := attrList.Update(
winapi.PROC_THREAD_ATTRIBUTE_JOB_LIST,
unsafe.Pointer(&job.handle),
unsafe.Sizeof(job.handle),
); err != nil {
return fmt.Errorf("failed to update proc thread attributes for job object: %w", err)
}
return nil
}
// Close closes the job object handle.
func (job *JobObject) Close() error {
job.handleLock.Lock()
defer job.handleLock.Unlock()
if job.handle == 0 {
return ErrAlreadyClosed
}
if err := windows.Close(job.handle); err != nil {
return err
}
if job.mq != nil {
job.mq.Close()
}
// Handles now invalid so if the map entry to receive notifications for this job still
// exists remove it so we can stop receiving notifications.
if _, ok := jobMap.Load(uintptr(job.handle)); ok {
jobMap.Delete(uintptr(job.handle))
}
job.handle = 0
return nil
}
// Assign assigns a process to the job object.
func (job *JobObject) Assign(pid uint32) error {
job.handleLock.RLock()
defer job.handleLock.RUnlock()
if job.handle == 0 {
return ErrAlreadyClosed
}
if pid == 0 {
return errors.New("invalid pid: 0")
}
hProc, err := windows.OpenProcess(winapi.PROCESS_ALL_ACCESS, true, pid)
if err != nil {
return err
}
defer windows.Close(hProc)
return windows.AssignProcessToJobObject(job.handle, hProc)
}
// Terminate terminates the job, essentially calls TerminateProcess on every process in the
// job.
func (job *JobObject) Terminate(exitCode uint32) error {
job.handleLock.RLock()
defer job.handleLock.RUnlock()
if job.handle == 0 {
return ErrAlreadyClosed
}
return windows.TerminateJobObject(job.handle, exitCode)
}
// Pids returns all of the process IDs in the job object.
func (job *JobObject) Pids() ([]uint32, error) {
job.handleLock.RLock()
defer job.handleLock.RUnlock()
if job.handle == 0 {
return nil, ErrAlreadyClosed
}
info := winapi.JOBOBJECT_BASIC_PROCESS_ID_LIST{}
err := winapi.QueryInformationJobObject(
job.handle,
winapi.JobObjectBasicProcessIdList,
uintptr(unsafe.Pointer(&info)),
uint32(unsafe.Sizeof(info)),
nil,
)
// This is either the case where there is only one process or no processes in
// the job. Any other case will result in ERROR_MORE_DATA. Check if info.NumberOfProcessIdsInList
// is 1 and just return this, otherwise return an empty slice.
if err == nil {
if info.NumberOfProcessIdsInList == 1 {
return []uint32{uint32(info.ProcessIdList[0])}, nil
}
// Return empty slice instead of nil to play well with the caller of this.
// Do not return an error if no processes are running inside the job
return []uint32{}, nil
}
if err != winapi.ERROR_MORE_DATA {
return nil, fmt.Errorf("failed initial query for PIDs in job object: %w", err)
}
jobBasicProcessIDListSize := unsafe.Sizeof(info) + (unsafe.Sizeof(info.ProcessIdList[0]) * uintptr(info.NumberOfAssignedProcesses-1))
buf := make([]byte, jobBasicProcessIDListSize)
if err = winapi.QueryInformationJobObject(
job.handle,
winapi.JobObjectBasicProcessIdList,
uintptr(unsafe.Pointer(&buf[0])),
uint32(len(buf)),
nil,
); err != nil {
return nil, fmt.Errorf("failed to query for PIDs in job object: %w", err)
}
bufInfo := (*winapi.JOBOBJECT_BASIC_PROCESS_ID_LIST)(unsafe.Pointer(&buf[0]))
pids := make([]uint32, bufInfo.NumberOfProcessIdsInList)
for i, bufPid := range bufInfo.AllPids() {
pids[i] = uint32(bufPid)
}
return pids, nil
}
// QueryMemoryStats gets the memory stats for the job object.
func (job *JobObject) QueryMemoryStats() (*winapi.JOBOBJECT_MEMORY_USAGE_INFORMATION, error) {
job.handleLock.RLock()
defer job.handleLock.RUnlock()
if job.handle == 0 {
return nil, ErrAlreadyClosed
}
info := winapi.JOBOBJECT_MEMORY_USAGE_INFORMATION{}
if err := winapi.QueryInformationJobObject(
job.handle,
winapi.JobObjectMemoryUsageInformation,
uintptr(unsafe.Pointer(&info)),
uint32(unsafe.Sizeof(info)),
nil,
); err != nil {
return nil, fmt.Errorf("failed to query for job object memory stats: %w", err)
}
return &info, nil
}
// QueryProcessorStats gets the processor stats for the job object.
func (job *JobObject) QueryProcessorStats() (*winapi.JOBOBJECT_BASIC_ACCOUNTING_INFORMATION, error) {
job.handleLock.RLock()
defer job.handleLock.RUnlock()
if job.handle == 0 {
return nil, ErrAlreadyClosed
}
info := winapi.JOBOBJECT_BASIC_ACCOUNTING_INFORMATION{}
if err := winapi.QueryInformationJobObject(
job.handle,
winapi.JobObjectBasicAccountingInformation,
uintptr(unsafe.Pointer(&info)),
uint32(unsafe.Sizeof(info)),
nil,
); err != nil {
return nil, fmt.Errorf("failed to query for job object process stats: %w", err)
}
return &info, nil
}
// QueryStorageStats gets the storage (I/O) stats for the job object.
func (job *JobObject) QueryStorageStats() (*winapi.JOBOBJECT_IO_ATTRIBUTION_INFORMATION, error) {
job.handleLock.RLock()
defer job.handleLock.RUnlock()
if job.handle == 0 {
return nil, ErrAlreadyClosed
}
info := winapi.JOBOBJECT_IO_ATTRIBUTION_INFORMATION{
ControlFlags: winapi.JOBOBJECT_IO_ATTRIBUTION_CONTROL_ENABLE,
}
if err := winapi.QueryInformationJobObject(
job.handle,
winapi.JobObjectIoAttribution,
uintptr(unsafe.Pointer(&info)),
uint32(unsafe.Sizeof(info)),
nil,
); err != nil {
return nil, fmt.Errorf("failed to query for job object storage stats: %w", err)
}
return &info, nil
}
// QueryPrivateWorkingSet returns the private working set size for the job. This is calculated by adding up the
// private working set for every process running in the job.
func (job *JobObject) QueryPrivateWorkingSet() (uint64, error) {
pids, err := job.Pids()
if err != nil {
return 0, err
}
openAndQueryWorkingSet := func(pid uint32) (uint64, error) {
h, err := windows.OpenProcess(windows.PROCESS_QUERY_LIMITED_INFORMATION, false, pid)
if err != nil {
// Continue to the next if OpenProcess doesn't return a valid handle (fails). Handles a
// case where one of the pids in the job exited before we open.
return 0, nil
}
defer func() {
_ = windows.Close(h)
}()
// Check if the process is actually running in the job still. There's a small chance
// that the process could have exited and had its pid re-used between grabbing the pids
// in the job and opening the handle to it above.
var inJob int32
if err := winapi.IsProcessInJob(h, job.handle, &inJob); err != nil {
// This shouldn't fail unless we have incorrect access rights which we control
// here so probably best to error out if this failed.
return 0, err
}
// Don't report stats for this process as it's not running in the job. This shouldn't be
// an error condition though.
if inJob == 0 {
return 0, nil
}
var vmCounters winapi.VM_COUNTERS_EX2
status := winapi.NtQueryInformationProcess(
h,
winapi.ProcessVmCounters,
uintptr(unsafe.Pointer(&vmCounters)),
uint32(unsafe.Sizeof(vmCounters)),
nil,
)
if !winapi.NTSuccess(status) {
return 0, fmt.Errorf("failed to query information for process: %w", winapi.RtlNtStatusToDosError(status))
}
return uint64(vmCounters.PrivateWorkingSetSize), nil
}
var jobWorkingSetSize uint64
for _, pid := range pids {
workingSet, err := openAndQueryWorkingSet(pid)
if err != nil {
return 0, err
}
jobWorkingSetSize += workingSet
}
return jobWorkingSetSize, nil
}

View File

@@ -0,0 +1,315 @@
package jobobject
import (
"errors"
"fmt"
"unsafe"
"github.com/Microsoft/hcsshim/internal/winapi"
"golang.org/x/sys/windows"
)
const (
memoryLimitMax uint64 = 0xffffffffffffffff
)
func isFlagSet(flag, controlFlags uint32) bool {
return (flag & controlFlags) == flag
}
// SetResourceLimits sets resource limits on the job object (cpu, memory, storage).
func (job *JobObject) SetResourceLimits(limits *JobLimits) error {
// Go through and check what limits were specified and apply them to the job.
if limits.MemoryLimitInBytes != 0 {
if err := job.SetMemoryLimit(limits.MemoryLimitInBytes); err != nil {
return fmt.Errorf("failed to set job object memory limit: %w", err)
}
}
if limits.CPULimit != 0 {
if err := job.SetCPULimit(RateBased, limits.CPULimit); err != nil {
return fmt.Errorf("failed to set job object cpu limit: %w", err)
}
} else if limits.CPUWeight != 0 {
if err := job.SetCPULimit(WeightBased, limits.CPUWeight); err != nil {
return fmt.Errorf("failed to set job object cpu limit: %w", err)
}
}
if limits.MaxBandwidth != 0 || limits.MaxIOPS != 0 {
if err := job.SetIOLimit(limits.MaxBandwidth, limits.MaxIOPS); err != nil {
return fmt.Errorf("failed to set io limit on job object: %w", err)
}
}
return nil
}
// SetTerminateOnLastHandleClose sets the job object flag that specifies that the job should terminate
// all processes in the job on the last open handle being closed.
func (job *JobObject) SetTerminateOnLastHandleClose() error {
info, err := job.getExtendedInformation()
if err != nil {
return err
}
info.BasicLimitInformation.LimitFlags |= windows.JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE
return job.setExtendedInformation(info)
}
// SetMemoryLimit sets the memory limit of the job object based on the given `memoryLimitInBytes`.
func (job *JobObject) SetMemoryLimit(memoryLimitInBytes uint64) error {
if memoryLimitInBytes >= memoryLimitMax {
return errors.New("memory limit specified exceeds the max size")
}
info, err := job.getExtendedInformation()
if err != nil {
return err
}
info.JobMemoryLimit = uintptr(memoryLimitInBytes)
info.BasicLimitInformation.LimitFlags |= windows.JOB_OBJECT_LIMIT_JOB_MEMORY
return job.setExtendedInformation(info)
}
// GetMemoryLimit gets the memory limit in bytes of the job object.
func (job *JobObject) GetMemoryLimit() (uint64, error) {
info, err := job.getExtendedInformation()
if err != nil {
return 0, err
}
return uint64(info.JobMemoryLimit), nil
}
// SetCPULimit sets the CPU limit depending on the specified `CPURateControlType` to
// `rateControlValue` for the job object.
func (job *JobObject) SetCPULimit(rateControlType CPURateControlType, rateControlValue uint32) error {
cpuInfo, err := job.getCPURateControlInformation()
if err != nil {
return err
}
switch rateControlType {
case WeightBased:
if rateControlValue < cpuWeightMin || rateControlValue > cpuWeightMax {
return fmt.Errorf("processor weight value of `%d` is invalid", rateControlValue)
}
cpuInfo.ControlFlags |= winapi.JOB_OBJECT_CPU_RATE_CONTROL_ENABLE | winapi.JOB_OBJECT_CPU_RATE_CONTROL_WEIGHT_BASED
cpuInfo.Value = rateControlValue
case RateBased:
if rateControlValue < cpuLimitMin || rateControlValue > cpuLimitMax {
return fmt.Errorf("processor rate of `%d` is invalid", rateControlValue)
}
cpuInfo.ControlFlags |= winapi.JOB_OBJECT_CPU_RATE_CONTROL_ENABLE | winapi.JOB_OBJECT_CPU_RATE_CONTROL_HARD_CAP
cpuInfo.Value = rateControlValue
default:
return errors.New("invalid job object cpu rate control type")
}
return job.setCPURateControlInfo(cpuInfo)
}
// GetCPULimit gets the cpu limits for the job object.
// `rateControlType` is used to indicate what type of cpu limit to query for.
func (job *JobObject) GetCPULimit(rateControlType CPURateControlType) (uint32, error) {
info, err := job.getCPURateControlInformation()
if err != nil {
return 0, err
}
if !isFlagSet(winapi.JOB_OBJECT_CPU_RATE_CONTROL_ENABLE, info.ControlFlags) {
return 0, errors.New("the job does not have cpu rate control enabled")
}
switch rateControlType {
case WeightBased:
if !isFlagSet(winapi.JOB_OBJECT_CPU_RATE_CONTROL_WEIGHT_BASED, info.ControlFlags) {
return 0, errors.New("cannot get cpu weight for job object without cpu weight option set")
}
case RateBased:
if !isFlagSet(winapi.JOB_OBJECT_CPU_RATE_CONTROL_HARD_CAP, info.ControlFlags) {
return 0, errors.New("cannot get cpu rate hard cap for job object without cpu rate hard cap option set")
}
default:
return 0, errors.New("invalid job object cpu rate control type")
}
return info.Value, nil
}
// SetCPUAffinity sets the processor affinity for the job object.
// The affinity is passed in as a bitmask.
func (job *JobObject) SetCPUAffinity(affinityBitMask uint64) error {
info, err := job.getExtendedInformation()
if err != nil {
return err
}
info.BasicLimitInformation.LimitFlags |= uint32(windows.JOB_OBJECT_LIMIT_AFFINITY)
info.BasicLimitInformation.Affinity = uintptr(affinityBitMask)
return job.setExtendedInformation(info)
}
// GetCPUAffinity gets the processor affinity for the job object.
// The returned affinity is a bitmask.
func (job *JobObject) GetCPUAffinity() (uint64, error) {
info, err := job.getExtendedInformation()
if err != nil {
return 0, err
}
return uint64(info.BasicLimitInformation.Affinity), nil
}
// SetIOLimit sets the IO limits specified on the job object.
func (job *JobObject) SetIOLimit(maxBandwidth, maxIOPS int64) error {
ioInfo, err := job.getIOLimit()
if err != nil {
return err
}
ioInfo.ControlFlags |= winapi.JOB_OBJECT_IO_RATE_CONTROL_ENABLE
if maxBandwidth != 0 {
ioInfo.MaxBandwidth = maxBandwidth
}
if maxIOPS != 0 {
ioInfo.MaxIops = maxIOPS
}
return job.setIORateControlInfo(ioInfo)
}
// GetIOMaxBandwidthLimit gets the max bandwidth for the job object.
func (job *JobObject) GetIOMaxBandwidthLimit() (int64, error) {
info, err := job.getIOLimit()
if err != nil {
return 0, err
}
return info.MaxBandwidth, nil
}
// GetIOMaxIopsLimit gets the max iops for the job object.
func (job *JobObject) GetIOMaxIopsLimit() (int64, error) {
info, err := job.getIOLimit()
if err != nil {
return 0, err
}
return info.MaxIops, nil
}
// Helper function for getting a job object's extended information.
func (job *JobObject) getExtendedInformation() (*windows.JOBOBJECT_EXTENDED_LIMIT_INFORMATION, error) {
job.handleLock.RLock()
defer job.handleLock.RUnlock()
if job.handle == 0 {
return nil, ErrAlreadyClosed
}
info := windows.JOBOBJECT_EXTENDED_LIMIT_INFORMATION{}
if err := winapi.QueryInformationJobObject(
job.handle,
windows.JobObjectExtendedLimitInformation,
uintptr(unsafe.Pointer(&info)),
uint32(unsafe.Sizeof(info)),
nil,
); err != nil {
return nil, fmt.Errorf("query %v returned error: %w", info, err)
}
return &info, nil
}
// Helper function for getting a job object's CPU rate control information.
func (job *JobObject) getCPURateControlInformation() (*winapi.JOBOBJECT_CPU_RATE_CONTROL_INFORMATION, error) {
job.handleLock.RLock()
defer job.handleLock.RUnlock()
if job.handle == 0 {
return nil, ErrAlreadyClosed
}
info := winapi.JOBOBJECT_CPU_RATE_CONTROL_INFORMATION{}
if err := winapi.QueryInformationJobObject(
job.handle,
windows.JobObjectCpuRateControlInformation,
uintptr(unsafe.Pointer(&info)),
uint32(unsafe.Sizeof(info)),
nil,
); err != nil {
return nil, fmt.Errorf("query %v returned error: %w", info, err)
}
return &info, nil
}
// Helper function for setting a job object's extended information.
func (job *JobObject) setExtendedInformation(info *windows.JOBOBJECT_EXTENDED_LIMIT_INFORMATION) error {
job.handleLock.RLock()
defer job.handleLock.RUnlock()
if job.handle == 0 {
return ErrAlreadyClosed
}
if _, err := windows.SetInformationJobObject(
job.handle,
windows.JobObjectExtendedLimitInformation,
uintptr(unsafe.Pointer(info)),
uint32(unsafe.Sizeof(*info)),
); err != nil {
return fmt.Errorf("failed to set Extended info %v on job object: %w", info, err)
}
return nil
}
// Helper function for querying job handle for IO limit information.
func (job *JobObject) getIOLimit() (*winapi.JOBOBJECT_IO_RATE_CONTROL_INFORMATION, error) {
job.handleLock.RLock()
defer job.handleLock.RUnlock()
if job.handle == 0 {
return nil, ErrAlreadyClosed
}
ioInfo := &winapi.JOBOBJECT_IO_RATE_CONTROL_INFORMATION{}
var blockCount uint32 = 1
if _, err := winapi.QueryIoRateControlInformationJobObject(
job.handle,
nil,
&ioInfo,
&blockCount,
); err != nil {
return nil, fmt.Errorf("query %v returned error: %w", ioInfo, err)
}
if !isFlagSet(winapi.JOB_OBJECT_IO_RATE_CONTROL_ENABLE, ioInfo.ControlFlags) {
return nil, fmt.Errorf("query %v cannot get IO limits for job object without IO rate control option set", ioInfo)
}
return ioInfo, nil
}
// Helper function for setting a job object's IO rate control information.
func (job *JobObject) setIORateControlInfo(ioInfo *winapi.JOBOBJECT_IO_RATE_CONTROL_INFORMATION) error {
job.handleLock.RLock()
defer job.handleLock.RUnlock()
if job.handle == 0 {
return ErrAlreadyClosed
}
if _, err := winapi.SetIoRateControlInformationJobObject(job.handle, ioInfo); err != nil {
return fmt.Errorf("failed to set IO limit info %v on job object: %w", ioInfo, err)
}
return nil
}
// Helper function for setting a job object's CPU rate control information.
func (job *JobObject) setCPURateControlInfo(cpuInfo *winapi.JOBOBJECT_CPU_RATE_CONTROL_INFORMATION) error {
job.handleLock.RLock()
defer job.handleLock.RUnlock()
if job.handle == 0 {
return ErrAlreadyClosed
}
if _, err := windows.SetInformationJobObject(
job.handle,
windows.JobObjectCpuRateControlInformation,
uintptr(unsafe.Pointer(cpuInfo)),
uint32(unsafe.Sizeof(cpuInfo)),
); err != nil {
return fmt.Errorf("failed to set cpu limit info %v on job object: %w", cpuInfo, err)
}
return nil
}

View File

@@ -0,0 +1,111 @@
package queue
import (
"errors"
"sync"
)
var (
ErrQueueClosed = errors.New("the queue is closed for reading and writing")
ErrQueueEmpty = errors.New("the queue is empty")
)
// MessageQueue represents a threadsafe message queue to be used to retrieve or
// write messages to.
type MessageQueue struct {
m *sync.RWMutex
c *sync.Cond
messages []interface{}
closed bool
}
// NewMessageQueue returns a new MessageQueue.
func NewMessageQueue() *MessageQueue {
m := &sync.RWMutex{}
return &MessageQueue{
m: m,
c: sync.NewCond(m),
messages: []interface{}{},
}
}
// Write writes `msg` to the queue.
func (mq *MessageQueue) Write(msg interface{}) error {
mq.m.Lock()
defer mq.m.Unlock()
if mq.closed {
return ErrQueueClosed
}
mq.messages = append(mq.messages, msg)
// Signal a waiter that there is now a value available in the queue.
mq.c.Signal()
return nil
}
// Read will read a value from the queue if available, otherwise return an error.
func (mq *MessageQueue) Read() (interface{}, error) {
mq.m.Lock()
defer mq.m.Unlock()
if mq.closed {
return nil, ErrQueueClosed
}
if mq.isEmpty() {
return nil, ErrQueueEmpty
}
val := mq.messages[0]
mq.messages[0] = nil
mq.messages = mq.messages[1:]
return val, nil
}
// ReadOrWait will read a value from the queue if available, else it will wait for a
// value to become available. This will block forever if nothing gets written or until
// the queue gets closed.
func (mq *MessageQueue) ReadOrWait() (interface{}, error) {
mq.m.Lock()
if mq.closed {
mq.m.Unlock()
return nil, ErrQueueClosed
}
if mq.isEmpty() {
for !mq.closed && mq.isEmpty() {
mq.c.Wait()
}
mq.m.Unlock()
return mq.Read()
}
val := mq.messages[0]
mq.messages[0] = nil
mq.messages = mq.messages[1:]
mq.m.Unlock()
return val, nil
}
// IsEmpty returns if the queue is empty
func (mq *MessageQueue) IsEmpty() bool {
mq.m.RLock()
defer mq.m.RUnlock()
return len(mq.messages) == 0
}
// Nonexported empty check that doesn't lock so we can call this in Read and Write.
func (mq *MessageQueue) isEmpty() bool {
return len(mq.messages) == 0
}
// Close closes the queue for future writes or reads. Any attempts to read or write from the
// queue after close will return ErrQueueClosed. This is safe to call multiple times.
func (mq *MessageQueue) Close() {
mq.m.Lock()
defer mq.m.Unlock()
// Already closed
if mq.closed {
return
}
mq.messages = nil
mq.closed = true
// If there's anybody currently waiting on a value from ReadOrWait, we need to
// broadcast so the read(s) can return ErrQueueClosed.
mq.c.Broadcast()
}

View File

@@ -1,3 +0,0 @@
package winapi
//sys GetQueuedCompletionStatus(cphandle windows.Handle, qty *uint32, key *uintptr, overlapped **windows.Overlapped, timeout uint32) (err error)

View File

@@ -24,7 +24,10 @@ const (
// Access rights for creating or opening job objects.
//
// https://docs.microsoft.com/en-us/windows/win32/procthread/job-object-security-and-access-rights
const JOB_OBJECT_ALL_ACCESS = 0x1F001F
const (
JOB_OBJECT_QUERY = 0x0004
JOB_OBJECT_ALL_ACCESS = 0x1F001F
)
// IO limit flags
//
@@ -93,7 +96,7 @@ type JOBOBJECT_BASIC_PROCESS_ID_LIST struct {
// AllPids returns all the process Ids in the job object.
func (p *JOBOBJECT_BASIC_PROCESS_ID_LIST) AllPids() []uintptr {
return (*[(1 << 27) - 1]uintptr)(unsafe.Pointer(&p.ProcessIdList[0]))[:p.NumberOfProcessIdsInList]
return (*[(1 << 27) - 1]uintptr)(unsafe.Pointer(&p.ProcessIdList[0]))[:p.NumberOfProcessIdsInList:p.NumberOfProcessIdsInList]
}
// https://docs.microsoft.com/en-us/windows/win32/api/winnt/ns-winnt-jobobject_basic_accounting_information
@@ -162,7 +165,7 @@ type JOBOBJECT_ASSOCIATE_COMPLETION_PORT struct {
// PBOOL Result
// );
//
//sys IsProcessInJob(procHandle windows.Handle, jobHandle windows.Handle, result *bool) (err error) = kernel32.IsProcessInJob
//sys IsProcessInJob(procHandle windows.Handle, jobHandle windows.Handle, result *int32) (err error) = kernel32.IsProcessInJob
// BOOL QueryInformationJobObject(
// HANDLE hJob,

View File

@@ -6,3 +6,60 @@ const (
PROC_THREAD_ATTRIBUTE_PSEUDOCONSOLE = 0x20016
PROC_THREAD_ATTRIBUTE_JOB_LIST = 0x2000D
)
// ProcessVmCounters corresponds to the _VM_COUNTERS_EX and _VM_COUNTERS_EX2 structures.
const ProcessVmCounters = 3
// __kernel_entry NTSTATUS NtQueryInformationProcess(
// [in] HANDLE ProcessHandle,
// [in] PROCESSINFOCLASS ProcessInformationClass,
// [out] PVOID ProcessInformation,
// [in] ULONG ProcessInformationLength,
// [out, optional] PULONG ReturnLength
// );
//
//sys NtQueryInformationProcess(processHandle windows.Handle, processInfoClass uint32, processInfo uintptr, processInfoLength uint32, returnLength *uint32) (status uint32) = ntdll.NtQueryInformationProcess
// typedef struct _VM_COUNTERS_EX
// {
// SIZE_T PeakVirtualSize;
// SIZE_T VirtualSize;
// ULONG PageFaultCount;
// SIZE_T PeakWorkingSetSize;
// SIZE_T WorkingSetSize;
// SIZE_T QuotaPeakPagedPoolUsage;
// SIZE_T QuotaPagedPoolUsage;
// SIZE_T QuotaPeakNonPagedPoolUsage;
// SIZE_T QuotaNonPagedPoolUsage;
// SIZE_T PagefileUsage;
// SIZE_T PeakPagefileUsage;
// SIZE_T PrivateUsage;
// } VM_COUNTERS_EX, *PVM_COUNTERS_EX;
//
type VM_COUNTERS_EX struct {
PeakVirtualSize uintptr
VirtualSize uintptr
PageFaultCount uint32
PeakWorkingSetSize uintptr
WorkingSetSize uintptr
QuotaPeakPagedPoolUsage uintptr
QuotaPagedPoolUsage uintptr
QuotaPeakNonPagedPoolUsage uintptr
QuotaNonPagedPoolUsage uintptr
PagefileUsage uintptr
PeakPagefileUsage uintptr
PrivateUsage uintptr
}
// typedef struct _VM_COUNTERS_EX2
// {
// VM_COUNTERS_EX CountersEx;
// SIZE_T PrivateWorkingSetSize;
// SIZE_T SharedCommitUsage;
// } VM_COUNTERS_EX2, *PVM_COUNTERS_EX2;
//
type VM_COUNTERS_EX2 struct {
CountersEx VM_COUNTERS_EX
PrivateWorkingSetSize uintptr
SharedCommitUsage uintptr
}

View File

@@ -2,4 +2,4 @@
// be thought of as an extension to golang.org/x/sys/windows.
package winapi
//go:generate go run ..\..\mksyscall_windows.go -output zsyscall_windows.go console.go system.go net.go path.go thread.go iocp.go jobobject.go logon.go memory.go process.go processor.go devices.go filesystem.go errors.go
//go:generate go run ..\..\mksyscall_windows.go -output zsyscall_windows.go user.go console.go system.go net.go path.go thread.go jobobject.go logon.go memory.go process.go processor.go devices.go filesystem.go errors.go

View File

@@ -50,7 +50,6 @@ var (
procSetJobCompartmentId = modiphlpapi.NewProc("SetJobCompartmentId")
procSearchPathW = modkernel32.NewProc("SearchPathW")
procCreateRemoteThread = modkernel32.NewProc("CreateRemoteThread")
procGetQueuedCompletionStatus = modkernel32.NewProc("GetQueuedCompletionStatus")
procIsProcessInJob = modkernel32.NewProc("IsProcessInJob")
procQueryInformationJobObject = modkernel32.NewProc("QueryInformationJobObject")
procOpenJobObjectW = modkernel32.NewProc("OpenJobObjectW")
@@ -61,6 +60,7 @@ var (
procLogonUserW = modadvapi32.NewProc("LogonUserW")
procLocalAlloc = modkernel32.NewProc("LocalAlloc")
procLocalFree = modkernel32.NewProc("LocalFree")
procNtQueryInformationProcess = modntdll.NewProc("NtQueryInformationProcess")
procGetActiveProcessorCount = modkernel32.NewProc("GetActiveProcessorCount")
procCM_Get_Device_ID_List_SizeA = modcfgmgr32.NewProc("CM_Get_Device_ID_List_SizeA")
procCM_Get_Device_ID_ListA = modcfgmgr32.NewProc("CM_Get_Device_ID_ListA")
@@ -140,19 +140,7 @@ func CreateRemoteThread(process windows.Handle, sa *windows.SecurityAttributes,
return
}
func GetQueuedCompletionStatus(cphandle windows.Handle, qty *uint32, key *uintptr, overlapped **windows.Overlapped, timeout uint32) (err error) {
r1, _, e1 := syscall.Syscall6(procGetQueuedCompletionStatus.Addr(), 5, uintptr(cphandle), uintptr(unsafe.Pointer(qty)), uintptr(unsafe.Pointer(key)), uintptr(unsafe.Pointer(overlapped)), uintptr(timeout), 0)
if r1 == 0 {
if e1 != 0 {
err = errnoErr(e1)
} else {
err = syscall.EINVAL
}
}
return
}
func IsProcessInJob(procHandle windows.Handle, jobHandle windows.Handle, result *bool) (err error) {
func IsProcessInJob(procHandle windows.Handle, jobHandle windows.Handle, result *int32) (err error) {
r1, _, e1 := syscall.Syscall(procIsProcessInJob.Addr(), 3, uintptr(procHandle), uintptr(jobHandle), uintptr(unsafe.Pointer(result)))
if r1 == 0 {
if e1 != 0 {
@@ -256,6 +244,12 @@ func LocalFree(ptr uintptr) {
return
}
func NtQueryInformationProcess(processHandle windows.Handle, processInfoClass uint32, processInfo uintptr, processInfoLength uint32, returnLength *uint32) (status uint32) {
r0, _, _ := syscall.Syscall6(procNtQueryInformationProcess.Addr(), 5, uintptr(processHandle), uintptr(processInfoClass), uintptr(processInfo), uintptr(processInfoLength), uintptr(unsafe.Pointer(returnLength)), 0)
status = uint32(r0)
return
}
func GetActiveProcessorCount(groupNumber uint16) (amount uint32) {
r0, _, _ := syscall.Syscall(procGetActiveProcessorCount.Addr(), 1, uintptr(groupNumber), 0, 0)
amount = uint32(r0)

View File

@@ -32,6 +32,7 @@ import (
"fmt"
"io"
"os"
"path/filepath"
"reflect"
"sort"
"strings"
@@ -1313,6 +1314,18 @@ func testWriteAndOpen(t *testing.T, controllers ...TestingController) {
),
wantFailOnLossLess: true,
},
{
name: "hardlink should be replaced to the destination entry",
in: tarOf(
dir("foo/"),
file("foo/foo1", "test"),
link("foolink", "foo/foo1"),
),
wantNumGz: 4, // dir, foo1 + link, TOC, footer
want: checks(
mustSameEntry("foo/foo1", "foolink"),
),
},
}
for _, tt := range tests {
@@ -1730,6 +1743,60 @@ func hasEntryOwner(entry string, owner owner) stargzCheck {
})
}
func mustSameEntry(files ...string) stargzCheck {
return stargzCheckFn(func(t *testing.T, r *Reader) {
var first *TOCEntry
for _, f := range files {
if first == nil {
var ok bool
first, ok = r.Lookup(f)
if !ok {
t.Errorf("unknown first file on Lookup: %q", f)
return
}
}
// Test Lookup
e, ok := r.Lookup(f)
if !ok {
t.Errorf("unknown file on Lookup: %q", f)
return
}
if e != first {
t.Errorf("Lookup: %+v(%p) != %+v(%p)", e, e, first, first)
return
}
// Test LookupChild
pe, ok := r.Lookup(filepath.Dir(filepath.Clean(f)))
if !ok {
t.Errorf("failed to get parent of %q", f)
return
}
e, ok = pe.LookupChild(filepath.Base(filepath.Clean(f)))
if !ok {
t.Errorf("failed to get %q as the child of %+v", f, pe)
return
}
if e != first {
t.Errorf("LookupChild: %+v(%p) != %+v(%p)", e, e, first, first)
return
}
// Test ForeachChild
pe.ForeachChild(func(baseName string, e *TOCEntry) bool {
if baseName == filepath.Base(filepath.Clean(f)) {
if e != first {
t.Errorf("ForeachChild: %+v(%p) != %+v(%p)", e, e, first, first)
return false
}
}
return true
})
}
})
}
func tarOf(s ...tarEntry) []tarEntry { return s }
type tarEntry interface {

View File

@@ -159,7 +159,8 @@ type TOCEntry struct {
// NumLink is the number of entry names pointing to this entry.
// Zero means one name references this entry.
NumLink int
// This field is calculated during runtime and not recorded in TOC JSON.
NumLink int `json:"-"`
// Xattrs are the extended attribute for the entry.
Xattrs map[string][]byte `json:"xattrs,omitempty"`

View File

@@ -3,6 +3,7 @@ package auth
import (
"bufio"
"context"
"errors"
"fmt"
"net/url"
"os"
@@ -14,7 +15,6 @@ import (
"github.com/containers/image/v5/pkg/docker/config"
"github.com/containers/image/v5/pkg/sysregistriesv2"
"github.com/containers/image/v5/types"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
terminal "golang.org/x/term"
)
@@ -26,8 +26,8 @@ func GetDefaultAuthFile() string {
if authfile := os.Getenv("REGISTRY_AUTH_FILE"); authfile != "" {
return authfile
}
if auth_env := os.Getenv("DOCKER_CONFIG"); auth_env != "" {
return filepath.Join(auth_env, "config.json")
if authEnv := os.Getenv("DOCKER_CONFIG"); authEnv != "" {
return filepath.Join(authEnv, "config.json")
}
return ""
}
@@ -39,7 +39,7 @@ func CheckAuthFile(authfile string) error {
return nil
}
if _, err := os.Stat(authfile); err != nil {
return errors.Wrap(err, "checking authfile")
return fmt.Errorf("checking authfile: %w", err)
}
return nil
}
@@ -97,12 +97,12 @@ func Login(ctx context.Context, systemContext *types.SystemContext, opts *LoginO
authConfig, err := config.GetCredentials(systemContext, key)
if err != nil {
return errors.Wrap(err, "get credentials")
return fmt.Errorf("get credentials: %w", err)
}
if opts.GetLoginSet {
if authConfig.Username == "" {
return errors.Errorf("not logged into %s", key)
return fmt.Errorf("not logged into %s", key)
}
fmt.Fprintf(opts.Stdout, "%s\n", authConfig.Username)
return nil
@@ -139,7 +139,7 @@ func Login(ctx context.Context, systemContext *types.SystemContext, opts *LoginO
username, password, err := getUserAndPass(opts, password, authConfig.Username)
if err != nil {
return errors.Wrap(err, "getting username and password")
return fmt.Errorf("getting username and password: %w", err)
}
if err = docker.CheckAuth(ctx, systemContext, username, password, registry); err == nil {
@@ -158,9 +158,9 @@ func Login(ctx context.Context, systemContext *types.SystemContext, opts *LoginO
}
if unauthorized, ok := err.(docker.ErrUnauthorizedForCredentials); ok {
logrus.Debugf("error logging into %q: %v", key, unauthorized)
return errors.Errorf("error logging into %q: invalid username/password", key)
return fmt.Errorf("error logging into %q: invalid username/password", key)
}
return errors.Wrapf(err, "authenticating creds for %q", key)
return fmt.Errorf("authenticating creds for %q: %w", key, err)
}
// parseCredentialsKey turns the provided argument into a valid credential key
@@ -191,10 +191,10 @@ func parseCredentialsKey(arg string, acceptRepositories bool) (key, registry str
// Ideally c/image should provide dedicated validation functionality.
ref, err := reference.ParseNormalizedNamed(key)
if err != nil {
return "", "", errors.Wrapf(err, "parse reference from %q", key)
return "", "", fmt.Errorf("parse reference from %q: %w", key, err)
}
if !reference.IsNameOnly(ref) {
return "", "", errors.Errorf("reference %q contains tag or digest", ref.String())
return "", "", fmt.Errorf("reference %q contains tag or digest", ref.String())
}
refRegistry := reference.Domain(ref)
if refRegistry != registry { // This should never happen, check just to make sure
@@ -232,7 +232,7 @@ func getUserAndPass(opts *LoginOptions, password, userFromAuthFile string) (user
}
username, err = reader.ReadString('\n')
if err != nil {
return "", "", errors.Wrap(err, "reading username")
return "", "", fmt.Errorf("reading username: %w", err)
}
// If the user just hit enter, use the displayed user from the
// the authentication file. This allows to do a lazy
@@ -246,7 +246,7 @@ func getUserAndPass(opts *LoginOptions, password, userFromAuthFile string) (user
fmt.Fprint(opts.Stdout, "Password: ")
pass, err := terminal.ReadPassword(int(os.Stdin.Fd()))
if err != nil {
return "", "", errors.Wrap(err, "reading password")
return "", "", fmt.Errorf("reading password: %w", err)
}
password = string(pass)
fmt.Fprintln(opts.Stdout)
@@ -298,14 +298,15 @@ func Logout(systemContext *types.SystemContext, opts *LogoutOptions, args []stri
}
err = config.RemoveAuthentication(systemContext, key)
switch errors.Cause(err) {
case nil:
if err == nil {
fmt.Fprintf(opts.Stdout, "Removed login credentials for %s\n", key)
return nil
case config.ErrNotLoggedIn:
}
if errors.Is(err, config.ErrNotLoggedIn) {
authConfig, err := config.GetCredentials(systemContext, key)
if err != nil {
return errors.Wrap(err, "get credentials")
return fmt.Errorf("get credentials: %w", err)
}
authInvalid := docker.CheckAuth(context.Background(), systemContext, authConfig.Username, authConfig.Password, registry)
@@ -313,10 +314,10 @@ func Logout(systemContext *types.SystemContext, opts *LogoutOptions, args []stri
fmt.Printf("Not logged into %s with current tool. Existing credentials were established via docker login. Please use docker logout instead.\n", key)
return nil
}
return errors.Errorf("Not logged into %s\n", key)
default:
return errors.Wrapf(err, "logging out of %q", key)
return fmt.Errorf("not logged into %s", key)
}
return fmt.Errorf("logging out of %q: %w", key, err)
}
// defaultRegistryWhenUnspecified returns first registry from search list of registry.conf
@@ -324,7 +325,7 @@ func Logout(systemContext *types.SystemContext, opts *LogoutOptions, args []stri
func defaultRegistryWhenUnspecified(systemContext *types.SystemContext) (string, error) {
registriesFromFile, err := sysregistriesv2.UnqualifiedSearchRegistries(systemContext)
if err != nil {
return "", errors.Wrap(err, "getting registry from registry.conf, please specify a registry")
return "", fmt.Errorf("getting registry from registry.conf, please specify a registry: %w", err)
}
if len(registriesFromFile) == 0 {
return "", errors.New("no registries found in registries.conf, a registry must be provided")

View File

@@ -6,11 +6,12 @@ package capabilities
// changed significantly to fit the needs of libpod.
import (
"errors"
"fmt"
"sort"
"strings"
"sync"
"github.com/pkg/errors"
"github.com/syndtr/gocapability/capability"
)
@@ -104,8 +105,8 @@ func AllCapabilities() []string {
// NormalizeCapabilities normalizes caps by adding a "CAP_" prefix (if not yet
// present).
func NormalizeCapabilities(caps []string) ([]string, error) {
normalized := make([]string, len(caps))
for i, c := range caps {
normalized := make([]string, 0, len(caps))
for _, c := range caps {
c = strings.ToUpper(c)
if c == All {
normalized = append(normalized, c)
@@ -115,9 +116,9 @@ func NormalizeCapabilities(caps []string) ([]string, error) {
c = "CAP_" + c
}
if !stringInSlice(c, capabilityList) {
return nil, errors.Wrapf(ErrUnknownCapability, "%q", c)
return nil, fmt.Errorf("%q: %w", c, ErrUnknownCapability)
}
normalized[i] = c
normalized = append(normalized, c)
}
sort.Strings(normalized)
return normalized, nil
@@ -127,7 +128,7 @@ func NormalizeCapabilities(caps []string) ([]string, error) {
func ValidateCapabilities(caps []string) error {
for _, c := range caps {
if !stringInSlice(c, capabilityList) {
return errors.Wrapf(ErrUnknownCapability, "%q", c)
return fmt.Errorf("%q: %w", c, ErrUnknownCapability)
}
}
return nil
@@ -140,8 +141,6 @@ func ValidateCapabilities(caps []string) error {
// "ALL" in capAdd adds returns known capabilities
// "All" in capDrop returns only the capabilities specified in capAdd
func MergeCapabilities(base, adds, drops []string) ([]string, error) {
var caps []string
// Normalize the base capabilities
base, err := NormalizeCapabilities(base)
if err != nil {
@@ -178,17 +177,18 @@ func MergeCapabilities(base, adds, drops []string) ([]string, error) {
} else {
for _, add := range capAdd {
if stringInSlice(add, capDrop) {
return nil, errors.Errorf("capability %q cannot be dropped and added", add)
return nil, fmt.Errorf("capability %q cannot be dropped and added", add)
}
}
}
for _, drop := range capDrop {
if stringInSlice(drop, capAdd) {
return nil, errors.Errorf("capability %q cannot be dropped and added", drop)
return nil, fmt.Errorf("capability %q cannot be dropped and added", drop)
}
}
caps := make([]string, 0, len(base)+len(capAdd))
// Drop any capabilities in capDrop that are in base
for _, cap := range base {
if stringInSlice(cap, capDrop) {

View File

@@ -51,7 +51,7 @@ func AutocompleteCapabilities(cmd *cobra.Command, args []string, toComplete stri
offset = 4
}
var completions []string
completions := make([]string, 0, len(caps))
for _, cap := range caps {
completions = append(completions, convertCase(cap)[offset:])
}

View File

@@ -12,25 +12,32 @@ import (
"github.com/docker/distribution/registry/api/errcode"
errcodev2 "github.com/docker/distribution/registry/api/v2"
"github.com/hashicorp/go-multierror"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
)
// RetryOptions defines the option to retry
type RetryOptions struct {
MaxRetry int // The number of times to possibly retry
Delay time.Duration // The delay to use between retries, if set
// Options defines the option to retry.
type Options struct {
MaxRetry int // The number of times to possibly retry.
Delay time.Duration // The delay to use between retries, if set.
}
// RetryIfNecessary retries the operation in exponential backoff with the retryOptions
func RetryIfNecessary(ctx context.Context, operation func() error, retryOptions *RetryOptions) error {
// RetryOptions is deprecated, use Options.
type RetryOptions = Options // nolint:revive
// RetryIfNecessary deprecated function use IfNecessary.
func RetryIfNecessary(ctx context.Context, operation func() error, options *Options) error { // nolint:revive
return IfNecessary(ctx, operation, options)
}
// IfNecessary retries the operation in exponential backoff with the retry Options.
func IfNecessary(ctx context.Context, operation func() error, options *Options) error {
err := operation()
for attempt := 0; err != nil && isRetryable(err) && attempt < retryOptions.MaxRetry; attempt++ {
for attempt := 0; err != nil && isRetryable(err) && attempt < options.MaxRetry; attempt++ {
delay := time.Duration(int(math.Pow(2, float64(attempt)))) * time.Second
if retryOptions.Delay != 0 {
delay = retryOptions.Delay
if options.Delay != 0 {
delay = options.Delay
}
logrus.Warnf("Failed, retrying in %s ... (%d/%d). Error: %v", delay, attempt+1, retryOptions.MaxRetry, err)
logrus.Warnf("Failed, retrying in %s ... (%d/%d). Error: %v", delay, attempt+1, options.MaxRetry, err)
select {
case <-time.After(delay):
break
@@ -43,8 +50,6 @@ func RetryIfNecessary(ctx context.Context, operation func() error, retryOptions
}
func isRetryable(err error) bool {
err = errors.Cause(err)
switch err {
case nil:
return false
@@ -91,6 +96,14 @@ func isRetryable(err error) bool {
}
}
return true
case net.Error:
if e.Timeout() {
return true
}
if unwrappable, ok := e.(unwrapper); ok {
err = unwrappable.Unwrap()
return isRetryable(err)
}
case unwrapper: // Test this last, because various error types might implement .Unwrap()
err = e.Unwrap()
return isRetryable(err)

View File

@@ -9,7 +9,6 @@ import (
"github.com/containers/image/v5/internal/private"
compressiontypes "github.com/containers/image/v5/pkg/compression/types"
"github.com/containers/image/v5/types"
perrors "github.com/pkg/errors"
"github.com/sirupsen/logrus"
)
@@ -36,7 +35,7 @@ func (ic *imageCopier) copyBlobFromStream(ctx context.Context, srcReader io.Read
// read stream to the end, and validation does not happen.
digestingReader, err := newDigestingReader(stream.reader, srcInfo.Digest)
if err != nil {
return types.BlobInfo{}, perrors.Wrapf(err, "preparing to verify blob %s", srcInfo.Digest)
return types.BlobInfo{}, fmt.Errorf("preparing to verify blob %s: %w", srcInfo.Digest, err)
}
stream.reader = digestingReader
@@ -107,7 +106,7 @@ func (ic *imageCopier) copyBlobFromStream(ctx context.Context, srcReader io.Read
}
uploadedInfo, err := ic.c.dest.PutBlobWithOptions(ctx, &errorAnnotationReader{stream.reader}, stream.info, options)
if err != nil {
return types.BlobInfo{}, perrors.Wrap(err, "writing blob")
return types.BlobInfo{}, fmt.Errorf("writing blob: %w", err)
}
uploadedInfo.Annotations = stream.info.Annotations
@@ -126,7 +125,7 @@ func (ic *imageCopier) copyBlobFromStream(ctx context.Context, srcReader io.Read
logrus.Debugf("Consuming rest of the original blob to satisfy getOriginalLayerCopyWriter")
_, err := io.Copy(io.Discard, originalLayerReader)
if err != nil {
return types.BlobInfo{}, perrors.Wrapf(err, "reading input blob %s", srcInfo.Digest)
return types.BlobInfo{}, fmt.Errorf("reading input blob %s: %w", srcInfo.Digest, err)
}
}
@@ -165,8 +164,8 @@ type errorAnnotationReader struct {
// Read annotates the error happened during read
func (r errorAnnotationReader) Read(b []byte) (n int, err error) {
n, err = r.reader.Read(b)
if err != io.EOF {
return n, perrors.Wrapf(err, "happened during read")
if err != nil && err != io.EOF {
return n, fmt.Errorf("happened during read: %w", err)
}
return n, err
}

View File

@@ -9,7 +9,6 @@ import (
"github.com/containers/image/v5/pkg/compression"
compressiontypes "github.com/containers/image/v5/pkg/compression/types"
"github.com/containers/image/v5/types"
perrors "github.com/pkg/errors"
"github.com/sirupsen/logrus"
)
@@ -28,7 +27,7 @@ func blobPipelineDetectCompressionStep(stream *sourceStream, srcInfo types.BlobI
// This requires us to “peek ahead” into the stream to read the initial part, which requires us to chain through another io.Reader returned by DetectCompression.
format, decompressor, reader, err := compression.DetectCompressionFormat(stream.reader) // We could skip this in some cases, but let's keep the code path uniform
if err != nil {
return bpDetectCompressionStepData{}, perrors.Wrapf(err, "reading blob %s", srcInfo.Digest)
return bpDetectCompressionStepData{}, fmt.Errorf("reading blob %s: %w", srcInfo.Digest, err)
}
stream.reader = reader

View File

@@ -19,7 +19,6 @@ import (
"github.com/containers/image/v5/internal/imagesource"
"github.com/containers/image/v5/internal/pkg/platform"
"github.com/containers/image/v5/internal/private"
internalsig "github.com/containers/image/v5/internal/signature"
"github.com/containers/image/v5/manifest"
"github.com/containers/image/v5/pkg/blobinfocache"
"github.com/containers/image/v5/pkg/compression"
@@ -30,7 +29,6 @@ import (
encconfig "github.com/containers/ocicrypt/config"
digest "github.com/opencontainers/go-digest"
imgspecv1 "github.com/opencontainers/image-spec/specs-go/v1"
perrors "github.com/pkg/errors"
"github.com/sirupsen/logrus"
"github.com/vbauerster/mpb/v7"
"golang.org/x/sync/semaphore"
@@ -208,29 +206,37 @@ func Image(ctx context.Context, policyContext *signature.PolicyContext, destRef,
publicDest, err := destRef.NewImageDestination(ctx, options.DestinationCtx)
if err != nil {
return nil, perrors.Wrapf(err, "initializing destination %s", transports.ImageName(destRef))
return nil, fmt.Errorf("initializing destination %s: %w", transports.ImageName(destRef), err)
}
dest := imagedestination.FromPublic(publicDest)
defer func() {
if err := dest.Close(); err != nil {
retErr = perrors.Wrapf(retErr, " (dest: %v)", err)
if retErr != nil {
retErr = fmt.Errorf(" (dest: %v): %w", err, retErr)
} else {
retErr = fmt.Errorf(" (dest: %v)", err)
}
}
}()
publicRawSource, err := srcRef.NewImageSource(ctx, options.SourceCtx)
if err != nil {
return nil, perrors.Wrapf(err, "initializing source %s", transports.ImageName(srcRef))
return nil, fmt.Errorf("initializing source %s: %w", transports.ImageName(srcRef), err)
}
rawSource := imagesource.FromPublic(publicRawSource)
defer func() {
if err := rawSource.Close(); err != nil {
retErr = perrors.Wrapf(retErr, " (src: %v)", err)
if retErr != nil {
retErr = fmt.Errorf(" (src: %v): %w", err, retErr)
} else {
retErr = fmt.Errorf(" (src: %v)", err)
}
}
}()
// If reportWriter is not a TTY (e.g., when piping to a file), do not
// print the progress bars to avoid long and hard to parse output.
// createProgressBar() will print a single line instead.
// Instead use printCopyInfo() to print single line "Copying ..." messages.
progressOutput := reportWriter
if !isTTY(reportWriter) {
progressOutput = io.Discard
@@ -281,7 +287,7 @@ func Image(ctx context.Context, policyContext *signature.PolicyContext, destRef,
unparsedToplevel := image.UnparsedInstance(rawSource, nil)
multiImage, err := isMultiImage(ctx, unparsedToplevel)
if err != nil {
return nil, perrors.Wrapf(err, "determining manifest MIME type for %s", transports.ImageName(srcRef))
return nil, fmt.Errorf("determining manifest MIME type for %s: %w", transports.ImageName(srcRef), err)
}
if !multiImage {
@@ -294,21 +300,21 @@ func Image(ctx context.Context, policyContext *signature.PolicyContext, destRef,
// matches the current system to copy, and copy it.
mfest, manifestType, err := unparsedToplevel.Manifest(ctx)
if err != nil {
return nil, perrors.Wrapf(err, "reading manifest for %s", transports.ImageName(srcRef))
return nil, fmt.Errorf("reading manifest for %s: %w", transports.ImageName(srcRef), err)
}
manifestList, err := manifest.ListFromBlob(mfest, manifestType)
if err != nil {
return nil, perrors.Wrapf(err, "parsing primary manifest as list for %s", transports.ImageName(srcRef))
return nil, fmt.Errorf("parsing primary manifest as list for %s: %w", transports.ImageName(srcRef), err)
}
instanceDigest, err := manifestList.ChooseInstance(options.SourceCtx) // try to pick one that matches options.SourceCtx
if err != nil {
return nil, perrors.Wrapf(err, "choosing an image from manifest list %s", transports.ImageName(srcRef))
return nil, fmt.Errorf("choosing an image from manifest list %s: %w", transports.ImageName(srcRef), err)
}
logrus.Debugf("Source is a manifest list; copying (only) instance %s for current system", instanceDigest)
unparsedInstance := image.UnparsedInstance(rawSource, &instanceDigest)
if copiedManifest, _, _, err = c.copyOneImage(ctx, policyContext, options, unparsedToplevel, unparsedInstance, nil); err != nil {
return nil, perrors.Wrap(err, "copying system image from manifest list")
return nil, fmt.Errorf("copying system image from manifest list: %w", err)
}
} else { /* options.ImageListSelection == CopyAllImages or options.ImageListSelection == CopySpecificImages, */
// If we were asked to copy multiple images and can't, that's an error.
@@ -328,7 +334,7 @@ func Image(ctx context.Context, policyContext *signature.PolicyContext, destRef,
}
if err := c.dest.Commit(ctx, unparsedToplevel); err != nil {
return nil, perrors.Wrap(err, "committing the finished image")
return nil, fmt.Errorf("committing the finished image: %w", err)
}
return copiedManifest, nil
@@ -355,7 +361,7 @@ func supportsMultipleImages(dest types.ImageDestination) bool {
func compareImageDestinationManifestEqual(ctx context.Context, options *Options, src *image.SourcedImage, targetInstance *digest.Digest, dest types.ImageDestination) (bool, []byte, string, digest.Digest, error) {
srcManifestDigest, err := manifest.Digest(src.ManifestBlob)
if err != nil {
return false, nil, "", "", perrors.Wrapf(err, "calculating manifest digest")
return false, nil, "", "", fmt.Errorf("calculating manifest digest: %w", err)
}
destImageSource, err := dest.Reference().NewImageSource(ctx, options.DestinationCtx)
@@ -372,7 +378,7 @@ func compareImageDestinationManifestEqual(ctx context.Context, options *Options,
destManifestDigest, err := manifest.Digest(destManifest)
if err != nil {
return false, nil, "", "", perrors.Wrapf(err, "calculating manifest digest")
return false, nil, "", "", fmt.Errorf("calculating manifest digest: %w", err)
}
logrus.Debugf("Comparing source and destination manifest digests: %v vs. %v", srcManifestDigest, destManifestDigest)
@@ -390,31 +396,19 @@ func (c *copier) copyMultipleImages(ctx context.Context, policyContext *signatur
// Parse the list and get a copy of the original value after it's re-encoded.
manifestList, manifestType, err := unparsedToplevel.Manifest(ctx)
if err != nil {
return nil, perrors.Wrapf(err, "reading manifest list")
return nil, fmt.Errorf("reading manifest list: %w", err)
}
originalList, err := manifest.ListFromBlob(manifestList, manifestType)
if err != nil {
return nil, perrors.Wrapf(err, "parsing manifest list %q", string(manifestList))
return nil, fmt.Errorf("parsing manifest list %q: %w", string(manifestList), err)
}
updatedList := originalList.Clone()
// Read and/or clear the set of signatures for this list.
var sigs []internalsig.Signature
if options.RemoveSignatures {
sigs = []internalsig.Signature{}
} else {
c.Printf("Getting image list signatures\n")
s, err := c.rawSource.GetSignaturesWithFormat(ctx, nil)
if err != nil {
return nil, perrors.Wrap(err, "reading signatures")
}
sigs = s
}
if len(sigs) != 0 {
c.Printf("Checking if image list destination supports signatures\n")
if err := c.dest.SupportsSignatures(ctx); err != nil {
return nil, perrors.Wrapf(err, "Can not copy signatures to %s", transports.ImageName(c.dest.Reference()))
}
sigs, err := c.sourceSignatures(ctx, unparsedToplevel, options,
"Getting image list signatures",
"Checking if image list destination supports signatures")
if err != nil {
return nil, err
}
// If the destination is a digested reference, make a note of that, determine what digest value we're
@@ -425,7 +419,7 @@ func (c *copier) copyMultipleImages(ctx context.Context, policyContext *signatur
destIsDigestedReference = true
matches, err := manifest.MatchesDigest(manifestList, digested.Digest())
if err != nil {
return nil, perrors.Wrapf(err, "computing digest of source image's manifest")
return nil, fmt.Errorf("computing digest of source image's manifest: %w", err)
}
if !matches {
return nil, errors.New("Digest of source image's manifest would not match destination reference")
@@ -457,7 +451,7 @@ func (c *copier) copyMultipleImages(ctx context.Context, policyContext *signatur
}
selectedListType, otherManifestMIMETypeCandidates, err := c.determineListConversion(manifestType, c.dest.SupportedManifestMIMETypes(), forceListMIMEType)
if err != nil {
return nil, perrors.Wrapf(err, "determining manifest list type to write to destination")
return nil, fmt.Errorf("determining manifest list type to write to destination: %w", err)
}
if selectedListType != originalList.MIMEType() {
if cannotModifyManifestListReason != "" {
@@ -499,7 +493,7 @@ func (c *copier) copyMultipleImages(ctx context.Context, policyContext *signatur
unparsedInstance := image.UnparsedInstance(c.rawSource, &instanceDigest)
updatedManifest, updatedManifestType, updatedManifestDigest, err := c.copyOneImage(ctx, policyContext, options, unparsedToplevel, unparsedInstance, &instanceDigest)
if err != nil {
return nil, perrors.Wrapf(err, "copying image %d/%d from manifest list", instancesCopied+1, imagesToCopy)
return nil, fmt.Errorf("copying image %d/%d from manifest list: %w", instancesCopied+1, imagesToCopy, err)
}
instancesCopied++
// Record the result of a possible conversion here.
@@ -513,7 +507,7 @@ func (c *copier) copyMultipleImages(ctx context.Context, policyContext *signatur
// Now reset the digest/size/types of the manifests in the list to account for any conversions that we made.
if err = updatedList.UpdateInstances(updates); err != nil {
return nil, perrors.Wrapf(err, "updating manifest list")
return nil, fmt.Errorf("updating manifest list: %w", err)
}
// Iterate through supported list types, preferred format first.
@@ -528,7 +522,7 @@ func (c *copier) copyMultipleImages(ctx context.Context, policyContext *signatur
if thisListType != updatedList.MIMEType() {
attemptedList, err = updatedList.ConvertToMIMEType(thisListType)
if err != nil {
return nil, perrors.Wrapf(err, "converting manifest list to list with MIME type %q", thisListType)
return nil, fmt.Errorf("converting manifest list to list with MIME type %q: %w", thisListType, err)
}
}
@@ -536,11 +530,11 @@ func (c *copier) copyMultipleImages(ctx context.Context, policyContext *signatur
// by serializing them both so that we can compare them.
attemptedManifestList, err := attemptedList.Serialize()
if err != nil {
return nil, perrors.Wrapf(err, "encoding updated manifest list (%q: %#v)", updatedList.MIMEType(), updatedList.Instances())
return nil, fmt.Errorf("encoding updated manifest list (%q: %#v): %w", updatedList.MIMEType(), updatedList.Instances(), err)
}
originalManifestList, err := originalList.Serialize()
if err != nil {
return nil, perrors.Wrapf(err, "encoding original manifest list for comparison (%q: %#v)", originalList.MIMEType(), originalList.Instances())
return nil, fmt.Errorf("encoding original manifest list for comparison (%q: %#v): %w", originalList.MIMEType(), originalList.Instances(), err)
}
// If we can't just use the original value, but we have to change it, flag an error.
@@ -587,7 +581,7 @@ func (c *copier) copyMultipleImages(ctx context.Context, policyContext *signatur
c.Printf("Storing list signatures\n")
if err := c.dest.PutSignaturesWithFormat(ctx, sigs, nil); err != nil {
return nil, perrors.Wrap(err, "writing signatures")
return nil, fmt.Errorf("writing signatures: %w", err)
}
return manifestList, nil
@@ -601,7 +595,7 @@ func (c *copier) copyOneImage(ctx context.Context, policyContext *signature.Poli
multiImage, err := isMultiImage(ctx, unparsedImage)
if err != nil {
// FIXME FIXME: How to name a reference for the sub-image?
return nil, "", "", perrors.Wrapf(err, "determining manifest MIME type for %s", transports.ImageName(unparsedImage.Reference()))
return nil, "", "", fmt.Errorf("determining manifest MIME type for %s: %w", transports.ImageName(unparsedImage.Reference()), err)
}
if multiImage {
return nil, "", "", fmt.Errorf("Unexpectedly received a manifest list instead of a manifest for a single image")
@@ -611,11 +605,11 @@ func (c *copier) copyOneImage(ctx context.Context, policyContext *signature.Poli
// (The multiImage check above only matches the MIME type, which we have received anyway.
// Actual parsing of anything should be deferred.)
if allowed, err := policyContext.IsRunningImageAllowed(ctx, unparsedImage); !allowed || err != nil { // Be paranoid and fail if either return value indicates so.
return nil, "", "", perrors.Wrap(err, "Source image rejected")
return nil, "", "", fmt.Errorf("Source image rejected: %w", err)
}
src, err := image.FromUnparsedImage(ctx, options.SourceCtx, unparsedImage)
if err != nil {
return nil, "", "", perrors.Wrapf(err, "initializing image from source %s", transports.ImageName(c.rawSource.Reference()))
return nil, "", "", fmt.Errorf("initializing image from source %s: %w", transports.ImageName(c.rawSource.Reference()), err)
}
// If the destination is a digested reference, make a note of that, determine what digest value we're
@@ -627,16 +621,16 @@ func (c *copier) copyOneImage(ctx context.Context, policyContext *signature.Poli
destIsDigestedReference = true
matches, err := manifest.MatchesDigest(src.ManifestBlob, digested.Digest())
if err != nil {
return nil, "", "", perrors.Wrapf(err, "computing digest of source image's manifest")
return nil, "", "", fmt.Errorf("computing digest of source image's manifest: %w", err)
}
if !matches {
manifestList, _, err := unparsedToplevel.Manifest(ctx)
if err != nil {
return nil, "", "", perrors.Wrapf(err, "reading manifest from source image")
return nil, "", "", fmt.Errorf("reading manifest from source image: %w", err)
}
matches, err = manifest.MatchesDigest(manifestList, digested.Digest())
if err != nil {
return nil, "", "", perrors.Wrapf(err, "computing digest of source image's manifest")
return nil, "", "", fmt.Errorf("computing digest of source image's manifest: %w", err)
}
if !matches {
return nil, "", "", errors.New("Digest of source image's manifest would not match destination reference")
@@ -649,22 +643,11 @@ func (c *copier) copyOneImage(ctx context.Context, policyContext *signature.Poli
return nil, "", "", err
}
var sigs []internalsig.Signature
if options.RemoveSignatures {
sigs = []internalsig.Signature{}
} else {
c.Printf("Getting image source signatures\n")
s, err := src.UntrustedSignatures(ctx)
if err != nil {
return nil, "", "", perrors.Wrap(err, "reading signatures")
}
sigs = s
}
if len(sigs) != 0 {
c.Printf("Checking if image destination supports signatures\n")
if err := c.dest.SupportsSignatures(ctx); err != nil {
return nil, "", "", perrors.Wrapf(err, "Can not copy signatures to %s", transports.ImageName(c.dest.Reference()))
}
sigs, err := c.sourceSignatures(ctx, src, options,
"Getting image source signatures",
"Checking if image destination supports signatures")
if err != nil {
return nil, "", "", err
}
// Determine if we're allowed to modify the manifest.
@@ -778,7 +761,7 @@ func (c *copier) copyOneImage(ctx context.Context, policyContext *signature.Poli
// With ic.cannotModifyManifestReason != "", that would just be a string of repeated failures for the same reason,
// so lets bail out early and with a better error message.
if ic.cannotModifyManifestReason != "" {
return nil, "", "", perrors.Wrapf(err, "Writing manifest failed and we cannot try conversions: %q", cannotModifyManifestReason)
return nil, "", "", fmt.Errorf("writing manifest failed and we cannot try conversions: %q: %w", cannotModifyManifestReason, err)
}
// errs is a list of errors when trying various manifest types. Also serves as an "upload succeeded" flag when set to nil.
@@ -825,7 +808,7 @@ func (c *copier) copyOneImage(ctx context.Context, policyContext *signature.Poli
c.Printf("Storing signatures\n")
if err := c.dest.PutSignaturesWithFormat(ctx, sigs, targetInstance); err != nil {
return nil, "", "", perrors.Wrap(err, "writing signatures")
return nil, "", "", fmt.Errorf("writing signatures: %w", err)
}
return manifestBytes, retManifestType, retManifestDigest, nil
@@ -845,11 +828,11 @@ func checkImageDestinationForCurrentRuntime(ctx context.Context, sys *types.Syst
if dest.MustMatchRuntimeOS() {
c, err := src.OCIConfig(ctx)
if err != nil {
return perrors.Wrapf(err, "parsing image configuration")
return fmt.Errorf("parsing image configuration: %w", err)
}
wantedPlatforms, err := platform.WantedPlatforms(sys)
if err != nil {
return perrors.Wrapf(err, "getting current platform information %#v", sys)
return fmt.Errorf("getting current platform information %#v: %w", sys, err)
}
options := newOrderedSet()
@@ -1057,13 +1040,13 @@ func (ic *imageCopier) copyUpdatedConfigAndManifest(ctx context.Context, instanc
}
pi, err := ic.src.UpdatedImage(ctx, *ic.manifestUpdates)
if err != nil {
return nil, "", perrors.Wrap(err, "creating an updated image manifest")
return nil, "", fmt.Errorf("creating an updated image manifest: %w", err)
}
pendingImage = pi
}
man, _, err := pendingImage.Manifest(ctx)
if err != nil {
return nil, "", perrors.Wrap(err, "reading manifest")
return nil, "", fmt.Errorf("reading manifest: %w", err)
}
if err := ic.copyConfig(ctx, pendingImage); err != nil {
@@ -1080,7 +1063,7 @@ func (ic *imageCopier) copyUpdatedConfigAndManifest(ctx context.Context, instanc
}
if err := ic.c.dest.PutManifest(ctx, man, instanceDigest); err != nil {
logrus.Debugf("Error %v while writing manifest %q", err, string(man))
return nil, "", perrors.Wrapf(err, "writing manifest")
return nil, "", fmt.Errorf("writing manifest: %w", err)
}
return man, manifestDigest, nil
}
@@ -1100,10 +1083,11 @@ func (ic *imageCopier) copyConfig(ctx context.Context, src types.Image) error {
defer progressPool.Wait()
bar := ic.c.createProgressBar(progressPool, false, srcInfo, "config", "done")
defer bar.Abort(false)
ic.c.printCopyInfo("config", srcInfo)
configBlob, err := src.ConfigBlob(ctx)
if err != nil {
return types.BlobInfo{}, perrors.Wrapf(err, "reading config blob %s", srcInfo.Digest)
return types.BlobInfo{}, fmt.Errorf("reading config blob %s: %w", srcInfo.Digest, err)
}
destInfo, err := ic.copyBlobFromStream(ctx, bytes.NewReader(configBlob), srcInfo, nil, true, false, bar, -1, false)
@@ -1155,6 +1139,8 @@ func (ic *imageCopier) copyLayer(ctx context.Context, srcInfo types.BlobInfo, to
}
}
ic.c.printCopyInfo("blob", srcInfo)
cachedDiffID := ic.c.blobInfoCache.UncompressedDigest(srcInfo.Digest) // May be ""
diffIDIsNeeded := ic.diffIDsAreNeeded && cachedDiffID == ""
// When encrypting to decrypting, only use the simple code path. We might be able to optimize more
@@ -1183,7 +1169,7 @@ func (ic *imageCopier) copyLayer(ctx context.Context, srcInfo types.BlobInfo, to
SrcRef: srcRef,
})
if err != nil {
return types.BlobInfo{}, "", perrors.Wrapf(err, "trying to reuse blob %s at destination", srcInfo.Digest)
return types.BlobInfo{}, "", fmt.Errorf("trying to reuse blob %s at destination: %w", srcInfo.Digest, err)
}
if reused {
logrus.Debugf("Skipping blob %s (already present):", srcInfo.Digest)
@@ -1257,7 +1243,7 @@ func (ic *imageCopier) copyLayer(ctx context.Context, srcInfo types.BlobInfo, to
srcStream, srcBlobSize, err := ic.c.rawSource.GetBlob(ctx, srcInfo, ic.c.blobInfoCache)
if err != nil {
return types.BlobInfo{}, "", perrors.Wrapf(err, "reading blob %s", srcInfo.Digest)
return types.BlobInfo{}, "", fmt.Errorf("reading blob %s: %w", srcInfo.Digest, err)
}
defer srcStream.Close()
@@ -1273,7 +1259,7 @@ func (ic *imageCopier) copyLayer(ctx context.Context, srcInfo types.BlobInfo, to
return types.BlobInfo{}, "", ctx.Err()
case diffIDResult := <-diffIDChan:
if diffIDResult.err != nil {
return types.BlobInfo{}, "", perrors.Wrap(diffIDResult.err, "computing layer DiffID")
return types.BlobInfo{}, "", fmt.Errorf("computing layer DiffID: %w", diffIDResult.err)
}
logrus.Debugf("Computed DiffID %s for layer %s", diffIDResult.digest, srcInfo.Digest)
// Dont record any associations that involve encrypted data. This is a bit crude,

View File

@@ -6,7 +6,6 @@ import (
"io"
digest "github.com/opencontainers/go-digest"
perrors "github.com/pkg/errors"
)
type digestingReader struct {
@@ -48,7 +47,7 @@ func (d *digestingReader) Read(p []byte) (int, error) {
// Coverage: This should not happen, the hash.Hash interface requires
// d.digest.Write to never return an error, and the io.Writer interface
// requires n2 == len(input) if no error is returned.
return 0, perrors.Wrapf(err, "updating digest during verification: %d vs. %d", n2, n)
return 0, fmt.Errorf("updating digest during verification: %d vs. %d: %w", n2, n, err)
}
}
if err == io.EOF {

View File

@@ -1,12 +1,12 @@
package copy
import (
"fmt"
"strings"
"github.com/containers/image/v5/types"
"github.com/containers/ocicrypt"
imgspecv1 "github.com/opencontainers/image-spec/specs-go/v1"
perrors "github.com/pkg/errors"
)
// isOciEncrypted returns a bool indicating if a mediatype is encrypted
@@ -41,7 +41,7 @@ func (c *copier) blobPipelineDecryptionStep(stream *sourceStream, srcInfo types.
}
reader, decryptedDigest, err := ocicrypt.DecryptLayer(c.ociDecryptConfig, stream.reader, desc, false)
if err != nil {
return nil, perrors.Wrapf(err, "decrypting layer %s", srcInfo.Digest)
return nil, fmt.Errorf("decrypting layer %s: %w", srcInfo.Digest, err)
}
stream.reader = reader
@@ -92,7 +92,7 @@ func (c *copier) blobPipelineEncryptionStep(stream *sourceStream, toEncrypt bool
}
reader, finalizer, err := ocicrypt.EncryptLayer(c.ociEncryptConfig, stream.reader, desc)
if err != nil {
return nil, perrors.Wrapf(err, "encrypting blob %s", srcInfo.Digest)
return nil, fmt.Errorf("encrypting blob %s: %w", srcInfo.Digest, err)
}
stream.reader = reader
@@ -116,7 +116,7 @@ func (d *bpEncryptionStepData) updateCryptoOperationAndAnnotations(operation *ty
encryptAnnotations, err := d.finalizer()
if err != nil {
return perrors.Wrap(err, "Unable to finalize encryption")
return fmt.Errorf("Unable to finalize encryption: %w", err)
}
*operation = types.Encrypt
if *annotations == nil {

View File

@@ -38,7 +38,8 @@ type progressBar struct {
}
// createProgressBar creates a progressBar in pool. Note that if the copier's reportWriter
// is io.Discard, the progress bar's output will be discarded
// is io.Discard, the progress bar's output will be discarded. Callers may call printCopyInfo()
// to print a single line instead.
//
// NOTE: Every progress bar created within a progress pool must either successfully
// complete or be aborted, or pool.Wait() will hang. That is typically done
@@ -95,15 +96,21 @@ func (c *copier) createProgressBar(pool *mpb.Progress, partial bool, info types.
),
)
}
if c.progressOutput == io.Discard {
c.Printf("Copying %s %s\n", kind, info.Digest)
}
return &progressBar{
Bar: bar,
originalSize: info.Size,
}
}
// printCopyInfo prints a "Copying ..." message on the copier if the output is
// set to `io.Discard`. In that case, the progress bars won't be rendered but
// we still want to indicate when blobs and configs are copied.
func (c *copier) printCopyInfo(kind string, info types.BlobInfo) {
if c.progressOutput == io.Discard {
c.Printf("Copying %s %s\n", kind, info.Digest)
}
}
// mark100PercentComplete marks the progres bars as 100% complete;
// it may do so by possibly advancing the current state if it is below the known total.
func (bar *progressBar) mark100PercentComplete() {

View File

@@ -1,25 +1,51 @@
package copy
import (
"context"
"fmt"
"github.com/containers/image/v5/docker/reference"
"github.com/containers/image/v5/internal/private"
internalsig "github.com/containers/image/v5/internal/signature"
"github.com/containers/image/v5/signature"
"github.com/containers/image/v5/signature/sigstore"
"github.com/containers/image/v5/transports"
perrors "github.com/pkg/errors"
)
// sourceSignatures returns signatures from unparsedSource based on options,
// and verifies that they can be used (to avoid copying a large image when we
// can tell in advance that it would ultimately fail)
func (c *copier) sourceSignatures(ctx context.Context, unparsed private.UnparsedImage, options *Options,
gettingSignaturesMessage, checkingDestMessage string) ([]internalsig.Signature, error) {
var sigs []internalsig.Signature
if options.RemoveSignatures {
sigs = []internalsig.Signature{}
} else {
c.Printf("%s\n", gettingSignaturesMessage)
s, err := unparsed.UntrustedSignatures(ctx)
if err != nil {
return nil, fmt.Errorf("reading signatures: %w", err)
}
sigs = s
}
if len(sigs) != 0 {
c.Printf("%s\n", checkingDestMessage)
if err := c.dest.SupportsSignatures(ctx); err != nil {
return nil, fmt.Errorf("Can not copy signatures to %s: %w", transports.ImageName(c.dest.Reference()), err)
}
}
return sigs, nil
}
// createSignature creates a new signature of manifest using keyIdentity.
func (c *copier) createSignature(manifest []byte, keyIdentity string, passphrase string, identity reference.Named) (internalsig.Signature, error) {
mech, err := signature.NewGPGSigningMechanism()
if err != nil {
return nil, perrors.Wrap(err, "initializing GPG")
return nil, fmt.Errorf("initializing GPG: %w", err)
}
defer mech.Close()
if err := mech.SupportsSigning(); err != nil {
return nil, perrors.Wrap(err, "Signing not supported")
return nil, fmt.Errorf("Signing not supported: %w", err)
}
if identity != nil {
@@ -36,7 +62,7 @@ func (c *copier) createSignature(manifest []byte, keyIdentity string, passphrase
c.Printf("Signing manifest using simple signing\n")
newSig, err := signature.SignDockerManifestWithOptions(manifest, identity.String(), mech, keyIdentity, &signature.SignOptions{Passphrase: passphrase})
if err != nil {
return nil, perrors.Wrap(err, "creating signature")
return nil, fmt.Errorf("creating signature: %w", err)
}
return internalsig.SimpleSigningFromBlob(newSig), nil
}

View File

@@ -16,7 +16,6 @@ import (
"github.com/containers/image/v5/internal/signature"
"github.com/containers/image/v5/types"
"github.com/opencontainers/go-digest"
perrors "github.com/pkg/errors"
"github.com/sirupsen/logrus"
)
@@ -56,7 +55,7 @@ func newImageDestination(sys *types.SystemContext, ref dirReference) (private.Im
// if the contents don't match throw an error
dirExists, err := pathExists(ref.resolvedPath)
if err != nil {
return nil, perrors.Wrapf(err, "checking for path %q", ref.resolvedPath)
return nil, fmt.Errorf("checking for path %q: %w", ref.resolvedPath, err)
}
if dirExists {
isEmpty, err := isDirEmpty(ref.resolvedPath)
@@ -67,7 +66,7 @@ func newImageDestination(sys *types.SystemContext, ref dirReference) (private.Im
if !isEmpty {
versionExists, err := pathExists(ref.versionPath())
if err != nil {
return nil, perrors.Wrapf(err, "checking if path exists %q", ref.versionPath())
return nil, fmt.Errorf("checking if path exists %q: %w", ref.versionPath(), err)
}
if versionExists {
contents, err := os.ReadFile(ref.versionPath())
@@ -83,20 +82,20 @@ func newImageDestination(sys *types.SystemContext, ref dirReference) (private.Im
}
// delete directory contents so that only one image is in the directory at a time
if err = removeDirContents(ref.resolvedPath); err != nil {
return nil, perrors.Wrapf(err, "erasing contents in %q", ref.resolvedPath)
return nil, fmt.Errorf("erasing contents in %q: %w", ref.resolvedPath, err)
}
logrus.Debugf("overwriting existing container image directory %q", ref.resolvedPath)
}
} else {
// create directory if it doesn't exist
if err := os.MkdirAll(ref.resolvedPath, 0755); err != nil {
return nil, perrors.Wrapf(err, "unable to create directory %q", ref.resolvedPath)
return nil, fmt.Errorf("unable to create directory %q: %w", ref.resolvedPath, err)
}
}
// create version file
err = os.WriteFile(ref.versionPath(), []byte(version), 0644)
if err != nil {
return nil, perrors.Wrapf(err, "creating version file %q", ref.versionPath())
return nil, fmt.Errorf("creating version file %q: %w", ref.versionPath(), err)
}
d := &dirImageDestination{

View File

@@ -7,7 +7,6 @@ import (
"github.com/containers/image/v5/docker/reference"
"github.com/containers/image/v5/transports"
"github.com/containers/image/v5/types"
perrors "github.com/pkg/errors"
)
// Reader manages a single Docker archive, allows listing its contents and accessing
@@ -75,7 +74,7 @@ func (r *Reader) List() ([][]types.ImageReference, error) {
for _, tag := range image.RepoTags {
parsedTag, err := reference.ParseNormalizedNamed(tag)
if err != nil {
return nil, perrors.Wrapf(err, "Invalid tag %#v in manifest item @%d", tag, imageIndex)
return nil, fmt.Errorf("Invalid tag %#v in manifest item @%d: %w", tag, imageIndex, err)
}
nt, ok := parsedTag.(reference.NamedTagged)
if !ok {
@@ -83,14 +82,14 @@ func (r *Reader) List() ([][]types.ImageReference, error) {
}
ref, err := newReference(r.path, nt, -1, r.archive, nil)
if err != nil {
return nil, perrors.Wrapf(err, "creating a reference for tag %#v in manifest item @%d", tag, imageIndex)
return nil, fmt.Errorf("creating a reference for tag %#v in manifest item @%d: %w", tag, imageIndex, err)
}
refs = append(refs, ref)
}
if len(refs) == 0 {
ref, err := newReference(r.path, nil, imageIndex, r.archive, nil)
if err != nil {
return nil, perrors.Wrapf(err, "creating a reference for manifest item @%d", imageIndex)
return nil, fmt.Errorf("creating a reference for manifest item @%d: %w", imageIndex, err)
}
refs = append(refs, ref)
}

View File

@@ -12,7 +12,6 @@ import (
ctrImage "github.com/containers/image/v5/internal/image"
"github.com/containers/image/v5/transports"
"github.com/containers/image/v5/types"
perrors "github.com/pkg/errors"
)
func init() {
@@ -73,7 +72,7 @@ func ParseReference(refString string) (types.ImageReference, error) {
if len(parts[1]) > 0 && parts[1][0] == '@' {
i, err := strconv.Atoi(parts[1][1:])
if err != nil {
return nil, perrors.Wrapf(err, "Invalid source index %s", parts[1])
return nil, fmt.Errorf("Invalid source index %s: %w", parts[1], err)
}
if i < 0 {
return nil, fmt.Errorf("Invalid source index @%d: must not be negative", i)
@@ -82,7 +81,7 @@ func ParseReference(refString string) (types.ImageReference, error) {
} else {
ref, err := reference.ParseNormalizedNamed(parts[1])
if err != nil {
return nil, perrors.Wrapf(err, "docker-archive parsing reference")
return nil, fmt.Errorf("docker-archive parsing reference: %w", err)
}
ref = reference.TagNameOnly(ref)
refTagged, isTagged := ref.(reference.NamedTagged)

View File

@@ -2,13 +2,13 @@ package archive
import (
"errors"
"fmt"
"io"
"os"
"github.com/containers/image/v5/docker/internal/tarfile"
"github.com/containers/image/v5/docker/reference"
"github.com/containers/image/v5/types"
perrors "github.com/pkg/errors"
)
// Writer manages a single in-progress Docker archive and allows adding images to it.
@@ -61,7 +61,7 @@ func openArchiveForWriting(path string) (*os.File, error) {
// only in a different way. Either way, its up to the user to not have two writers to the same path.)
fh, err := os.OpenFile(path, os.O_WRONLY|os.O_CREATE, 0644)
if err != nil {
return nil, perrors.Wrapf(err, "opening file %q", path)
return nil, fmt.Errorf("opening file %q: %w", path, err)
}
succeeded := false
defer func() {
@@ -71,7 +71,7 @@ func openArchiveForWriting(path string) (*os.File, error) {
}()
fhStat, err := fh.Stat()
if err != nil {
return nil, perrors.Wrapf(err, "statting file %q", path)
return nil, fmt.Errorf("statting file %q: %w", path, err)
}
if fhStat.Mode().IsRegular() && fhStat.Size() != 0 {

View File

@@ -11,7 +11,6 @@ import (
"github.com/containers/image/v5/internal/private"
"github.com/containers/image/v5/types"
"github.com/docker/docker/client"
perrors "github.com/pkg/errors"
"github.com/sirupsen/logrus"
)
@@ -45,7 +44,7 @@ func newImageDestination(ctx context.Context, sys *types.SystemContext, ref daem
c, err := newDockerClient(sys)
if err != nil {
return nil, perrors.Wrap(err, "initializing docker engine client")
return nil, fmt.Errorf("initializing docker engine client: %w", err)
}
reader, writer := io.Pipe()
@@ -87,7 +86,7 @@ func imageLoadGoroutine(ctx context.Context, c *client.Client, reader *io.PipeRe
resp, err := c.ImageLoad(ctx, reader, true)
if err != nil {
err = perrors.Wrap(err, "saving image to docker engine")
err = fmt.Errorf("saving image to docker engine: %w", err)
return
}
defer resp.Body.Close()

View File

@@ -2,11 +2,11 @@ package daemon
import (
"context"
"fmt"
"github.com/containers/image/v5/docker/internal/tarfile"
"github.com/containers/image/v5/internal/private"
"github.com/containers/image/v5/types"
perrors "github.com/pkg/errors"
)
type daemonImageSource struct {
@@ -26,13 +26,13 @@ type daemonImageSource struct {
func newImageSource(ctx context.Context, sys *types.SystemContext, ref daemonReference) (private.ImageSource, error) {
c, err := newDockerClient(sys)
if err != nil {
return nil, perrors.Wrap(err, "initializing docker engine client")
return nil, fmt.Errorf("initializing docker engine client: %w", err)
}
// Per NewReference(), ref.StringWithinTransport() is either an image ID (config digest), or a !reference.NameOnly() reference.
// Either way ImageSave should create a tarball with exactly one image.
inputStream, err := c.ImageSave(ctx, []string{ref.StringWithinTransport()})
if err != nil {
return nil, perrors.Wrap(err, "loading image from docker engine")
return nil, fmt.Errorf("loading image from docker engine: %w", err)
}
defer inputStream.Close()

View File

@@ -1,6 +1,7 @@
package docker
import (
"bytes"
"context"
"crypto/tls"
"encoding/json"
@@ -31,7 +32,6 @@ import (
"github.com/docker/go-connections/tlsconfig"
digest "github.com/opencontainers/go-digest"
imgspecv1 "github.com/opencontainers/image-spec/specs-go/v1"
perrors "github.com/pkg/errors"
"github.com/sirupsen/logrus"
)
@@ -218,7 +218,7 @@ func dockerCertDir(sys *types.SystemContext, hostPort string) (string, error) {
func newDockerClientFromRef(sys *types.SystemContext, ref dockerReference, registryConfig *registryConfiguration, write bool, actions string) (*dockerClient, error) {
auth, err := config.GetCredentialsForRef(sys, ref.ref)
if err != nil {
return nil, perrors.Wrapf(err, "getting username and password")
return nil, fmt.Errorf("getting username and password: %w", err)
}
sigBase, err := registryConfig.lookasideStorageBaseURL(ref, write)
@@ -273,7 +273,7 @@ func newDockerClient(sys *types.SystemContext, registry, reference string) (*doc
skipVerify := false
reg, err := sysregistriesv2.FindRegistry(sys, reference)
if err != nil {
return nil, perrors.Wrapf(err, "loading registries")
return nil, fmt.Errorf("loading registries: %w", err)
}
if reg != nil {
if reg.Blocked {
@@ -301,7 +301,7 @@ func newDockerClient(sys *types.SystemContext, registry, reference string) (*doc
func CheckAuth(ctx context.Context, sys *types.SystemContext, username, password, registry string) error {
client, err := newDockerClient(sys, registry, registry)
if err != nil {
return perrors.Wrapf(err, "creating new docker client")
return fmt.Errorf("creating new docker client: %w", err)
}
client.auth = types.DockerAuthConfig{
Username: username,
@@ -350,7 +350,7 @@ func SearchRegistry(ctx context.Context, sys *types.SystemContext, registry, ima
// We can't use GetCredentialsForRef here because we want to search the whole registry.
auth, err := config.GetCredentials(sys, registry)
if err != nil {
return nil, perrors.Wrapf(err, "getting username and password")
return nil, fmt.Errorf("getting username and password: %w", err)
}
// The /v2/_catalog endpoint has been disabled for docker.io therefore
@@ -364,7 +364,7 @@ func SearchRegistry(ctx context.Context, sys *types.SystemContext, registry, ima
client, err := newDockerClient(sys, hostname, registry)
if err != nil {
return nil, perrors.Wrapf(err, "creating new docker client")
return nil, fmt.Errorf("creating new docker client: %w", err)
}
client.auth = auth
if sys != nil {
@@ -407,13 +407,13 @@ func SearchRegistry(ctx context.Context, sys *types.SystemContext, registry, ima
resp, err := client.makeRequest(ctx, http.MethodGet, path, nil, nil, v2Auth, nil)
if err != nil {
logrus.Debugf("error getting search results from v2 endpoint %q: %v", registry, err)
return nil, perrors.Wrapf(err, "couldn't search registry %q", registry)
return nil, fmt.Errorf("couldn't search registry %q: %w", registry, err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
err := httpResponseToError(resp, "")
logrus.Errorf("error getting search results from v2 endpoint %q: %v", registry, err)
return nil, perrors.Wrapf(err, "couldn't search registry %q", registry)
return nil, fmt.Errorf("couldn't search registry %q: %w", registry, err)
}
v2Res := &V2Results{}
if err := json.NewDecoder(resp.Body).Decode(v2Res); err != nil {
@@ -767,7 +767,7 @@ func (c *dockerClient) detectPropertiesHelper(ctx context.Context) error {
err = ping("http")
}
if err != nil {
err = perrors.Wrapf(err, "pinging container registry %s", c.registry)
err = fmt.Errorf("pinging container registry %s: %w", c.registry, err)
if c.sys != nil && c.sys.DockerDisableV1Ping {
return err
}
@@ -819,7 +819,7 @@ func (c *dockerClient) fetchManifest(ctx context.Context, ref dockerReference, t
logrus.Debugf("Content-Type from manifest GET is %q", res.Header.Get("Content-Type"))
defer res.Body.Close()
if res.StatusCode != http.StatusOK {
return nil, "", perrors.Wrapf(registryHTTPResponseToError(res), "reading manifest %s in %s", tagOrDigest, ref.ref.Name())
return nil, "", fmt.Errorf("reading manifest %s in %s: %w", tagOrDigest, ref.ref.Name(), registryHTTPResponseToError(res))
}
manblob, err := iolimits.ReadAtMost(res.Body, iolimits.MaxManifestBodySize)
@@ -922,15 +922,25 @@ func (c *dockerClient) getOCIDescriptorContents(ctx context.Context, ref dockerR
// isManifestUnknownError returns true iff err from fetchManifest is a “manifest unknown” error.
func isManifestUnknownError(err error) bool {
var errs errcode.Errors
if !errors.As(err, &errs) || len(errs) == 0 {
return false
if errors.As(err, &errs) && len(errs) != 0 {
firstErr := errs[0]
// docker/distribution, and as defined in the spec
var ec errcode.ErrorCoder
if errors.As(firstErr, &ec) && ec.ErrorCode() == v2.ErrorCodeManifestUnknown {
return true
}
// registry.redhat.io as of October 2022
var e errcode.Error
if errors.As(firstErr, &e) && e.ErrorCode() == errcode.ErrorCodeUnknown && e.Message == "Not Found" {
return true
}
}
err = errs[0]
ec, ok := err.(errcode.ErrorCoder)
if !ok {
return false
// ALSO registry.redhat.io as of October 2022
var unexpected *clientLib.UnexpectedHTTPResponseError
if errors.As(err, &unexpected) && unexpected.StatusCode == http.StatusNotFound && bytes.Contains(unexpected.Response, []byte("Not found")) {
return true
}
return ec.ErrorCode() == v2.ErrorCodeManifestUnknown
return false
}
// getSigstoreAttachmentManifest loads and parses the manifest for sigstore attachments for
@@ -978,7 +988,7 @@ func (c *dockerClient) getExtensionsSignatures(ctx context.Context, ref dockerRe
defer res.Body.Close()
if res.StatusCode != http.StatusOK {
return nil, perrors.Wrapf(clientLib.HandleErrorResponse(res), "downloading signatures for %s in %s", manifestDigest, ref.ref.Name())
return nil, fmt.Errorf("downloading signatures for %s in %s: %w", manifestDigest, ref.ref.Name(), clientLib.HandleErrorResponse(res))
}
body, err := iolimits.ReadAtMost(res.Body, iolimits.MaxSignatureListBodySize)
@@ -988,7 +998,7 @@ func (c *dockerClient) getExtensionsSignatures(ctx context.Context, ref dockerRe
var parsedBody extensionSignatureList
if err := json.Unmarshal(body, &parsedBody); err != nil {
return nil, perrors.Wrapf(err, "decoding signature list")
return nil, fmt.Errorf("decoding signature list: %w", err)
}
return &parsedBody, nil
}

View File

@@ -14,7 +14,6 @@ import (
"github.com/containers/image/v5/manifest"
"github.com/containers/image/v5/types"
"github.com/opencontainers/go-digest"
perrors "github.com/pkg/errors"
)
// Image is a Docker-specific implementation of types.ImageCloser with a few extra methods
@@ -67,7 +66,7 @@ func GetRepositoryTags(ctx context.Context, sys *types.SystemContext, ref types.
path := fmt.Sprintf(tagsPath, reference.Path(dr.ref))
client, err := newDockerClientFromRef(sys, dr, registryConfig, false, "pull")
if err != nil {
return nil, perrors.Wrap(err, "failed to create client")
return nil, fmt.Errorf("failed to create client: %w", err)
}
tags := make([]string, 0)
@@ -135,7 +134,7 @@ func GetDigest(ctx context.Context, sys *types.SystemContext, ref types.ImageRef
}
client, err := newDockerClientFromRef(sys, dr, registryConfig, false, "pull")
if err != nil {
return "", perrors.Wrap(err, "failed to create client")
return "", fmt.Errorf("failed to create client: %w", err)
}
path := fmt.Sprintf(manifestPath, reference.Path(dr.ref), tagOrDigest)
@@ -150,7 +149,7 @@ func GetDigest(ctx context.Context, sys *types.SystemContext, ref types.ImageRef
defer res.Body.Close()
if res.StatusCode != http.StatusOK {
return "", perrors.Wrapf(registryHTTPResponseToError(res), "reading digest %s in %s", tagOrDigest, dr.ref.Name())
return "", fmt.Errorf("reading digest %s in %s: %w", tagOrDigest, dr.ref.Name(), registryHTTPResponseToError(res))
}
dig, err := digest.Parse(res.Header.Get("Docker-Content-Digest"))

View File

@@ -31,7 +31,6 @@ import (
v2 "github.com/docker/distribution/registry/api/v2"
"github.com/opencontainers/go-digest"
imgspecv1 "github.com/opencontainers/image-spec/specs-go/v1"
perrors "github.com/pkg/errors"
"github.com/sirupsen/logrus"
)
@@ -167,11 +166,11 @@ func (d *dockerImageDestination) PutBlobWithOptions(ctx context.Context, stream
defer res.Body.Close()
if res.StatusCode != http.StatusAccepted {
logrus.Debugf("Error initiating layer upload, response %#v", *res)
return types.BlobInfo{}, perrors.Wrapf(registryHTTPResponseToError(res), "initiating layer upload to %s in %s", uploadPath, d.c.registry)
return types.BlobInfo{}, fmt.Errorf("initiating layer upload to %s in %s: %w", uploadPath, d.c.registry, registryHTTPResponseToError(res))
}
uploadLocation, err := res.Location()
if err != nil {
return types.BlobInfo{}, perrors.Wrap(err, "determining upload URL")
return types.BlobInfo{}, fmt.Errorf("determining upload URL: %w", err)
}
digester, stream := putblobdigest.DigestIfCanonicalUnknown(stream, inputInfo)
@@ -190,11 +189,11 @@ func (d *dockerImageDestination) PutBlobWithOptions(ctx context.Context, stream
}
defer res.Body.Close()
if !successStatus(res.StatusCode) {
return nil, perrors.Wrapf(registryHTTPResponseToError(res), "uploading layer chunked")
return nil, fmt.Errorf("uploading layer chunked: %w", registryHTTPResponseToError(res))
}
uploadLocation, err := res.Location()
if err != nil {
return nil, perrors.Wrap(err, "determining upload URL")
return nil, fmt.Errorf("determining upload URL: %w", err)
}
return uploadLocation, nil
}()
@@ -215,7 +214,7 @@ func (d *dockerImageDestination) PutBlobWithOptions(ctx context.Context, stream
defer res.Body.Close()
if res.StatusCode != http.StatusCreated {
logrus.Debugf("Error uploading layer, response %#v", *res)
return types.BlobInfo{}, perrors.Wrapf(registryHTTPResponseToError(res), "uploading layer to %s", uploadLocation)
return types.BlobInfo{}, fmt.Errorf("uploading layer to %s: %w", uploadLocation, registryHTTPResponseToError(res))
}
logrus.Debugf("Upload of layer %s complete", blobDigest)
@@ -240,7 +239,7 @@ func (d *dockerImageDestination) blobExists(ctx context.Context, repo reference.
return true, getBlobSize(res), nil
case http.StatusUnauthorized:
logrus.Debugf("... not authorized")
return false, -1, perrors.Wrapf(registryHTTPResponseToError(res), "checking whether a blob %s exists in %s", digest, repo.Name())
return false, -1, fmt.Errorf("checking whether a blob %s exists in %s: %w", digest, repo.Name(), registryHTTPResponseToError(res))
case http.StatusNotFound:
logrus.Debugf("... not present")
return false, -1, nil
@@ -274,7 +273,7 @@ func (d *dockerImageDestination) mountBlob(ctx context.Context, srcRepo referenc
// NOTE: This does not really work in docker/distribution servers, which incorrectly require the "delete" action in the token's scope, and is thus entirely untested.
uploadLocation, err := res.Location()
if err != nil {
return perrors.Wrap(err, "determining upload URL after a mount attempt")
return fmt.Errorf("determining upload URL after a mount attempt: %w", err)
}
logrus.Debugf("... started an upload instead of mounting, trying to cancel at %s", uploadLocation.Redacted())
res2, err := d.c.makeRequestToResolvedURL(ctx, http.MethodDelete, uploadLocation, nil, nil, -1, v2Auth, extraScope)
@@ -290,7 +289,7 @@ func (d *dockerImageDestination) mountBlob(ctx context.Context, srcRepo referenc
return fmt.Errorf("Mounting %s from %s to %s started an upload instead", srcDigest, srcRepo.Name(), d.ref.ref.Name())
default:
logrus.Debugf("Error mounting, response %#v", *res)
return perrors.Wrapf(registryHTTPResponseToError(res), "mounting %s from %s to %s", srcDigest, srcRepo.Name(), d.ref.ref.Name())
return fmt.Errorf("mounting %s from %s to %s: %w", srcDigest, srcRepo.Name(), d.ref.ref.Name(), registryHTTPResponseToError(res))
}
}
@@ -416,12 +415,12 @@ func (d *dockerImageDestination) PutManifest(ctx context.Context, m []byte, inst
// Double-check that the manifest we've been given matches the digest we've been given.
matches, err := manifest.MatchesDigest(m, *instanceDigest)
if err != nil {
return perrors.Wrapf(err, "digesting manifest in PutManifest")
return fmt.Errorf("digesting manifest in PutManifest: %w", err)
}
if !matches {
manifestDigest, merr := manifest.Digest(m)
if merr != nil {
return perrors.Wrapf(err, "Attempted to PutManifest using an explicitly specified digest (%q) that didn't match the manifest's digest (%v attempting to compute it)", instanceDigest.String(), merr)
return fmt.Errorf("Attempted to PutManifest using an explicitly specified digest (%q) that didn't match the manifest's digest: %w", instanceDigest.String(), merr)
}
return fmt.Errorf("Attempted to PutManifest using an explicitly specified digest (%q) that didn't match the manifest's digest (%q)", instanceDigest.String(), manifestDigest.String())
}
@@ -460,7 +459,7 @@ func (d *dockerImageDestination) uploadManifest(ctx context.Context, m []byte, t
defer res.Body.Close()
if !successStatus(res.StatusCode) {
rawErr := registryHTTPResponseToError(res)
err := perrors.Wrapf(rawErr, "uploading manifest %s to %s", tagOrDigest, d.ref.ref.Name())
err := fmt.Errorf("uploading manifest %s to %s: %w", tagOrDigest, d.ref.ref.Name(), rawErr)
if isManifestInvalidError(rawErr) {
err = types.ManifestTypeRejectedError{Err: err}
}
@@ -653,6 +652,7 @@ func (d *dockerImageDestination) putSignaturesToSigstoreAttachments(ctx context.
Digest: "", // We will fill this in later.
Size: 0,
}, nil)
ociConfig.RootFS.Type = "layers"
} else {
logrus.Debugf("Fetching sigstore attachment config %s", ociManifest.Config.Digest.String())
// We dont benefit from a real BlobInfoCache here because we never try to reuse/mount configs.
@@ -832,7 +832,7 @@ sigExists:
randBytes := make([]byte, 16)
n, err := rand.Read(randBytes)
if err != nil || n != 16 {
return perrors.Wrapf(err, "generating random signature len %d", n)
return fmt.Errorf("generating random signature len %d: %w", n, err)
}
signatureName = fmt.Sprintf("%s@%032x", manifestDigest.String(), randBytes)
if _, ok := existingSigNames[signatureName]; !ok {
@@ -858,7 +858,7 @@ sigExists:
defer res.Body.Close()
if res.StatusCode != http.StatusCreated {
logrus.Debugf("Error uploading signature, status %d, %#v", res.StatusCode, res)
return perrors.Wrapf(registryHTTPResponseToError(res), "uploading signature to %s in %s", path, d.c.registry)
return fmt.Errorf("uploading signature to %s in %s: %w", path, d.c.registry, registryHTTPResponseToError(res))
}
}

View File

@@ -25,7 +25,6 @@ import (
"github.com/containers/image/v5/pkg/sysregistriesv2"
"github.com/containers/image/v5/types"
digest "github.com/opencontainers/go-digest"
perrors "github.com/pkg/errors"
"github.com/sirupsen/logrus"
)
@@ -52,7 +51,7 @@ func newImageSource(ctx context.Context, sys *types.SystemContext, ref dockerRef
}
registry, err := sysregistriesv2.FindRegistry(sys, ref.ref.Name())
if err != nil {
return nil, perrors.Wrapf(err, "loading registries configuration")
return nil, fmt.Errorf("loading registries configuration: %w", err)
}
if registry == nil {
// No configuration was found for the provided reference, so use the
@@ -109,7 +108,7 @@ func newImageSource(ctx context.Context, sys *types.SystemContext, ref dockerRef
// The paired [] at least have some chance of being unambiguous.
extras = append(extras, fmt.Sprintf("[%s: %v]", attempts[i].ref.String(), attempts[i].err))
}
return nil, perrors.Wrapf(primary.err, "(Mirrors also failed: %s): %s", strings.Join(extras, "\n"), primary.ref.String())
return nil, fmt.Errorf("(Mirrors also failed: %s): %s: %w", strings.Join(extras, "\n"), primary.ref.String(), primary.err)
}
}

View File

@@ -54,7 +54,8 @@ func registryHTTPResponseToError(res *http.Response) error {
if len(response) > 50 {
response = response[:50] + "..."
}
err = fmt.Errorf("StatusCode: %d, %s", e.StatusCode, response)
// %.0w makes e visible to error.Unwrap() without including any text
err = fmt.Errorf("StatusCode: %d, %s%.0w", e.StatusCode, response, e)
}
return err
}

View File

@@ -5,6 +5,7 @@ import (
"context"
"encoding/json"
"errors"
"fmt"
"io"
"github.com/containers/image/v5/docker/reference"
@@ -16,7 +17,6 @@ import (
"github.com/containers/image/v5/manifest"
"github.com/containers/image/v5/types"
"github.com/opencontainers/go-digest"
perrors "github.com/pkg/errors"
"github.com/sirupsen/logrus"
)
@@ -108,11 +108,11 @@ func (d *Destination) PutBlobWithOptions(ctx context.Context, stream io.Reader,
if options.IsConfig {
buf, err := iolimits.ReadAtMost(stream, iolimits.MaxConfigBodySize)
if err != nil {
return types.BlobInfo{}, perrors.Wrap(err, "reading Config file stream")
return types.BlobInfo{}, fmt.Errorf("reading Config file stream: %w", err)
}
d.config = buf
if err := d.archive.sendFileLocked(d.archive.configPath(inputInfo.Digest), inputInfo.Size, bytes.NewReader(buf)); err != nil {
return types.BlobInfo{}, perrors.Wrap(err, "writing Config file")
return types.BlobInfo{}, fmt.Errorf("writing Config file: %w", err)
}
} else {
if err := d.archive.sendFileLocked(d.archive.physicalLayerPath(inputInfo.Digest), inputInfo.Size, stream); err != nil {
@@ -153,7 +153,7 @@ func (d *Destination) PutManifest(ctx context.Context, m []byte, instanceDigest
// so the caller trying a different manifest kind would be pointless.
var man manifest.Schema2
if err := json.Unmarshal(m, &man); err != nil {
return perrors.Wrap(err, "parsing manifest")
return fmt.Errorf("parsing manifest: %w", err)
}
if man.SchemaVersion != 2 || man.MediaType != manifest.DockerV2Schema2MediaType {
return errors.New("Unsupported manifest type, need a Docker schema 2 manifest")

View File

@@ -14,7 +14,6 @@ import (
"github.com/containers/image/v5/internal/tmpdir"
"github.com/containers/image/v5/pkg/compression"
"github.com/containers/image/v5/types"
perrors "github.com/pkg/errors"
)
// Reader is a ((docker save)-formatted) tar archive that allows random access to any component.
@@ -31,7 +30,7 @@ type Reader struct {
func NewReaderFromFile(sys *types.SystemContext, path string) (*Reader, error) {
file, err := os.Open(path)
if err != nil {
return nil, perrors.Wrapf(err, "opening file %q", path)
return nil, fmt.Errorf("opening file %q: %w", path, err)
}
defer file.Close()
@@ -39,7 +38,7 @@ func NewReaderFromFile(sys *types.SystemContext, path string) (*Reader, error) {
// as a source. Otherwise we pass the stream to NewReaderFromStream.
stream, isCompressed, err := compression.AutoDecompress(file)
if err != nil {
return nil, perrors.Wrapf(err, "detecting compression for file %q", path)
return nil, fmt.Errorf("detecting compression for file %q: %w", path, err)
}
defer stream.Close()
if !isCompressed {
@@ -56,7 +55,7 @@ func NewReaderFromStream(sys *types.SystemContext, inputStream io.Reader) (*Read
// Save inputStream to a temporary file
tarCopyFile, err := os.CreateTemp(tmpdir.TemporaryDirectoryForBigFiles(sys), "docker-tar")
if err != nil {
return nil, perrors.Wrap(err, "creating temporary file")
return nil, fmt.Errorf("creating temporary file: %w", err)
}
defer tarCopyFile.Close()
@@ -72,7 +71,7 @@ func NewReaderFromStream(sys *types.SystemContext, inputStream io.Reader) (*Read
// giving users really confusing "invalid tar header" errors).
uncompressedStream, _, err := compression.AutoDecompress(inputStream)
if err != nil {
return nil, perrors.Wrap(err, "auto-decompressing input")
return nil, fmt.Errorf("auto-decompressing input: %w", err)
}
defer uncompressedStream.Close()
@@ -81,7 +80,7 @@ func NewReaderFromStream(sys *types.SystemContext, inputStream io.Reader) (*Read
// TODO: This can take quite some time, and should ideally be cancellable
// using a context.Context.
if _, err := io.Copy(tarCopyFile, uncompressedStream); err != nil {
return nil, perrors.Wrapf(err, "copying contents to temporary file %q", tarCopyFile.Name())
return nil, fmt.Errorf("copying contents to temporary file %q: %w", tarCopyFile.Name(), err)
}
succeeded = true
@@ -114,7 +113,7 @@ func newReader(path string, removeOnClose bool) (*Reader, error) {
return nil, err
}
if err := json.Unmarshal(bytes, &r.Manifest); err != nil {
return nil, perrors.Wrap(err, "decoding tar manifest.json")
return nil, fmt.Errorf("decoding tar manifest.json: %w", err)
}
succeeded = true
@@ -147,7 +146,7 @@ func (r *Reader) ChooseManifestItem(ref reference.NamedTagged, sourceIndex int)
for tagIndex, tag := range r.Manifest[i].RepoTags {
parsedTag, err := reference.ParseNormalizedNamed(tag)
if err != nil {
return nil, -1, perrors.Wrapf(err, "Invalid tag %#v in manifest.json item @%d", tag, i)
return nil, -1, fmt.Errorf("Invalid tag %#v in manifest.json item @%d: %w", tag, i, err)
}
if parsedTag.String() == refString {
return &r.Manifest[i], tagIndex, nil
@@ -259,7 +258,7 @@ func findTarComponent(inputFile io.Reader, componentPath string) (*tar.Reader, *
func (r *Reader) readTarComponent(path string, limit int) ([]byte, error) {
file, err := r.openTarComponent(path)
if err != nil {
return nil, perrors.Wrapf(err, "loading tar component %s", path)
return nil, fmt.Errorf("loading tar component %s: %w", path, err)
}
defer file.Close()
bytes, err := iolimits.ReadAtMost(file, limit)

View File

@@ -20,7 +20,6 @@ import (
"github.com/containers/image/v5/pkg/compression"
"github.com/containers/image/v5/types"
digest "github.com/opencontainers/go-digest"
perrors "github.com/pkg/errors"
)
// Source is a partial implementation of types.ImageSource for reading from tarPath.
@@ -96,7 +95,7 @@ func (s *Source) ensureCachedDataIsPresentPrivate() error {
}
var parsedConfig manifest.Schema2Image // There's a lot of info there, but we only really care about layer DiffIDs.
if err := json.Unmarshal(configBytes, &parsedConfig); err != nil {
return perrors.Wrapf(err, "decoding tar config %s", tarManifest.Config)
return fmt.Errorf("decoding tar config %s: %w", tarManifest.Config, err)
}
if parsedConfig.RootFS == nil {
return fmt.Errorf("Invalid image config (rootFS is not set): %s", tarManifest.Config)
@@ -180,7 +179,7 @@ func (s *Source) prepareLayerData(tarManifest *ManifestItem, parsedConfig *manif
// the slower method of checking if it's compressed.
uncompressedStream, isCompressed, err := compression.AutoDecompress(t)
if err != nil {
return nil, perrors.Wrapf(err, "auto-decompressing %s to determine its size", layerPath)
return nil, fmt.Errorf("auto-decompressing %s to determine its size: %w", layerPath, err)
}
defer uncompressedStream.Close()
@@ -188,7 +187,7 @@ func (s *Source) prepareLayerData(tarManifest *ManifestItem, parsedConfig *manif
if isCompressed {
uncompressedSize, err = io.Copy(io.Discard, uncompressedStream)
if err != nil {
return nil, perrors.Wrapf(err, "reading %s to find its size", layerPath)
return nil, fmt.Errorf("reading %s to find its size: %w", layerPath, err)
}
}
li.size = uncompressedSize
@@ -303,7 +302,7 @@ func (s *Source) GetBlob(ctx context.Context, info types.BlobInfo, cache types.B
uncompressedStream, _, err := compression.AutoDecompress(underlyingStream)
if err != nil {
return nil, 0, perrors.Wrapf(err, "auto-decompressing blob %s", info.Digest)
return nil, 0, fmt.Errorf("auto-decompressing blob %s: %w", info.Digest, err)
}
newStream := uncompressedReadCloser{

View File

@@ -16,7 +16,6 @@ import (
"github.com/containers/image/v5/manifest"
"github.com/containers/image/v5/types"
"github.com/opencontainers/go-digest"
perrors "github.com/pkg/errors"
"github.com/sirupsen/logrus"
)
@@ -95,16 +94,16 @@ func (w *Writer) ensureSingleLegacyLayerLocked(layerID string, layerDigest diges
// See also the comment in physicalLayerPath.
physicalLayerPath := w.physicalLayerPath(layerDigest)
if err := w.sendSymlinkLocked(filepath.Join(layerID, legacyLayerFileName), filepath.Join("..", physicalLayerPath)); err != nil {
return perrors.Wrap(err, "creating layer symbolic link")
return fmt.Errorf("creating layer symbolic link: %w", err)
}
b := []byte("1.0")
if err := w.sendBytesLocked(filepath.Join(layerID, legacyVersionFileName), b); err != nil {
return perrors.Wrap(err, "writing VERSION file")
return fmt.Errorf("writing VERSION file: %w", err)
}
if err := w.sendBytesLocked(filepath.Join(layerID, legacyConfigFileName), configBytes); err != nil {
return perrors.Wrap(err, "writing config json file")
return fmt.Errorf("writing config json file: %w", err)
}
w.legacyLayers[layerID] = struct{}{}
@@ -129,7 +128,7 @@ func (w *Writer) writeLegacyMetadataLocked(layerDescriptors []manifest.Schema2De
var config map[string]*json.RawMessage
err := json.Unmarshal(configBytes, &config)
if err != nil {
return perrors.Wrap(err, "unmarshaling config")
return fmt.Errorf("unmarshaling config: %w", err)
}
for _, attr := range [7]string{"architecture", "config", "container", "container_config", "created", "docker_version", "os"} {
layerConfig[attr] = config[attr]
@@ -153,7 +152,7 @@ func (w *Writer) writeLegacyMetadataLocked(layerDescriptors []manifest.Schema2De
layerConfig["layer_id"] = chainID
b, err := json.Marshal(layerConfig) // Note that layerConfig["id"] is not set yet at this point.
if err != nil {
return perrors.Wrap(err, "marshaling layer config")
return fmt.Errorf("marshaling layer config: %w", err)
}
delete(layerConfig, "layer_id")
layerID := digest.Canonical.FromBytes(b).Hex()
@@ -161,7 +160,7 @@ func (w *Writer) writeLegacyMetadataLocked(layerDescriptors []manifest.Schema2De
configBytes, err := json.Marshal(layerConfig)
if err != nil {
return perrors.Wrap(err, "marshaling layer config")
return fmt.Errorf("marshaling layer config: %w", err)
}
if err := w.ensureSingleLegacyLayerLocked(layerID, l.Digest, configBytes); err != nil {
@@ -281,10 +280,10 @@ func (w *Writer) Close() error {
b, err = json.Marshal(w.repositories)
if err != nil {
return perrors.Wrap(err, "marshaling repositories")
return fmt.Errorf("marshaling repositories: %w", err)
}
if err := w.sendBytesLocked(legacyRepositoriesFileName, b); err != nil {
return perrors.Wrap(err, "writing config json file")
return fmt.Errorf("writing config json file: %w", err)
}
if err := w.tar.Close(); err != nil {

View File

@@ -15,7 +15,6 @@ import (
"github.com/containers/storage/pkg/homedir"
"github.com/ghodss/yaml"
"github.com/opencontainers/go-digest"
perrors "github.com/pkg/errors"
"github.com/sirupsen/logrus"
)
@@ -135,7 +134,7 @@ func loadAndMergeConfig(dirPath string) (*registryConfiguration, error) {
var config registryConfiguration
err = yaml.Unmarshal(configBytes, &config)
if err != nil {
return nil, perrors.Wrapf(err, "parsing %s", configPath)
return nil, fmt.Errorf("parsing %s: %w", configPath, err)
}
if config.DefaultDocker != nil {
@@ -168,7 +167,7 @@ func (config *registryConfiguration) lookasideStorageBaseURL(dr dockerReference,
if topLevel != "" {
u, err := url.Parse(topLevel)
if err != nil {
return nil, perrors.Wrapf(err, "Invalid signature storage URL %s", topLevel)
return nil, fmt.Errorf("Invalid signature storage URL %s: %w", topLevel, err)
}
url = u
} else {

View File

@@ -6,26 +6,25 @@ import (
"github.com/containers/image/v5/manifest"
"github.com/containers/image/v5/types"
perrors "github.com/pkg/errors"
)
func manifestSchema2FromManifestList(ctx context.Context, sys *types.SystemContext, src types.ImageSource, manblob []byte) (genericManifest, error) {
list, err := manifest.Schema2ListFromManifest(manblob)
if err != nil {
return nil, perrors.Wrapf(err, "parsing schema2 manifest list")
return nil, fmt.Errorf("parsing schema2 manifest list: %w", err)
}
targetManifestDigest, err := list.ChooseInstance(sys)
if err != nil {
return nil, perrors.Wrapf(err, "choosing image instance")
return nil, fmt.Errorf("choosing image instance: %w", err)
}
manblob, mt, err := src.GetManifest(ctx, &targetManifestDigest)
if err != nil {
return nil, perrors.Wrapf(err, "fetching target platform image selected from manifest list")
return nil, fmt.Errorf("fetching target platform image selected from manifest list: %w", err)
}
matches, err := manifest.MatchesDigest(manblob, targetManifestDigest)
if err != nil {
return nil, perrors.Wrap(err, "computing manifest digest")
return nil, fmt.Errorf("computing manifest digest: %w", err)
}
if !matches {
return nil, fmt.Errorf("Image manifest does not match selected manifest digest %s", targetManifestDigest)

View File

@@ -17,7 +17,6 @@ import (
"github.com/containers/image/v5/types"
"github.com/opencontainers/go-digest"
imgspecv1 "github.com/opencontainers/image-spec/specs-go/v1"
perrors "github.com/pkg/errors"
"github.com/sirupsen/logrus"
)
@@ -294,7 +293,7 @@ func (m *manifestSchema2) convertToManifestSchema1(ctx context.Context, options
// and anyway this blob is so small that its easier to just copy it than to worry about figuring out another location where to get it.
info, err := dest.PutBlob(ctx, bytes.NewReader(GzippedEmptyLayer), emptyLayerBlobInfo, none.NoCache, false)
if err != nil {
return nil, perrors.Wrap(err, "uploading empty layer")
return nil, fmt.Errorf("uploading empty layer: %w", err)
}
if info.Digest != emptyLayerBlobInfo.Digest {
return nil, fmt.Errorf("Internal error: Uploaded empty layer has digest %#v instead of %s", info.Digest, emptyLayerBlobInfo.Digest)

View File

@@ -6,26 +6,25 @@ import (
"github.com/containers/image/v5/manifest"
"github.com/containers/image/v5/types"
perrors "github.com/pkg/errors"
)
func manifestOCI1FromImageIndex(ctx context.Context, sys *types.SystemContext, src types.ImageSource, manblob []byte) (genericManifest, error) {
index, err := manifest.OCI1IndexFromManifest(manblob)
if err != nil {
return nil, perrors.Wrapf(err, "parsing OCI1 index")
return nil, fmt.Errorf("parsing OCI1 index: %w", err)
}
targetManifestDigest, err := index.ChooseInstance(sys)
if err != nil {
return nil, perrors.Wrapf(err, "choosing image instance")
return nil, fmt.Errorf("choosing image instance: %w", err)
}
manblob, mt, err := src.GetManifest(ctx, &targetManifestDigest)
if err != nil {
return nil, perrors.Wrapf(err, "fetching target platform image selected from image index")
return nil, fmt.Errorf("fetching target platform image selected from image index: %w", err)
}
matches, err := manifest.MatchesDigest(manblob, targetManifestDigest)
if err != nil {
return nil, perrors.Wrap(err, "computing manifest digest")
return nil, fmt.Errorf("computing manifest digest: %w", err)
}
if !matches {
return nil, fmt.Errorf("Image manifest does not match selected manifest digest %s", targetManifestDigest)

View File

@@ -11,7 +11,6 @@ import (
"github.com/containers/image/v5/manifest"
"github.com/containers/image/v5/types"
"github.com/opencontainers/go-digest"
perrors "github.com/pkg/errors"
)
// UnparsedImage implements types.UnparsedImage .
@@ -61,7 +60,7 @@ func (i *UnparsedImage) Manifest(ctx context.Context) ([]byte, string, error) {
if digest, haveDigest := i.expectedManifestDigest(); haveDigest {
matches, err := manifest.MatchesDigest(m, digest)
if err != nil {
return nil, "", perrors.Wrap(err, "computing manifest digest")
return nil, "", fmt.Errorf("computing manifest digest: %w", err)
}
if !matches {
return nil, "", fmt.Errorf("Manifest does not match provided manifest digest %s", digest)

View File

@@ -12,7 +12,6 @@ import (
"github.com/containers/image/v5/types"
"github.com/docker/docker/api/types/versions"
"github.com/opencontainers/go-digest"
perrors "github.com/pkg/errors"
)
// Schema1FSLayers is an entry of the "fsLayers" array in docker/distribution schema 1.
@@ -115,7 +114,7 @@ func (m *Schema1) initialize() error {
m.ExtractedV1Compatibility = make([]Schema1V1Compatibility, len(m.History))
for i, h := range m.History {
if err := json.Unmarshal([]byte(h.V1Compatibility), &m.ExtractedV1Compatibility[i]); err != nil {
return perrors.Wrapf(err, "parsing v2s1 history entry %d", i)
return fmt.Errorf("parsing v2s1 history entry %d: %w", i, err)
}
}
return nil
@@ -248,14 +247,14 @@ func (m *Schema1) ToSchema2Config(diffIDs []digest.Digest) ([]byte, error) {
config := []byte(m.History[0].V1Compatibility)
err := json.Unmarshal(config, &s1)
if err != nil {
return nil, perrors.Wrapf(err, "decoding configuration")
return nil, fmt.Errorf("decoding configuration: %w", err)
}
// Images created with versions prior to 1.8.3 require us to re-encode the encoded object,
// adding some fields that aren't "omitempty".
if s1.DockerVersion != "" && versions.LessThan(s1.DockerVersion, "1.8.3") {
config, err = json.Marshal(&s1)
if err != nil {
return nil, perrors.Wrapf(err, "re-encoding compat image config %#v", s1)
return nil, fmt.Errorf("re-encoding compat image config %#v: %w", s1, err)
}
}
// Build the history.
@@ -282,7 +281,7 @@ func (m *Schema1) ToSchema2Config(diffIDs []digest.Digest) ([]byte, error) {
raw := make(map[string]*json.RawMessage)
err = json.Unmarshal(config, &raw)
if err != nil {
return nil, perrors.Wrapf(err, "re-decoding compat image config %#v", s1)
return nil, fmt.Errorf("re-decoding compat image config %#v: %w", s1, err)
}
// Drop some fields.
delete(raw, "id")

View File

@@ -9,7 +9,6 @@ import (
"github.com/containers/image/v5/pkg/strslice"
"github.com/containers/image/v5/types"
"github.com/opencontainers/go-digest"
perrors "github.com/pkg/errors"
)
// Schema2Descriptor is a “descriptor” in docker/distribution schema 2.
@@ -246,7 +245,7 @@ func (m *Schema2) UpdateLayerInfos(layerInfos []types.BlobInfo) error {
}
mimeType, err := updatedMIMEType(schema2CompressionMIMETypeSets, mimeType, info)
if err != nil {
return perrors.Wrapf(err, "preparing updated manifest, layer %q", info.Digest)
return fmt.Errorf("preparing updated manifest, layer %q: %w", info.Digest, err)
}
m.LayersDescriptors[i].MediaType = mimeType
m.LayersDescriptors[i].Digest = info.Digest

View File

@@ -8,7 +8,6 @@ import (
"github.com/containers/image/v5/types"
"github.com/opencontainers/go-digest"
imgspecv1 "github.com/opencontainers/image-spec/specs-go/v1"
perrors "github.com/pkg/errors"
)
// Schema2PlatformSpec describes the platform which a particular manifest is
@@ -71,7 +70,7 @@ func (list *Schema2List) UpdateInstances(updates []ListUpdate) error {
}
for i := range updates {
if err := updates[i].Digest.Validate(); err != nil {
return perrors.Wrapf(err, "update %d of %d passed to Schema2List.UpdateInstances contained an invalid digest", i+1, len(updates))
return fmt.Errorf("update %d of %d passed to Schema2List.UpdateInstances contained an invalid digest: %w", i+1, len(updates), err)
}
list.Manifests[i].Digest = updates[i].Digest
if updates[i].Size < 0 {
@@ -91,7 +90,7 @@ func (list *Schema2List) UpdateInstances(updates []ListUpdate) error {
func (list *Schema2List) ChooseInstance(ctx *types.SystemContext) (digest.Digest, error) {
wantedPlatforms, err := platform.WantedPlatforms(ctx)
if err != nil {
return "", perrors.Wrapf(err, "getting platform information %#v", ctx)
return "", fmt.Errorf("getting platform information %#v: %w", ctx, err)
}
for _, wantedPlatform := range wantedPlatforms {
for _, d := range list.Manifests {
@@ -115,7 +114,7 @@ func (list *Schema2List) ChooseInstance(ctx *types.SystemContext) (digest.Digest
func (list *Schema2List) Serialize() ([]byte, error) {
buf, err := json.Marshal(list)
if err != nil {
return nil, perrors.Wrapf(err, "marshaling Schema2List %#v", list)
return nil, fmt.Errorf("marshaling Schema2List %#v: %w", list, err)
}
return buf, nil
}
@@ -190,7 +189,7 @@ func Schema2ListFromManifest(manifest []byte) (*Schema2List, error) {
Manifests: []Schema2ManifestDescriptor{},
}
if err := json.Unmarshal(manifest, &list); err != nil {
return nil, perrors.Wrapf(err, "unmarshaling Schema2List %q", string(manifest))
return nil, fmt.Errorf("unmarshaling Schema2List %q: %w", string(manifest), err)
}
if err := validateUnambiguousManifestFormat(manifest, DockerV2ListMediaType,
allowedFieldManifests); err != nil {

View File

@@ -12,7 +12,6 @@ import (
"github.com/opencontainers/go-digest"
"github.com/opencontainers/image-spec/specs-go"
imgspecv1 "github.com/opencontainers/image-spec/specs-go/v1"
perrors "github.com/pkg/errors"
)
// BlobInfoFromOCI1Descriptor returns a types.BlobInfo based on the input OCI1 descriptor.
@@ -139,7 +138,7 @@ func (m *OCI1) UpdateLayerInfos(layerInfos []types.BlobInfo) error {
}
mimeType, err := updatedMIMEType(oci1CompressionMIMETypeSets, mimeType, info)
if err != nil {
return perrors.Wrapf(err, "preparing updated manifest, layer %q", info.Digest)
return fmt.Errorf("preparing updated manifest, layer %q: %w", info.Digest, err)
}
if info.CryptoOperation == types.Encrypt {
encMediaType, err := getEncryptedMediaType(mimeType)

View File

@@ -10,7 +10,6 @@ import (
"github.com/opencontainers/go-digest"
imgspec "github.com/opencontainers/image-spec/specs-go"
imgspecv1 "github.com/opencontainers/image-spec/specs-go/v1"
perrors "github.com/pkg/errors"
)
// OCI1Index is just an alias for the OCI index type, but one which we can
@@ -55,7 +54,7 @@ func (index *OCI1Index) UpdateInstances(updates []ListUpdate) error {
}
for i := range updates {
if err := updates[i].Digest.Validate(); err != nil {
return perrors.Wrapf(err, "update %d of %d passed to OCI1Index.UpdateInstances contained an invalid digest", i+1, len(updates))
return fmt.Errorf("update %d of %d passed to OCI1Index.UpdateInstances contained an invalid digest: %w", i+1, len(updates), err)
}
index.Manifests[i].Digest = updates[i].Digest
if updates[i].Size < 0 {
@@ -75,7 +74,7 @@ func (index *OCI1Index) UpdateInstances(updates []ListUpdate) error {
func (index *OCI1Index) ChooseInstance(ctx *types.SystemContext) (digest.Digest, error) {
wantedPlatforms, err := platform.WantedPlatforms(ctx)
if err != nil {
return "", perrors.Wrapf(err, "getting platform information %#v", ctx)
return "", fmt.Errorf("getting platform information %#v: %w", ctx, err)
}
for _, wantedPlatform := range wantedPlatforms {
for _, d := range index.Manifests {
@@ -108,7 +107,7 @@ func (index *OCI1Index) ChooseInstance(ctx *types.SystemContext) (digest.Digest,
func (index *OCI1Index) Serialize() ([]byte, error) {
buf, err := json.Marshal(index)
if err != nil {
return nil, perrors.Wrapf(err, "marshaling OCI1Index %#v", index)
return nil, fmt.Errorf("marshaling OCI1Index %#v: %w", index, err)
}
return buf, nil
}
@@ -202,7 +201,7 @@ func OCI1IndexFromManifest(manifest []byte) (*OCI1Index, error) {
},
}
if err := json.Unmarshal(manifest, &index); err != nil {
return nil, perrors.Wrapf(err, "unmarshaling OCI1Index %q", string(manifest))
return nil, fmt.Errorf("unmarshaling OCI1Index %q: %w", string(manifest), err)
}
if err := validateUnambiguousManifestFormat(manifest, imgspecv1.MediaTypeImageIndex,
allowedFieldManifests); err != nil {

View File

@@ -2,6 +2,7 @@ package archive
import (
"context"
"fmt"
"io"
"os"
@@ -13,7 +14,6 @@ import (
"github.com/containers/image/v5/types"
"github.com/containers/storage/pkg/archive"
digest "github.com/opencontainers/go-digest"
perrors "github.com/pkg/errors"
"github.com/sirupsen/logrus"
)
@@ -29,12 +29,12 @@ type ociArchiveImageDestination struct {
func newImageDestination(ctx context.Context, sys *types.SystemContext, ref ociArchiveReference) (private.ImageDestination, error) {
tempDirRef, err := createOCIRef(sys, ref.image)
if err != nil {
return nil, perrors.Wrapf(err, "creating oci reference")
return nil, fmt.Errorf("creating oci reference: %w", err)
}
unpackedDest, err := tempDirRef.ociRefExtracted.NewImageDestination(ctx, sys)
if err != nil {
if err := tempDirRef.deleteTempDir(); err != nil {
return nil, perrors.Wrapf(err, "deleting temp directory %q", tempDirRef.tempDirectory)
return nil, fmt.Errorf("deleting temp directory %q: %w", tempDirRef.tempDirectory, err)
}
return nil, err
}
@@ -158,7 +158,7 @@ func (d *ociArchiveImageDestination) PutSignaturesWithFormat(ctx context.Context
// after the directory is made, it is tarred up into a file and the directory is deleted
func (d *ociArchiveImageDestination) Commit(ctx context.Context, unparsedToplevel types.UnparsedImage) error {
if err := d.unpackedDest.Commit(ctx, unparsedToplevel); err != nil {
return perrors.Wrapf(err, "storing image %q", d.ref.image)
return fmt.Errorf("storing image %q: %w", d.ref.image, err)
}
// path of directory to tar up
@@ -173,13 +173,13 @@ func tarDirectory(src, dst string) error {
// input is a stream of bytes from the archive of the directory at path
input, err := archive.Tar(src, archive.Uncompressed)
if err != nil {
return perrors.Wrapf(err, "retrieving stream of bytes from %q", src)
return fmt.Errorf("retrieving stream of bytes from %q: %w", src, err)
}
// creates the tar file
outFile, err := os.Create(dst)
if err != nil {
return perrors.Wrapf(err, "creating tar file %q", dst)
return fmt.Errorf("creating tar file %q: %w", dst, err)
}
defer outFile.Close()

View File

@@ -3,6 +3,7 @@ package archive
import (
"context"
"errors"
"fmt"
"io"
"github.com/containers/image/v5/internal/imagesource"
@@ -13,7 +14,6 @@ import (
"github.com/containers/image/v5/types"
digest "github.com/opencontainers/go-digest"
imgspecv1 "github.com/opencontainers/image-spec/specs-go/v1"
perrors "github.com/pkg/errors"
"github.com/sirupsen/logrus"
)
@@ -30,13 +30,13 @@ type ociArchiveImageSource struct {
func newImageSource(ctx context.Context, sys *types.SystemContext, ref ociArchiveReference) (private.ImageSource, error) {
tempDirRef, err := createUntarTempDir(sys, ref)
if err != nil {
return nil, perrors.Wrap(err, "creating temp directory")
return nil, fmt.Errorf("creating temp directory: %w", err)
}
unpackedSrc, err := tempDirRef.ociRefExtracted.NewImageSource(ctx, sys)
if err != nil {
if err := tempDirRef.deleteTempDir(); err != nil {
return nil, perrors.Wrapf(err, "deleting temp directory %q", tempDirRef.tempDirectory)
return nil, fmt.Errorf("deleting temp directory %q: %w", tempDirRef.tempDirectory, err)
}
return nil, err
}
@@ -63,7 +63,7 @@ func LoadManifestDescriptorWithContext(sys *types.SystemContext, imgRef types.Im
}
tempDirRef, err := createUntarTempDir(sys, ociArchRef)
if err != nil {
return imgspecv1.Descriptor{}, perrors.Wrap(err, "creating temp directory")
return imgspecv1.Descriptor{}, fmt.Errorf("creating temp directory: %w", err)
}
defer func() {
err := tempDirRef.deleteTempDir()
@@ -72,7 +72,7 @@ func LoadManifestDescriptorWithContext(sys *types.SystemContext, imgRef types.Im
descriptor, err := ocilayout.LoadManifestDescriptor(tempDirRef.ociRefExtracted)
if err != nil {
return imgspecv1.Descriptor{}, perrors.Wrap(err, "loading index")
return imgspecv1.Descriptor{}, fmt.Errorf("loading index: %w", err)
}
return descriptor, nil
}

View File

@@ -16,7 +16,6 @@ import (
"github.com/containers/image/v5/transports"
"github.com/containers/image/v5/types"
"github.com/containers/storage/pkg/archive"
perrors "github.com/pkg/errors"
)
func init() {
@@ -159,7 +158,7 @@ func (t *tempDirOCIRef) deleteTempDir() error {
func createOCIRef(sys *types.SystemContext, image string) (tempDirOCIRef, error) {
dir, err := os.MkdirTemp(tmpdir.TemporaryDirectoryForBigFiles(sys), "oci")
if err != nil {
return tempDirOCIRef{}, perrors.Wrapf(err, "creating temp directory")
return tempDirOCIRef{}, fmt.Errorf("creating temp directory: %w", err)
}
ociRef, err := ocilayout.NewReference(dir, image)
if err != nil {
@@ -174,7 +173,7 @@ func createOCIRef(sys *types.SystemContext, image string) (tempDirOCIRef, error)
func createUntarTempDir(sys *types.SystemContext, ref ociArchiveReference) (tempDirOCIRef, error) {
tempDirRef, err := createOCIRef(sys, ref.image)
if err != nil {
return tempDirOCIRef{}, perrors.Wrap(err, "creating oci reference")
return tempDirOCIRef{}, fmt.Errorf("creating oci reference: %w", err)
}
src := ref.resolvedFile
dst := tempDirRef.tempDirectory
@@ -186,9 +185,9 @@ func createUntarTempDir(sys *types.SystemContext, ref ociArchiveReference) (temp
defer arch.Close()
if err := archive.NewDefaultArchiver().Untar(arch, dst, &archive.TarOptions{NoLchown: true}); err != nil {
if err := tempDirRef.deleteTempDir(); err != nil {
return tempDirOCIRef{}, perrors.Wrapf(err, "deleting temp directory %q", tempDirRef.tempDirectory)
return tempDirOCIRef{}, fmt.Errorf("deleting temp directory %q: %w", tempDirRef.tempDirectory, err)
}
return tempDirOCIRef{}, perrors.Wrapf(err, "untarring file %q", tempDirRef.tempDirectory)
return tempDirOCIRef{}, fmt.Errorf("untarring file %q: %w", tempDirRef.tempDirectory, err)
}
return tempDirRef, nil
}

View File

@@ -3,6 +3,7 @@ package layout
import (
"context"
"errors"
"fmt"
"io"
"net/http"
"net/url"
@@ -18,7 +19,6 @@ import (
"github.com/docker/go-connections/tlsconfig"
"github.com/opencontainers/go-digest"
imgspecv1 "github.com/opencontainers/image-spec/specs-go/v1"
perrors "github.com/pkg/errors"
)
type ociImageSource struct {
@@ -170,19 +170,19 @@ func (s *ociImageSource) getExternalBlob(ctx context.Context, urls []string) (io
hasSupportedURL = true
req, err := http.NewRequestWithContext(ctx, http.MethodGet, u, nil)
if err != nil {
errWrap = perrors.Wrapf(errWrap, "fetching %s failed %s", u, err.Error())
errWrap = fmt.Errorf("fetching %s failed %s: %w", u, err.Error(), errWrap)
continue
}
resp, err := s.client.Do(req)
if err != nil {
errWrap = perrors.Wrapf(errWrap, "fetching %s failed %s", u, err.Error())
errWrap = fmt.Errorf("fetching %s failed %s: %w", u, err.Error(), errWrap)
continue
}
if resp.StatusCode != http.StatusOK {
resp.Body.Close()
errWrap = perrors.Wrapf(errWrap, "fetching %s failed, response code not 200", u)
errWrap = fmt.Errorf("fetching %s failed, response code not 200: %w", u, errWrap)
continue
}

View File

@@ -17,7 +17,6 @@ import (
"github.com/containers/image/v5/types"
"github.com/opencontainers/go-digest"
imgspecv1 "github.com/opencontainers/image-spec/specs-go/v1"
perrors "github.com/pkg/errors"
)
func init() {
@@ -251,7 +250,7 @@ func (ref ociReference) indexPath() string {
// blobPath returns a path for a blob within a directory using OCI image-layout conventions.
func (ref ociReference) blobPath(digest digest.Digest, sharedBlobDir string) (string, error) {
if err := digest.Validate(); err != nil {
return "", perrors.Wrapf(err, "unexpected digest reference %s", digest)
return "", fmt.Errorf("unexpected digest reference %s: %w", digest, err)
}
blobDir := filepath.Join(ref.dir, "blobs")
if sharedBlobDir != "" {

View File

@@ -19,7 +19,6 @@ import (
"github.com/containers/storage/pkg/homedir"
"github.com/ghodss/yaml"
"github.com/imdario/mergo"
perrors "github.com/pkg/errors"
"github.com/sirupsen/logrus"
"golang.org/x/net/http2"
)
@@ -579,7 +578,7 @@ func (rules *clientConfigLoadingRules) Load() (*clientcmdConfig, error) {
continue
}
if err != nil {
errlist = append(errlist, perrors.Wrapf(err, "loading config file \"%s\"", filename))
errlist = append(errlist, fmt.Errorf("loading config file \"%s\": %w", filename, err))
continue
}
@@ -692,7 +691,7 @@ func resolveLocalPaths(config *clientcmdConfig) error {
}
base, err := filepath.Abs(filepath.Dir(cluster.LocationOfOrigin))
if err != nil {
return perrors.Wrapf(err, "Could not determine the absolute path of config file %s", cluster.LocationOfOrigin)
return fmt.Errorf("Could not determine the absolute path of config file %s: %w", cluster.LocationOfOrigin, err)
}
if err := resolvePaths(getClusterFileReferences(cluster), base); err != nil {
@@ -705,7 +704,7 @@ func resolveLocalPaths(config *clientcmdConfig) error {
}
base, err := filepath.Abs(filepath.Dir(authInfo.LocationOfOrigin))
if err != nil {
return perrors.Wrapf(err, "Could not determine the absolute path of config file %s", authInfo.LocationOfOrigin)
return fmt.Errorf("Could not determine the absolute path of config file %s: %w", authInfo.LocationOfOrigin, err)
}
if err := resolvePaths(getAuthInfoFileReferences(authInfo), base); err != nil {

View File

@@ -21,7 +21,6 @@ import (
"github.com/containers/image/v5/manifest"
"github.com/containers/image/v5/types"
"github.com/opencontainers/go-digest"
perrors "github.com/pkg/errors"
)
type openshiftImageDestination struct {
@@ -206,7 +205,7 @@ sigExists:
randBytes := make([]byte, 16)
n, err := rand.Read(randBytes)
if err != nil || n != 16 {
return perrors.Wrapf(err, "generating random signature len %d", n)
return fmt.Errorf("generating random signature len %d: %w", n, err)
}
signatureName = fmt.Sprintf("%s@%032x", imageStreamImageName, randBytes)
if _, ok := existingSigNames[signatureName]; !ok {

View File

@@ -12,7 +12,6 @@ import (
genericImage "github.com/containers/image/v5/internal/image"
"github.com/containers/image/v5/transports"
"github.com/containers/image/v5/types"
perrors "github.com/pkg/errors"
)
func init() {
@@ -60,7 +59,7 @@ type openshiftReference struct {
func ParseReference(ref string) (types.ImageReference, error) {
r, err := reference.ParseNormalizedNamed(ref)
if err != nil {
return nil, perrors.Wrapf(err, "failed to parse image reference %q", ref)
return nil, fmt.Errorf("failed to parse image reference %q: %w", ref, err)
}
tagged, ok := r.(reference.NamedTagged)
if !ok {

View File

@@ -33,7 +33,6 @@ import (
"github.com/opencontainers/go-digest"
selinux "github.com/opencontainers/selinux/go-selinux"
"github.com/ostreedev/ostree-go/pkg/otbuiltin"
perrors "github.com/pkg/errors"
"github.com/vbatts/tar-split/tar/asm"
"github.com/vbatts/tar-split/tar/storage"
)
@@ -203,7 +202,7 @@ func fixFiles(selinuxHnd *C.struct_selabel_handle, root string, dir string, user
res, err := C.selabel_lookup_raw(selinuxHnd, &context, relPathC, C.int(info.Mode()&os.ModePerm))
if int(res) < 0 && err != syscall.ENOENT {
return perrors.Wrapf(err, "cannot selabel_lookup_raw %s", relPath)
return fmt.Errorf("cannot selabel_lookup_raw %s: %w", relPath, err)
}
if int(res) == 0 {
defer C.freecon(context)
@@ -211,7 +210,7 @@ func fixFiles(selinuxHnd *C.struct_selabel_handle, root string, dir string, user
defer C.free(unsafe.Pointer(fullpathC))
res, err = C.lsetfilecon_raw(fullpathC, context)
if int(res) < 0 {
return perrors.Wrapf(err, "cannot setfilecon_raw %s to %s", fullpath, C.GoString(context))
return fmt.Errorf("cannot setfilecon_raw %s to %s: %w", fullpath, C.GoString(context), err)
}
}
}
@@ -446,7 +445,7 @@ func (d *ostreeImageDestination) Commit(context.Context, types.UnparsedImage) er
if os.Getuid() == 0 && selinux.GetEnabled() {
selinuxHnd, err = C.selabel_open(C.SELABEL_CTX_FILE, nil, 0)
if selinuxHnd == nil {
return perrors.Wrapf(err, "cannot open the SELinux DB")
return fmt.Errorf("cannot open the SELinux DB: %w", err)
}
defer C.selabel_close(selinuxHnd)

View File

@@ -10,7 +10,6 @@ import (
"github.com/containers/image/v5/pkg/compression/types"
"github.com/containers/storage/pkg/chunked/compressor"
"github.com/klauspost/pgzip"
perrors "github.com/pkg/errors"
"github.com/sirupsen/logrus"
"github.com/ulikunitz/xz"
)
@@ -151,13 +150,13 @@ func DetectCompression(input io.Reader) (DecompressorFunc, io.Reader, error) {
func AutoDecompress(stream io.Reader) (io.ReadCloser, bool, error) {
decompressor, stream, err := DetectCompression(stream)
if err != nil {
return nil, false, perrors.Wrapf(err, "detecting compression")
return nil, false, fmt.Errorf("detecting compression: %w", err)
}
var res io.ReadCloser
if decompressor != nil {
res, err = decompressor(stream)
if err != nil {
return nil, false, perrors.Wrapf(err, "initializing decompression")
return nil, false, fmt.Errorf("initializing decompression: %w", err)
}
} else {
res = io.NopCloser(stream)

View File

@@ -19,7 +19,6 @@ import (
helperclient "github.com/docker/docker-credential-helpers/client"
"github.com/docker/docker-credential-helpers/credentials"
"github.com/hashicorp/go-multierror"
perrors "github.com/pkg/errors"
"github.com/sirupsen/logrus"
)
@@ -150,7 +149,7 @@ func GetAllCredentials(sys *types.SystemContext) (map[string]types.DockerAuthCon
// readJSONFile returns an empty map in case the path doesn't exist.
auths, err := readJSONFile(path.path, path.legacyFormat)
if err != nil {
return nil, perrors.Wrapf(err, "reading JSON file %q", path.path)
return nil, fmt.Errorf("reading JSON file %q: %w", path.path, err)
}
// Credential helpers in the auth file have a
// direct mapping to a registry, so we can just
@@ -357,7 +356,7 @@ func getAuthenticationWithHomeDir(sys *types.SystemContext, key, homeDir string)
return "", "", err
}
if auth.IdentityToken != "" {
return "", "", perrors.Wrap(ErrNotSupported, "non-empty identity token found and this API doesn't support it")
return "", "", fmt.Errorf("non-empty identity token found and this API doesn't support it: %w", ErrNotSupported)
}
return auth.Username, auth.Password, nil
}
@@ -396,7 +395,7 @@ func RemoveAuthentication(sys *types.SystemContext, key string) error {
return
}
}
multiErr = multierror.Append(multiErr, perrors.Wrapf(err, "removing credentials for %s from credential helper %s", key, helper))
multiErr = multierror.Append(multiErr, fmt.Errorf("removing credentials for %s from credential helper %s: %w", key, helper, err))
}
for _, helper := range helpers {
@@ -529,7 +528,7 @@ func getPathToAuthWithOS(sys *types.SystemContext, goOS string) (string, bool, e
// This means the user set the XDG_RUNTIME_DIR variable and either forgot to create the directory
// or made a typo while setting the environment variable,
// so return an error referring to $XDG_RUNTIME_DIR instead of xdgRuntimeDirPath inside.
return "", false, perrors.Wrapf(err, "%q directory set by $XDG_RUNTIME_DIR does not exist. Either create the directory or unset $XDG_RUNTIME_DIR.", runtimeDir)
return "", false, fmt.Errorf("%q directory set by $XDG_RUNTIME_DIR does not exist. Either create the directory or unset $XDG_RUNTIME_DIR.: %w", runtimeDir, err)
} // else ignore err and let the caller fail accessing xdgRuntimeDirPath.
return filepath.Join(runtimeDir, xdgRuntimeDirPath), false, nil
}
@@ -553,13 +552,13 @@ func readJSONFile(path string, legacyFormat bool) (dockerConfigFile, error) {
if legacyFormat {
if err = json.Unmarshal(raw, &auths.AuthConfigs); err != nil {
return dockerConfigFile{}, perrors.Wrapf(err, "unmarshaling JSON at %q", path)
return dockerConfigFile{}, fmt.Errorf("unmarshaling JSON at %q: %w", path, err)
}
return auths, nil
}
if err = json.Unmarshal(raw, &auths); err != nil {
return dockerConfigFile{}, perrors.Wrapf(err, "unmarshaling JSON at %q", path)
return dockerConfigFile{}, fmt.Errorf("unmarshaling JSON at %q: %w", path, err)
}
if auths.AuthConfigs == nil {
@@ -591,21 +590,21 @@ func modifyJSON(sys *types.SystemContext, editor func(auths *dockerConfigFile) (
auths, err := readJSONFile(path, false)
if err != nil {
return "", perrors.Wrapf(err, "reading JSON file %q", path)
return "", fmt.Errorf("reading JSON file %q: %w", path, err)
}
updated, err := editor(&auths)
if err != nil {
return "", perrors.Wrapf(err, "updating %q", path)
return "", fmt.Errorf("updating %q: %w", path, err)
}
if updated {
newData, err := json.MarshalIndent(auths, "", "\t")
if err != nil {
return "", perrors.Wrapf(err, "marshaling JSON %q", path)
return "", fmt.Errorf("marshaling JSON %q: %w", path, err)
}
if err = ioutils.AtomicWriteFile(path, newData, 0600); err != nil {
return "", perrors.Wrapf(err, "writing to file %q", path)
return "", fmt.Errorf("writing to file %q: %w", path, err)
}
}
@@ -659,7 +658,7 @@ func deleteAuthFromCredHelper(credHelper, registry string) error {
func findCredentialsInFile(key, registry, path string, legacyFormat bool) (types.DockerAuthConfig, error) {
auths, err := readJSONFile(path, legacyFormat)
if err != nil {
return types.DockerAuthConfig{}, perrors.Wrapf(err, "reading JSON file %q", path)
return types.DockerAuthConfig{}, fmt.Errorf("reading JSON file %q: %w", path, err)
}
// First try cred helpers. They should always be normalized.

View File

@@ -13,7 +13,6 @@ import (
"github.com/containers/image/v5/types"
"github.com/containers/storage/pkg/homedir"
"github.com/containers/storage/pkg/lockfile"
perrors "github.com/pkg/errors"
"github.com/sirupsen/logrus"
)
@@ -211,7 +210,7 @@ func RemoveShortNameAlias(ctx *types.SystemContext, name string) error {
func parseShortNameValue(alias string) (reference.Named, error) {
ref, err := reference.Parse(alias)
if err != nil {
return nil, perrors.Wrapf(err, "parsing alias %q", alias)
return nil, fmt.Errorf("parsing alias %q: %w", alias, err)
}
if _, ok := ref.(reference.Digested); ok {
@@ -243,7 +242,7 @@ func parseShortNameValue(alias string) (reference.Named, error) {
func validateShortName(name string) error {
repo, err := reference.Parse(name)
if err != nil {
return perrors.Wrapf(err, "cannot parse short name: %q", name)
return fmt.Errorf("cannot parse short name: %q: %w", name, err)
}
if _, ok := repo.(reference.Digested); ok {
@@ -299,7 +298,7 @@ func newShortNameAliasCache(path string, conf *shortNameAliasConf) (*shortNameAl
if len(errs) > 0 {
err := errs[0]
for i := 1; i < len(errs); i++ {
err = perrors.Wrapf(err, "%v\n", errs[i])
err = fmt.Errorf("%v\n: %w", errs[i], err)
}
return nil, err
}
@@ -320,7 +319,7 @@ func loadShortNameAliasConf(confPath string) (*shortNameAliasConf, *shortNameAli
meta, err := toml.DecodeFile(confPath, &conf)
if err != nil && !os.IsNotExist(err) {
// It's okay if the config doesn't exist. Other errors are not.
return nil, nil, perrors.Wrapf(err, "loading short-name aliases config file %q", confPath)
return nil, nil, fmt.Errorf("loading short-name aliases config file %q: %w", confPath, err)
}
if keys := meta.Undecoded(); len(keys) > 0 {
logrus.Debugf("Failed to decode keys %q from %q", keys, confPath)
@@ -330,7 +329,7 @@ func loadShortNameAliasConf(confPath string) (*shortNameAliasConf, *shortNameAli
// file could still be corrupted by another process or user.
cache, err := newShortNameAliasCache(confPath, &conf)
if err != nil {
return nil, nil, perrors.Wrapf(err, "loading short-name aliases config file %q", confPath)
return nil, nil, fmt.Errorf("loading short-name aliases config file %q: %w", confPath, err)
}
return &conf, cache, nil

View File

@@ -15,7 +15,6 @@ import (
"github.com/containers/image/v5/docker/reference"
"github.com/containers/image/v5/types"
"github.com/containers/storage/pkg/homedir"
perrors "github.com/pkg/errors"
"github.com/sirupsen/logrus"
)
@@ -103,7 +102,7 @@ func (e *Endpoint) rewriteReference(ref reference.Named, prefix string) (referen
newNamedRef = e.Location + refString[prefixLen:]
newParsedRef, err := reference.ParseNamed(newNamedRef)
if err != nil {
return nil, perrors.Wrapf(err, "rewriting reference")
return nil, fmt.Errorf("rewriting reference: %w", err)
}
return newParsedRef, nil
@@ -666,7 +665,7 @@ func dropInConfigs(wrapper configWrapper) ([]string, error) {
if err != nil && !os.IsNotExist(err) {
// Ignore IsNotExist errors: most systems won't have a registries.conf.d
// directory.
return nil, perrors.Wrapf(err, "reading registries.conf.d")
return nil, fmt.Errorf("reading registries.conf.d: %w", err)
}
}
@@ -708,7 +707,7 @@ func tryUpdatingCache(ctx *types.SystemContext, wrapper configWrapper) (*parsedC
return nil, err // Should never happen
}
} else {
return nil, perrors.Wrapf(err, "loading registries configuration %q", wrapper.configPath)
return nil, fmt.Errorf("loading registries configuration %q: %w", wrapper.configPath, err)
}
}
@@ -721,7 +720,7 @@ func tryUpdatingCache(ctx *types.SystemContext, wrapper configWrapper) (*parsedC
// Enforce v2 format for drop-in-configs.
dropIn, err := loadConfigFile(path, true)
if err != nil {
return nil, perrors.Wrapf(err, "loading drop-in registries configuration %q", path)
return nil, fmt.Errorf("loading drop-in registries configuration %q: %w", path, err)
}
config.updateWithConfigurationFrom(dropIn)
}
@@ -975,7 +974,7 @@ func loadConfigFile(path string, forceV2 bool) (*parsedConfig, error) {
// Parse and validate short-name aliases.
cache, err := newShortNameAliasCache(path, &res.partialV2.shortNameAliasConf)
if err != nil {
return nil, perrors.Wrap(err, "validating short-name aliases")
return nil, fmt.Errorf("validating short-name aliases: %w", err)
}
res.aliasCache = cache
// Clear conf.partialV2.shortNameAliasConf to make it available for garbage collection and

View File

@@ -12,7 +12,6 @@ import (
"github.com/docker/go-connections/sockets"
"github.com/docker/go-connections/tlsconfig"
perrors "github.com/pkg/errors"
"github.com/sirupsen/logrus"
)
@@ -50,7 +49,7 @@ func SetupCertificates(dir string, tlsc *tls.Config) error {
if tlsc.RootCAs == nil {
systemPool, err := tlsconfig.SystemCertPool()
if err != nil {
return perrors.Wrap(err, "unable to get system cert pool")
return fmt.Errorf("unable to get system cert pool: %w", err)
}
tlsc.RootCAs = systemPool
}

View File

@@ -65,7 +65,7 @@ func NewGPGSigningMechanism() (SigningMechanism, error) {
// of these keys.
// The caller must call .Close() on the returned SigningMechanism.
func NewEphemeralGPGSigningMechanism(blob []byte) (SigningMechanism, []string, error) {
return newEphemeralGPGSigningMechanism(blob)
return newEphemeralGPGSigningMechanism([][]byte{blob})
}
// gpgUntrustedSignatureContents returns UNTRUSTED contents of the signature WITHOUT ANY VERIFICATION,

View File

@@ -33,10 +33,10 @@ func newGPGSigningMechanismInDirectory(optionalDir string) (signingMechanismWith
}
// newEphemeralGPGSigningMechanism returns a new GPG/OpenPGP signing mechanism which
// recognizes _only_ public keys from the supplied blob, and returns the identities
// recognizes _only_ public keys from the supplied blobs, and returns the identities
// of these keys.
// The caller must call .Close() on the returned SigningMechanism.
func newEphemeralGPGSigningMechanism(blob []byte) (signingMechanismWithPassphrase, []string, error) {
func newEphemeralGPGSigningMechanism(blobs [][]byte) (signingMechanismWithPassphrase, []string, error) {
dir, err := os.MkdirTemp("", "containers-ephemeral-gpg-")
if err != nil {
return nil, nil, err
@@ -55,9 +55,13 @@ func newEphemeralGPGSigningMechanism(blob []byte) (signingMechanismWithPassphras
ctx: ctx,
ephemeralDir: dir,
}
keyIdentities, err := mech.importKeysFromBytes(blob)
if err != nil {
return nil, nil, err
keyIdentities := []string{}
for _, blob := range blobs {
ki, err := mech.importKeysFromBytes(blob)
if err != nil {
return nil, nil, err
}
keyIdentities = append(keyIdentities, ki...)
}
removeDir = false

View File

@@ -63,14 +63,19 @@ func newGPGSigningMechanismInDirectory(optionalDir string) (signingMechanismWith
// recognizes _only_ public keys from the supplied blob, and returns the identities
// of these keys.
// The caller must call .Close() on the returned SigningMechanism.
func newEphemeralGPGSigningMechanism(blob []byte) (signingMechanismWithPassphrase, []string, error) {
func newEphemeralGPGSigningMechanism(blobs [][]byte) (signingMechanismWithPassphrase, []string, error) {
m := &openpgpSigningMechanism{
keyring: openpgp.EntityList{},
}
keyIdentities, err := m.importKeysFromBytes(blob)
if err != nil {
return nil, nil, err
keyIdentities := []string{}
for _, blob := range blobs {
ki, err := m.importKeysFromBytes(blob)
if err != nil {
return nil, nil, err
}
keyIdentities = append(keyIdentities, ki...)
}
return m, keyIdentities, nil
}

View File

@@ -26,7 +26,6 @@ import (
"github.com/containers/image/v5/transports"
"github.com/containers/image/v5/types"
"github.com/containers/storage/pkg/homedir"
perrors "github.com/pkg/errors"
)
// systemDefaultPolicyPath is the policy path used for DefaultPolicy().
@@ -83,7 +82,7 @@ func NewPolicyFromFile(fileName string) (*Policy, error) {
}
policy, err := NewPolicyFromBytes(contents)
if err != nil {
return nil, perrors.Wrapf(err, "invalid policy in %q", fileName)
return nil, fmt.Errorf("invalid policy in %q: %w", fileName, err)
}
return policy, nil
}
@@ -316,12 +315,22 @@ func (pr *prReject) UnmarshalJSON(data []byte) error {
}
// newPRSignedBy returns a new prSignedBy if parameters are valid.
func newPRSignedBy(keyType sbKeyType, keyPath string, keyData []byte, signedIdentity PolicyReferenceMatch) (*prSignedBy, error) {
func newPRSignedBy(keyType sbKeyType, keyPath string, keyPaths []string, keyData []byte, signedIdentity PolicyReferenceMatch) (*prSignedBy, error) {
if !keyType.IsValid() {
return nil, InvalidPolicyFormatError(fmt.Sprintf("invalid keyType \"%s\"", keyType))
}
if len(keyPath) > 0 && len(keyData) > 0 {
return nil, InvalidPolicyFormatError("keyType and keyData cannot be used simultaneously")
keySources := 0
if keyPath != "" {
keySources++
}
if keyPaths != nil {
keySources++
}
if keyData != nil {
keySources++
}
if keySources != 1 {
return nil, InvalidPolicyFormatError("exactly one of keyPath, keyPaths and keyData must be specified")
}
if signedIdentity == nil {
return nil, InvalidPolicyFormatError("signedIdentity not specified")
@@ -330,6 +339,7 @@ func newPRSignedBy(keyType sbKeyType, keyPath string, keyData []byte, signedIden
prCommon: prCommon{Type: prTypeSignedBy},
KeyType: keyType,
KeyPath: keyPath,
KeyPaths: keyPaths,
KeyData: keyData,
SignedIdentity: signedIdentity,
}, nil
@@ -337,7 +347,7 @@ func newPRSignedBy(keyType sbKeyType, keyPath string, keyData []byte, signedIden
// newPRSignedByKeyPath is NewPRSignedByKeyPath, except it returns the private type.
func newPRSignedByKeyPath(keyType sbKeyType, keyPath string, signedIdentity PolicyReferenceMatch) (*prSignedBy, error) {
return newPRSignedBy(keyType, keyPath, nil, signedIdentity)
return newPRSignedBy(keyType, keyPath, nil, nil, signedIdentity)
}
// NewPRSignedByKeyPath returns a new "signedBy" PolicyRequirement using a KeyPath
@@ -345,9 +355,19 @@ func NewPRSignedByKeyPath(keyType sbKeyType, keyPath string, signedIdentity Poli
return newPRSignedByKeyPath(keyType, keyPath, signedIdentity)
}
// newPRSignedByKeyPaths is NewPRSignedByKeyPaths, except it returns the private type.
func newPRSignedByKeyPaths(keyType sbKeyType, keyPaths []string, signedIdentity PolicyReferenceMatch) (*prSignedBy, error) {
return newPRSignedBy(keyType, "", keyPaths, nil, signedIdentity)
}
// NewPRSignedByKeyPaths returns a new "signedBy" PolicyRequirement using KeyPaths
func NewPRSignedByKeyPaths(keyType sbKeyType, keyPaths []string, signedIdentity PolicyReferenceMatch) (PolicyRequirement, error) {
return newPRSignedByKeyPaths(keyType, keyPaths, signedIdentity)
}
// newPRSignedByKeyData is NewPRSignedByKeyData, except it returns the private type.
func newPRSignedByKeyData(keyType sbKeyType, keyData []byte, signedIdentity PolicyReferenceMatch) (*prSignedBy, error) {
return newPRSignedBy(keyType, "", keyData, signedIdentity)
return newPRSignedBy(keyType, "", nil, keyData, signedIdentity)
}
// NewPRSignedByKeyData returns a new "signedBy" PolicyRequirement using a KeyData
@@ -362,7 +382,7 @@ var _ json.Unmarshaler = (*prSignedBy)(nil)
func (pr *prSignedBy) UnmarshalJSON(data []byte) error {
*pr = prSignedBy{}
var tmp prSignedBy
var gotKeyPath, gotKeyData = false, false
var gotKeyPath, gotKeyPaths, gotKeyData = false, false, false
var signedIdentity json.RawMessage
if err := internal.ParanoidUnmarshalJSONObject(data, func(key string) interface{} {
switch key {
@@ -373,6 +393,9 @@ func (pr *prSignedBy) UnmarshalJSON(data []byte) error {
case "keyPath":
gotKeyPath = true
return &tmp.KeyPath
case "keyPaths":
gotKeyPaths = true
return &tmp.KeyPaths
case "keyData":
gotKeyData = true
return &tmp.KeyData
@@ -401,16 +424,16 @@ func (pr *prSignedBy) UnmarshalJSON(data []byte) error {
var res *prSignedBy
var err error
switch {
case gotKeyPath && gotKeyData:
return InvalidPolicyFormatError("keyPath and keyData cannot be used simultaneously")
case gotKeyPath && !gotKeyData:
case gotKeyPath && !gotKeyPaths && !gotKeyData:
res, err = newPRSignedByKeyPath(tmp.KeyType, tmp.KeyPath, tmp.SignedIdentity)
case !gotKeyPath && gotKeyData:
case !gotKeyPath && gotKeyPaths && !gotKeyData:
res, err = newPRSignedByKeyPaths(tmp.KeyType, tmp.KeyPaths, tmp.SignedIdentity)
case !gotKeyPath && !gotKeyPaths && gotKeyData:
res, err = newPRSignedByKeyData(tmp.KeyType, tmp.KeyData, tmp.SignedIdentity)
case !gotKeyPath && !gotKeyData:
return InvalidPolicyFormatError("At least one of keyPath and keyData mus be specified")
default: // Coverage: This should never happen
return fmt.Errorf("Impossible keyPath/keyData presence combination!?")
case !gotKeyPath && !gotKeyPaths && !gotKeyData:
return InvalidPolicyFormatError("Exactly one of keyPath, keyPaths and keyData must be specified, none of them present")
default:
return fmt.Errorf("Exactly one of keyPath, keyPaths and keyData must be specified, more than one present")
}
if err != nil {
return err
@@ -582,7 +605,7 @@ func (pr *prSigstoreSigned) UnmarshalJSON(data []byte) error {
case !gotKeyPath && gotKeyData:
res, err = newPRSigstoreSignedKeyData(tmp.KeyData, tmp.SignedIdentity)
case !gotKeyPath && !gotKeyData:
return InvalidPolicyFormatError("At least one of keyPath and keyData mus be specified")
return InvalidPolicyFormatError("At least one of keyPath and keyData must be specified")
default: // Coverage: This should never happen
return fmt.Errorf("Impossible keyPath/keyData presence combination!?")
}

View File

@@ -25,23 +25,38 @@ func (pr *prSignedBy) isSignatureAuthorAccepted(ctx context.Context, image priva
return sarRejected, nil, fmt.Errorf(`Unknown "keyType" value "%s"`, string(pr.KeyType))
}
if pr.KeyPath != "" && pr.KeyData != nil {
return sarRejected, nil, errors.New(`Internal inconsistency: both "keyPath" and "keyData" specified`)
}
// FIXME: move this to per-context initialization
var data []byte
if pr.KeyData != nil {
data = pr.KeyData
} else {
var data [][]byte
keySources := 0
if pr.KeyPath != "" {
keySources++
d, err := os.ReadFile(pr.KeyPath)
if err != nil {
return sarRejected, nil, err
}
data = d
data = [][]byte{d}
}
if pr.KeyPaths != nil {
keySources++
data = [][]byte{}
for _, path := range pr.KeyPaths {
d, err := os.ReadFile(path)
if err != nil {
return sarRejected, nil, err
}
data = append(data, d)
}
}
if pr.KeyData != nil {
keySources++
data = [][]byte{pr.KeyData}
}
if keySources != 1 {
return sarRejected, nil, errors.New(`Internal inconsistency: not exactly one of "keyPath", "keyPaths" and "keyData" specified`)
}
// FIXME: move this to per-context initialization
mech, trustedIdentities, err := NewEphemeralGPGSigningMechanism(data)
mech, trustedIdentities, err := newEphemeralGPGSigningMechanism(data)
if err != nil {
return sarRejected, nil, err
}

View File

@@ -67,14 +67,16 @@ type prReject struct {
type prSignedBy struct {
prCommon
// KeyType specifies what kind of key reference KeyPath/KeyData is.
// KeyType specifies what kind of key reference KeyPath/KeyPaths/KeyData is.
// Acceptable values are “GPGKeys” | “signedByGPGKeys” “X.509Certificates” | “signedByX.509CAs”
// FIXME: eventually also support GPGTOFU, X.509TOFU, with KeyPath only
KeyType sbKeyType `json:"keyType"`
// KeyPath is a pathname to a local file containing the trusted key(s). Exactly one of KeyPath and KeyData must be specified.
// KeyPath is a pathname to a local file containing the trusted key(s). Exactly one of KeyPath, KeyPaths and KeyData must be specified.
KeyPath string `json:"keyPath,omitempty"`
// KeyData contains the trusted key(s), base64-encoded. Exactly one of KeyPath and KeyData must be specified.
// KeyPaths if a set of pathnames to local files containing the trusted key(s). Exactly one of KeyPath, KeyPaths and KeyData must be specified.
KeyPaths []string `json:"keyPaths,omitempty"`
// KeyData contains the trusted key(s), base64-encoded. Exactly one of KeyPath, KeyPaths and KeyData must be specified.
KeyData []byte `json:"keyData,omitempty"`
// SignedIdentity specifies what image identity the signature must be claiming about the image.

View File

@@ -33,7 +33,6 @@ import (
"github.com/containers/storage/pkg/ioutils"
digest "github.com/opencontainers/go-digest"
imgspecv1 "github.com/opencontainers/image-spec/specs-go/v1"
perrors "github.com/pkg/errors"
"github.com/sirupsen/logrus"
)
@@ -90,7 +89,7 @@ type storageImageDestination struct {
func newImageDestination(sys *types.SystemContext, imageRef storageReference) (*storageImageDestination, error) {
directory, err := os.MkdirTemp(tmpdir.TemporaryDirectoryForBigFiles(sys), "storage")
if err != nil {
return nil, perrors.Wrapf(err, "creating a temporary directory")
return nil, fmt.Errorf("creating a temporary directory: %w", err)
}
dest := &storageImageDestination{
PropertyMethodsInitialize: impl.PropertyMethods(impl.Properties{
@@ -189,7 +188,7 @@ func (s *storageImageDestination) putBlobToPendingFile(ctx context.Context, stre
filename := s.computeNextBlobCacheFile()
file, err := os.OpenFile(filename, os.O_CREATE|os.O_TRUNC|os.O_WRONLY|os.O_EXCL, 0600)
if err != nil {
return errorBlobInfo, perrors.Wrapf(err, "creating temporary file %q", filename)
return errorBlobInfo, fmt.Errorf("creating temporary file %q: %w", filename, err)
}
defer file.Close()
counter := ioutils.NewWriteCounter(file)
@@ -197,7 +196,7 @@ func (s *storageImageDestination) putBlobToPendingFile(ctx context.Context, stre
digester, stream := putblobdigest.DigestIfUnknown(stream, blobinfo)
decompressed, err := archive.DecompressStream(stream)
if err != nil {
return errorBlobInfo, perrors.Wrap(err, "setting up to decompress blob")
return errorBlobInfo, fmt.Errorf("setting up to decompress blob: %w", err)
}
diffID := digest.Canonical.Digester()
@@ -206,7 +205,7 @@ func (s *storageImageDestination) putBlobToPendingFile(ctx context.Context, stre
_, err = io.Copy(diffID.Hash(), decompressed)
decompressed.Close()
if err != nil {
return errorBlobInfo, perrors.Wrapf(err, "storing blob to file %q", filename)
return errorBlobInfo, fmt.Errorf("storing blob to file %q: %w", filename, err)
}
// Determine blob properties, and fail if information that we were given about the blob
@@ -320,7 +319,7 @@ func (s *storageImageDestination) tryReusingBlobAsPending(ctx context.Context, b
// Check if we have the layer in the underlying additional layer store.
aLayer, err := s.imageRef.transport.store.LookupAdditionalLayer(blobinfo.Digest, options.SrcRef.String())
if err != nil && !errors.Is(err, storage.ErrLayerUnknown) {
return false, types.BlobInfo{}, perrors.Wrapf(err, `looking for compressed layers with digest %q and labels`, blobinfo.Digest)
return false, types.BlobInfo{}, fmt.Errorf(`looking for compressed layers with digest %q and labels: %w`, blobinfo.Digest, err)
} else if err == nil {
// Record the uncompressed value so that we can use it to calculate layer IDs.
s.blobDiffIDs[blobinfo.Digest] = aLayer.UncompressedDigest()
@@ -337,7 +336,7 @@ func (s *storageImageDestination) tryReusingBlobAsPending(ctx context.Context, b
return false, types.BlobInfo{}, errors.New(`Can not check for a blob with unknown digest`)
}
if err := blobinfo.Digest.Validate(); err != nil {
return false, types.BlobInfo{}, perrors.Wrapf(err, `Can not check for a blob with invalid digest`)
return false, types.BlobInfo{}, fmt.Errorf("Can not check for a blob with invalid digest: %w", err)
}
// Check if we've already cached it in a file.
@@ -352,7 +351,7 @@ func (s *storageImageDestination) tryReusingBlobAsPending(ctx context.Context, b
// Check if we have a wasn't-compressed layer in storage that's based on that blob.
layers, err := s.imageRef.transport.store.LayersByUncompressedDigest(blobinfo.Digest)
if err != nil && !errors.Is(err, storage.ErrLayerUnknown) {
return false, types.BlobInfo{}, perrors.Wrapf(err, `looking for layers with digest %q`, blobinfo.Digest)
return false, types.BlobInfo{}, fmt.Errorf(`looking for layers with digest %q: %w`, blobinfo.Digest, err)
}
if len(layers) > 0 {
// Save this for completeness.
@@ -367,7 +366,7 @@ func (s *storageImageDestination) tryReusingBlobAsPending(ctx context.Context, b
// Check if we have a was-compressed layer in storage that's based on that blob.
layers, err = s.imageRef.transport.store.LayersByCompressedDigest(blobinfo.Digest)
if err != nil && !errors.Is(err, storage.ErrLayerUnknown) {
return false, types.BlobInfo{}, perrors.Wrapf(err, `looking for compressed layers with digest %q`, blobinfo.Digest)
return false, types.BlobInfo{}, fmt.Errorf(`looking for compressed layers with digest %q: %w`, blobinfo.Digest, err)
}
if len(layers) > 0 {
// Record the uncompressed value so that we can use it to calculate layer IDs.
@@ -386,7 +385,7 @@ func (s *storageImageDestination) tryReusingBlobAsPending(ctx context.Context, b
if uncompressedDigest := options.Cache.UncompressedDigest(blobinfo.Digest); uncompressedDigest != "" && uncompressedDigest != blobinfo.Digest {
layers, err := s.imageRef.transport.store.LayersByUncompressedDigest(uncompressedDigest)
if err != nil && !errors.Is(err, storage.ErrLayerUnknown) {
return false, types.BlobInfo{}, perrors.Wrapf(err, `looking for layers with digest %q`, uncompressedDigest)
return false, types.BlobInfo{}, fmt.Errorf(`looking for layers with digest %q: %w`, uncompressedDigest, err)
}
if len(layers) > 0 {
if blobinfo.Size != -1 {
@@ -455,13 +454,13 @@ func (s *storageImageDestination) getConfigBlob(info types.BlobInfo) ([]byte, er
return nil, errors.New(`no digest supplied when reading blob`)
}
if err := info.Digest.Validate(); err != nil {
return nil, perrors.Wrapf(err, `invalid digest supplied when reading blob`)
return nil, fmt.Errorf("invalid digest supplied when reading blob: %w", err)
}
// Assume it's a file, since we're only calling this from a place that expects to read files.
if filename, ok := s.filenames[info.Digest]; ok {
contents, err2 := os.ReadFile(filename)
if err2 != nil {
return nil, perrors.Wrapf(err2, `reading blob from file %q`, filename)
return nil, fmt.Errorf(`reading blob from file %q: %w`, filename, err2)
}
return contents, nil
}
@@ -563,7 +562,7 @@ func (s *storageImageDestination) commitLayer(ctx context.Context, blob manifest
// NOTE: use `TryReusingBlob` to prevent recursion.
has, _, err := s.TryReusingBlob(ctx, blob.BlobInfo, none.NoCache, false)
if err != nil {
return perrors.Wrapf(err, "checking for a layer based on blob %q", blob.Digest.String())
return fmt.Errorf("checking for a layer based on blob %q: %w", blob.Digest.String(), err)
}
if !has {
return fmt.Errorf("error determining uncompressed digest for blob %q", blob.Digest.String())
@@ -611,7 +610,7 @@ func (s *storageImageDestination) commitLayer(ctx context.Context, blob manifest
if ok {
layer, err := al.PutAs(id, lastLayer, nil)
if err != nil && !errors.Is(err, storage.ErrDuplicateID) {
return perrors.Wrapf(err, "failed to put layer from digest and labels")
return fmt.Errorf("failed to put layer from digest and labels: %w", err)
}
lastLayer = layer.ID
s.indexToStorageID[index] = &lastLayer
@@ -636,7 +635,7 @@ func (s *storageImageDestination) commitLayer(ctx context.Context, blob manifest
}
}
if layer == "" {
return perrors.Wrapf(err2, "locating layer for blob %q", blob.Digest)
return fmt.Errorf("locating layer for blob %q: %w", blob.Digest, err2)
}
// Read the layer's contents.
noCompression := archive.Uncompressed
@@ -645,7 +644,7 @@ func (s *storageImageDestination) commitLayer(ctx context.Context, blob manifest
}
diff, err2 := s.imageRef.transport.store.Diff("", layer, diffOptions)
if err2 != nil {
return perrors.Wrapf(err2, "reading layer %q for blob %q", layer, blob.Digest)
return fmt.Errorf("reading layer %q for blob %q: %w", layer, blob.Digest, err2)
}
// Copy the layer diff to a file. Diff() takes a lock that it holds
// until the ReadCloser that it returns is closed, and PutLayer() wants
@@ -655,7 +654,7 @@ func (s *storageImageDestination) commitLayer(ctx context.Context, blob manifest
file, err := os.OpenFile(filename, os.O_CREATE|os.O_TRUNC|os.O_WRONLY|os.O_EXCL, 0600)
if err != nil {
diff.Close()
return perrors.Wrapf(err, "creating temporary file %q", filename)
return fmt.Errorf("creating temporary file %q: %w", filename, err)
}
// Copy the data to the file.
// TODO: This can take quite some time, and should ideally be cancellable using
@@ -664,7 +663,7 @@ func (s *storageImageDestination) commitLayer(ctx context.Context, blob manifest
diff.Close()
file.Close()
if err != nil {
return perrors.Wrapf(err, "storing blob to file %q", filename)
return fmt.Errorf("storing blob to file %q: %w", filename, err)
}
// Make sure that we can find this file later, should we need the layer's
// contents again.
@@ -675,7 +674,7 @@ func (s *storageImageDestination) commitLayer(ctx context.Context, blob manifest
// Read the cached blob and use it as a diff.
file, err := os.Open(filename)
if err != nil {
return perrors.Wrapf(err, "opening file %q", filename)
return fmt.Errorf("opening file %q: %w", filename, err)
}
defer file.Close()
// Build the new layer using the diff, regardless of where it came from.
@@ -685,7 +684,7 @@ func (s *storageImageDestination) commitLayer(ctx context.Context, blob manifest
UncompressedDigest: diffID,
}, file)
if err != nil && !errors.Is(err, storage.ErrDuplicateID) {
return perrors.Wrapf(err, "adding layer with blob %q", blob.Digest)
return fmt.Errorf("adding layer with blob %q: %w", blob.Digest, err)
}
s.indexToStorageID[index] = &layer.ID
@@ -705,7 +704,7 @@ func (s *storageImageDestination) Commit(ctx context.Context, unparsedToplevel t
}
toplevelManifest, _, err := unparsedToplevel.Manifest(ctx)
if err != nil {
return perrors.Wrapf(err, "retrieving top-level manifest")
return fmt.Errorf("retrieving top-level manifest: %w", err)
}
// If the name we're saving to includes a digest, then check that the
// manifests that we're about to save all either match the one from the
@@ -730,7 +729,7 @@ func (s *storageImageDestination) Commit(ctx context.Context, unparsedToplevel t
// Find the list of layer blobs.
man, err := manifest.FromBlob(s.manifest, manifest.GuessMIMEType(s.manifest))
if err != nil {
return perrors.Wrapf(err, "parsing manifest")
return fmt.Errorf("parsing manifest: %w", err)
}
layerBlobs := man.LayerInfos()
@@ -766,15 +765,15 @@ func (s *storageImageDestination) Commit(ctx context.Context, unparsedToplevel t
if err != nil {
if !errors.Is(err, storage.ErrDuplicateID) {
logrus.Debugf("error creating image: %q", err)
return perrors.Wrapf(err, "creating image %q", intendedID)
return fmt.Errorf("creating image %q: %w", intendedID, err)
}
img, err = s.imageRef.transport.store.Image(intendedID)
if err != nil {
return perrors.Wrapf(err, "reading image %q", intendedID)
return fmt.Errorf("reading image %q: %w", intendedID, err)
}
if img.TopLayer != lastLayer {
logrus.Debugf("error creating image: image with ID %q exists, but uses different layers", intendedID)
return perrors.Wrapf(storage.ErrDuplicateID, "image with ID %q already exists, but uses a different top layer", intendedID)
return fmt.Errorf("image with ID %q already exists, but uses a different top layer: %w", intendedID, storage.ErrDuplicateID)
}
logrus.Debugf("reusing image ID %q", img.ID)
oldNames = append(oldNames, img.Names...)
@@ -806,23 +805,23 @@ func (s *storageImageDestination) Commit(ctx context.Context, unparsedToplevel t
for blob := range dataBlobs {
v, err := os.ReadFile(s.filenames[blob])
if err != nil {
return perrors.Wrapf(err, "copying non-layer blob %q to image", blob)
return fmt.Errorf("copying non-layer blob %q to image: %w", blob, err)
}
if err := s.imageRef.transport.store.SetImageBigData(img.ID, blob.String(), v, manifest.Digest); err != nil {
logrus.Debugf("error saving big data %q for image %q: %v", blob.String(), img.ID, err)
return perrors.Wrapf(err, "saving big data %q for image %q", blob.String(), img.ID)
return fmt.Errorf("saving big data %q for image %q: %w", blob.String(), img.ID, err)
}
}
// Save the unparsedToplevel's manifest if it differs from the per-platform one, which is saved below.
if len(toplevelManifest) != 0 && !bytes.Equal(toplevelManifest, s.manifest) {
manifestDigest, err := manifest.Digest(toplevelManifest)
if err != nil {
return perrors.Wrapf(err, "digesting top-level manifest")
return fmt.Errorf("digesting top-level manifest: %w", err)
}
key := manifestBigDataKey(manifestDigest)
if err := s.imageRef.transport.store.SetImageBigData(img.ID, key, toplevelManifest, manifest.Digest); err != nil {
logrus.Debugf("error saving top-level manifest for image %q: %v", img.ID, err)
return perrors.Wrapf(err, "saving top-level manifest for image %q", img.ID)
return fmt.Errorf("saving top-level manifest for image %q: %w", img.ID, err)
}
}
// Save the image's manifest. Allow looking it up by digest by using the key convention defined by the Store.
@@ -831,37 +830,37 @@ func (s *storageImageDestination) Commit(ctx context.Context, unparsedToplevel t
key := manifestBigDataKey(s.manifestDigest)
if err := s.imageRef.transport.store.SetImageBigData(img.ID, key, s.manifest, manifest.Digest); err != nil {
logrus.Debugf("error saving manifest for image %q: %v", img.ID, err)
return perrors.Wrapf(err, "saving manifest for image %q", img.ID)
return fmt.Errorf("saving manifest for image %q: %w", img.ID, err)
}
key = storage.ImageDigestBigDataKey
if err := s.imageRef.transport.store.SetImageBigData(img.ID, key, s.manifest, manifest.Digest); err != nil {
logrus.Debugf("error saving manifest for image %q: %v", img.ID, err)
return perrors.Wrapf(err, "saving manifest for image %q", img.ID)
return fmt.Errorf("saving manifest for image %q: %w", img.ID, err)
}
// Save the signatures, if we have any.
if len(s.signatures) > 0 {
if err := s.imageRef.transport.store.SetImageBigData(img.ID, "signatures", s.signatures, manifest.Digest); err != nil {
logrus.Debugf("error saving signatures for image %q: %v", img.ID, err)
return perrors.Wrapf(err, "saving signatures for image %q", img.ID)
return fmt.Errorf("saving signatures for image %q: %w", img.ID, err)
}
}
for instanceDigest, signatures := range s.signatureses {
key := signatureBigDataKey(instanceDigest)
if err := s.imageRef.transport.store.SetImageBigData(img.ID, key, signatures, manifest.Digest); err != nil {
logrus.Debugf("error saving signatures for image %q: %v", img.ID, err)
return perrors.Wrapf(err, "saving signatures for image %q", img.ID)
return fmt.Errorf("saving signatures for image %q: %w", img.ID, err)
}
}
// Save our metadata.
metadata, err := json.Marshal(s)
if err != nil {
logrus.Debugf("error encoding metadata for image %q: %v", img.ID, err)
return perrors.Wrapf(err, "encoding metadata for image %q", img.ID)
return fmt.Errorf("encoding metadata for image %q: %w", img.ID, err)
}
if len(metadata) != 0 {
if err = s.imageRef.transport.store.SetMetadata(img.ID, string(metadata)); err != nil {
logrus.Debugf("error saving metadata for image %q: %v", img.ID, err)
return perrors.Wrapf(err, "saving metadata for image %q", img.ID)
return fmt.Errorf("saving metadata for image %q: %w", img.ID, err)
}
logrus.Debugf("saved image metadata %q", string(metadata))
}
@@ -869,7 +868,7 @@ func (s *storageImageDestination) Commit(ctx context.Context, unparsedToplevel t
// values because AddNames() will deduplicate the list that we pass to it.
if name := s.imageRef.DockerReference(); name != nil {
if err := s.imageRef.transport.store.AddNames(img.ID, []string{name.String()}); err != nil {
return perrors.Wrapf(err, "adding names %v to image %q", name, img.ID)
return fmt.Errorf("adding names %v to image %q: %w", name, img.ID, err)
}
logrus.Debugf("added name %q to image %q", name, img.ID)
}

View File

@@ -5,6 +5,7 @@ package storage
import (
"context"
"fmt"
"strings"
"github.com/containers/image/v5/docker/reference"
@@ -12,7 +13,6 @@ import (
"github.com/containers/image/v5/types"
"github.com/containers/storage"
digest "github.com/opencontainers/go-digest"
perrors "github.com/pkg/errors"
"github.com/sirupsen/logrus"
)
@@ -31,11 +31,11 @@ func newReference(transport storageTransport, named reference.Named, id string)
return nil, ErrInvalidReference
}
if named != nil && reference.IsNameOnly(named) {
return nil, perrors.Wrapf(ErrInvalidReference, "reference %s has neither a tag nor a digest", named.String())
return nil, fmt.Errorf("reference %s has neither a tag nor a digest: %w", named.String(), ErrInvalidReference)
}
if id != "" {
if err := validateImageID(id); err != nil {
return nil, perrors.Wrapf(ErrInvalidReference, "invalid ID value %q: %v", id, err)
return nil, fmt.Errorf("invalid ID value %q: %v: %w", id, err, ErrInvalidReference)
}
}
// We take a copy of the transport, which contains a pointer to the
@@ -145,12 +145,12 @@ func (s *storageReference) resolveImage(sys *types.SystemContext) (*storage.Imag
}
if s.id == "" {
logrus.Debugf("reference %q does not resolve to an image ID", s.StringWithinTransport())
return nil, perrors.Wrapf(ErrNoSuchImage, "reference %q does not resolve to an image ID", s.StringWithinTransport())
return nil, fmt.Errorf("reference %q does not resolve to an image ID: %w", s.StringWithinTransport(), ErrNoSuchImage)
}
if loadedImage == nil {
img, err := s.transport.store.Image(s.id)
if err != nil {
return nil, perrors.Wrapf(err, "reading image %q", s.id)
return nil, fmt.Errorf("reading image %q: %w", s.id, err)
}
loadedImage = img
}

View File

@@ -26,7 +26,6 @@ import (
"github.com/containers/storage/pkg/ioutils"
digest "github.com/opencontainers/go-digest"
imgspecv1 "github.com/opencontainers/image-spec/specs-go/v1"
perrors "github.com/pkg/errors"
"github.com/sirupsen/logrus"
)
@@ -70,7 +69,7 @@ func newImageSource(ctx context.Context, sys *types.SystemContext, imageRef stor
image.Compat = impl.AddCompat(image)
if img.Metadata != "" {
if err := json.Unmarshal([]byte(img.Metadata), image); err != nil {
return nil, perrors.Wrap(err, "decoding metadata for source image")
return nil, fmt.Errorf("decoding metadata for source image: %w", err)
}
}
return image, nil
@@ -183,7 +182,7 @@ func (s *storageImageSource) GetManifest(ctx context.Context, instanceDigest *di
key := manifestBigDataKey(*instanceDigest)
blob, err := s.imageRef.transport.store.ImageBigData(s.image.ID, key)
if err != nil {
return nil, "", perrors.Wrapf(err, "reading manifest for image instance %q", *instanceDigest)
return nil, "", fmt.Errorf("reading manifest for image instance %q: %w", *instanceDigest, err)
}
return blob, manifest.GuessMIMEType(blob), err
}
@@ -220,14 +219,14 @@ func (s *storageImageSource) GetManifest(ctx context.Context, instanceDigest *di
func (s *storageImageSource) LayerInfosForCopy(ctx context.Context, instanceDigest *digest.Digest) ([]types.BlobInfo, error) {
manifestBlob, manifestType, err := s.GetManifest(ctx, instanceDigest)
if err != nil {
return nil, perrors.Wrapf(err, "reading image manifest for %q", s.image.ID)
return nil, fmt.Errorf("reading image manifest for %q: %w", s.image.ID, err)
}
if manifest.MIMETypeIsMultiImage(manifestType) {
return nil, errors.New("can't copy layers for a manifest list (shouldn't be attempted)")
}
man, err := manifest.FromBlob(manifestBlob, manifestType)
if err != nil {
return nil, perrors.Wrapf(err, "parsing image manifest for %q", s.image.ID)
return nil, fmt.Errorf("parsing image manifest for %q: %w", s.image.ID, err)
}
uncompressedLayerType := ""
@@ -243,7 +242,7 @@ func (s *storageImageSource) LayerInfosForCopy(ctx context.Context, instanceDige
for layerID != "" {
layer, err := s.imageRef.transport.store.Layer(layerID)
if err != nil {
return nil, perrors.Wrapf(err, "reading layer %q in image %q", layerID, s.image.ID)
return nil, fmt.Errorf("reading layer %q in image %q: %w", layerID, s.image.ID, err)
}
if layer.UncompressedDigest == "" {
return nil, fmt.Errorf("uncompressed digest for layer %q is unknown", layerID)
@@ -262,7 +261,7 @@ func (s *storageImageSource) LayerInfosForCopy(ctx context.Context, instanceDige
res, err := buildLayerInfosForCopy(man.LayerInfos(), physicalBlobInfos)
if err != nil {
return nil, perrors.Wrapf(err, "creating LayerInfosForCopy of image %q", s.image.ID)
return nil, fmt.Errorf("creating LayerInfosForCopy of image %q: %w", s.image.ID, err)
}
return res, nil
}
@@ -313,7 +312,7 @@ func (s *storageImageSource) GetSignaturesWithFormat(ctx context.Context, instan
if len(signatureSizes) > 0 {
data, err := s.imageRef.transport.store.ImageBigData(s.image.ID, key)
if err != nil {
return nil, perrors.Wrapf(err, "looking up signatures data for image %q (%s)", s.image.ID, instance)
return nil, fmt.Errorf("looking up signatures data for image %q (%s): %w", s.image.ID, instance, err)
}
signatureBlobs = data
}
@@ -342,12 +341,12 @@ func (s *storageImageSource) getSize() (int64, error) {
// Size up the data blobs.
dataNames, err := s.imageRef.transport.store.ListImageBigData(s.image.ID)
if err != nil {
return -1, perrors.Wrapf(err, "reading image %q", s.image.ID)
return -1, fmt.Errorf("reading image %q: %w", s.image.ID, err)
}
for _, dataName := range dataNames {
bigSize, err := s.imageRef.transport.store.ImageBigDataSize(s.image.ID, dataName)
if err != nil {
return -1, perrors.Wrapf(err, "reading data blob size %q for %q", dataName, s.image.ID)
return -1, fmt.Errorf("reading data blob size %q for %q: %w", dataName, s.image.ID, err)
}
sum += bigSize
}

View File

@@ -15,7 +15,6 @@ import (
"github.com/containers/storage"
"github.com/containers/storage/pkg/idtools"
digest "github.com/opencontainers/go-digest"
perrors "github.com/pkg/errors"
"github.com/sirupsen/logrus"
)
@@ -118,13 +117,13 @@ func (s *storageTransport) DefaultGIDMap() []idtools.IDMap {
// relative to the given store, and returns it in a reference object.
func (s storageTransport) ParseStoreReference(store storage.Store, ref string) (*storageReference, error) {
if ref == "" {
return nil, perrors.Wrapf(ErrInvalidReference, "%q is an empty reference", ref)
return nil, fmt.Errorf("%q is an empty reference: %w", ref, ErrInvalidReference)
}
if ref[0] == '[' {
// Ignore the store specifier.
closeIndex := strings.IndexRune(ref, ']')
if closeIndex < 1 {
return nil, perrors.Wrapf(ErrInvalidReference, "store specifier in %q did not end", ref)
return nil, fmt.Errorf("store specifier in %q did not end: %w", ref, ErrInvalidReference)
}
ref = ref[closeIndex+1:]
}
@@ -136,7 +135,7 @@ func (s storageTransport) ParseStoreReference(store storage.Store, ref string) (
if split != -1 {
possibleID := ref[split+1:]
if possibleID == "" {
return nil, perrors.Wrapf(ErrInvalidReference, "empty trailing digest or ID in %q", ref)
return nil, fmt.Errorf("empty trailing digest or ID in %q: %w", ref, ErrInvalidReference)
}
// If it looks like a digest, leave it alone for now.
if _, err := digest.Parse(possibleID); err != nil {
@@ -148,7 +147,7 @@ func (s storageTransport) ParseStoreReference(store storage.Store, ref string) (
// so we might as well use the expanded value.
id = img.ID
} else {
return nil, perrors.Wrapf(ErrInvalidReference, "%q does not look like an image ID or digest", possibleID)
return nil, fmt.Errorf("%q does not look like an image ID or digest: %w", possibleID, ErrInvalidReference)
}
// We have recognized an image ID; peel it off.
ref = ref[:split]
@@ -174,7 +173,7 @@ func (s storageTransport) ParseStoreReference(store storage.Store, ref string) (
var err error
named, err = reference.ParseNormalizedNamed(ref)
if err != nil {
return nil, perrors.Wrapf(err, "parsing named reference %q", ref)
return nil, fmt.Errorf("parsing named reference %q: %w", ref, err)
}
named = reference.TagNameOnly(named)
}

View File

@@ -6,12 +6,12 @@ const (
// VersionMajor is for an API incompatible changes
VersionMajor = 5
// VersionMinor is for functionality in a backwards-compatible manner
VersionMinor = 21
VersionMinor = 22
// VersionPatch is for backwards-compatible bug fixes
VersionPatch = 2
VersionPatch = 1
// VersionDev indicates development branch. Releases will be empty string.
VersionDev = "-dev"
VersionDev = ""
)
// Version is the specification version that the package types support.

View File

@@ -17,14 +17,14 @@ env:
####
#### Cache-image names to test with (double-quotes around names are critical)
###
FEDORA_NAME: "fedora-35"
PRIOR_FEDORA_NAME: "fedora-34"
UBUNTU_NAME: "ubuntu-2104"
FEDORA_NAME: "fedora-36"
PRIOR_FEDORA_NAME: "fedora-35"
UBUNTU_NAME: "ubuntu-2204"
# GCE project where images live
IMAGE_PROJECT: "libpod-218412"
# VM Image built in containers/automation_images
IMAGE_SUFFIX: "c4512539143831552"
IMAGE_SUFFIX: "c5878804328480768"
FEDORA_CACHE_IMAGE_NAME: "fedora-${IMAGE_SUFFIX}"
PRIOR_FEDORA_CACHE_IMAGE_NAME: "prior-fedora-${IMAGE_SUFFIX}"
UBUNTU_CACHE_IMAGE_NAME: "ubuntu-${IMAGE_SUFFIX}"

View File

@@ -1 +1 @@
1.41.0
1.42.0

View File

@@ -1,6 +1,7 @@
package storage
import (
"errors"
"fmt"
"io/ioutil"
"os"
@@ -13,7 +14,6 @@ import (
"github.com/containers/storage/pkg/stringid"
"github.com/containers/storage/pkg/truncindex"
digest "github.com/opencontainers/go-digest"
"github.com/pkg/errors"
)
// A Container is a reference to a read-write layer with metadata.
@@ -329,8 +329,7 @@ func (r *containerStore) Create(id string, names []string, image, layer, metadat
names = dedupeNames(names)
for _, name := range names {
if _, nameInUse := r.byname[name]; nameInUse {
return nil, errors.Wrapf(ErrDuplicateName,
fmt.Sprintf("the container name \"%s\" is already in use by \"%s\". You have to remove that container to be able to reuse that name.", name, r.byname[name].ID))
return nil, fmt.Errorf("the container name %q is already in use by %s. You have to remove that container to be able to reuse that name: %w", name, r.byname[name].ID, ErrDuplicateName)
}
}
if err := hasOverlappingRanges(options.UIDMap); err != nil {
@@ -479,7 +478,7 @@ func (r *containerStore) Exists(id string) bool {
func (r *containerStore) BigData(id, key string) ([]byte, error) {
if key == "" {
return nil, errors.Wrapf(ErrInvalidBigDataName, "can't retrieve container big data value for empty name")
return nil, fmt.Errorf("can't retrieve container big data value for empty name: %w", ErrInvalidBigDataName)
}
c, ok := r.lookup(id)
if !ok {
@@ -490,7 +489,7 @@ func (r *containerStore) BigData(id, key string) ([]byte, error) {
func (r *containerStore) BigDataSize(id, key string) (int64, error) {
if key == "" {
return -1, errors.Wrapf(ErrInvalidBigDataName, "can't retrieve size of container big data with empty name")
return -1, fmt.Errorf("can't retrieve size of container big data with empty name: %w", ErrInvalidBigDataName)
}
c, ok := r.lookup(id)
if !ok {
@@ -520,7 +519,7 @@ func (r *containerStore) BigDataSize(id, key string) (int64, error) {
func (r *containerStore) BigDataDigest(id, key string) (digest.Digest, error) {
if key == "" {
return "", errors.Wrapf(ErrInvalidBigDataName, "can't retrieve digest of container big data value with empty name")
return "", fmt.Errorf("can't retrieve digest of container big data value with empty name: %w", ErrInvalidBigDataName)
}
c, ok := r.lookup(id)
if !ok {
@@ -558,7 +557,7 @@ func (r *containerStore) BigDataNames(id string) ([]string, error) {
func (r *containerStore) SetBigData(id, key string, data []byte) error {
if key == "" {
return errors.Wrapf(ErrInvalidBigDataName, "can't set empty name for container big data item")
return fmt.Errorf("can't set empty name for container big data item: %w", ErrInvalidBigDataName)
}
c, ok := r.lookup(id)
if !ok {

View File

@@ -25,6 +25,7 @@ package aufs
import (
"bufio"
"errors"
"fmt"
"io"
"io/fs"
@@ -48,7 +49,6 @@ import (
"github.com/containers/storage/pkg/system"
"github.com/opencontainers/runc/libcontainer/userns"
"github.com/opencontainers/selinux/go-selinux/label"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
"github.com/vbatts/tar-split/tar/storage"
"golang.org/x/sys/unix"
@@ -56,9 +56,9 @@ import (
var (
// ErrAufsNotSupported is returned if aufs is not supported by the host.
ErrAufsNotSupported = fmt.Errorf("AUFS was not found in /proc/filesystems")
ErrAufsNotSupported = fmt.Errorf("aufs was not found in /proc/filesystems")
// ErrAufsNested means aufs cannot be used bc we are in a user namespace
ErrAufsNested = fmt.Errorf("AUFS cannot be used in non-init user namespace")
ErrAufsNested = fmt.Errorf("aufs cannot be used in non-init user namespace")
backingFs = "<unknown>"
enableDirpermLock sync.Once
@@ -91,7 +91,7 @@ func Init(home string, options graphdriver.Options) (graphdriver.Driver, error)
// Try to load the aufs kernel module
if err := supportsAufs(); err != nil {
return nil, errors.Wrap(graphdriver.ErrNotSupported, "kernel does not support aufs")
return nil, fmt.Errorf("kernel does not support aufs: %w", graphdriver.ErrNotSupported)
}
@@ -106,7 +106,7 @@ func Init(home string, options graphdriver.Options) (graphdriver.Driver, error)
switch fsMagic {
case graphdriver.FsMagicAufs, graphdriver.FsMagicBtrfs, graphdriver.FsMagicEcryptfs:
logrus.Errorf("AUFS is not supported over %s", backingFs)
return nil, errors.Wrapf(graphdriver.ErrIncompatibleFS, "AUFS is not supported over %q", backingFs)
return nil, fmt.Errorf("aufs is not supported over %q: %w", backingFs, graphdriver.ErrIncompatibleFS)
}
var mountOptions string
@@ -374,10 +374,10 @@ func (a *Driver) Remove(id string) error {
}
if err != unix.EBUSY {
return errors.Wrapf(err, "aufs: unmount error: %s", mountpoint)
return fmt.Errorf("aufs: unmount error: %s: %w", mountpoint, err)
}
if retries >= 5 {
return errors.Wrapf(err, "aufs: unmount error after retries: %s", mountpoint)
return fmt.Errorf("aufs: unmount error after retries: %s: %w", mountpoint, err)
}
// If unmount returns EBUSY, it could be a transient error. Sleep and retry.
retries++
@@ -387,21 +387,21 @@ func (a *Driver) Remove(id string) error {
// Remove the layers file for the id
if err := os.Remove(path.Join(a.rootPath(), "layers", id)); err != nil && !os.IsNotExist(err) {
return errors.Wrapf(err, "error removing layers dir for %s", id)
return fmt.Errorf("removing layers dir for %s: %w", id, err)
}
if err := atomicRemove(a.getDiffPath(id)); err != nil {
return errors.Wrapf(err, "could not remove diff path for id %s", id)
return fmt.Errorf("could not remove diff path for id %s: %w", id, err)
}
// Atomically remove each directory in turn by first moving it out of the
// way (so that container runtime doesn't find it anymore) before doing removal of
// the whole tree.
if err := atomicRemove(mountpoint); err != nil {
if errors.Cause(err) == unix.EBUSY {
if errors.Is(err, unix.EBUSY) {
logger.WithField("dir", mountpoint).WithError(err).Warn("error performing atomic remove due to EBUSY")
}
return errors.Wrapf(err, "could not remove mountpoint for id %s", id)
return fmt.Errorf("could not remove mountpoint for id %s: %w", id, err)
}
a.pathCacheLock.Lock()
@@ -419,10 +419,10 @@ func atomicRemove(source string) error {
case os.IsExist(err):
// Got error saying the target dir already exists, maybe the source doesn't exist due to a previous (failed) remove
if _, e := os.Stat(source); !os.IsNotExist(e) {
return errors.Wrapf(err, "target rename dir '%s' exists but should not, this needs to be manually cleaned up", target)
return fmt.Errorf("target rename dir '%s' exists but should not, this needs to be manually cleaned up: %w", target, err)
}
default:
return errors.Wrapf(err, "error preparing atomic delete")
return fmt.Errorf("preparing atomic delete: %w", err)
}
return system.EnsureRemoveAll(target)
@@ -626,7 +626,7 @@ func (a *Driver) mount(id string, target string, layers []string, options graphd
rw := a.getDiffPath(id)
if err := a.aufsMount(layers, rw, target, options); err != nil {
return fmt.Errorf("error creating aufs mount to %s: %v", target, err)
return fmt.Errorf("creating aufs mount to %s: %w", target, err)
}
return nil
}

View File

@@ -36,7 +36,6 @@ import (
"github.com/containers/storage/pkg/system"
"github.com/docker/go-units"
"github.com/opencontainers/selinux/go-selinux/label"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
"golang.org/x/sys/unix"
)
@@ -62,7 +61,7 @@ func Init(home string, options graphdriver.Options) (graphdriver.Driver, error)
}
if fsMagic != graphdriver.FsMagicBtrfs {
return nil, errors.Wrapf(graphdriver.ErrPrerequisites, "%q is not on a btrfs filesystem", home)
return nil, fmt.Errorf("%q is not on a btrfs filesystem: %w", home, graphdriver.ErrPrerequisites)
}
rootUID, rootGID, err := idtools.GetRootUIDGID(options.UIDMaps, options.GIDMaps)
@@ -118,7 +117,7 @@ func parseOptions(opt []string) (btrfsOptions, bool, error) {
case "btrfs.mountopt":
return options, userDiskQuota, fmt.Errorf("btrfs driver does not support mount options")
default:
return options, userDiskQuota, fmt.Errorf("Unknown option %s", key)
return options, userDiskQuota, fmt.Errorf("unknown option %s", key)
}
}
return options, userDiskQuota, nil
@@ -174,7 +173,7 @@ func openDir(path string) (*C.DIR, error) {
dir := C.opendir(Cpath)
if dir == nil {
return nil, fmt.Errorf("Can't open dir")
return nil, fmt.Errorf("can't open dir %s", path)
}
return dir, nil
}
@@ -204,7 +203,7 @@ func subvolCreate(path, name string) error {
_, _, errno := unix.Syscall(unix.SYS_IOCTL, getDirFd(dir), C.BTRFS_IOC_SUBVOL_CREATE,
uintptr(unsafe.Pointer(&args)))
if errno != 0 {
return fmt.Errorf("Failed to create btrfs subvolume: %v", errno.Error())
return fmt.Errorf("failed to create btrfs subvolume: %w", errno)
}
return nil
}
@@ -232,7 +231,7 @@ func subvolSnapshot(src, dest, name string) error {
_, _, errno := unix.Syscall(unix.SYS_IOCTL, getDirFd(destDir), C.BTRFS_IOC_SNAP_CREATE_V2,
uintptr(unsafe.Pointer(&args)))
if errno != 0 {
return fmt.Errorf("Failed to create btrfs snapshot: %v", errno.Error())
return fmt.Errorf("failed to create btrfs snapshot: %w", errno)
}
return nil
}
@@ -265,25 +264,25 @@ func subvolDelete(dirpath, name string, quotaEnabled bool) error {
// since it's gone anyway, we don't care
return nil
}
return fmt.Errorf("error walking subvolumes: %v", err)
return fmt.Errorf("walking subvolumes: %w", err)
}
// we want to check children only so skip itself
// it will be removed after the filepath walk anyways
if d.IsDir() && p != fullPath {
sv, err := isSubvolume(p)
if err != nil {
return fmt.Errorf("Failed to test if %s is a btrfs subvolume: %v", p, err)
return fmt.Errorf("failed to test if %s is a btrfs subvolume: %w", p, err)
}
if sv {
if err := subvolDelete(path.Dir(p), d.Name(), quotaEnabled); err != nil {
return fmt.Errorf("Failed to destroy btrfs child subvolume (%s) of parent (%s): %v", p, dirpath, err)
return fmt.Errorf("failed to destroy btrfs child subvolume (%s) of parent (%s): %w", p, dirpath, err)
}
}
}
return nil
}
if err := filepath.WalkDir(path.Join(dirpath, name), walkSubvolumes); err != nil {
return fmt.Errorf("Recursively walking subvolumes for %s failed: %v", dirpath, err)
return fmt.Errorf("recursively walking subvolumes for %s failed: %w", dirpath, err)
}
if quotaEnabled {
@@ -309,7 +308,7 @@ func subvolDelete(dirpath, name string, quotaEnabled bool) error {
_, _, errno := unix.Syscall(unix.SYS_IOCTL, getDirFd(dir), C.BTRFS_IOC_SNAP_DESTROY,
uintptr(unsafe.Pointer(&args)))
if errno != 0 {
return fmt.Errorf("Failed to destroy btrfs snapshot %s for %s: %v", dirpath, name, errno.Error())
return fmt.Errorf("failed to destroy btrfs snapshot %s for %s: %w", dirpath, name, errno)
}
return nil
}
@@ -345,7 +344,7 @@ func (d *Driver) enableQuota() error {
_, _, errno := unix.Syscall(unix.SYS_IOCTL, getDirFd(dir), C.BTRFS_IOC_QUOTA_CTL,
uintptr(unsafe.Pointer(&args)))
if errno != 0 {
return fmt.Errorf("Failed to enable btrfs quota for %s: %v", dir, errno.Error())
return fmt.Errorf("failed to enable btrfs quota for %s: %w", dir, errno)
}
d.quotaEnabled = true
@@ -370,7 +369,7 @@ func (d *Driver) subvolRescanQuota() error {
_, _, errno := unix.Syscall(unix.SYS_IOCTL, getDirFd(dir), C.BTRFS_IOC_QUOTA_RESCAN_WAIT,
uintptr(unsafe.Pointer(&args)))
if errno != 0 {
return fmt.Errorf("Failed to rescan btrfs quota for %s: %v", dir, errno.Error())
return fmt.Errorf("failed to rescan btrfs quota for %s: %w", dir, errno)
}
return nil
@@ -389,7 +388,7 @@ func subvolLimitQgroup(path string, size uint64) error {
_, _, errno := unix.Syscall(unix.SYS_IOCTL, getDirFd(dir), C.BTRFS_IOC_QGROUP_LIMIT,
uintptr(unsafe.Pointer(&args)))
if errno != 0 {
return fmt.Errorf("Failed to limit qgroup for %s: %v", dir, errno.Error())
return fmt.Errorf("failed to limit qgroup for %s: %w", dir, errno)
}
return nil
@@ -418,11 +417,11 @@ func qgroupStatus(path string) error {
_, _, errno := unix.Syscall(unix.SYS_IOCTL, getDirFd(dir), C.BTRFS_IOC_TREE_SEARCH,
uintptr(unsafe.Pointer(&args)))
if errno != 0 {
return fmt.Errorf("Failed to search qgroup for %s: %v", path, errno.Error())
return fmt.Errorf("failed to search qgroup for %s: %w", path, errno)
}
sh := (*C.struct_btrfs_ioctl_search_header)(unsafe.Pointer(&args.buf))
if sh._type != C.BTRFS_QGROUP_STATUS_KEY {
return fmt.Errorf("Invalid qgroup search header type for %s: %v", path, sh._type)
return fmt.Errorf("invalid qgroup search header type for %s: %v", path, sh._type)
}
return nil
}
@@ -440,10 +439,10 @@ func subvolLookupQgroup(path string) (uint64, error) {
_, _, errno := unix.Syscall(unix.SYS_IOCTL, getDirFd(dir), C.BTRFS_IOC_INO_LOOKUP,
uintptr(unsafe.Pointer(&args)))
if errno != 0 {
return 0, fmt.Errorf("Failed to lookup qgroup for %s: %v", dir, errno.Error())
return 0, fmt.Errorf("failed to lookup qgroup for %s: %w", dir, errno)
}
if args.treeid == 0 {
return 0, fmt.Errorf("Invalid qgroup id for %s: 0", dir)
return 0, fmt.Errorf("invalid qgroup id for %s: 0", dir)
}
return uint64(args.treeid), nil
@@ -559,7 +558,7 @@ func (d *Driver) parseStorageOpt(storageOpt map[string]string, driver *Driver) e
}
driver.options.size = uint64(size)
default:
return fmt.Errorf("Unknown option %s", key)
return fmt.Errorf("unknown option %s", key)
}
}

View File

@@ -87,13 +87,13 @@ func ChownPathByMaps(path string, toContainer, toHost *idtools.IDMappings) error
cmd.Stdin = bytes.NewReader(config)
output, err := cmd.CombinedOutput()
if len(output) > 0 && err != nil {
return fmt.Errorf("%v: %s", err, string(output))
return fmt.Errorf("%s: %w", string(output), err)
}
if err != nil {
return err
}
if len(output) > 0 {
return fmt.Errorf("%s", string(output))
return fmt.Errorf(string(output))
}
return nil

View File

@@ -0,0 +1,109 @@
//go:build darwin
// +build darwin
package graphdriver
import (
"errors"
"fmt"
"os"
"sync"
"syscall"
"github.com/containers/storage/pkg/idtools"
"github.com/containers/storage/pkg/system"
)
type inode struct {
Dev uint64
Ino uint64
}
type platformChowner struct {
mutex sync.Mutex
inodes map[inode]bool
}
func newLChowner() *platformChowner {
return &platformChowner{
inodes: make(map[inode]bool),
}
}
func (c *platformChowner) LChown(path string, info os.FileInfo, toHost, toContainer *idtools.IDMappings) error {
st, ok := info.Sys().(*syscall.Stat_t)
if !ok {
return nil
}
i := inode{
Dev: uint64(st.Dev),
Ino: uint64(st.Ino),
}
c.mutex.Lock()
_, found := c.inodes[i]
if !found {
c.inodes[i] = true
}
c.mutex.Unlock()
if found {
return nil
}
// Map an on-disk UID/GID pair from host to container
// using the first map, then back to the host using the
// second map. Skip that first step if they're 0, to
// compensate for cases where a parent layer should
// have had a mapped value, but didn't.
uid, gid := int(st.Uid), int(st.Gid)
if toContainer != nil {
pair := idtools.IDPair{
UID: uid,
GID: gid,
}
mappedUID, mappedGID, err := toContainer.ToContainer(pair)
if err != nil {
if (uid != 0) || (gid != 0) {
return fmt.Errorf("mapping host ID pair %#v for %q to container: %w", pair, path, err)
}
mappedUID, mappedGID = uid, gid
}
uid, gid = mappedUID, mappedGID
}
if toHost != nil {
pair := idtools.IDPair{
UID: uid,
GID: gid,
}
mappedPair, err := toHost.ToHostOverflow(pair)
if err != nil {
return fmt.Errorf("mapping container ID pair %#v for %q to host: %w", pair, path, err)
}
uid, gid = mappedPair.UID, mappedPair.GID
}
if uid != int(st.Uid) || gid != int(st.Gid) {
cap, err := system.Lgetxattr(path, "security.capability")
if err != nil && !errors.Is(err, system.EOPNOTSUPP) && err != system.ErrNotSupportedPlatform {
return fmt.Errorf("%s: %w", os.Args[0], err)
}
// Make the change.
if err := system.Lchown(path, uid, gid); err != nil {
return fmt.Errorf("%s: %w", os.Args[0], err)
}
// Restore the SUID and SGID bits if they were originally set.
if (info.Mode()&os.ModeSymlink == 0) && info.Mode()&(os.ModeSetuid|os.ModeSetgid) != 0 {
if err := system.Chmod(path, info.Mode()); err != nil {
return fmt.Errorf("%s: %w", os.Args[0], err)
}
}
if cap != nil {
if err := system.Lsetxattr(path, "security.capability", cap, 0); err != nil {
return fmt.Errorf("%s: %w", os.Args[0], err)
}
}
}
return nil
}

Some files were not shown because too many files have changed in this diff Show More