Compare commits

...

415 Commits

Author SHA1 Message Date
Antonio Murdaca
416ff71bed bump version to 0.1.15
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-09-26 17:55:40 +02:00
Jonathan Boulle
4ed4525155 Update README.md 2016-09-26 17:55:40 +02:00
Antonio Murdaca
c813de92d8 Merge pull request #216 from runcom/progress-fix
fix containers/image progress
2016-09-26 17:42:06 +02:00
Antonio Murdaca
0420fa0299 fix containers/image progress
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-09-26 17:21:08 +02:00
Miloslav Trmač
5b7fcc8eca Merge pull request #215 from runcom/progress
add progress bars during copy
2016-09-26 16:21:04 +02:00
Antonio Murdaca
c84203bdd5 add progress bars during copy
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-09-26 15:59:37 +02:00
Miloslav Trmač
98bfef9072 Merge pull request #211 from mtrmac/compress
Layer compression
2016-09-19 22:06:14 +02:00
Miloslav Trmač
4ec3b64c84 Add an integration tests for compression during upload
Sadly, most of the cases are disabled for now; hopefully this will get
fixed soon.
2016-09-19 21:39:59 +02:00
Miloslav Trmač
d705644f22 Vendor after merging mtrmac/image:compress 2016-09-19 21:39:46 +02:00
Miloslav Trmač
9c1cb79754 Merge pull request #210 from mtrmac/api-changes
Vendor after merging mtrmac/image:api-changes and update API use
2016-09-19 17:11:22 +02:00
Miloslav Trmač
459ab05b22 Vendor after merging mtrmac/image:api-changes and update API use 2016-09-19 16:44:47 +02:00
Miloslav Trmač
2e7b6c9d14 Merge pull request #205 from mtrmac/tls-verification
TLS verification in docker registries
2016-09-13 19:48:07 +02:00
Miloslav Trmač
2a8ffee621 Flip --tls-verify default to true
Document better what --tls-verify does

... and sprinkle --tls-verify=false over integration tests.
2016-09-13 19:26:21 +02:00
Miloslav Trmač
623865c159 Vendor after merging mtrmac/image:tls-verification 2016-09-13 19:25:42 +02:00
Antonio Murdaca
58ec828eab Merge pull request #204 from mtrmac/registries.d
Create /etc/containers/registries.d in (make install)
2016-09-13 18:30:07 +02:00
Miloslav Trmač
9835ae579b Create /etc/containers/registries.d in (make install) 2016-09-13 18:08:25 +02:00
Antonio Murdaca
14847101c0 Merge pull request #202 from runcom/change-os-uri
Change atomic URI
2016-09-13 17:11:07 +02:00
Antonio Murdaca
3980ac5894 vendor containers/image#81
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-09-13 16:50:28 +02:00
Miloslav Trmač
d6be447ce9 Merge pull request #170 from mtrmac/docker-lookaside
Implement a lookaside storage for signatures of images in Docker registries
2016-09-12 21:39:39 +02:00
Miloslav Trmač
b6fdea03f2 Add a global --registries.d option to skopeo
This is added pretty much only for integration tests right now;
though, it might be useful also for non-root operation.

Also makes a tiny cleanup of contextFromGlobalOptions, removing a
variable.
2016-09-12 21:13:53 +02:00
Miloslav Trmač
f46da343e2 Vendor after merging in mtrmac/image:docker-lookaside 2016-09-12 21:13:34 +02:00
Antonio Murdaca
d1d1d6533e Merge pull request #201 from runcom/fix-198
vendor containers/image#84
2016-09-12 17:47:00 +02:00
Antonio Murdaca
890c073526 vendor containers/image#84
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-09-12 17:25:52 +02:00
Antonio Murdaca
7e69022723 Merge pull request #196 from runcom/crane-fix
vendor containers/image to fix RH
2016-09-09 11:55:37 +02:00
Antonio Murdaca
1c16cd5e9d vendor containers/image to fix RH
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-09-09 11:34:58 +02:00
Miloslav Trmač
362bfc5fe3 Merge pull request #195 from runcom/vendor-cont/images
vendor containers/image, OCI/image-spec
2016-09-08 14:03:43 +02:00
Antonio Murdaca
81d67eab92 vendor containers/image, OCI/image-spec
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-09-08 13:23:41 +02:00
Miloslav Trmač
fc0c5be08d Merge pull request #192 from rhatdan/install
Refer to the policy file as a trust policy file.
2016-09-07 17:42:29 +02:00
Dan Walsh
824853d85d Refer to the signature trust policy.
The policy file is actualy indicatiting the signatures that the
user trusts.  This patch changes the documentation and error messages
to indicate this trust.
2016-09-07 10:18:14 -04:00
Antonio Murdaca
2c78131d1d Merge pull request #171 from aweiteka/makefile
Fix selinux perms in Makefile binary build
2016-09-06 23:03:55 +02:00
Aaron Weitekamp
157b9c0f3b disable selinux for binary build 2016-09-06 16:28:07 -04:00
Antonio Murdaca
ee89d2c6a4 Merge pull request #190 from runcom/fix-putblob
vendor containers/image for PutBlob returns
2016-09-06 20:10:16 +02:00
Antonio Murdaca
4e40830eae vendor containers/image for PutBlob returns
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-09-06 19:47:59 +02:00
Miloslav Trmač
46ffaa8e51 Merge pull request #188 from runcom/vendor-image-spec
vendor containers/image and OCI/image-spec
2016-09-06 16:50:18 +02:00
Antonio Murdaca
649ea391a4 vendor containers/image and OCI/image-spec
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-09-06 16:19:52 +02:00
Miloslav Trmač
4421e7ea2f Merge pull request #187 from mtrmac/api-changes
Update for mtrmac/image:api-changes
2016-09-06 16:03:28 +02:00
Miloslav Trmač
e8794bd9ff Vendor after merging in mtrmac/image:api-changes
... and update for the API changes.
2016-09-06 15:37:39 +02:00
Antonio Murdaca
136fd1d8a6 Merge pull request #185 from mtrmac/remove-signatures
Add --remove-signatures to (skopeo copy)
2016-09-05 19:34:42 +02:00
Miloslav Trmač
f627fc6045 Add --remove-signatures to (skopeo copy)
This is necessary to allow copying signed images into destinations which
don't support signatures.
2016-09-01 22:34:13 +02:00
Miloslav Trmač
7c2a47f8b9 Vendor after merging mtrmac/image:remove-signatures 2016-09-01 22:17:04 +02:00
Antonio Murdaca
1bfb549f7f Merge pull request #182 from runcom/fix-oci
vendor containers/image for oci dest fix
2016-09-01 18:01:44 +02:00
Antonio Murdaca
9914de1bf4 vendor containers/image for oci dest fix
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-09-01 17:38:17 +02:00
Antonio Murdaca
f37d72d964 Merge pull request #175 from mtrmac/copy
Move copy implementation into containers/image
2016-09-01 16:55:34 +02:00
Miloslav Trmač
3e3748a800 Move the core of the (skopeo copy) implementation to containers/image 2016-09-01 16:27:38 +02:00
Miloslav Trmač
61158ce7f4 Vendor after merging mtrmac/image:copy 2016-09-01 16:27:22 +02:00
Miloslav Trmač
7c17614143 Fix an ambiguity in (git reset)
This is necessary to be able to check out a branch named "clone",
otherwise we get
> fatal: ambiguous argument 'copy': both revision and filename
2016-08-31 22:10:47 +02:00
Miloslav Trmač
d24cdcbcf3 Merge pull request #180 from mtrmac/api-changes
Vendor in API changes from https://github.com/containers/image/pull/64
2016-08-31 22:04:20 +02:00
Miloslav Trmač
4055442da5 Vendor after merging mtrmac/image:api-changes
... and update for the API changes.

NOTE: This keeps the old dangerous tlsVerify semantics.
2016-08-31 21:26:42 +02:00
Antonio Murdaca
fb5e5a79f6 Merge pull request #176 from rhatdan/install
Fix install command to create directories
2016-08-25 21:06:50 +02:00
Dan Walsh
88bec961af Fix install command to create directories 2016-08-25 14:37:35 -04:00
Miloslav Trmač
fc843adca9 Merge pull request #158 from mtrmac/copy-signing-integration-tests
Copy signing integration tests
2016-08-25 20:35:39 +02:00
Miloslav Trmač
3d42f226c2 Add integration tests for signature handling in (skopeo copy)
Note the need for openshiftCluster.relaxImageSignerPermissions.
2016-08-25 20:11:31 +02:00
Miloslav Trmač
821f938a11 Merge pull request #157 from mtrmac/verify-on-pull
Verify signatures on pull
2016-08-25 20:02:45 +02:00
Miloslav Trmač
76a14985d6 Implement policy enforcement in (skopeo copy)
Finally, load and enforce the policy.

NOTE that this breaks a simple ./skopeo from a built directory if you
don't have /etc/atomic/policy.json installed for other reasons;
use (./skopeo --policy default-policy.json) instead.
2016-08-25 19:39:21 +02:00
Miloslav Trmač
d4462330a5 Add a default policy file, install it in (make install) and integration tests
(skopeo copy) will soon ALWAYS require a present policy file.  So,
install one by (make install), and ensure that integration tests do so
as well.

Also simplifies the usage of install(1) a bit.
2016-08-25 19:39:21 +02:00
Miloslav Trmač
d5d6bc28f7 Add a new --policy flag.
This ordinarily uses the compiled-in default, but allows per-command
override.  No users yet.

Note that this adds an URL to policy documentation within
containers/image, and that URL does not exist at the moment.
2016-08-25 19:39:15 +02:00
Miloslav Trmač
8826f09cf4 Vendor after merging mtrmac/image:default-policy 2016-08-25 19:36:29 +02:00
Daniel J Walsh
e6886e4afc Merge pull request #173 from mikebrow/auto-completions
add support for completions
2016-08-25 18:13:18 +02:00
Mike Brown
a40d7b53aa add support for completions
Signed-off-by: Mike Brown <brownwm@us.ibm.com>
2016-08-25 10:45:24 -05:00
Miloslav Trmač
e0d44861af Merge pull request #165 from mtrmac/manifest-digest
Improve manifest digest handling
2016-08-25 17:28:59 +02:00
Miloslav Trmač
c236b29c75 Add (skopeo manifest-digest)
A plain sha256sum and the like is insufficient because we need to strip
signatures from v2s1 manifests; so, add a subcommand.

This can be used together with (skopeo inspect --raw) to download a
manifest from a source untrusted to modify it under us; we download a
manifest once using (skopeo inspect --raw), compute a digest using
(skopeo manifest-digest), and then do all future operations using a
digest reference.
2016-08-25 16:49:02 +02:00
Miloslav Trmač
e4315e82b0 Output the original raw manifest in (skopeo inspect --raw)
We need (skopeo inspect --raw > manifest.json) to save the unmodified
original: no extra new lines, no undetected truncation, nothing.
2016-08-25 16:49:02 +02:00
Miloslav Trmač
91b722fec8 Merge pull request #169 from mtrmac/makefile-cleanup
Makefile cleanup
2016-08-25 16:47:12 +02:00
Miloslav Trmač
406ab86104 Clean up and fix minor bugs in DEBUG/GOGCFLAGS handling
* Use “override GOGCFLAGS+=” so that (make GOGCFLAGS=… DEBUG=1)
  does not ignore the appending to GOGCFLAGS
* Move quoting of -gcflags from the variable to its use,
  so that (make GOGCFLAGS=… DEBUG=1) is correctly quoted
* Now that GOGCFLAGS and DEBUG are both handled correctly when
  completely empty, simplify by dropping the DEBUG!=1 branch.
* Beautify the command line by not using DEBUG= if DEBUG is unset.
2016-08-25 15:51:45 +02:00
Antonio Murdaca
2c90120ce6 Merge pull request #146 from mtrmac/update-openshift
Update OpenShift
2016-08-25 12:01:36 +02:00
Miloslav Trmač
47d74dba90 Update OpenShift after the final version of https://github.com/openshift/origin/pull/9181
Uses a tag created after merging that PR.  (git clone -b …) does not
work with commit IDs, and we like to use a released version anyway.
2016-08-22 16:43:07 +02:00
Miloslav Trmač
aafe2a7337 Merge pull request #161 from mikebrow/debug-build
add a source debug build
2016-08-18 17:02:44 +02:00
Mike Brown
63f4f3413f add a source debug build
Signed-off-by: Mike Brown <brownwm@us.ibm.com>
2016-08-18 09:22:42 -05:00
Daniel J Walsh
50f45932f9 Merge pull request #166 from mtrmac/error-pasto
Fix a pasto in an error message
2016-08-16 19:24:25 +02:00
Miloslav Trmač
da298638a2 Fix a pasto in an error message 2016-08-16 18:44:51 +02:00
Miloslav Trmač
cd0cef8442 Merge pull request #160 from mikebrow/make-dependencies
fix dependencies
2016-08-15 16:59:19 +02:00
Mike Brown
1f30fd7bf3 fix dependencies
Signed-off-by: Mike Brown <brownwm@us.ibm.com>
2016-08-13 19:45:32 -05:00
Daniel J Walsh
3da98694a0 Merge pull request #163 from mtrmac/install-dependencies
Make the install-* targets depend on things they are installing
2016-08-12 13:33:12 +02:00
Miloslav Trmač
9abac5b134 Make the install-* targets depend on things they are installing
This ensures that we are not installing e.g. an obsolete version of the
man page after the Markdown version is updated.

Note that this greatly benefits from the "skopeo" target being
non-phony, otherwise (make install) would rebuild the binary.
2016-08-11 18:55:12 +02:00
Miloslav Trmač
ffe92ed2bb Merge pull request #159 from mikebrow/man-build-update
minor cleanup for build issues related to the manual
2016-08-11 15:17:23 +02:00
Mike Brown
6f6c2b9c73 minor cleanup for build issues
Signed-off-by: Mike Brown <brownwm@us.ibm.com>
2016-08-10 19:46:06 -05:00
Miloslav Trmač
e44bd98fa4 Merge pull request #156 from mtrmac/gitignore
Add the generated man page to .gitignore
2016-08-11 00:06:46 +02:00
Miloslav Trmač
e6049802ba Add the generated man page to .gitignore
… and reorder it alphabetically.
2016-08-10 22:58:07 +02:00
Miloslav Trmač
43273caab1 Merge pull request #153 from jwhonce/wip/issue-151
Convert man page to markdown format
2016-08-10 21:18:37 +02:00
Lokesh Mandvekar
ad3e26d042 install manpages using the install-docs target
The MANINSTALL/man1 dir needs to be installed first before installing manpages
into it.

Signed-off-by: Lokesh Mandvekar <lsm5@fedoraproject.org>
2016-08-10 11:52:02 -07:00
Jhon Honce
9a8529667d Convert man page to markdown format
Signed-off-by: Jhon Honce <jhonce@redhat.com>
2016-08-10 11:50:39 -07:00
Miloslav Trmač
6becbb2c66 Merge pull request #149 from mtrmac/docs-and-help
Improve man page and --help
2016-08-09 15:41:27 +02:00
Miloslav Trmač
8a239596a9 Improve --help output
- Use ArgsUsage to document the non-option arguments
- Refer to ArgsUsage placeholders in Usage
- Use named placeholders in flag documentation

Fixes #137, more or less.
2016-08-09 00:40:08 +02:00
Miloslav Trmač
68faefed61 Comprehensively rework the man page
Among other minor changes:
- Do not duplicate synopses of the subcommands; use a generic synopsis
  at the top, and detailed subcommand synopses only when documenting the
  subcommands.
- Use the conventions documented in man-pages(7), in particular using
  italic for replaceable values.
- Add a section documenting the transport:details reference format,
  and list the supported transports.
- Relax the warning about standalone-sign.
2016-08-09 00:40:08 +02:00
Miloslav Trmač
09399a9ac1 Merge pull request #148 from mtrmac/verify-blobs
Verify blobs
2016-08-05 18:44:33 +02:00
Miloslav Trmač
23c96cb998 Verify blobs against the expected digests while copying them.
Note that this requires ImageDestination.PutBlob to fail and delete
any unfinished data if stream.Read() fails.

We do not have to trust PutBlob to correctly handle a validation error,
so we don't; but we can't do the storage cleanup for PutBlob.
2016-08-04 19:58:08 +02:00
Miloslav Trmač
6e2cd739da Vendor after merging mtrmac/image:PutBlob-error-handling 2016-08-04 19:57:40 +02:00
Antonio Murdaca
5197c8dba0 Merge pull request #147 from duglin/contrib
Add a CONTRIBUTING.md file
2016-08-02 15:12:26 +02:00
Doug Davis
e17b1f97ca Add a CONTRIBUTING.md file
Signed-off-by: Doug Davis <dug@us.ibm.com>
2016-08-02 05:46:43 -07:00
Miloslav Trmač
e4982ea82a Merge pull request #93 from mtrmac/openshift-native-signatures
OpenShift native signatures
2016-08-01 22:05:51 +02:00
Miloslav Trmač
c9fbb6c1ab Vendor after merging mtrmac/image:openshift-native-signatures and update API use
Update copy.go for signature implementation change

Now we need to push the manifest first, and only afterwards the
signatures.
2016-08-01 20:44:16 +02:00
Miloslav Trmač
ecc745d124 Merge pull request #138 from mtrmac/location-namespaced-signatures
Use transport abstraction and transport-abstracted references
2016-07-18 21:39:45 +02:00
Miloslav Trmač
b806001e18 Vendor after merging mtrmac/image:reference-abstraction and update API use
directory.NewReference now can fail.
2016-07-18 21:20:51 +02:00
Miloslav Trmač
177463ed03 Merge pull request #144 from mtrmac/reference-abstraction
Reference abstraction
2016-07-18 16:45:59 +02:00
Miloslav Trmač
9ad71d27e0 Vendor after merging mtrmac/image:reference-abstraction and update API use
- Use transports.ParseImageReference instead of dealing with individual
  transports
- CanonicalDockerReference replaced by Reference.DockerReference, can't
  fail but can be unsupported
- directory.NewImageDestination replaced by
  directory.NewReference.NewImageDestination
2016-07-18 16:22:48 +02:00
Miloslav Trmač
5b550a7b37 Merge pull request #143 from mtrmac/docker-references
Clean up Docker reference handling
2016-07-12 15:55:56 +02:00
Miloslav Trmač
8adb5f56de Update for docker-references PR and API changes
Pull in https://github.com/containers/image/pull/37 , and
update for CanonicalDockerReference() returning a reference.Named
2016-07-12 15:24:24 +02:00
Antonio Murdaca
29d76eb5ca Merge pull request #142 from runcom/vendor-contimage
Vendor containers/image
2016-07-04 12:46:10 +02:00
Antonio Murdaca
d7cafe671b Vendor containers/image 9d6b8fc4ae35b9843e4c4397fe0002c8edda7314
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-07-04 12:31:31 +02:00
Antonio Murdaca
ed3016b1c1 Merge pull request #140 from runcom/no-docker-ref
vendor containers/image 5c10ea7c3f0b1f2e36164c15667cc847b1784e16
2016-07-02 12:18:40 +02:00
Antonio Murdaca
09586bb08f vendor containers/image 5c10ea7c3f0b1f2e36164c15667cc847b1784e16
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-07-02 12:02:33 +02:00
Antonio Murdaca
9adb76bf15 Merge pull request #141 from mtrmac/fix-centos-build
Fix integration tests on CentOS
2016-07-01 23:51:37 +02:00
Miloslav Trmač
0cb6cc6222 Fix integration tests on CentOS
This fixes --version integration test on CentOS, as noticed by
https://github.com/projectatomic/skopeo/pull/91 .  The underlying cause
is:
- Makefile builds with -ldflags "-X var=value", while go 1.4.2 only
  supports "-X var value".  This causes CentOS builds to be built
  without the specific commit information
- The --version integration test assumes that commit information will
  always follow the version number.

Changing either one of these would fix the build, changing the
integration test has the advantage that we don't have to use the
obsolete -X syntax and suffer warnings on newer Go versions.
2016-07-01 23:31:07 +02:00
Antonio Murdaca
123891de32 Merge pull request #133 from runcom/oci-3
add possibility to download to OCI image-layout
2016-07-01 22:37:28 +02:00
Antonio Murdaca
6942920ee8 add possibility to download to OCI image-layout
- vendor containers/image c703326038d30c3422168dd9a1a5afaf51740331
- fix copy tests relying on v2s1 manifests

Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-07-01 22:14:14 +02:00
Miloslav Trmač
891c46ed59 Merge pull request #139 from mtrmac/openshift-old-api
Keep using an old version of https://github.com/openshift/origin/pull/9181
2016-07-01 22:08:59 +02:00
Miloslav Trmač
7f9c56ab05 Keep using an old version of https://github.com/openshift/origin/pull/9181
I don’t know how to checkout a specific untagged commit (
9ff4bf43548c758b6767b639b335681285fece48 ) from the original repo, so
I have forked the project and fetched that commit from a cached Docker
image.

We should instead update the containers/image client for the new API ASAP,
and then the github.com/mtrmac/origin repo should be removed.
2016-07-01 20:42:40 +02:00
Antonio Murdaca
a82c64b397 Merge pull request #136 from runcom/fix-version
cmd: skopeo: fix version
2016-06-30 18:02:25 +02:00
Antonio Murdaca
064d37134b cmd: skopeo: fix version
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-06-30 17:46:45 +02:00
Antonio Murdaca
6d7c93acf7 Merge pull request #123 from duglin/modBuild
Build binary in a docker container
2016-06-29 13:43:02 +02:00
Doug Davis
4f7a49ed78 Build binary in a docker container
So that people don't need to install all dependencies just to build.

Make it so that "make binary" does nothing if nothing changed.

Remove ${DEST}

Signed-off-by: Doug Davis <dug@us.ibm.com>
2016-06-29 04:27:54 -07:00
Antonio Murdaca
18223121dd Merge pull request #129 from mtrmac/api-update
Update for changed images.Type API
2016-06-28 20:29:46 +02:00
Miloslav Trmač
fe6c392d45 Update for changed images.Type API 2016-06-28 20:14:15 +02:00
Antonio Murdaca
4558575d9e Merge pull request #127 from runcom/refactor-layers
cmd/skopeo: refactor layers command
2016-06-28 17:41:38 +02:00
Antonio Murdaca
6c4eab8a07 vendor containers/image b95a6b8688d7702cf5906debf87f01cfd849a67a
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-06-28 17:24:20 +02:00
Antonio Murdaca
9900b79eb6 cmd/skopeo: refactor layers command
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-06-28 09:50:50 +02:00
Antonio Murdaca
f420d6867b Merge pull request #126 from runcom/move-containers-image
*: move to containers/image
2016-06-27 17:48:58 +02:00
Antonio Murdaca
2e8bcf65f6 *: move to containers/image
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-06-27 17:17:13 +02:00
Antonio Murdaca
e7a76f750b Merge pull request #124 from duglin/license
Move to Apache 2 license
2016-06-24 23:06:49 +02:00
Doug Davis
de42d88d2c Move to Apache 2 license
Signed-off-by: Doug Davis <dug@us.ibm.com>
2016-06-24 11:35:34 -07:00
Antonio Murdaca
1d5e38454e Merge pull request #121 from vbatts/shorten_build_steps
README: fewer build steps
2016-06-23 23:48:55 +02:00
Vincent Batts
11a0108456 README: fewer build steps
Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2016-06-23 15:19:27 -04:00
Antonio Murdaca
1cf2b63483 Merge pull request #103 from mtrmac/image-layer-digests
Move parsing layer digests from copy.go to types.Image
2016-06-23 18:53:06 +02:00
Miloslav Trmač
5b1ca76131 Only copy each layer once in (skopeo layers)
... using the new uniqueLayerDigests().
2016-06-23 18:09:57 +02:00
Miloslav Trmač
a23befcbf4 Add types.Image.LayerDigests, use it in (skopeo copy)
To do so, have (skopeo copy) work with a types.Image, and replace uses
of types.ImageSource with types.Image where possible to allow the
caching in types.Image to work.

This is a slight behavior change:
- The manifest is now processed through fixManifestLayers
- Duplicate layers (created e.g. when a non-filesystem-altering command is used
  in a Dockerfile) are only copied once.
2016-06-23 18:09:57 +02:00
Miloslav Trmač
c81541de0a Rename types.Image.Layers to LayersCommand
The .Layers() method name is too nice to contain this layering
violation; make it more explicit in the naming.
2016-06-23 15:53:14 +02:00
Miloslav Trmač
206a8e3eed Remove a FIXME? about types.Image.Manifest.
Per the discussion in https://github.com/projectatomic/skopeo/pull/73 ,
types.Image.Manifest should not need to expose MIME types:

ImageSource.GetManifest allows supplying MIME types; the intent is
for clients who want to parse the manifests to use an ImageSource.

Clients who want to use skopeo’s parsing should use types.Image, and
then they don’t need to care about MIME types. In fact, types.Image
needs to decide among the various manifest alternatives which one to
parse (and which one to match against the provided or signed manifest
digest). So, Image.Manifest will not be all that useful for parsing the
contents, it is basically useful only for verifying against a digest.
2016-06-23 15:53:14 +02:00
Antonio Murdaca
b3bcf49d46 Merge pull request #112 from runcom/manifest-pkg
move manifests stuff to its own pkg and add OCI mime types
2016-06-23 12:31:39 +02:00
Antonio Murdaca
705f393109 move manifests stuff to its own pkg and add OCI mime types
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-06-23 12:12:48 +02:00
Antonio Murdaca
6841ee321c Merge pull request #26 from mtrmac/signing-unit-tests
Signing unit tests
2016-06-23 11:55:51 +02:00
Antonio Murdaca
a45e8e1e87 Merge pull request #16 from mtrmac/cmd-test
Support for in-process command testing
2016-06-23 11:55:29 +02:00
Miloslav Trmač
ba2dabe62d Add unit tests for standalone-sign and standalone-verify commands 2016-06-22 20:54:18 +02:00
Miloslav Trmač
aa627d0844 Merge branch 'cmd-test' into HEAD 2016-06-22 20:54:17 +02:00
Miloslav Trmač
df076baf56 Use cli.Context.App.Writer in the "inspect" and "standalone-verify" commands
This will make the implementations testable in the future, and prevent
spreading the untestable code via copy&paste.
2016-06-22 20:53:00 +02:00
Miloslav Trmač
59f7abe749 Add infrastructure for testing cli.Command.Action handlers
Also split creation of cli.App from main(), and add a test helper
function.

This does not change behavior at the moment, but will allow writing
tests of the command handlers.
2016-06-22 20:31:04 +02:00
Antonio Murdaca
6bec0699cb Merge pull request #117 from mtrmac/atomic-registry-in-tests
Add Atomic registry integration tests
2016-06-22 18:29:14 +02:00
Miloslav Trmač
7d379cf87a Add integration tests for (skopeo copy) against the Atomic Registry
This builds from the image-signatures-rest branch for
https://github.com/openshift/origin/pull/9181 .

Testing push, pull, streaming.

Does not test working with the other Docker registries built in
Dockerfile; I will leave that to the author of that code :)

Note that this relies on an internet connection for pulling from the
Docker Hub (which is incidentally tested by that); pushing to no Docker
Registry, neither local nor Hub, is tested by this.

The tests only run in a container because the (oc login) / (docker
login)-like code modifies files in a home directory; the new
SKOPEO_CONTAINER_TESTS environment variable should protect against
accidental non-container runs.
2016-06-22 16:19:59 +02:00
Miloslav Trmač
39b06cb31c Add more helpers for running skopeo, use them in existing tests
- consumeAndLogOutputs
- assertSkopeoSucceeds
- assertSkopeoFails
- runCommandWithInput
All of these allow running commands as one-liners with no call-site
error handling, making tests much more readable.

Also modifies TestNoNeedAuthToPrivateRegistryV2ImageNotFound to use
check.Matches instead of manual strings.Contains conditions, which is
shorter and more consistent with the assertSkopeo... calls.
2016-06-22 16:19:59 +02:00
Miloslav Trmač
601f76f96d Fix consumeAndLogOutput
Primarily, make it actually work; reading into a non-zero-capacity but
zero-length slice would just return 0, the goroutine would terminate,
and even the producer of the output could fail with EPIPE/SIGPIPE.

Also make the logged output readable, converting it into a string
instead of a series of hexadecimal byte values.
2016-06-22 16:19:59 +02:00
Miloslav Trmač
2f2a688026 Move ConsumeAndLogOutput to integration/utils.go
This will be used also by non-signing tests.

No code changes besides removing the initial capital letter in the
function name; this is a separate commit only to make reviewing of
future changes to this function easier.
2016-06-22 16:19:59 +02:00
Antonio Murdaca
a2e9d08e38 Merge pull request #116 from runcom/fix-readme.md
update README.md
2016-06-22 16:02:53 +02:00
Antonio Murdaca
b6a38edfcb update README.md
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-06-22 15:52:29 +02:00
Antonio Murdaca
9a92a10bba Merge pull request #114 from GrantSeltzer/Codegangsta-to-urfave-cli
Codegangsta to urfave cli
2016-06-22 15:51:49 +02:00
Grantseltzer
5ae0402bf0 Changed usage of actions to return errors instead of using logrus.Fatal() 2016-06-22 09:42:52 -04:00
Grantseltzer
313dafe928 update github.com/urfave/cli to v1.17.0
Signed-off-by: Antonio Murdaca <runcom@redhat.com>

Updated action function signatures to return errors
2016-06-20 14:57:00 -04:00
Antonio Murdaca
f4ddde7f47 Merge pull request #115 from mtrmac/skopeo.1
Remove /skopeo.1 from .gitignore
2016-06-20 20:46:07 +02:00
Miloslav Trmač
4a2a78b63b Remove /skopeo.1 from .gitignore
/skopeo.1 was a generated file before #35; now this path is not used
(replaced by man1/skopeo.1); if the generated file is left around, it is
obsolete (and confusingly empty).  Remove it from .gitignore to nudge
developers like me to clean up.
2016-06-20 20:35:18 +02:00
Antonio Murdaca
35dd662fea Merge pull request #104 from projectatomic/expose-blob-size
expose blob size
2016-06-15 20:01:24 +02:00
Antonio Murdaca
7769a21cef expose blob size
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-06-15 19:52:16 +02:00
Antonio Murdaca
a50211ce2a Merge pull request #101 from projectatomic/oci-prep-1
Generalize [Get|Put]Layer
2016-06-14 14:01:16 +02:00
Antonio Murdaca
d54a10f490 Image[Source|Destination]: generalize [Get|Put]Layer into [Get|Put]Blob
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-06-14 13:53:53 +02:00
Antonio Murdaca
3098898a98 Merge pull request #99 from mtrmac/fixManifestLayers-docs
Add minimal comments to fixManifestLayers
2016-06-13 18:25:07 +02:00
Miloslav Trmač
cab18e48ad Add minimal comments to fixManifestLayers
This does not really go into why duplicate layers can happen or why it
is worth supporting that; the code originates from
504e67b867 ,
which does not explain either.
2016-06-13 18:07:03 +02:00
Antonio Murdaca
a8a3cc3525 Merge pull request #98 from mtrmac/generic-image
Move docker.genericImage to a separate skopeo/image subpackage
2016-06-13 11:36:58 +02:00
Miloslav Trmač
96d6a58052 Move docker.genericImage to a separate skopeo/image subpackage
... making image.FromSource a public, stable, API.
2016-06-11 10:48:57 +02:00
Miloslav Trmač
e15276232e Make docker.Image unaware of genericImage internals
This will allow us to cleanly move genericImage into a separate package.

This costs an extra pointer, but also allows us to rely on the type
system and drop handling "certainly impossible" errors, worth it just
for this simplification anyway.
2016-06-11 10:48:57 +02:00
Antonio Murdaca
daeb358572 Merge pull request #96 from mtrmac/update-readme
Update README.md
2016-06-11 09:52:31 +02:00
Miloslav Trmač
55622350c4 Show (skopeo copy) and (skopeo delete) in README.md 2016-06-11 03:34:53 +02:00
Miloslav Trmač
29d189b581 Recommend (make check) instead of (make test-integration)
... so that we also run validate-* and unit tests.
2016-06-11 03:20:27 +02:00
Miloslav Trmač
d947d90bf7 Merge pull request #95 from jwhonce/wip/delete-image
Card container_security_113 - Delete image support
2016-06-11 03:02:43 +02:00
Jhon Honce
f3efa063e3 Card container_security_113 - Delete image support
Add support to mark images for deletion from repository

Requires:
  * V2 API and schema
  * registry configured to allow deletes
  * run registry garbage collection to free up disk space

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2016-06-09 15:23:02 -07:00
Antonio Murdaca
0ff261802b Merge pull request #94 from projectatomic/readme-tweak
README.md: fix examples
2016-06-07 18:41:40 +02:00
Antonio Murdaca
fb236c85af README.md: fix examples
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-06-07 18:27:31 +02:00
Antonio Murdaca
9c4ceeb147 Merge pull request #92 from mtrmac/fix-blob-upload
Fix uploading layer blobs to Docker registry
2016-06-06 23:12:48 +02:00
Miloslav Trmač
fc761ed74f Fix uploading layer blobs to Docker registry
Implement a client to the chunked API, instead of the nonexistent
one-shot API (per
2a4deee441
).

Adds a FIXME to DELETE the pending upload on failure; the uploads are
supposed to time out so this is not immediately critical.

Fixes #64 .
2016-06-06 23:00:58 +02:00
Antonio Murdaca
e66541f7d0 Merge pull request #90 from mtrmac/cleanups
Another random cleanup
2016-06-02 21:44:14 +02:00
Miloslav Trmač
000f31fb73 Better test diagnostics 2016-06-02 21:16:56 +02:00
Miloslav Trmač
bc8041add8 Merge pull request #88 from mtrmac/policy-eval
Add a policy evaluation library
2016-06-02 16:25:23 +02:00
Miloslav Trmač
21229685cf Add PolicyContext, with GetSignaturesWithAcceptedAuthor and IsRunningImageAllowed
PolicyContext is intended to be the primary API for skopeo/signature:
supply a policy and an image, and ask specific, well-defined
(preferably yes/no) questions.
2016-06-02 16:12:10 +02:00
Miloslav Trmač
fd9c615d88 Add PolicyRequirement implementations
Also move the declaration of the type from the mostly-public
policy_types.go to policy_eval.go.
2016-06-02 16:12:10 +02:00
Miloslav Trmač
90361256bc Add PolicyReferenceMatch implementations
Also move the declaration of the type from the mostly-public
policy_types.go to policy_eval.go.
2016-06-02 16:12:10 +02:00
Miloslav Trmač
677f711c6c Redefine Policy.Specific scopes to use fully expanded hostname/namespace/repo format
Using the canonical minimized format of Docker references introduces too
many ambiguities.

This also removes some validation of the scope string, but all that was
really doing was rejecting completely invalid input like uppercase.

Sadly it is not qutie obvious that we can detect and reject mistakes like
using "busybox" as a scope instead of the correct
"docker.io/library/busybox".  Perhaps require at least one dot or port
number in the host name?
2016-06-02 16:12:10 +02:00
Miloslav Trmač
488a535aa0 Use callbacks instead of single expected values in verifyAndExtractSignature
To support verification of signatures when more than one key, or more
than one identity, are accepted, have verifyAndExtract signature accept
callbacks (in a struct so that they are explicitly named).

verifyAndExtractSignature now also validates the manifest digest.  It is
intended to become THE SINGLE PLACE where untrusted signature blobs
have signatures verified, are validated against other expectations, and
parsed, and converted into internal data structures available to other
code.

Also:
- Modifies VerifyDockerManifestSignature to use utils.ManifestMatchesDigest.
- Adds a test for Docker reference mismatch in VerifyDockerManifestSignature.
2016-06-02 16:12:10 +02:00
Miloslav Trmač
e2839c38c5 Add a test for valid signature using an unknown public key
(The key was one-time-generated in a temporary directory,
and is, intentionally, not available.)

This is not conceptually related to the rest of the PR, just adding a
missing case to the test, except that the added fixture will be reused
in a prSignedBy test.
2016-06-02 16:12:10 +02:00
Antonio Murdaca
ee7c5ebae9 Merge pull request #75 from mtrmac/matches-manifest-digest
Add docker/utils.ManifestMatchesDigest
2016-06-02 11:27:07 +02:00
Miloslav Trmač
938478e702 Add docker.utils.ManifestMatchesDigest
As opposed to callers just calling utils.ManifestDigest(), this is
a forward-compatible interface, allowing other digest algorithms to
be added in the future.

Right now, we only support SHA-256, so the underlying implementation
does not change anything.
2016-06-01 16:38:11 +02:00
Antonio Murdaca
837fc231a9 Merge pull request #87 from mtrmac/cleanups
Cleanups
2016-05-31 18:15:02 +02:00
Miloslav Trmač
429a4b0aec Do not drop the underlying error message when a Docker reference is invalid 2016-05-31 17:10:34 +02:00
Miloslav Trmač
e332d0e5d7 Fix a typo 2016-05-31 17:10:34 +02:00
Antonio Murdaca
2e917cf146 Merge pull request #86 from projectatomic/bump-again-v0.1.14-dev
bump v0.1.14-dev
2016-05-31 17:03:01 +02:00
Antonio Murdaca
e7020c2d8c bump v0.1.14-dev
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-05-31 16:45:19 +02:00
Antonio Murdaca
9e971b4937 Merge pull request #85 from runcom/bump-v0.1.13
bump to v0.1.13
2016-05-31 16:43:55 +02:00
Antonio Murdaca
bd018696bd bump to v0.1.13
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-05-31 16:35:08 +02:00
Antonio Murdaca
ad7eb5d221 Merge pull request #84 from mtrmac/gpgme-32bit
Rerun hack/vendor.sh to fix build on 32-bit systems
2016-05-31 16:28:26 +02:00
Miloslav Trmač
80ccbaa021 Rerun hack/vendor.sh to fix build on 32-bit systems
i.e. to pick up https://github.com/proglottis/gpgme/pull/10

Fixes #80.
2016-05-31 16:12:44 +02:00
Antonio Murdaca
c24b42177e Merge pull request #83 from projectatomic/remove-from-api
Remove ManifestMIMETypes
2016-05-31 11:28:51 +02:00
Antonio Murdaca
6fc6d809e0 Remove ManifestMIMETypes
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-05-31 11:19:09 +02:00
Miloslav Trmač
0d95328125 Merge pull request #79 from mtrmac/image-from-imagesource
Make types.Image Docker-independent, add docker.GenericImageFromSource
2016-05-30 17:35:34 +02:00
Miloslav Trmač
41dbbc9b50 Support dir:… as an image specification in (skopeo {inspect,layers})
This is not expected to be that useful in production; for now it serves
as a demonstration of the concept, and it allows (skopeo inspect) to be
clumsily used as parser of stand-alone manifests (by creating a dir:
structure with that manifest).

(skopeo layers) support follows naturally, but is even less useful.
2016-05-28 02:11:32 +02:00
Miloslav Trmač
323b56a049 Make types.Image Docker-independent
The remaining uses of the dependencies, in (skopeo inspect), now check
whether their types.Image is a docker.Image and call the docker.Image
functions directly.

This does not change behavior for Docker images.

For non-Docker images (which can't happen yet), the Name field is
removed; RepoTags remain and are reported as empty, because using
json:",omitempty" would also omit an empty list for Docker images.
2016-05-28 02:11:32 +02:00
Miloslav Trmač
ea643e8658 Use types.ImageSource instead of *dockerImageSource in genericImage
This finally makes genericImage Docker-independent.

(dockerImage is still the only implementation of types.Image.)
2016-05-28 02:11:32 +02:00
Miloslav Trmač
cada464c90 Split dockerImage to genericImage and docker.Image
The code not dependent on specifics of DockerImageSource now lives in
docker.genericImage; the rest directly in docker.Image.

docker.Image remains the only implementation of types.Image at this
point, but that will change.
2016-05-28 02:09:22 +02:00
Miloslav Trmač
0da4307aea Split SourceRefFullName from types.Image.Inspect
This is the only Docker-specific aspect of types.Image.Inspect.

This does not change behavior; plausibly we might want to replace the
Name value in (skopeo inspect) by something else which is not dependent
on Docker, but that can be a separate work later.

Adds a FIXME? in docker_image.go for consistency with
dockerImage.GetRepositoryTags, both will be removed later in the
patchset.
2016-05-28 02:08:12 +02:00
Miloslav Trmač
143e3602ae Split Docker-independent parts of dockerImage into docker/image.go
This does not change the code at all, only moving things around now.
2016-05-28 02:04:48 +02:00
Miloslav Trmač
0314fdb49e Remove temporary variables in (skopeo inspect)
We abort on failure to get the data anyway, so there is no need to use
temporaries to avoid modifying outputData on failure.

This is not a simplification yet, but handling optional (e.g.
Docker-specific) data this way will be simpler, and handling
non-optional data the same way will be more consistent.
2016-05-28 02:04:48 +02:00
Antonio Murdaca
847b5bff85 Merge pull request #78 from runcom/bump-0113dev
bump to v0.1.13-dev
2016-05-27 12:42:29 +02:00
Antonio Murdaca
864568bbd9 bump to v0.1.13-dev
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-05-27 12:26:54 +02:00
Antonio Murdaca
015f1c8c9a Merge pull request #77 from runcom/bump-0112
bump to v0.1.12
2016-05-27 12:25:54 +02:00
Antonio Murdaca
46bb9a0698 bump to v0.1.12
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-05-27 12:16:25 +02:00
Antonio Murdaca
62af96c5c9 Merge pull request #74 from mtrmac/key-import
Add SigningMechanism.ImportKeysFromBytes
2016-05-25 16:35:42 +02:00
Miloslav Trmač
aee0abb5d2 Add SigningMechanism.ImportKeysFromBytes
This will be needed for verification against specified public keys.

Also rerun hack/vendor.sh to pick up import support from
github.com/mtrmac/gpgme .
2016-05-25 16:04:20 +02:00
Miloslav Trmač
721a628f4a Merge pull request #76 from mtrmac/policy-config
Update a comment for prInsecureAcceptAnything
2016-05-25 16:03:30 +02:00
Miloslav Trmač
10280f2e0d Update a comment for prInsecureAcceptAnything 2016-05-25 15:53:12 +02:00
Miloslav Trmač
9ccfc6a423 Merge pull request #55 from mtrmac/policy-config
Add policy configuration data structures, construction and parsing
2016-05-25 15:46:53 +02:00
Miloslav Trmač
d9b1c229e5 Add policy configuration data structures, construction and parsing 2016-05-24 20:24:15 +02:00
Miloslav Trmač
7a8602c54c Add paranoidUnmarshalJSONObject() helper
This allows unmarshaling JSON data and refusing any ambiguous input, to
make sure users don't make mistakes when writing policy.

This might be a bit easier with reflection, but we will need the
non-reflection variant (for unmarshaling a map type) anyway, and quite a
few users which do ultimately unmarshal into a struct need to override
the type of one or more fields, so reflection would force them to define
temporary fields - not necessarily all that better.
2016-05-24 18:16:33 +02:00
Antonio Murdaca
dbb47e6bb6 Merge pull request #72 from runcom/godoc
Godoc
2016-05-24 17:37:53 +02:00
Antonio Murdaca
67119f4875 add doc.go stub
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-05-24 17:29:12 +02:00
Miloslav Trmač
3d1201007e Merge pull request #73 from runcom/mimetypes-choose
add the possibility to choose image's MIME type
2016-05-24 17:10:07 +02:00
Antonio Murdaca
15f478e26b add the possibility to choose image's MIME type
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-05-24 16:52:39 +02:00
Miloslav Trmač
0abbb9a2ce Merge pull request #69 from runcom/re-mimetypes
add mimetypes
2016-05-23 21:00:28 +02:00
Antonio Murdaca
7d12b66fb8 add mimetypes
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-05-23 20:40:52 +02:00
Miloslav Trmač
814a2a6f94 Merge pull request #70 from projectatomic/cleanups
Cleanups
2016-05-23 19:29:06 +02:00
Antonio Murdaca
4036b3543e cleanup API
moving stuff around (godoc.org review)

Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-05-23 17:20:55 +02:00
Miloslav Trmač
1cf55db9be Merge pull request #61 from runcom/fix-godoc
signature: remove pkg fixtures
2016-05-23 16:40:12 +02:00
Antonio Murdaca
7c5db83261 Remove signature/fixtures subpackage
This will make the output of godoc cleaner, we can't filter out the
subpackage otherwise.

Also copy the needed fixture into the integration subpackage, instead of
referring to it using ../signature/fixtures (and we can't import
signature/fixtures_info-test.go now).

Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-05-21 21:21:58 +02:00
Antonio Murdaca
7522e6c99c Merge pull request #68 from mtrmac/cleanups
Random cleanups
2016-05-21 10:51:42 +02:00
Miloslav Trmač
09f33a7c2c Remove a redundant check
reference.WithDefaultTag is already calling reference.IsNameOnly, so we
don't need to guard it on the outside.
2016-05-21 04:48:33 +02:00
Miloslav Trmač
521e3ce0eb Remove duplicated test 2016-05-21 04:46:57 +02:00
Miloslav Trmač
532fae24ac Merge pull request #65 from runcom/fix-headers
provide a way to pass multi values-single key headers
2016-05-19 18:43:25 +02:00
Antonio Murdaca
c661fad3eb provide a way to pass multi values-signle key headers
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-05-18 17:02:53 +02:00
Miloslav Trmač
09f82a5ad2 Merge pull request #60 from projectatomic/more-cleanups
move dockerutils under docker
2016-05-17 17:52:10 +02:00
Antonio Murdaca
e775248b96 move dockerutils under docker/utils
also remove fixtures pkg as it would clutter godoc (there's not need
to have a .go files with fixtures)

Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-05-17 17:35:32 +02:00
Antonio Murdaca
df618e5f7a Merge pull request #58 from mtrmac/api-general
General image API cleanups
2016-05-17 15:58:08 +02:00
Miloslav Trmač
6a357b6fcc Add comments on the use of the API and the general direction. 2016-05-17 15:50:17 +02:00
Miloslav Trmač
7598aab521 Clean up Image.Inspect
This does not change behavior.

Rename types.DockerImageManifest to types.ImageInspectInfo.

This naming more accurately reflects what the function does and how it is
expected to be used.

(The only outstanding non-inspection piece is the Name field, which is
kind of a subset of GetIntendedDockerReference() right now. Not sure
whether that is intentional.)

Also fold makeImageManifest into its only user.
2016-05-16 20:50:46 +02:00
Miloslav Trmač
9766f72760 Move listing of repository tags from Image.Manifest to a separate Image.GetRepositoryTags
This does not change behavior.

Splits listing of repository tags, which is not a property of an image,
from the image.Manifest gathering of information about an image.
2016-05-16 20:50:46 +02:00
Miloslav Trmač
60e8d63712 Compute the digest in (skopeo inspect) instead of trusting the registry
Compute the digest ourselves, the registry is in general untrusted and
computing it ourserlves is easy enough.

The stop passing the unverifiedCanonicalDigest value around, simplifying
ImageSource.GetManifest and related code.  In particular, remove
retrieveRawManifest and have internal users just call Manifest() now that
we don't need the digest.
2016-05-16 20:50:45 +02:00
Miloslav Trmač
e3d257e7b5 Decouple (skopeo inspect) output formatting from types.Image.Manifest
Does not change behavior.

This will allow us to move collecting some of the data to the (skopeo
inspect) code and to have a more focused types.Image API, where
types.Image.Manifest() does not return a grab bag of manifest-unrelated
data, eventually.

For how it actually makes the coupling more explicit by having
types.Image.Manifest() return a types.DockerImageManifest instead of the
too generic types.ImageManifest.  We will need to think about which
parts of DockerImageManifest are truly generic, later.
2016-05-16 20:50:45 +02:00
Antonio Murdaca
c75f0f6780 Merge pull request #59 from mtrmac/api-for-signatures
API for signatures - drop Get prefixes on quick getters
2016-05-16 20:43:48 +02:00
Miloslav Trmač
c38ed76969 Rename GetIndendedDockerReference to IntendedDockerReference 2016-05-16 20:33:13 +02:00
Miloslav Trmač
119609b871 Drop the Get prefix from types.Image.GetManifest and GetSignatures
Keeps the Get prefix on the equivalent methods on types.ImageSource, to
hint that they may be slow.
2016-05-16 20:29:52 +02:00
Miloslav Trmač
dc7a05ebf9 Rename types.Image.Manifest to types.Image.Inspect
Does not change behavior.

This better expresses the purpose of this method (it is working with
more, currently much more, than the manifest), and frees up the Manifest
method name for a simple getter of the raw blob.
2016-05-16 20:24:39 +02:00
Antonio Murdaca
d4eb69e1ab Merge pull request #57 from mtrmac/api-for-signatures
API for signatures
2016-05-16 20:10:03 +02:00
Miloslav Trmač
e4913bd0b0 Add GetIntendedDockerReference to types.Image and types.ImageSource
This will be necessary for signature verification and related policy
evaluation in the future.
2016-05-16 19:25:11 +02:00
Miloslav Trmač
feb9de4845 Add GetManifest and GetSignatures to types.Image
No change in behavior.

These functions are guaranteed-cached versions of the same method in
types.ImageSource.  Both will be needed for signature policy evaluation,
and the symmetry with ImageSource is nice.

Also replaces the equivalent RawManifest method, preferring to keep
the same naming convention as types.ImageSource.
2016-05-16 19:25:11 +02:00
Antonio Murdaca
a39474c817 Merge pull request #56 from mtrmac/sub-pkgs
Move directory, docker and openshift from cmd/skopeo to their own subpackages
2016-05-16 18:44:18 +02:00
Miloslav Trmač
f526328b30 Move directory, docker and openshift from cmd/skopeo to their own subpackages
Does not change behavior.  This is a straightforward move and update of
package references, except for:

- Adding a duplicate definition of manifestSchema1 to
  cmd/skopeo/copy.go.  This will need to be cleaned up later, for now
  preferring to make no design changes in this commit.
- Renaming parseDockerImage to NewDockerImage, to both make it public
  and consistent with common golang conventions.
2016-05-16 18:32:32 +02:00
Antonio Murdaca
b48c78b154 Merge pull request #54 from mtrmac/cleanups
signature cleanups
2016-05-16 16:01:12 +02:00
Miloslav Trmač
c8d8608b57 Move the x() helper from signature_test.go to json_test.go
It will be used in other tests as well.
2016-05-16 14:57:05 +02:00
Miloslav Trmač
35ba0edf0d Remove an unused savedEnvironment type 2016-05-16 14:57:05 +02:00
Miloslav Trmač
345d0c3e2b Reset a json.Unmarshal destination right before the call
… similar to how we do it in other places.
2016-05-16 14:57:05 +02:00
Miloslav Trmač
2ddaa122ab s/tryUnmarshalModified/tryUnmarshalModifiedSignature/g
We will be adding similar test helpers for other types as well, so avoid
the naming conflict.
2016-05-16 14:57:05 +02:00
Miloslav Trmač
7f7c71836c Move strict JSON parsing utilities into a separate file.
No semantic change, only a reorganization: The utilities now return
jsonFormatError instead of InvalidSignatureError, but their only
caller maps it back.
2016-05-16 14:57:05 +02:00
Miloslav Trmač
b5e8413d22 Add compile-time checks that privateSignature implements json.Marshaler and json.Unmarshaler 2016-05-16 14:57:05 +02:00
Miloslav Trmač
14686616c1 Add an API stability warning to mechanism.go 2016-05-16 14:57:05 +02:00
Antonio Murdaca
c89bc5cc4a Merge pull request #53 from mtrmac/copy-no-digest
Don‘t write the mainfest digest on stdout in (skopeo copy)
2016-05-14 10:01:00 +02:00
Miloslav Trmač
6db8872406 Don‘t write the mainfest digest on stdout in (skopeo copy)
The dir: source type does not return the value, the value is
untrusted/not validated, and it is not at all clear why we should print
it in the first place.
2016-05-11 17:36:02 +02:00
Miloslav Trmač
a4fba7b0a0 Merge pull request #51 from projectatomic/refactor-for-lib
*: move pkg main into cmd/skopeo/
2016-05-10 12:29:11 +02:00
Antonio Murdaca
3dc3957607 *: move pkg main into cmd/skopeo/
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-05-10 11:04:03 +02:00
Antonio Murdaca
0bd8bea9ec Merge pull request #50 from jwhonce/wip/manpage
Correct man page formatting
2016-05-09 23:20:40 +02:00
Jhon Honce
780bd132ac Correct man page formatting
Signed-off-by: Jhon Honce <jhonce@redhat.com>
2016-05-09 13:48:31 -07:00
Miloslav Trmač
dbdb03eddb Merge pull request #49 from projectatomic/split-docker
*: split docker.go for future pkg creation
2016-05-09 21:30:38 +02:00
Antonio Murdaca
f43a92a78f *: split docker.go for future pkg creation
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-05-09 21:13:17 +02:00
Antonio Murdaca
9229d72a37 Merge pull request #48 from mtrmac/gpgme-update
Rerun hack/vendor.sh to pick up gpgme changes
2016-05-07 11:35:49 +02:00
Miloslav Trmač
5a2b4005bb Rerun hack/vendor.sh to pick up gpgme changes
In particular, https://github.com/proglottis/gpgme/pull/8 .
2016-05-07 02:33:51 +02:00
Antonio Murdaca
9d24de4c57 Merge pull request #45 from mtrmac/gpgme-update
Rerun hack/vendor.sh to pick up gpgme changes
2016-05-06 22:58:53 +02:00
Miloslav Trmač
fe37c71a4f Rerun hack/vendor.sh to pick up gpgme changes
See https://github.com/proglottis/gpgme/pull/7 for the full discussion.

Resolves #42 .
2016-05-06 22:44:30 +02:00
Antonio Murdaca
28973c0a2d Merge pull request #43 from mtrmac/openshift-copypasta
Add Atomic Registry support for push and pull, and a new “copy” command
2016-05-05 15:39:28 +02:00
Miloslav Trmač
026acb2a57 Add a --sign-by flag to the (skopeo copy) command.
This expects a GPG key fingerprint as a value of the argument (though
other key identification methods, like mitr@volny.cz, happen to work).

Do we need to namespace this (gpg:…)?

Note that this is unusable at the moment because only the dir: backend
implements storing signatures, and this backend does can not determine
the canonical Docker reference to use as a signed image identity.
2016-05-04 17:32:51 +02:00
Miloslav Trmač
da24e319af Add CanonicalDockerReference to ImageDestination
This is necessary to resolve the canonical form of a reference for
signing it.
2016-05-04 17:32:51 +02:00
Miloslav Trmač
2e48975b8b Add a "copy" command for copying images
This copies an image from ImageSource to ImageDestination, e.g.

skopeo copy atomic:mitr/busybox:latest dir:t-down # pull
skopeo copy dir:t-up atomic:mitr/busybox:latest # push
2016-05-04 17:32:51 +02:00
Miloslav Trmač
56f9c987a2 Add utilities for parsing Docker URIs into ImageSource and ImageDestination objects
This finally uses all of the ImageSource and ImageDestination
implementations, though these utilities are in turn not used yet.

Adds unresolved FIXME (FIXME!!) notes for the tlsVerify default value;
for now, the code follows the existing parseImage semantics.

Also note the naming inconsistency: dir:…, atomic:…, but
docker://… .  I think the non-// names are cleaner, but if we are
committed to docker://…, just being consistent might be better.
2016-05-04 17:32:51 +02:00
Miloslav Trmač
36d4353229 Add OpenShift implementations of ImageSource and ImageDestination
Note that this assumes that both (docker login) and (oc login) has
happened, the credentials can be read from the usual config files,
and that the default OpenShift instance should be used.

This includes copy&pasted/modified/simplified code from OpenShift
and Kubernetes, primarily for config file parsing and setting up
TLS and HTTP authentication.

This is much smaller than linking to the upstream OpenShift client
libraries, which via various abstractions and registration drag in much
(dozens of megabytes) more code.

The primary loss from this simplification is automatic conversions
between various versions of the API objects, both for the REST API and
for local configuration storage.

This does not contain downloading/uploading signatures, which depends on
server-side support.
2016-05-04 17:32:51 +02:00
Miloslav Trmač
935eee7592 Add an ImageDestination implementation for the Docker Registry
Note that this does not allow uploading under new tags; Docker Registry
requires the tag to be present within the manifest, i.e. we might need
to modify the (possibly signed) manifest.

For now, uploading manifests only identified by a digest is sufficient
for the Atomic Registry; tagging happens in OpenShift imagestreams.
2016-05-04 17:32:51 +02:00
Miloslav Trmač
0587501ff0 Split dockerClient from dockerImageSource
The dockerClient encapsulates makeRequest and authentication setup, and
will be shared between the pull and push code.

This is only a restructuring, does not change behavior.

The dockerImage->dockerImageSource->dockerClient inclusion chain is
somewhat ugly, hopefully eventually we will move the remaining
dockerImage functionality either to dockerutils or to the top level, and
then eliminate it.
2016-05-04 17:32:51 +02:00
Miloslav Trmač
2790d9a1c3 Make dockerutils.GuessManifestMIMEType public
The Docker Registry manifest upload should supply a Content-Type, and
guessing from the contents is the easiest we can do right now.

Also eliminate dockerutils.manifestMIMEType, it is making it too
difficult to use the returned value to be worth the extra safety.
2016-05-04 17:32:51 +02:00
Antonio Murdaca
696eb74918 Merge pull request #30 from mtrmac/cleanups
Cleanups
2016-05-04 17:29:14 +02:00
Miloslav Trmač
654050b7e8 Fix handling of !tlsVerify when certPath is not set 2016-05-04 17:19:59 +02:00
Miloslav Trmač
7bee2da169 Use the provided method in dockerImageSource.makeRequest instead of hard-coding GET 2016-05-04 17:19:59 +02:00
Miloslav Trmač
60fbdd3988 Do not assume GetManifest is called before other dockerImageSource methods
Call dockerImageSource.ping() in .makeRequest() if needed, instead of
expecting a caller to do it (which only happened in GetManifest).

This required splitting the URLs into the baseURL (dependent on .ping()
result) and the suffix (independent of it), which was a simplification
anyway.

Also rename WWWAuthenticate to wwwAuthenticate, it is a private cache
field.
2016-05-04 17:19:59 +02:00
Miloslav Trmač
14bc664b48 Remove a redundant dockerImageSource.makeRequest parameter
It is always computed in the same, or equivalent, way.

Also remove pingResponse.needsAuth, only used in the above.
2016-05-04 17:19:59 +02:00
Antonio Murdaca
749a8e3b82 Merge pull request #38 from jwhonce/wip/manpage
Add disclaimer to man page for sign-* commands
2016-05-04 17:10:06 +02:00
Jhon Honce
0470e7fb0f Add disclaimer to man page for sign-* commands
Signed-off-by: Jhon Honce <jhonce@redhat.com>
2016-05-04 07:36:38 -07:00
Antonio Murdaca
266f0b8487 Merge pull request #29 from mtrmac/source-dest
Add ImageSource and ImageDestination abstractions
2016-05-04 13:03:23 +02:00
Miloslav Trmač
fd41449410 Use dirImageDestination for writing to local files in docker.go
This will hopefully allow better reuse of the "copy images" code from
docker.go in the future.

No behavior change, the dirImageDestination code was based on the code
this commit is replacing.
2016-05-02 19:43:16 +02:00
Miloslav Trmač
af126bc68c Add an ImageSource and ImageDestination implementation for local directories
This is consistent with the (skopeo layers) storage layout; otherwise it
is expected to be used primarily as an a debugging aid when working on
more complex image transfers (e.g. directly from OpenShift to a running
Docker daemon), allowing them to be split to two simpler problems
between one complex storage mechanism and a simple directory.

Not used yet, users will be added in future commits.
2016-05-02 19:43:16 +02:00
Miloslav Trmač
e169c311d3 Add an ImageSource implementation to docker.go
The ImageSource type does not provide all of the functionality of
docker.go, but we will be able to reuse the ImageSource parts in an
OpenShift client.

This is only a restructuring, does not change behavior.
2016-05-02 19:43:16 +02:00
Miloslav Trmač
a4aedae063 Add types.ImageSource and types.ImageDestination
Right now, only a declaration.

This will allow writing generalized push/pull between various storage
mechanisms, and reuse of the Docker Registry client code for the Docker
Registry embedded in OpenShift.
2016-05-02 19:43:16 +02:00
Antonio Murdaca
aff6aa7c2c Merge pull request #41 from projectatomic/fix-url
docker.go: do not concatenate url in ping
2016-04-29 16:51:38 +02:00
Antonio Murdaca
6d74750bba docker.go: do not concatenate url in ping
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-04-29 15:23:06 +02:00
Antonio Murdaca
2b3a4cfdfe Merge pull request #39 from mtrmac/update-gpgme
Update mtrmac/gpgme vendor to fix build on CentOS 7
2016-04-26 18:30:20 +02:00
Miloslav Trmač
e76eecd533 Update mtrmac/gpgme vendor to fix build on CentOS 7 2016-04-26 17:57:40 +02:00
Antonio Murdaca
dfc6352108 Merge pull request #37 from mtrmac/v2s1-manifest-followup
v2s1 manifest followup
2016-04-25 18:08:54 +02:00
Miloslav Trmač
23899acadd Create a new subpackage "dockerutils", starting with manifest computation
Move the manifest computation (with v2s1 signature stripping) out of
skopeo/signature into a separate package; it is necessary in the
OpenShift client as well, unrelated to signatures.

Other Docker-specific utilities, like getting a list of layer blobsums
from a manifest, may be also moved here in the future.
2016-04-25 17:27:51 +02:00
Miloslav Trmač
7a7dd84818 Fix fixture file name
It is “manifest version 2, schema 1”, not v1.
2016-04-25 17:27:51 +02:00
Antonio Murdaca
8374928f74 Merge pull request #35 from jwhonce/wip/manpage
Update man page
2016-04-22 09:13:31 +02:00
Jhon Honce
b52d3c85c6 * Update Authors 2016-04-21 13:44:12 -07:00
Jhon Honce
eab73f3d51 Update man page
Resolves https://github.com/projectatomic/skopeo/issues/12

* Convert man page from markdown to nroff
* Fill out man page
* Remove TODO's from go code regarding man page
* Additional information on building instructions
* Update Makfile

Signed-off-by: Jhon Honce <jhonce@redhat.com>
2016-04-21 09:46:02 -07:00
Antonio Murdaca
918a4d9110 Merge pull request #36 from projectatomic/fix-creds
fix invalid credentials error
2016-04-21 15:33:14 +02:00
Antonio Murdaca
c7be79e190 fix invalid credentials error
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-04-21 14:32:49 +02:00
Antonio Murdaca
28f2fedab9 Merge pull request #34 from mtrmac/v1s1-manifest-digest
Strip signatures from v1s1 manifests before computing the digest
2016-04-19 18:03:57 +02:00
Miloslav Trmač
4e19770a1b Strip signatures from v1s1 manifests before computing the digest 2016-04-19 17:37:04 +02:00
Antonio Murdaca
68a614d463 Merge pull request #33 from mtrmac/cgo-pthread-ordering-workaround
Add a workaround for a glibc bug when -lphtread precedes -lgpgme
2016-04-13 22:40:58 +02:00
Miloslav Trmač
e782275c2e Add a workaround for a glibc bug when -lphtread precedes -lgpgme 2016-04-13 21:42:19 +02:00
Antonio Murdaca
9d10b0b4ea Merge pull request #32 from mtrmac/integration-diagnostics
Test command output before eror status in signing integration tests
2016-04-12 10:11:39 +02:00
Miloslav Trmač
dd7c2d44fa Log command output on failures in signing integration tests 2016-04-11 17:41:57 +02:00
Antonio Murdaca
c4e48c8f85 Merge pull request #31 from mtrmac/vendor-fixes
Vendor fixes
2016-04-07 09:00:01 +02:00
Miloslav Trmač
f7b81b5627 Fix dependency computation
Set GOPATH to start with ./vendor so that we use the dependencies in our
vendored versions instead of dependencies in whatever other version is
elsewhere in GOPATH.

And then undo it when trying to list the non-vendor subpackages in the
current directory.
2016-04-05 17:46:58 +02:00
Miloslav Trmač
96b96735ed Allow keeping vendor subdirectories in vendored packages
github.com/coreos/etcd as of v2.2.5 uses a Godeps subdirectory, and
imports packages by including the Godeps path fragments directly in the
package name; so we can't just remove the subdirectory and vendor the
included package directly.  So, add a flag to clone() to surpress
removing the vendor subdirectories.
2016-04-05 17:46:53 +02:00
Antonio Murdaca
d7ae061a83 Merge pull request #28 from runcom/fix-gitcommit
remove cmd/ subdir
2016-03-25 12:21:26 +01:00
Antonio Murdaca
1423aab202 remove cmd/ subdir
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-25 12:14:43 +01:00
Antonio Murdaca
9e982f0b1d Merge pull request #25 from mtrmac/signing
Signing
2016-03-24 17:40:30 +01:00
Miloslav Trmač
03f6cb89e6 Add standalone-sign and standalone-verify commands 2016-03-24 17:06:30 +01:00
Miloslav Trmač
69d5a131c9 Add signing and verification to the signature package 2016-03-24 11:32:23 +01:00
Miloslav Trmač
9595b3336f Signature JSON encoding/decoding
Adds stretchr/testify dependency.
2016-03-24 11:32:23 +01:00
Antonio Murdaca
10a41bd0fc Merge pull request #24 from mtrmac/all-dependencies
Do not clean test-only dependencies from vendor packages
2016-03-24 00:00:46 +01:00
Miloslav Trmač
e6841d0a27 Do not clean test-only dependencies from vendor packages
Instead of only checking dependencies of the "main" packages, include
also test dependencies of all subpackages of the project, and their
transitive dependencies.
2016-03-23 17:17:21 +01:00
Antonio Murdaca
79f09478b4 Merge pull request #22 from runcom/tls
support --cert-path and --tls-verify
2016-03-23 15:40:32 +01:00
Antonio Murdaca
1ce21cd233 support cert-path and tls-verify flags
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-23 15:35:07 +01:00
Antonio Murdaca
70a6c7b21d urls const(s)
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-23 12:40:32 +01:00
Antonio Murdaca
3a09e2bf8e clean vendors and bootstrap tls verify
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-23 10:27:59 +01:00
Antonio Murdaca
d204183544 Merge pull request #21 from runcom/fix-makefile
fix makefile
2016-03-22 18:21:01 +01:00
Antonio Murdaca
e78053938b fix makefile
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-22 18:10:59 +01:00
Antonio Murdaca
37ebb81936 Merge pull request #15 from runcom/enhanc
drop docker/ code
2016-03-22 16:34:37 +01:00
Antonio Murdaca
c02155340e include needed deps
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-22 16:29:16 +01:00
Antonio Murdaca
fed651449e remove vendors ftw
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-22 16:02:55 +01:00
Antonio Murdaca
aa6d271975 refactor image manifest
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-22 15:50:41 +01:00
Antonio Murdaca
50a2ed1124 fix CI and remove docker/ dir
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-22 15:50:41 +01:00
Antonio Murdaca
103420769f drop docker/ code
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-22 15:50:41 +01:00
Antonio Murdaca
7be01242a8 remove duplicate code
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-22 15:50:40 +01:00
Antonio Murdaca
60c5561c84 Merge pull request #20 from mtrmac/fix-vendor
Fix hack/.vendor-helper.sh for main package move.
2016-03-22 14:41:39 +01:00
Miloslav Trmač
f7ebc0a595 Fix hack/.vendor-helper.sh for main package move.
Otherwise the "clean" step of hack/vendor.sh would drop most .go files
from vendor/ as unused.

Also commits refreshed versions of a few of the vendored packages.
2016-03-22 14:33:15 +01:00
Antonio Murdaca
ffcb8f862f Merge pull request #19 from mtrmac/unit-tests
Add infrastructure for running unit tests
2016-03-22 14:22:36 +01:00
Miloslav Trmač
b815271f16 Add collective test targets:
- (make check): GNU coding standards-compliant primary entry point,
  running all available tests in the best environment (i.e. Docker
  container).
- (make test-all-local): Local entry point, running only tests
  which do not require a special environment; intended for IDE
  integration and quick turnaround cycles.

Also modifies the Travis configuration to run (make check), to prevent
duplication.
2016-03-22 14:12:56 +01:00
Miloslav Trmač
a4fd447146 Run unit tests in Travis 2016-03-22 14:12:56 +01:00
Miloslav Trmač
ab36f62d59 Add 'test-unit' and 'test-unit-local' Makefile targets
Running unit tests without the integration tests is non-trivial, so add
a Makefile target to help with this.
2016-03-22 14:12:56 +01:00
Antonio Murdaca
fde4c74547 Merge pull request #17 from mtrmac/validate-uncommitted
Use contents of local checkout instead of last commit for validation
2016-03-21 15:41:39 +01:00
Miloslav Trmač
d08a3812d2 Use contents of local checkout instead of last commit for validation
Validating only committed files is not useful in the natural
  $test_everything_passes; commit; push
workflow; the failures will not be caught locally, only by Travis later
(and only if PRs are used instead of direct commits to master).

So, use the working directory state instead of last commit for
validations; and remove misleading comments in checks which already use
the working directory state.
2016-03-21 14:55:32 +01:00
Antonio Murdaca
0d94172288 fix gitignore
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-19 09:53:56 +01:00
Antonio Murdaca
a73078ea75 refactor structure and print git commit in version
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-19 09:32:59 +01:00
Antonio Murdaca
8cf22b9ca2 fix make install target
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-18 16:46:25 +01:00
Antonio Murdaca
5f4a5653ac add todo
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-17 12:02:03 +01:00
Antonio Murdaca
648f2f8bc5 output raw manifest for v2 registries
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-17 11:55:49 +01:00
Antonio Murdaca
6a00ce47d2 import cifetch code and add layers command
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-17 11:48:04 +01:00
Antonio Murdaca
d8fbf24c25 types: more on interfaces
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-16 17:54:24 +01:00
Antonio Murdaca
b97d7fc68f thoughts on interfaces
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-16 13:24:42 +01:00
Antonio Murdaca
5d740b4611 fix to interfaces
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-16 13:23:08 +01:00
Antonio Murdaca
0440473c63 add comment to interfaces
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-16 11:52:24 +01:00
Antonio Murdaca
73509ef227 fix
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-16 11:51:20 +01:00
Antonio Murdaca
41329ca504 attempt abstract interface
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-16 11:46:39 +01:00
Antonio Murdaca
b46d977403 add todos
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-16 10:43:46 +01:00
Antonio Murdaca
0109708048 fix missing comma in tests
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-16 10:26:54 +01:00
Antonio Murdaca
adbf487541 fix readme, tests and vendors
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-16 10:21:36 +01:00
Antonio Murdaca
3fd3adc58e support multi commands
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-16 10:06:31 +01:00
Antonio Murdaca
d0fd876d7e update codegangsta/cli + fix Travis + todos
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-09 08:16:45 +01:00
Antonio Murdaca
c9d544c8fb fix golint RUN
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-09 08:08:00 +01:00
Antonio Murdaca
0715f36de8 add golint
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-09 07:50:04 +01:00
Antonio Murdaca
beef95f21a update readme
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-08 16:31:47 +01:00
Antonio Murdaca
ae27cb93db change license to GPLv2
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-08 16:25:51 +01:00
Antonio Murdaca
213b0505dc bump to v0.1.12-dev
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-08 09:49:51 +01:00
Antonio Murdaca
8094910c9a adapt code for projectatomic github
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-08 09:32:20 +01:00
Antonio Murdaca
82b121caf1 update docker code and adapt code
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-04 16:50:37 +01:00
Antonio Murdaca
89631ab4f1 fix building readme
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-04 13:08:51 +01:00
Antonio Murdaca
4d7ac1999e add another todo about v2 only
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-04 13:04:49 +01:00
Antonio Murdaca
370f1bc685 reg v1 setup wip
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-04 13:02:52 +01:00
Antonio Murdaca
51c104bde0 registry binary v1 working
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-04 12:57:28 +01:00
Antonio Murdaca
286ab4e144 build registry v1 for testing
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-04 11:48:16 +01:00
Antonio Murdaca
34fed9cabc update todo
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-04 10:54:15 +01:00
Antonio Murdaca
01b3c23ffa add todo
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-04 09:06:19 +01:00
Antonio Murdaca
7fc29e2323 fix readme
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-04 08:55:45 +01:00
Antonio Murdaca
b03817412b add dnf for fedora installation
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-04 08:55:03 +01:00
Antonio Murdaca
03b19aa069 fix readme
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-04 08:53:10 +01:00
Antonio Murdaca
69b84ce650 fix building section in readme
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-03 08:10:26 +01:00
Antonio Murdaca
044e9b5390 bump v0.1.10-dev
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-02-29 12:20:40 +01:00
Antonio Murdaca
30db2ad7fc bump v0.1.9
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-02-29 12:20:12 +01:00
Antonio Murdaca
b8d3588b54 bump v0.1.9-dev
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-02-29 12:18:52 +01:00
Antonio Murdaca
3eefe215e0 support insecure registries
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-02-29 11:44:51 +01:00
Antonio Murdaca
e5d9bee80c remove img-type
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-02-29 11:27:43 +01:00
Antonio Murdaca
7d7fb3c4b0 remote sudo_user, upstream has it
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-02-27 11:49:51 +01:00
Antonio Murdaca
8449c6ae07 bump docker to 1.10.2
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-02-23 00:25:23 +01:00
Antonio Murdaca
b77535feee add todo authconfigs fallthrough
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-02-22 15:04:55 +01:00
Antonio Murdaca
a09019661e support $SUDO_USER
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-02-22 14:39:12 +01:00
Antonio Murdaca
2986c6d0d6 return nicer error
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-02-07 18:14:24 +01:00
Antonio Murdaca
01b47e3681 default to inspect docker images
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-02-07 18:13:30 +01:00
Antonio Murdaca
1d452edce6 make space for appc img inspect :)
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-02-07 18:07:01 +01:00
Antonio Murdaca
3de433ea4a re-enable and fix failing tests
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-02-02 13:51:10 +01:00
Antonio Murdaca
143ff7165d fix readme
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-02-02 13:38:21 +01:00
Antonio Murdaca
6cfd14f0c0 lots of todo and fixes
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-02-02 11:44:49 +01:00
Antonio Murdaca
8d22756979 move infof to debugf to not clutter stderr
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-02-01 17:45:19 +01:00
Antonio Murdaca
55804fad16 add todo
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-01-29 10:30:22 +01:00
Antonio Murdaca
71e0fec389 bump v0.1.5-dev
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-01-29 10:15:09 +01:00
Antonio Murdaca
98aca9c188 bump v0.1.4
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-01-29 10:12:37 +01:00
Antonio Murdaca
a13bacf8ad fix by Jan
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-01-28 23:58:58 +01:00
Antonio Murdaca
45007a3006 spec for v0.1.3 & bump v0.1.4-dev
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-01-28 19:02:28 +01:00
Antonio Murdaca
fdb5cac7f5 bump v0.1.3
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-01-28 18:51:50 +01:00
Antonio Murdaca
0478e50fba new spec file
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-01-28 18:41:49 +01:00
Antonio Murdaca
572a6b6f53 fix wording
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-01-26 19:33:07 +01:00
Antonio Murdaca
6b836f71d4 fix for make man
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-01-26 19:30:23 +01:00
Antonio Murdaca
eee94e7a5a fix readme for go 1.5 vendoring
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-01-26 19:20:26 +01:00
Antonio Murdaca
354fb08916 Merge pull request #5 from vdemeester/travis-with-root
Switch to travis sudo & enable docker service
2016-01-25 21:59:53 +01:00
Vincent Demeester
728700068e Switch to travis sudo & enable docker service
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2016-01-25 21:50:17 +01:00
Vincent Demeester
80a3391c48 Merge pull request #4 from runcom/make
validate scripts in container
2016-01-24 12:34:00 +01:00
Antonio Murdaca
c2b6a42855 validate scripts in container
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-01-24 12:27:53 +01:00
Vincent Demeester
5e024e079d Merge pull request #3 from runcom/add-test
add-tests
2016-01-24 09:55:36 +01:00
Vincent Demeester
27f4f9a51f Add validation script and update travis config.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2016-01-24 09:50:30 +01:00
Antonio Murdaca
e17927ca1d add basic test infrastructure
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-01-23 18:31:03 +01:00
Antonio Murdaca
6a675e6966 add Dockerfile
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-01-23 12:50:59 +01:00
Antonio Murdaca
9dabd2e2cb bump v0.1.3-dev
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-01-23 11:50:15 +01:00
Antonio Murdaca
e604ed368d bump version v0.1.2
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-01-23 11:49:31 +01:00
Antonio Murdaca
a6742bceff bump v0.1.2-dev
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-01-23 11:46:50 +01:00
Antonio Murdaca
13ec8c78d2 bump version 0.1.1
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-01-23 11:44:47 +01:00
Antonio Murdaca
4967efa951 fix spec for GO15VENDOREXPERIMENT
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-01-23 11:44:22 +01:00
Antonio Murdaca
3d25e18181 finally fix master
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-01-23 11:37:28 +01:00
Antonio Murdaca
38cb7a9d27 fix master, missing dep
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-01-23 10:45:51 +01:00
Antonio Murdaca
a94193a469 fix master?
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-01-23 10:22:41 +01:00
Antonio Murdaca
a7ae2a03f6 again remove Azure/go-ansiterm for now
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-01-23 09:41:42 +01:00
Antonio Murdaca
c6eea504d8 shrink vendors
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-01-23 09:40:50 +01:00
Vincent Demeester
16caf459e5 Merge pull request #1 from vdemeester/pimp-my-vendor
Update vendor using docker vendor script
2016-01-22 21:36:11 +01:00
Vincent Demeester
e6ae8238aa Fix golint & gofmt :) 2016-01-22 21:26:52 +01:00
Vincent Demeester
0a8fce39ae Add a simple travis yaml
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2016-01-22 21:26:52 +01:00
Vincent Demeester
30b1077a25 Update vendor using docker vendor script
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2016-01-22 21:26:46 +01:00
Antonio Murdaca
f227c1b6d9 bump to 0.1.1-dev
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-01-22 20:37:37 +01:00
Antonio Murdaca
c07c167006 bump godeps and lots of fixes...
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-01-22 11:45:53 +01:00
Antonio Murdaca
f2aedbe15f todo(s)
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-01-21 16:20:46 +01:00
994 changed files with 71092 additions and 82073 deletions

5
.gitignore vendored
View File

@@ -1,2 +1,3 @@
skopeo
skopeo.1
/docs/skopeo.1
/layers-*
/skopeo

10
.travis.yml Normal file
View File

@@ -0,0 +1,10 @@
sudo: required
services:
- docker
notifications:
email: false
script:
- make check

142
CONTRIBUTING.md Normal file
View File

@@ -0,0 +1,142 @@
# Contributing to Skopeo
We'd love to have you join the community! Below summarizes the processes
that we follow.
## Topics
* [Reporting Issues](#reporting-issues)
* [Submitting Pull Requests](#submitting-pull-requests)
* [Communications](#communications)
* [Becomign a Maintainer](#becoming-a-maintainer)
## Reporting Issues
Before reporting an issue, check our backlog of
[open issues](https://github.com/projectatomic/skopeo/issues)
to see if someone else has already reported it. If so, feel free to add
your scenario, or additional information, to the discussion. Or simply
"subscribe" to it to be notified when it is updated.
If you find a new issue with the project we'd love to hear about it! The most
important aspect of a bug report is that it includes enough information for
us to reproduce it. So, please include as much detail as possible and try
to remove the extra stuff that doesn't really relate to the issue itself.
The easier it is for us to reproduce it, the faster it'll be fixed!
Please don't include any private/sensitive information in your issue!
## Submitting Pull Requests
No Pull Request (PR) is too small! Typos, additional comments in the code,
new testcases, bug fixes, new features, more documentation, ... it's all
welcome!
While bug fixes can first be identified via an "issue", that is not required.
It's ok to just open up a PR with the fix, but make sure you include the same
information you would have included in an issue - like how to reproduce it.
PRs for new features should include some background on what use cases the
new code is trying to address. And, when possible and it makes, try to break-up
larger PRs into smaller ones - it's easier to review smaller
code changes. But, only if those smaller ones make sense as stand-alone PRs.
Regardless of the type of PR, all PRs should include:
* well documented code changes
* additional testcases. Ideally, they should fail w/o your code change applied
* documentation changes
Squash your commits into logical pieces of work that might want to be reviewed
separate from the rest of the PRs. But, squashing down to just one commit is ok
too since in the end the entire PR will be reviewed anyway. When in doubt,
squash.
PRs that fix issues should include a reference like `Closes #XXXX` in the
commit message so that github will automatically close the referenced issue
when the PR is merged.
<!--
All PRs require at least two LGTMs (Looks Good To Me) from maintainers.
-->
### Sign your PRs
The sign-off is a line at the end of the explanation for the patch. Your
signature certifies that you wrote the patch or otherwise have the right to pass
it on as an open-source patch. The rules are simple: if you can certify
the below (from [developercertificate.org](http://developercertificate.org/)):
```
Developer Certificate of Origin
Version 1.1
Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
660 York Street, Suite 102,
San Francisco, CA 94110 USA
Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.
Developer's Certificate of Origin 1.1
By making a contribution to this project, I certify that:
(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or
(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or
(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.
(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.
```
Then you just add a line to every git commit message:
Signed-off-by: Joe Smith <joe.smith@email.com>
Use your real name (sorry, no pseudonyms or anonymous contributions.)
If you set your `user.name` and `user.email` git configs, you can sign your
commit automatically with `git commit -s`.
## Communications
For general questions, or dicsussions, please use the
IRC group on `irc.freenode.net` called `container-projects`
that has been setup.
For discussions around issues/bugs and features, you can use the github
[issues](https://github.com/projectatomic/skopeo/issues)
and
[PRs](https://github.com/projectatomic/skopeo/pulls)
tracking system.
<!--
## Becoming a Maintainer
To become a maintainer you must first be nominated by an existing maintainer.
If a majority (>50%) of maintainers agree then the proposal is adopted and
you will be added to the list.
Removing a maintainer requires at least 75% of the remaining maintainers
approval, or if the person requests to be removed then it is automatic.
Normally, a maintainer will only be removed if they are considered to be
inactive for a long period of time or are viewed as disruptive to the community.
The current list of maintainers can be found in the
[MAINTAINERS](MAINTAINERS) file.
-->

58
Dockerfile Normal file
View File

@@ -0,0 +1,58 @@
FROM fedora
RUN dnf -y update && dnf install -y make git golang golang-github-cpuguy83-go-md2man \
# gpgme bindings deps
libassuan-devel gpgme-devel \
gnupg \
# registry v1 deps
xz-devel \
python-devel \
python-pip \
swig \
redhat-rpm-config \
openssl-devel \
patch
# Install three versions of the registry. The first is an older version that
# only supports schema1 manifests. The second is a newer version that supports
# both. This allows integration-cli tests to cover push/pull with both schema1
# and schema2 manifests. Install registry v1 also.
ENV REGISTRY_COMMIT_SCHEMA1 ec87e9b6971d831f0eff752ddb54fb64693e51cd
ENV REGISTRY_COMMIT 47a064d4195a9b56133891bbb13620c3ac83a827
RUN set -x \
&& export GOPATH="$(mktemp -d)" \
&& git clone https://github.com/docker/distribution.git "$GOPATH/src/github.com/docker/distribution" \
&& (cd "$GOPATH/src/github.com/docker/distribution" && git checkout -q "$REGISTRY_COMMIT") \
&& GOPATH="$GOPATH/src/github.com/docker/distribution/Godeps/_workspace:$GOPATH" \
go build -o /usr/local/bin/registry-v2 github.com/docker/distribution/cmd/registry \
&& (cd "$GOPATH/src/github.com/docker/distribution" && git checkout -q "$REGISTRY_COMMIT_SCHEMA1") \
&& GOPATH="$GOPATH/src/github.com/docker/distribution/Godeps/_workspace:$GOPATH" \
go build -o /usr/local/bin/registry-v2-schema1 github.com/docker/distribution/cmd/registry \
&& rm -rf "$GOPATH" \
&& export DRV1="$(mktemp -d)" \
&& git clone https://github.com/docker/docker-registry.git "$DRV1" \
# no need for setuptools since we have a version conflict with fedora
&& sed -i.bak s/setuptools==5.8//g "$DRV1/requirements/main.txt" \
&& sed -i.bak s/setuptools==5.8//g "$DRV1/depends/docker-registry-core/requirements/main.txt" \
&& pip install "$DRV1/depends/docker-registry-core" \
&& pip install file://"$DRV1#egg=docker-registry[bugsnag,newrelic,cors]" \
&& patch $(python -c 'import boto; import os; print os.path.dirname(boto.__file__)')/connection.py \
< "$DRV1/contrib/boto_header_patch.diff" \
&& dnf -y update && dnf install -y m2crypto
RUN set -x \
&& yum install -y which git tar wget hostname util-linux bsdtar socat ethtool device-mapper iptables tree findutils nmap-ncat e2fsprogs xfsprogs lsof docker iproute \
&& export GOPATH=$(mktemp -d) \
&& git clone -b v1.3.0-alpha.3 git://github.com/openshift/origin "$GOPATH/src/github.com/openshift/origin" \
&& (cd "$GOPATH/src/github.com/openshift/origin" && make clean build && make all WHAT=cmd/dockerregistry) \
&& cp -a "$GOPATH/src/github.com/openshift/origin/_output/local/bin/linux"/*/* /usr/local/bin \
&& cp "$GOPATH/src/github.com/openshift/origin/images/dockerregistry/config.yml" /atomic-registry-config.yml \
&& mkdir /registry
ENV GOPATH /usr/share/gocode:/go
ENV PATH $GOPATH/bin:/usr/share/gocode/bin:$PATH
RUN go get github.com/golang/lint/golint
WORKDIR /go/src/github.com/projectatomic/skopeo
COPY . /go/src/github.com/projectatomic/skopeo
#ENTRYPOINT ["hack/dind"]

5
Dockerfile.build Normal file
View File

@@ -0,0 +1,5 @@
FROM ubuntu:16.04
RUN apt-get update
RUN apt-get install -y golang git-core libgpgme11-dev
ENV GOPATH=/
WORKDIR /src/github.com/projectatomic/skopeo

240
Godeps/Godeps.json generated
View File

@@ -1,240 +0,0 @@
{
"ImportPath": "github.com/runcom/skopeo",
"GoVersion": "go1.5.3",
"Packages": [
"."
],
"Deps": [
{
"ImportPath": "github.com/Sirupsen/logrus",
"Comment": "v0.8.7-55-gf7f79f7",
"Rev": "f7f79f729e0fbe2fcc061db48a9ba0263f588252"
},
{
"ImportPath": "github.com/codegangsta/cli",
"Comment": "1.2.0-187-gc31a797",
"Rev": "c31a7975863e7810c92e2e288a9ab074f9a88f29"
},
{
"ImportPath": "github.com/docker/distribution",
"Comment": "v2.2.0-199-g0865082",
"Rev": "08650825fef9f21ea819972fb2ed875c0832a255"
},
{
"ImportPath": "github.com/docker/docker/api",
"Comment": "v1.4.1-9391-g5537a92",
"Rev": "5537a92e450ea56e2002f83ff50bb70fdb2cc25e"
},
{
"ImportPath": "github.com/docker/docker/cliconfig",
"Comment": "v1.4.1-9391-g5537a92",
"Rev": "5537a92e450ea56e2002f83ff50bb70fdb2cc25e"
},
{
"ImportPath": "github.com/docker/docker/daemon/graphdriver",
"Comment": "v1.4.1-9391-g5537a92",
"Rev": "5537a92e450ea56e2002f83ff50bb70fdb2cc25e"
},
{
"ImportPath": "github.com/docker/docker/distribution",
"Comment": "v1.4.1-9391-g5537a92",
"Rev": "5537a92e450ea56e2002f83ff50bb70fdb2cc25e"
},
{
"ImportPath": "github.com/docker/docker/dockerversion",
"Comment": "v1.4.1-9391-g5537a92",
"Rev": "5537a92e450ea56e2002f83ff50bb70fdb2cc25e"
},
{
"ImportPath": "github.com/docker/docker/image",
"Comment": "v1.4.1-9391-g5537a92",
"Rev": "5537a92e450ea56e2002f83ff50bb70fdb2cc25e"
},
{
"ImportPath": "github.com/docker/docker/layer",
"Comment": "v1.4.1-9391-g5537a92",
"Rev": "5537a92e450ea56e2002f83ff50bb70fdb2cc25e"
},
{
"ImportPath": "github.com/docker/docker/opts",
"Comment": "v1.4.1-9391-g5537a92",
"Rev": "5537a92e450ea56e2002f83ff50bb70fdb2cc25e"
},
{
"ImportPath": "github.com/docker/docker/pkg/archive",
"Comment": "v1.4.1-9391-g5537a92",
"Rev": "5537a92e450ea56e2002f83ff50bb70fdb2cc25e"
},
{
"ImportPath": "github.com/docker/docker/pkg/chrootarchive",
"Comment": "v1.4.1-9391-g5537a92",
"Rev": "5537a92e450ea56e2002f83ff50bb70fdb2cc25e"
},
{
"ImportPath": "github.com/docker/docker/pkg/fileutils",
"Comment": "v1.4.1-9391-g5537a92",
"Rev": "5537a92e450ea56e2002f83ff50bb70fdb2cc25e"
},
{
"ImportPath": "github.com/docker/docker/pkg/homedir",
"Comment": "v1.4.1-9391-g5537a92",
"Rev": "5537a92e450ea56e2002f83ff50bb70fdb2cc25e"
},
{
"ImportPath": "github.com/docker/docker/pkg/httputils",
"Comment": "v1.4.1-9391-g5537a92",
"Rev": "5537a92e450ea56e2002f83ff50bb70fdb2cc25e"
},
{
"ImportPath": "github.com/docker/docker/pkg/idtools",
"Comment": "v1.4.1-9391-g5537a92",
"Rev": "5537a92e450ea56e2002f83ff50bb70fdb2cc25e"
},
{
"ImportPath": "github.com/docker/docker/pkg/ioutils",
"Comment": "v1.4.1-9391-g5537a92",
"Rev": "5537a92e450ea56e2002f83ff50bb70fdb2cc25e"
},
{
"ImportPath": "github.com/docker/docker/pkg/jsonlog",
"Comment": "v1.4.1-9391-g5537a92",
"Rev": "5537a92e450ea56e2002f83ff50bb70fdb2cc25e"
},
{
"ImportPath": "github.com/docker/docker/pkg/jsonmessage",
"Comment": "v1.4.1-9391-g5537a92",
"Rev": "5537a92e450ea56e2002f83ff50bb70fdb2cc25e"
},
{
"ImportPath": "github.com/docker/docker/pkg/mflag",
"Comment": "v1.4.1-9391-g5537a92",
"Rev": "5537a92e450ea56e2002f83ff50bb70fdb2cc25e"
},
{
"ImportPath": "github.com/docker/docker/pkg/parsers/kernel",
"Comment": "v1.4.1-9391-g5537a92",
"Rev": "5537a92e450ea56e2002f83ff50bb70fdb2cc25e"
},
{
"ImportPath": "github.com/docker/docker/pkg/pools",
"Comment": "v1.4.1-9391-g5537a92",
"Rev": "5537a92e450ea56e2002f83ff50bb70fdb2cc25e"
},
{
"ImportPath": "github.com/docker/docker/pkg/progress",
"Comment": "v1.4.1-9391-g5537a92",
"Rev": "5537a92e450ea56e2002f83ff50bb70fdb2cc25e"
},
{
"ImportPath": "github.com/docker/docker/pkg/promise",
"Comment": "v1.4.1-9391-g5537a92",
"Rev": "5537a92e450ea56e2002f83ff50bb70fdb2cc25e"
},
{
"ImportPath": "github.com/docker/docker/pkg/random",
"Comment": "v1.4.1-9391-g5537a92",
"Rev": "5537a92e450ea56e2002f83ff50bb70fdb2cc25e"
},
{
"ImportPath": "github.com/docker/docker/pkg/reexec",
"Comment": "v1.4.1-9391-g5537a92",
"Rev": "5537a92e450ea56e2002f83ff50bb70fdb2cc25e"
},
{
"ImportPath": "github.com/docker/docker/pkg/stringid",
"Comment": "v1.4.1-9391-g5537a92",
"Rev": "5537a92e450ea56e2002f83ff50bb70fdb2cc25e"
},
{
"ImportPath": "github.com/docker/docker/pkg/system",
"Comment": "v1.4.1-9391-g5537a92",
"Rev": "5537a92e450ea56e2002f83ff50bb70fdb2cc25e"
},
{
"ImportPath": "github.com/docker/docker/pkg/tarsum",
"Comment": "v1.4.1-9391-g5537a92",
"Rev": "5537a92e450ea56e2002f83ff50bb70fdb2cc25e"
},
{
"ImportPath": "github.com/docker/docker/pkg/term",
"Comment": "v1.4.1-9391-g5537a92",
"Rev": "5537a92e450ea56e2002f83ff50bb70fdb2cc25e"
},
{
"ImportPath": "github.com/docker/docker/pkg/useragent",
"Comment": "v1.4.1-9391-g5537a92",
"Rev": "5537a92e450ea56e2002f83ff50bb70fdb2cc25e"
},
{
"ImportPath": "github.com/docker/docker/pkg/version",
"Comment": "v1.4.1-9391-g5537a92",
"Rev": "5537a92e450ea56e2002f83ff50bb70fdb2cc25e"
},
{
"ImportPath": "github.com/docker/docker/reference",
"Comment": "v1.4.1-9391-g5537a92",
"Rev": "5537a92e450ea56e2002f83ff50bb70fdb2cc25e"
},
{
"ImportPath": "github.com/docker/docker/registry",
"Comment": "v1.4.1-9391-g5537a92",
"Rev": "5537a92e450ea56e2002f83ff50bb70fdb2cc25e"
},
{
"ImportPath": "github.com/docker/engine-api/types",
"Comment": "v0.2.2-13-ge98ada9",
"Rev": "e98ada9b4fa351554aa7ee7cf857a2d6d26b1453"
},
{
"ImportPath": "github.com/docker/go-connections/nat",
"Comment": "v0.1.2-2-g082e382",
"Rev": "082e382836465660fe3fd21de06624a205f986eb"
},
{
"ImportPath": "github.com/docker/go-connections/tlsconfig",
"Comment": "v0.1.2-2-g082e382",
"Rev": "082e382836465660fe3fd21de06624a205f986eb"
},
{
"ImportPath": "github.com/docker/go-units",
"Comment": "v0.1.0-21-g0bbddae",
"Rev": "0bbddae09c5a5419a8c6dcdd7ff90da3d450393b"
},
{
"ImportPath": "github.com/docker/libtrust",
"Rev": "9cbd2a1374f46905c68a4eb3694a130610adc62a"
},
{
"ImportPath": "github.com/gorilla/context",
"Rev": "1c83b3eabd45b6d76072b66b746c20815fb2872d"
},
{
"ImportPath": "github.com/gorilla/mux",
"Rev": "26a6070f849969ba72b72256e9f14cf519751690"
},
{
"ImportPath": "github.com/opencontainers/runc/libcontainer/user",
"Comment": "v0.0.4-226-gbc46574",
"Rev": "bc465742ac61aad50dec8e9769a75f01d8d8e502"
},
{
"ImportPath": "github.com/vbatts/tar-split/archive/tar",
"Comment": "v0.9.11-1-gd50e5c9",
"Rev": "d50e5c9283da469398d84078519de569f617be6f"
},
{
"ImportPath": "github.com/vbatts/tar-split/tar/asm",
"Comment": "v0.9.11-1-gd50e5c9",
"Rev": "d50e5c9283da469398d84078519de569f617be6f"
},
{
"ImportPath": "github.com/vbatts/tar-split/tar/storage",
"Comment": "v0.9.11-1-gd50e5c9",
"Rev": "d50e5c9283da469398d84078519de569f617be6f"
},
{
"ImportPath": "golang.org/x/net/context",
"Rev": "b2ed34f6fc8d65cc6a090fb87692ea6b1162fddd"
}
]
}

5
Godeps/Readme generated
View File

@@ -1,5 +0,0 @@
This directory tree is generated automatically by godep.
Please do not edit.
See https://github.com/tools/godep for more information.

2
Godeps/_workspace/.gitignore generated vendored
View File

@@ -1,2 +0,0 @@
/pkg
/bin

View File

@@ -1,7 +0,0 @@
language: go
go:
- 1.3
- 1.4
- tip
install:
- go get -t ./...

View File

@@ -1,50 +0,0 @@
package main
import (
"github.com/Sirupsen/logrus"
)
var log = logrus.New()
func init() {
log.Formatter = new(logrus.JSONFormatter)
log.Formatter = new(logrus.TextFormatter) // default
log.Level = logrus.DebugLevel
}
func main() {
defer func() {
err := recover()
if err != nil {
log.WithFields(logrus.Fields{
"omg": true,
"err": err,
"number": 100,
}).Fatal("The ice breaks!")
}
}()
log.WithFields(logrus.Fields{
"animal": "walrus",
"number": 8,
}).Debug("Started observing beach")
log.WithFields(logrus.Fields{
"animal": "walrus",
"size": 10,
}).Info("A group of walrus emerges from the ocean")
log.WithFields(logrus.Fields{
"omg": true,
"number": 122,
}).Warn("The group's number increased tremendously!")
log.WithFields(logrus.Fields{
"temperature": -4,
}).Debug("Temperature changes")
log.WithFields(logrus.Fields{
"animal": "orca",
"size": 9009,
}).Panic("It's over 9000!")
}

View File

@@ -1,30 +0,0 @@
package main
import (
"github.com/Sirupsen/logrus"
"gopkg.in/gemnasium/logrus-airbrake-hook.v2"
)
var log = logrus.New()
func init() {
log.Formatter = new(logrus.TextFormatter) // default
log.Hooks.Add(airbrake.NewHook(123, "xyz", "development"))
}
func main() {
log.WithFields(logrus.Fields{
"animal": "walrus",
"size": 10,
}).Info("A group of walrus emerges from the ocean")
log.WithFields(logrus.Fields{
"omg": true,
"number": 122,
}).Warn("The group's number increased tremendously!")
log.WithFields(logrus.Fields{
"omg": true,
"number": 100,
}).Fatal("The ice breaks!")
}

View File

@@ -1,61 +0,0 @@
package logstash
import (
"encoding/json"
"fmt"
"github.com/Sirupsen/logrus"
)
// Formatter generates json in logstash format.
// Logstash site: http://logstash.net/
type LogstashFormatter struct {
Type string // if not empty use for logstash type field.
// TimestampFormat sets the format used for timestamps.
TimestampFormat string
}
func (f *LogstashFormatter) Format(entry *logrus.Entry) ([]byte, error) {
fields := make(logrus.Fields)
for k, v := range entry.Data {
fields[k] = v
}
fields["@version"] = 1
if f.TimestampFormat == "" {
f.TimestampFormat = logrus.DefaultTimestampFormat
}
fields["@timestamp"] = entry.Time.Format(f.TimestampFormat)
// set message field
v, ok := entry.Data["message"]
if ok {
fields["fields.message"] = v
}
fields["message"] = entry.Message
// set level field
v, ok = entry.Data["level"]
if ok {
fields["fields.level"] = v
}
fields["level"] = entry.Level.String()
// set type field
if f.Type != "" {
v, ok = entry.Data["type"]
if ok {
fields["fields.type"] = v
}
fields["type"] = f.Type
}
serialized, err := json.Marshal(fields)
if err != nil {
return nil, fmt.Errorf("Failed to marshal fields to JSON, %v", err)
}
return append(serialized, '\n'), nil
}

View File

@@ -1,39 +0,0 @@
# Syslog Hooks for Logrus <img src="http://i.imgur.com/hTeVwmJ.png" width="40" height="40" alt=":walrus:" class="emoji" title=":walrus:"/>
## Usage
```go
import (
"log/syslog"
"github.com/Sirupsen/logrus"
logrus_syslog "github.com/Sirupsen/logrus/hooks/syslog"
)
func main() {
log := logrus.New()
hook, err := logrus_syslog.NewSyslogHook("udp", "localhost:514", syslog.LOG_INFO, "")
if err == nil {
log.Hooks.Add(hook)
}
}
```
If you want to connect to local syslog (Ex. "/dev/log" or "/var/run/syslog" or "/var/run/log"). Just assign empty string to the first two parameters of `NewSyslogHook`. It should look like the following.
```go
import (
"log/syslog"
"github.com/Sirupsen/logrus"
logrus_syslog "github.com/Sirupsen/logrus/hooks/syslog"
)
func main() {
log := logrus.New()
hook, err := logrus_syslog.NewSyslogHook("", "", syslog.LOG_INFO, "")
if err == nil {
log.Hooks.Add(hook)
}
}
```

View File

@@ -1,61 +0,0 @@
// +build !windows,!nacl,!plan9
package logrus_syslog
import (
"fmt"
"github.com/Sirupsen/logrus"
"log/syslog"
"os"
)
// SyslogHook to send logs via syslog.
type SyslogHook struct {
Writer *syslog.Writer
SyslogNetwork string
SyslogRaddr string
}
// Creates a hook to be added to an instance of logger. This is called with
// `hook, err := NewSyslogHook("udp", "localhost:514", syslog.LOG_DEBUG, "")`
// `if err == nil { log.Hooks.Add(hook) }`
func NewSyslogHook(network, raddr string, priority syslog.Priority, tag string) (*SyslogHook, error) {
w, err := syslog.Dial(network, raddr, priority, tag)
return &SyslogHook{w, network, raddr}, err
}
func (hook *SyslogHook) Fire(entry *logrus.Entry) error {
line, err := entry.String()
if err != nil {
fmt.Fprintf(os.Stderr, "Unable to read entry, %v", err)
return err
}
switch entry.Level {
case logrus.PanicLevel:
return hook.Writer.Crit(line)
case logrus.FatalLevel:
return hook.Writer.Crit(line)
case logrus.ErrorLevel:
return hook.Writer.Err(line)
case logrus.WarnLevel:
return hook.Writer.Warning(line)
case logrus.InfoLevel:
return hook.Writer.Info(line)
case logrus.DebugLevel:
return hook.Writer.Debug(line)
default:
return nil
}
}
func (hook *SyslogHook) Levels() []logrus.Level {
return []logrus.Level{
logrus.PanicLevel,
logrus.FatalLevel,
logrus.ErrorLevel,
logrus.WarnLevel,
logrus.InfoLevel,
logrus.DebugLevel,
}
}

View File

@@ -1,5 +0,0 @@
autoload -U compinit && compinit
autoload -U bashcompinit && bashcompinit
script_dir=$(dirname $0)
source ${script_dir}/bash_autocomplete

View File

@@ -1,38 +0,0 @@
image: dmp42/go:stable
script:
# To be spoofed back into the test image
- go get github.com/modocache/gover
- go get -t ./...
# Go fmt
- test -z "$(gofmt -s -l -w . | tee /dev/stderr)"
# Go lint
- test -z "$(golint ./... | tee /dev/stderr)"
# Go vet
- go vet ./...
# Go test
- go test -v -race -cover ./...
# Helper to concatenate reports
- gover
# Send to coverall
- goveralls -service drone.io -coverprofile=gover.coverprofile -repotoken {{COVERALLS_TOKEN}}
# Do we want these as well?
# - go get code.google.com/p/go.tools/cmd/goimports
# - test -z "$(goimports -l -w ./... | tee /dev/stderr)"
# http://labix.org/gocheck
notify:
email:
recipients:
- distribution@docker.com
slack:
team: docker
channel: "#dt"
username: mom
token: {{SLACK_TOKEN}}
on_success: true
on_failure: true

View File

@@ -1,37 +0,0 @@
# Compiled Object files, Static and Dynamic libs (Shared Objects)
*.o
*.a
*.so
# Folders
_obj
_test
# Architecture specific extensions/prefixes
*.[568vq]
[568vq].out
*.cgo1.go
*.cgo2.c
_cgo_defun.c
_cgo_gotypes.go
_cgo_export.*
_testmain.go
*.exe
*.test
*.prof
# never checkin from the bin file (for now)
bin/*
# Test key files
*.pem
# Cover profiles
*.out
# Editor/IDE specific files.
*.sublime-project
*.sublime-workspace

View File

@@ -1,14 +0,0 @@
Stephen J Day <stephen.day@docker.com> Stephen Day <stevvooe@users.noreply.github.com>
Stephen J Day <stephen.day@docker.com> Stephen Day <stevvooe@gmail.com>
Olivier Gambier <olivier@docker.com> Olivier Gambier <dmp42@users.noreply.github.com>
Brian Bland <brian.bland@docker.com> Brian Bland <r4nd0m1n4t0r@gmail.com>
Josh Hawn <josh.hawn@docker.com> Josh Hawn <jlhawn@berkeley.edu>
Richard Scothern <richard.scothern@docker.com> Richard <richard.scothern@gmail.com>
Richard Scothern <richard.scothern@docker.com> Richard Scothern <richard.scothern@gmail.com>
Andrew Meredith <andymeredith@gmail.com> Andrew Meredith <kendru@users.noreply.github.com>
harche <p.harshal@gmail.com> harche <harche@users.noreply.github.com>
Jessie Frazelle <jessie@docker.com> <jfrazelle@users.noreply.github.com>
Sharif Nassar <sharif@mrwacky.com> Sharif Nassar <mrwacky42@users.noreply.github.com>
Sven Dowideit <SvenDowideit@home.org.au> Sven Dowideit <SvenDowideit@users.noreply.github.com>
Vincent Giersch <vincent.giersch@ovh.net> Vincent Giersch <vincent@giersch.fr>
davidli <wenquan.li@hp.com> davidli <wenquan.li@hpe.com>

View File

@@ -1,104 +0,0 @@
Aaron Lehmann <aaron.lehmann@docker.com>
Aaron Vinson <avinson.public@gmail.com>
Adam Enger <adamenger@gmail.com>
Adrian Mouat <adrian.mouat@gmail.com>
Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
Alex Chan <alex.chan@metaswitch.com>
Alex Elman <aelman@indeed.com>
amitshukla <ashukla73@hotmail.com>
Amy Lindburg <amy.lindburg@docker.com>
Andrew Meredith <andymeredith@gmail.com>
Andrey Kostov <kostov.andrey@gmail.com>
Andy Goldstein <agoldste@redhat.com>
Anton Tiurin <noxiouz@yandex.ru>
Antonio Mercado <amercado@thinknode.com>
Arnaud Porterie <arnaud.porterie@docker.com>
Arthur Baars <arthur@semmle.com>
Avi Miller <avi.miller@oracle.com>
Ayose Cazorla <ayosec@gmail.com>
BadZen <dave.trombley@gmail.com>
Ben Firshman <ben@firshman.co.uk>
bin liu <liubin0329@gmail.com>
Brian Bland <brian.bland@docker.com>
burnettk <burnettk@gmail.com>
Chris Dillon <squarism@gmail.com>
Daisuke Fujita <dtanshi45@gmail.com>
Darren Shepherd <darren@rancher.com>
Dave Trombley <dave.trombley@gmail.com>
Dave Tucker <dt@docker.com>
David Lawrence <david.lawrence@docker.com>
David Verhasselt <david@crowdway.com>
David Xia <dxia@spotify.com>
davidli <wenquan.li@hp.com>
Dejan Golja <dejan@golja.org>
Derek McGowan <derek@mcgstyle.net>
Diogo Mónica <diogo.monica@gmail.com>
Donald Huang <don.hcd@gmail.com>
Doug Davis <dug@us.ibm.com>
farmerworking <farmerworking@gmail.com>
Florentin Raud <florentin.raud@gmail.com>
Frederick F. Kautz IV <fkautz@alumni.cmu.edu>
harche <p.harshal@gmail.com>
Henri Gomez <henri.gomez@gmail.com>
Hu Keping <hukeping@huawei.com>
Hua Wang <wanghua.humble@gmail.com>
Ian Babrou <ibobrik@gmail.com>
Jack Griffin <jackpg14@gmail.com>
Jason Freidman <jason.freidman@gmail.com>
Jeff Nickoloff <jeff@allingeek.com>
Jessie Frazelle <jessie@docker.com>
Jianqing Wang <tsing@jianqing.org>
Jon Poler <jonathan.poler@apcera.com>
Jonathan Boulle <jonathanboulle@gmail.com>
Jordan Liggitt <jliggitt@redhat.com>
Josh Hawn <josh.hawn@docker.com>
Julien Fernandez <julien.fernandez@gmail.com>
Kelsey Hightower <kelsey.hightower@gmail.com>
Kenneth Lim <kennethlimcp@gmail.com>
Li Yi <denverdino@gmail.com>
Louis Kottmann <louis.kottmann@gmail.com>
Luke Carpenter <x@rubynerd.net>
Mary Anthony <mary@docker.com>
Matt Bentley <mbentley@mbentley.net>
Matt Moore <mattmoor@google.com>
Matt Robenolt <matt@ydekproductions.com>
Michael Prokop <mika@grml.org>
Miquel Sabaté <msabate@suse.com>
Morgan Bauer <mbauer@us.ibm.com>
moxiegirl <mary@docker.com>
Nathan Sullivan <nathan@nightsys.net>
nevermosby <robolwq@qq.com>
Nghia Tran <tcnghia@gmail.com>
Nuutti Kotivuori <nuutti.kotivuori@poplatek.fi>
Oilbeater <liumengxinfly@gmail.com>
Olivier Gambier <olivier@docker.com>
Olivier Jacques <olivier.jacques@hp.com>
Patrick Devine <patrick.devine@docker.com>
Philip Misiowiec <philip@atlashealth.com>
Richard Scothern <richard.scothern@docker.com>
Rusty Conover <rusty@luckydinosaur.com>
Sebastiaan van Stijn <github@gone.nl>
Sharif Nassar <sharif@mrwacky.com>
Shawn Falkner-Horine <dreadpirateshawn@gmail.com>
Shreyas Karnik <karnik.shreyas@gmail.com>
Simon Thulbourn <simon+github@thulbourn.com>
Spencer Rinehart <anubis@overthemonkey.com>
Stephen J Day <stephen.day@docker.com>
Sungho Moon <sungho.moon@navercorp.com>
Sven Dowideit <SvenDowideit@home.org.au>
Sylvain Baubeau <sbaubeau@redhat.com>
Ted Reed <ted.reed@gmail.com>
tgic <farmer1992@gmail.com>
Thomas Sjögren <konstruktoid@users.noreply.github.com>
Tianon Gravi <admwiggin@gmail.com>
Tibor Vass <teabee89@gmail.com>
Tonis Tiigi <tonistiigi@gmail.com>
Troels Thomsen <troels@thomsen.io>
Vincent Batts <vbatts@redhat.com>
Vincent Demeester <vincent@sbr.pm>
Vincent Giersch <vincent.giersch@ovh.net>
W. Trevor King <wking@tremily.us>
xg.song <xg.song@venusource.com>
xiekeyang <xiekeyang@huawei.com>
Yann ROBERT <yann.robert@anantaplex.fr>
yuzou <zouyu7@huawei.com>

View File

@@ -1,140 +0,0 @@
# Contributing to the registry
## Before reporting an issue...
### If your problem is with...
- automated builds
- your account on the [Docker Hub](https://hub.docker.com/)
- any other [Docker Hub](https://hub.docker.com/) issue
Then please do not report your issue here - you should instead report it to [https://support.docker.com](https://support.docker.com)
### If you...
- need help setting up your registry
- can't figure out something
- are not sure what's going on or what your problem is
Then please do not open an issue here yet - you should first try one of the following support forums:
- irc: #docker-distribution on freenode
- mailing-list: <distribution@dockerproject.org> or https://groups.google.com/a/dockerproject.org/forum/#!forum/distribution
## Reporting an issue properly
By following these simple rules you will get better and faster feedback on your issue.
- search the bugtracker for an already reported issue
### If you found an issue that describes your problem:
- please read other user comments first, and confirm this is the same issue: a given error condition might be indicative of different problems - you may also find a workaround in the comments
- please refrain from adding "same thing here" or "+1" comments
- you don't need to comment on an issue to get notified of updates: just hit the "subscribe" button
- comment if you have some new, technical and relevant information to add to the case
- __DO NOT__ comment on closed issues or merged PRs. If you think you have a related problem, open up a new issue and reference the PR or issue.
### If you have not found an existing issue that describes your problem:
1. create a new issue, with a succinct title that describes your issue:
- bad title: "It doesn't work with my docker"
- good title: "Private registry push fail: 400 error with E_INVALID_DIGEST"
2. copy the output of:
- `docker version`
- `docker info`
- `docker exec <registry-container> registry -version`
3. copy the command line you used to launch your Registry
4. restart your docker daemon in debug mode (add `-D` to the daemon launch arguments)
5. reproduce your problem and get your docker daemon logs showing the error
6. if relevant, copy your registry logs that show the error
7. provide any relevant detail about your specific Registry configuration (e.g., storage backend used)
8. indicate if you are using an enterprise proxy, Nginx, or anything else between you and your Registry
## Contributing a patch for a known bug, or a small correction
You should follow the basic GitHub workflow:
1. fork
2. commit a change
3. make sure the tests pass
4. PR
Additionally, you must [sign your commits](https://github.com/docker/docker/blob/master/CONTRIBUTING.md#sign-your-work). It's very simple:
- configure your name with git: `git config user.name "Real Name" && git config user.email mail@example.com`
- sign your commits using `-s`: `git commit -s -m "My commit"`
Some simple rules to ensure quick merge:
- clearly point to the issue(s) you want to fix in your PR comment (e.g., `closes #12345`)
- prefer multiple (smaller) PRs addressing individual issues over a big one trying to address multiple issues at once
- if you need to amend your PR following comments, please squash instead of adding more commits
## Contributing new features
You are heavily encouraged to first discuss what you want to do. You can do so on the irc channel, or by opening an issue that clearly describes the use case you want to fulfill, or the problem you are trying to solve.
If this is a major new feature, you should then submit a proposal that describes your technical solution and reasoning.
If you did discuss it first, this will likely be greenlighted very fast. It's advisable to address all feedback on this proposal before starting actual work.
Then you should submit your implementation, clearly linking to the issue (and possible proposal).
Your PR will be reviewed by the community, then ultimately by the project maintainers, before being merged.
It's mandatory to:
- interact respectfully with other community members and maintainers - more generally, you are expected to abide by the [Docker community rules](https://github.com/docker/docker/blob/master/CONTRIBUTING.md#docker-community-guidelines)
- address maintainers' comments and modify your submission accordingly
- write tests for any new code
Complying to these simple rules will greatly accelerate the review process, and will ensure you have a pleasant experience in contributing code to the Registry.
Have a look at a great, succesful contribution: the [Ceph driver PR](https://github.com/docker/distribution/pull/443)
## Coding Style
Unless explicitly stated, we follow all coding guidelines from the Go
community. While some of these standards may seem arbitrary, they somehow seem
to result in a solid, consistent codebase.
It is possible that the code base does not currently comply with these
guidelines. We are not looking for a massive PR that fixes this, since that
goes against the spirit of the guidelines. All new contributions should make a
best effort to clean up and make the code base better than they left it.
Obviously, apply your best judgement. Remember, the goal here is to make the
code base easier for humans to navigate and understand. Always keep that in
mind when nudging others to comply.
The rules:
1. All code should be formatted with `gofmt -s`.
2. All code should pass the default levels of
[`golint`](https://github.com/golang/lint).
3. All code should follow the guidelines covered in [Effective
Go](http://golang.org/doc/effective_go.html) and [Go Code Review
Comments](https://github.com/golang/go/wiki/CodeReviewComments).
4. Comment the code. Tell us the why, the history and the context.
5. Document _all_ declarations and methods, even private ones. Declare
expectations, caveats and anything else that may be important. If a type
gets exported, having the comments already there will ensure it's ready.
6. Variable name length should be proportional to its context and no longer.
`noCommaALongVariableNameLikeThisIsNotMoreClearWhenASimpleCommentWouldDo`.
In practice, short methods will have short variable names and globals will
have longer names.
7. No underscores in package names. If you need a compound name, step back,
and re-examine why you need a compound name. If you still think you need a
compound name, lose the underscore.
8. No utils or helpers packages. If a function is not general enough to
warrant its own package, it has not been written generally enough to be a
part of a util package. Just leave it unexported and well-documented.
9. All tests should run with `go test` and outside tooling should not be
required. No, we don't need another unit testing framework. Assertion
packages are acceptable if they provide _real_ incremental value.
10. Even though we call these "rules" above, they are actually just
guidelines. Since you've read all the rules, you now know that.
If you are having trouble getting into the mood of idiomatic Go, we recommend
reading through [Effective Go](http://golang.org/doc/effective_go.html). The
[Go Blog](http://blog.golang.org/) is also a great resource. Drinking the
kool-aid is a lot easier than going thirsty.

View File

@@ -1,19 +0,0 @@
FROM golang:1.5.3
RUN apt-get update && \
apt-get install -y librados-dev apache2-utils && \
rm -rf /var/lib/apt/lists/*
ENV DISTRIBUTION_DIR /go/src/github.com/docker/distribution
ENV GOPATH $DISTRIBUTION_DIR/Godeps/_workspace:$GOPATH
ENV DOCKER_BUILDTAGS include_rados include_oss include_gcs
WORKDIR $DISTRIBUTION_DIR
COPY . $DISTRIBUTION_DIR
COPY cmd/registry/config-dev.yml /etc/docker/registry/config.yml
RUN make PREFIX=/go clean binaries
VOLUME ["/var/lib/registry"]
EXPOSE 5000
ENTRYPOINT ["registry"]
CMD ["/etc/docker/registry/config.yml"]

View File

@@ -1,181 +0,0 @@
{
"ImportPath": "github.com/docker/distribution",
"GoVersion": "go1.4.2",
"Packages": [
"./..."
],
"Deps": [
{
"ImportPath": "golang.org/x/oauth2",
"Rev": "8914e5017ca260f2a3a1575b1e6868874050d95e"
},
{
"ImportPath": "golang.org/x/oauth2/google",
"Rev": "8914e5017ca260f2a3a1575b1e6868874050d95e"
},
{
"ImportPath": "google.golang.org/api/storage/v1",
"Rev": "18450f4e95c7e76ce3a5dc3a8cb7178ab6d56121"
},
{
"ImportPath": "google.golang.org/cloud",
"Rev": "2400193c85c3561d13880d34e0e10c4315bb02af"
},
{
"ImportPath": "google.golang.org/api",
"Rev": "18450f4e95c7e76ce3a5dc3a8cb7178ab6d56121"
},
{
"ImportPath": "google.golang.org/grpc",
"Rev": "91c8b79535eb6045d70ec671d302213f88a3ab95"
},
{
"ImportPath": "github.com/bradfitz/http2",
"Rev": "f8202bc903bda493ebba4aa54922d78430c2c42f"
},
{
"ImportPath": "github.com/golang/protobuf",
"Rev": "0f7a9caded1fb3c9cc5a9b4bcf2ff633cc8ae644"
},
{
"ImportPath": "google.golang.org/cloud/storage",
"Rev": "2400193c85c3561d13880d34e0e10c4315bb02af"
},
{
"ImportPath": "github.com/AdRoll/goamz/aws",
"Rev": "aa6e716d710a0c7941cb2075cfbb9661f16d21f1"
},
{
"ImportPath": "github.com/AdRoll/goamz/cloudfront",
"Rev": "aa6e716d710a0c7941cb2075cfbb9661f16d21f1"
},
{
"ImportPath": "github.com/AdRoll/goamz/s3",
"Rev": "aa6e716d710a0c7941cb2075cfbb9661f16d21f1"
},
{
"ImportPath": "github.com/Azure/azure-sdk-for-go/storage",
"Rev": "97d9593768bbbbd316f9c055dfc5f780933cd7fc"
},
{
"ImportPath": "github.com/Sirupsen/logrus",
"Comment": "v0.7.3",
"Rev": "55eb11d21d2a31a3cc93838241d04800f52e823d"
},
{
"ImportPath": "github.com/bugsnag/bugsnag-go",
"Comment": "v1.0.2-5-gb1d1530",
"Rev": "b1d153021fcd90ca3f080db36bec96dc690fb274"
},
{
"ImportPath": "github.com/bugsnag/osext",
"Rev": "0dd3f918b21bec95ace9dc86c7e70266cfc5c702"
},
{
"ImportPath": "github.com/bugsnag/panicwrap",
"Rev": "e2c28503fcd0675329da73bf48b33404db873782"
},
{
"ImportPath": "github.com/denverdino/aliyungo/oss",
"Rev": "6ffb587da9da6d029d0ce517b85fecc82172d502"
},
{
"ImportPath": "github.com/denverdino/aliyungo/util",
"Rev": "6ffb587da9da6d029d0ce517b85fecc82172d502"
},
{
"ImportPath": "github.com/denverdino/aliyungo/common",
"Rev": "6ffb587da9da6d029d0ce517b85fecc82172d502"
},
{
"ImportPath": "github.com/docker/libtrust",
"Rev": "fa567046d9b14f6aa788882a950d69651d230b21"
},
{
"ImportPath": "github.com/garyburd/redigo/internal",
"Rev": "535138d7bcd717d6531c701ef5933d98b1866257"
},
{
"ImportPath": "github.com/garyburd/redigo/redis",
"Rev": "535138d7bcd717d6531c701ef5933d98b1866257"
},
{
"ImportPath": "github.com/gorilla/context",
"Rev": "14f550f51af52180c2eefed15e5fd18d63c0a64a"
},
{
"ImportPath": "github.com/gorilla/handlers",
"Rev": "60c7bfde3e33c201519a200a4507a158cc03a17b"
},
{
"ImportPath": "github.com/gorilla/mux",
"Rev": "e444e69cbd2e2e3e0749a2f3c717cec491552bbf"
},
{
"ImportPath": "github.com/inconshreveable/mousetrap",
"Rev": "76626ae9c91c4f2a10f34cad8ce83ea42c93bb75"
},
{
"ImportPath": "github.com/mitchellh/mapstructure",
"Rev": "482a9fd5fa83e8c4e7817413b80f3eb8feec03ef"
},
{
"ImportPath": "github.com/ncw/swift",
"Rev": "c54732e87b0b283d1baf0a18db689d0aea460ba3"
},
{
"ImportPath": "github.com/noahdesu/go-ceph/rados",
"Comment": "v.0.3.0-29-gb15639c",
"Rev": "b15639c44c05368348355229070361395d9152ee"
},
{
"ImportPath": "github.com/spf13/cobra",
"Rev": "312092086bed4968099259622145a0c9ae280064"
},
{
"ImportPath": "github.com/spf13/pflag",
"Rev": "5644820622454e71517561946e3d94b9f9db6842"
},
{
"ImportPath": "github.com/stevvooe/resumable",
"Rev": "51ad44105773cafcbe91927f70ac68e1bf78f8b4"
},
{
"ImportPath": "github.com/yvasiyarov/go-metrics",
"Rev": "57bccd1ccd43f94bb17fdd8bf3007059b802f85e"
},
{
"ImportPath": "github.com/yvasiyarov/gorelic",
"Comment": "v0.0.6-8-ga9bba5b",
"Rev": "a9bba5b9ab508a086f9a12b8c51fab68478e2128"
},
{
"ImportPath": "github.com/yvasiyarov/newrelic_platform_go",
"Rev": "b21fdbd4370f3717f3bbd2bf41c223bc273068e6"
},
{
"ImportPath": "golang.org/x/crypto/bcrypt",
"Rev": "c10c31b5e94b6f7a0283272dc2bb27163dcea24b"
},
{
"ImportPath": "golang.org/x/crypto/blowfish",
"Rev": "c10c31b5e94b6f7a0283272dc2bb27163dcea24b"
},
{
"ImportPath": "golang.org/x/net/context",
"Rev": "2cba614e8ff920c60240d2677bc019af32ee04e5"
},
{
"ImportPath": "golang.org/x/net/trace",
"Rev": "2cba614e8ff920c60240d2677bc019af32ee04e5"
},
{
"ImportPath": "gopkg.in/check.v1",
"Rev": "64131543e7896d5bcc6bd5a76287eb75ea96c673"
},
{
"ImportPath": "gopkg.in/yaml.v2",
"Rev": "bef53efd0c76e49e6de55ead051f886bea7e9420"
}
]
}

View File

@@ -1,5 +0,0 @@
This directory tree is generated automatically by godep.
Please do not edit.
See https://github.com/tools/godep for more information.

View File

@@ -1,58 +0,0 @@
# Distribution maintainers file
#
# This file describes who runs the docker/distribution project and how.
# This is a living document - if you see something out of date or missing, speak up!
#
# It is structured to be consumable by both humans and programs.
# To extract its contents programmatically, use any TOML-compliant parser.
#
# This file is compiled into the MAINTAINERS file in docker/opensource.
#
[Org]
[Org."Core maintainers"]
people = [
"aaronlehmann",
"dmcgowan",
"dmp42",
"richardscothern",
"shykes",
"stevvooe",
]
[people]
# A reference list of all people associated with the project.
# All other sections should refer to people by their canonical key
# in the people section.
# ADD YOURSELF HERE IN ALPHABETICAL ORDER
[people.aaronlehmann]
Name = "Aaron Lehmann"
Email = "aaron.lehmann@docker.com"
GitHub = "aaronlehmann"
[people.dmcgowan]
Name = "Derek McGowan"
Email = "derek@mcgstyle.net"
GitHub = "dmcgowan"
[people.dmp42]
Name = "Olivier Gambier"
Email = "olivier@docker.com"
GitHub = "dmp42"
[people.richardscothern]
Name = "Richard Scothern"
Email = "richard.scothern@gmail.com"
GitHub = "richardscothern"
[people.shykes]
Name = "Solomon Hykes"
Email = "solomon@docker.com"
GitHub = "shykes"
[people.stevvooe]
Name = "Stephen Day"
Email = "stephen.day@docker.com"
GitHub = "stevvooe"

View File

@@ -1,74 +0,0 @@
# Set an output prefix, which is the local directory if not specified
PREFIX?=$(shell pwd)
# Used to populate version variable in main package.
VERSION=$(shell git describe --match 'v[0-9]*' --dirty='.m' --always)
# Allow turning off function inlining and variable registerization
ifeq (${DISABLE_OPTIMIZATION},true)
GO_GCFLAGS=-gcflags "-N -l"
VERSION:="$(VERSION)-noopt"
endif
GO_LDFLAGS=-ldflags "-X `go list ./version`.Version=$(VERSION)"
.PHONY: clean all fmt vet lint build test binaries
.DEFAULT: default
all: AUTHORS clean fmt vet fmt lint build test binaries
AUTHORS: .mailmap .git/HEAD
git log --format='%aN <%aE>' | sort -fu > $@
# This only needs to be generated by hand when cutting full releases.
version/version.go:
./version/version.sh > $@
${PREFIX}/bin/registry: version/version.go $(shell find . -type f -name '*.go')
@echo "+ $@"
@go build -tags "${DOCKER_BUILDTAGS}" -o $@ ${GO_LDFLAGS} ${GO_GCFLAGS} ./cmd/registry
${PREFIX}/bin/digest: version/version.go $(shell find . -type f -name '*.go')
@echo "+ $@"
@go build -tags "${DOCKER_BUILDTAGS}" -o $@ ${GO_LDFLAGS} ${GO_GCFLAGS} ./cmd/digest
${PREFIX}/bin/registry-api-descriptor-template: version/version.go $(shell find . -type f -name '*.go')
@echo "+ $@"
@go build -o $@ ${GO_LDFLAGS} ${GO_GCFLAGS} ./cmd/registry-api-descriptor-template
docs/spec/api.md: docs/spec/api.md.tmpl ${PREFIX}/bin/registry-api-descriptor-template
./bin/registry-api-descriptor-template $< > $@
# Depends on binaries because vet will silently fail if it can't load compiled
# imports
vet: binaries
@echo "+ $@"
@go vet ./...
fmt:
@echo "+ $@"
@test -z "$$(gofmt -s -l . | grep -v Godeps/_workspace/src/ | tee /dev/stderr)" || \
echo "+ please format Go code with 'gofmt -s'"
lint:
@echo "+ $@"
@test -z "$$(golint ./... | grep -v Godeps/_workspace/src/ | tee /dev/stderr)"
build:
@echo "+ $@"
@go build -tags "${DOCKER_BUILDTAGS}" -v ${GO_LDFLAGS} ./...
test:
@echo "+ $@"
@go test -test.short -tags "${DOCKER_BUILDTAGS}" ./...
test-full:
@echo "+ $@"
@go test ./...
binaries: ${PREFIX}/bin/registry ${PREFIX}/bin/digest ${PREFIX}/bin/registry-api-descriptor-template
@echo "+ $@"
clean:
@echo "+ $@"
@rm -rf "${PREFIX}/bin/registry" "${PREFIX}/bin/registry-api-descriptor-template"

View File

@@ -1,131 +0,0 @@
# Distribution
The Docker toolset to pack, ship, store, and deliver content.
This repository's main product is the Docker Registry 2.0 implementation
for storing and distributing Docker images. It supersedes the
[docker/docker-registry](https://github.com/docker/docker-registry)
project with a new API design, focused around security and performance.
<img src="https://www.docker.com/sites/default/files/oyster-registry-3.png" width=200px/>
[![Circle CI](https://circleci.com/gh/docker/distribution/tree/master.svg?style=svg)](https://circleci.com/gh/docker/distribution/tree/master)
[![GoDoc](https://godoc.org/github.com/docker/distribution?status.svg)](https://godoc.org/github.com/docker/distribution)
This repository contains the following components:
|**Component** |Description |
|--------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| **registry** | An implementation of the [Docker Registry HTTP API V2](docs/spec/api.md) for use with docker 1.6+. |
| **libraries** | A rich set of libraries for interacting with,distribution components. Please see [godoc](https://godoc.org/github.com/docker/distribution) for details. **Note**: These libraries are **unstable**. |
| **specifications** | _Distribution_ related specifications are available in [docs/spec](docs/spec) |
| **documentation** | Docker's full documentation set is available at [docs.docker.com](https://docs.docker.com). This repository [contains the subset](docs/index.md) related just to the registry. |
### How does this integrate with Docker engine?
This project should provide an implementation to a V2 API for use in the [Docker
core project](https://github.com/docker/docker). The API should be embeddable
and simplify the process of securely pulling and pushing content from `docker`
daemons.
### What are the long term goals of the Distribution project?
The _Distribution_ project has the further long term goal of providing a
secure tool chain for distributing content. The specifications, APIs and tools
should be as useful with Docker as they are without.
Our goal is to design a professional grade and extensible content distribution
system that allow users to:
* Enjoy an efficient, secured and reliable way to store, manage, package and
exchange content
* Hack/roll their own on top of healthy open-source components
* Implement their own home made solution through good specs, and solid
extensions mechanism.
## More about Registry 2.0
The new registry implementation provides the following benefits:
- faster push and pull
- new, more efficient implementation
- simplified deployment
- pluggable storage backend
- webhook notifications
For information on upcoming functionality, please see [ROADMAP.md](ROADMAP.md).
### Who needs to deploy a registry?
By default, Docker users pull images from Docker's public registry instance.
[Installing Docker](https://docs.docker.com/engine/installation/) gives users this
ability. Users can also push images to a repository on Docker's public registry,
if they have a [Docker Hub](https://hub.docker.com/) account.
For some users and even companies, this default behavior is sufficient. For
others, it is not.
For example, users with their own software products may want to maintain a
registry for private, company images. Also, you may wish to deploy your own
image repository for images used to test or in continuous integration. For these
use cases and others, [deploying your own registry instance](docs/deploying.md)
may be the better choice.
### Migration to Registry 2.0
For those who have previously deployed their own registry based on the Registry
1.0 implementation and wish to deploy a Registry 2.0 while retaining images,
data migration is required. A tool to assist with migration efforts has been
created. For more information see [docker/migrator]
(https://github.com/docker/migrator).
## Contribute
Please see [CONTRIBUTING.md](CONTRIBUTING.md) for details on how to contribute
issues, fixes, and patches to this project. If you are contributing code, see
the instructions for [building a development environment](docs/building.md).
## Support
If any issues are encountered while using the _Distribution_ project, several
avenues are available for support:
<table>
<tr>
<th align="left">
IRC
</th>
<td>
#docker-distribution on FreeNode
</td>
</tr>
<tr>
<th align="left">
Issue Tracker
</th>
<td>
github.com/docker/distribution/issues
</td>
</tr>
<tr>
<th align="left">
Google Groups
</th>
<td>
https://groups.google.com/a/dockerproject.org/forum/#!forum/distribution
</td>
</tr>
<tr>
<th align="left">
Mailing List
</th>
<td>
docker@dockerproject.org
</td>
</tr>
</table>
## License
This project is distributed under [Apache License, Version 2.0](LICENSE.md).

View File

@@ -1,267 +0,0 @@
# Roadmap
The Distribution Project consists of several components, some of which are
still being defined. This document defines the high-level goals of the
project, identifies the current components, and defines the release-
relationship to the Docker Platform.
* [Distribution Goals](#distribution-goals)
* [Distribution Components](#distribution-components)
* [Project Planning](#project-planning): release-relationship to the Docker Platform.
This road map is a living document, providing an overview of the goals and
considerations made in respect of the future of the project.
## Distribution Goals
- Replace the existing [docker registry](github.com/docker/docker-registry)
implementation as the primary implementation.
- Replace the existing push and pull code in the docker engine with the
distribution package.
- Define a strong data model for distributing docker images
- Provide a flexible distribution tool kit for use in the docker platform
- Unlock new distribution models
## Distribution Components
Components of the Distribution Project are managed via github [milestones](https://github.com/docker/distribution/milestones). Upcoming
features and bugfixes for a component will be added to the relevant milestone. If a feature or
bugfix is not part of a milestone, it is currently unscheduled for
implementation.
* [Registry](#registry)
* [Distribution Package](#distribution-package)
***
### Registry
The new Docker registry is the main portion of the distribution repository.
Registry 2.0 is the first release of the next-generation registry. This was
primarily focused on implementing the [new registry
API](https://github.com/docker/distribution/blob/master/docs/spec/api.md),
with a focus on security and performance.
Following from the Distribution project goals above, we have a set of goals
for registry v2 that we would like to follow in the design. New features
should be compared against these goals.
#### Data Storage and Distribution First
The registry's first goal is to provide a reliable, consistent storage
location for Docker images. The registry should only provide the minimal
amount of indexing required to fetch image data and no more.
This means we should be selective in new features and API additions, including
those that may require expensive, ever growing indexes. Requests should be
servable in "constant time".
#### Content Addressability
All data objects used in the registry API should be content addressable.
Content identifiers should be secure and verifiable. This provides a secure,
reliable base from which to build more advanced content distribution systems.
#### Content Agnostic
In the past, changes to the image format would require large changes in Docker
and the Registry. By decoupling the distribution and image format, we can
allow the formats to progress without having to coordinate between the two.
This means that we should be focused on decoupling Docker from the registry
just as much as decoupling the registry from Docker. Such an approach will
allow us to unlock new distribution models that haven't been possible before.
We can take this further by saying that the new registry should be content
agnostic. The registry provides a model of names, tags, manifests and content
addresses and that model can be used to work with content.
#### Simplicity
The new registry should be closer to a microservice component than its
predecessor. This means it should have a narrower API and a low number of
service dependencies. It should be easy to deploy.
This means that other solutions should be explored before changing the API or
adding extra dependencies. If functionality is required, can it be added as an
extension or companion service.
#### Extensibility
The registry should provide extension points to add functionality. By keeping
the scope narrow, but providing the ability to add functionality.
Features like search, indexing, synchronization and registry explorers fall
into this category. No such feature should be added unless we've found it
impossible to do through an extension.
#### Active Feature Discussions
The following are feature discussions that are currently active.
If you don't see your favorite, unimplemented feature, feel free to contact us
via IRC or the mailing list and we can talk about adding it. The goal here is
to make sure that new features go through a rigid design process before
landing in the registry.
##### Proxying to other Registries
A _pull-through caching_ mode exists for the registry, but is restricted from
within the docker client to only mirror the official Docker Hub. This functionality
can be expanded when image provenance has been specified and implemented in the
distribution project.
##### Metadata storage
Metadata for the registry is currently stored with the manifest and layer data on
the storage backend. While this is a big win for simplicity and reliably maintaining
state, it comes with the cost of consistency and high latency. The mutable registry
metadata operations should be abstracted behind an API which will allow ACID compliant
storage systems to handle metadata.
##### Peer to Peer transfer
Discussion has started here: https://docs.google.com/document/d/1rYDpSpJiQWmCQy8Cuiaa3NH-Co33oK_SC9HeXYo87QA/edit
##### Indexing, Search and Discovery
The original registry provided some implementation of search for use with
private registries. Support has been elided from V2 since we'd like to both
decouple search functionality from the registry. The makes the registry
simpler to deploy, especially in use cases where search is not needed, and
let's us decouple the image format from the registry.
There are explorations into using the catalog API and notification system to
build external indexes. The current line of thought is that we will define a
common search API to index and query docker images. Such a system could be run
as a companion to a registry or set of registries to power discovery.
The main issue with search and discovery is that there are so many ways to
accomplish it. There are two aspects to this project. The first is deciding on
how it will be done, including an API definition that can work with changing
data formats. The second is the process of integrating with `docker search`.
We expect that someone attempts to address the problem with the existing tools
and propose it as a standard search API or uses it to inform a standardization
process. Once this has been explored, we integrate with the docker client.
Please see the following for more detail:
- https://github.com/docker/distribution/issues/206
##### Deletes
> __NOTE:__ Deletes are a much asked for feature. Before requesting this
feature or participating in discussion, we ask that you read this section in
full and understand the problems behind deletes.
While, at first glance, implementing deleting seems simple, there are a number
mitigating factors that make many solutions not ideal or even pathological in
the context of a registry. The following paragraph discuss the background and
approaches that could be applied to a arrive at a solution.
The goal of deletes in any system is to remove unused or unneeded data. Only
data requested for deletion should be removed and no other data. Removing
unintended data is worse than _not_ removing data that was requested for
removal but ideally, both are supported. Generally, according to this rule, we
err on holding data longer than needed, ensuring that it is only removed when
we can be certain that it can be removed. With the current behavior, we opt to
hold onto the data forever, ensuring that data cannot be incorrectly removed.
To understand the problems with implementing deletes, one must understand the
data model. All registry data is stored in a filesystem layout, implemented on
a "storage driver", effectively a _virtual file system_ (VFS). The storage
system must assume that this VFS layer will be eventually consistent and has
poor read- after-write consistency, since this is the lower common denominator
among the storage drivers. This is mitigated by writing values in reverse-
dependent order, but makes wider transactional operations unsafe.
Layered on the VFS model is a content-addressable _directed, acyclic graph_
(DAG) made up of blobs. Manifests reference layers. Tags reference manifests.
Since the same data can be referenced by multiple manifests, we only store
data once, even if it is in different repositories. Thus, we have a set of
blobs, referenced by tags and manifests. If we want to delete a blob we need
to be certain that it is no longer referenced by another manifest or tag. When
we delete a manifest, we also can try to delete the referenced blobs. Deciding
whether or not a blob has an active reference is the crux of the problem.
Conceptually, deleting a manifest and its resources is quite simple. Just find
all the manifests, enumerate the referenced blobs and delete the blobs not in
that set. An astute observer will recognize this as a garbage collection
problem. As with garbage collection in programming languages, this is very
simple when one always has a consistent view. When one adds parallelism and an
inconsistent view of data, it becomes very challenging.
A simple example can demonstrate this. Let's say we are deleting a manifest
_A_ in one process. We scan the manifest and decide that all the blobs are
ready for deletion. Concurrently, we have another process accepting a new
manifest _B_ referencing one or more blobs from the manifest _A_. Manifest _B_
is accepted and all the blobs are considered present, so the operation
proceeds. The original process then deletes the referenced blobs, assuming
they were unreferenced. The manifest _B_, which we thought had all of its data
present, can no longer be served by the registry, since the dependent data has
been deleted.
Deleting data from the registry safely requires some way to coordinate this
operation. The following approaches are being considered:
- _Reference Counting_ - Maintain a count of references to each blob. This is
challenging for a number of reasons: 1. maintaining a consistent consensus
of reference counts across a set of Registries and 2. Building the initial
list of reference counts for an existing registry. These challenges can be
met with a consensus protocol like Paxos or Raft in the first case and a
necessary but simple scan in the second..
- _Lock the World GC_ - Halt all writes to the data store. Walk the data store
and find all blob references. Delete all unreferenced blobs. This approach
is very simple but requires disabling writes for a period of time while the
service reads all data. This is slow and expensive but very accurate and
effective.
- _Generational GC_ - Do something similar to above but instead of blocking
writes, writes are sent to another storage backend while reads are broadcast
to the new and old backends. GC is then performed on the read-only portion.
Because writes land in the new backend, the data in the read-only section
can be safely deleted. The main drawbacks of this approach are complexity
and coordination.
- _Centralized Oracle_ - Using a centralized, transactional database, we can
know exactly which data is referenced at any given time. This avoids
coordination problem by managing this data in a single location. We trade
off metadata scalability for simplicity and performance. This is a very good
option for most registry deployments. This would create a bottleneck for
registry metadata. However, metadata is generally not the main bottleneck
when serving images.
Please let us know if other solutions exist that we have yet to enumerate.
Note that for any approach, implementation is a massive consideration. For
example, a mark-sweep based solution may seem simple but the amount of work in
coordination offset the extra work it might take to build a _Centralized
Oracle_. We'll accept proposals for any solution but please coordinate with us
before dropping code.
At this time, we have traded off simplicity and ease of deployment for disk
space. Simplicity and ease of deployment tend to reduce developer involvement,
which is currently the most expensive resource in software engineering. Taking
on any solution for deletes will greatly effect these factors, trading off
very cheap disk space for a complex deployment and operational story.
Please see the following issues for more detail:
- https://github.com/docker/distribution/issues/422
- https://github.com/docker/distribution/issues/461
- https://github.com/docker/distribution/issues/462
### Distribution Package
At its core, the Distribution Project is a set of Go packages that make up
Distribution Components. At this time, most of these packages make up the
Registry implementation.
The package itself is considered unstable. If you're using it, please take care to vendor the dependent version.
For feature additions, please see the Registry section. In the future, we may break out a
separate Roadmap for distribution-specific features that apply to more than
just the registry.
***
### Project Planning
An [Open-Source Planning Process](https://github.com/docker/distribution/wiki/Open-Source-Planning-Process) is used to define the Roadmap. [Project Pages](https://github.com/docker/distribution/wiki) define the goals for each Milestone and identify current progress.

View File

@@ -1,233 +0,0 @@
package distribution
import (
"errors"
"fmt"
"io"
"net/http"
"time"
"github.com/docker/distribution/context"
"github.com/docker/distribution/digest"
"github.com/docker/distribution/reference"
)
var (
// ErrBlobExists returned when blob already exists
ErrBlobExists = errors.New("blob exists")
// ErrBlobDigestUnsupported when blob digest is an unsupported version.
ErrBlobDigestUnsupported = errors.New("unsupported blob digest")
// ErrBlobUnknown when blob is not found.
ErrBlobUnknown = errors.New("unknown blob")
// ErrBlobUploadUnknown returned when upload is not found.
ErrBlobUploadUnknown = errors.New("blob upload unknown")
// ErrBlobInvalidLength returned when the blob has an expected length on
// commit, meaning mismatched with the descriptor or an invalid value.
ErrBlobInvalidLength = errors.New("blob invalid length")
)
// ErrBlobInvalidDigest returned when digest check fails.
type ErrBlobInvalidDigest struct {
Digest digest.Digest
Reason error
}
func (err ErrBlobInvalidDigest) Error() string {
return fmt.Sprintf("invalid digest for referenced layer: %v, %v",
err.Digest, err.Reason)
}
// ErrBlobMounted returned when a blob is mounted from another repository
// instead of initiating an upload session.
type ErrBlobMounted struct {
From reference.Canonical
Descriptor Descriptor
}
func (err ErrBlobMounted) Error() string {
return fmt.Sprintf("blob mounted from: %v to: %v",
err.From, err.Descriptor)
}
// Descriptor describes targeted content. Used in conjunction with a blob
// store, a descriptor can be used to fetch, store and target any kind of
// blob. The struct also describes the wire protocol format. Fields should
// only be added but never changed.
type Descriptor struct {
// MediaType describe the type of the content. All text based formats are
// encoded as utf-8.
MediaType string `json:"mediaType,omitempty"`
// Size in bytes of content.
Size int64 `json:"size,omitempty"`
// Digest uniquely identifies the content. A byte stream can be verified
// against against this digest.
Digest digest.Digest `json:"digest,omitempty"`
// NOTE: Before adding a field here, please ensure that all
// other options have been exhausted. Much of the type relationships
// depend on the simplicity of this type.
}
// Descriptor returns the descriptor, to make it satisfy the Describable
// interface. Note that implementations of Describable are generally objects
// which can be described, not simply descriptors; this exception is in place
// to make it more convenient to pass actual descriptors to functions that
// expect Describable objects.
func (d Descriptor) Descriptor() Descriptor {
return d
}
// BlobStatter makes blob descriptors available by digest. The service may
// provide a descriptor of a different digest if the provided digest is not
// canonical.
type BlobStatter interface {
// Stat provides metadata about a blob identified by the digest. If the
// blob is unknown to the describer, ErrBlobUnknown will be returned.
Stat(ctx context.Context, dgst digest.Digest) (Descriptor, error)
}
// BlobDeleter enables deleting blobs from storage.
type BlobDeleter interface {
Delete(ctx context.Context, dgst digest.Digest) error
}
// BlobDescriptorService manages metadata about a blob by digest. Most
// implementations will not expose such an interface explicitly. Such mappings
// should be maintained by interacting with the BlobIngester. Hence, this is
// left off of BlobService and BlobStore.
type BlobDescriptorService interface {
BlobStatter
// SetDescriptor assigns the descriptor to the digest. The provided digest and
// the digest in the descriptor must map to identical content but they may
// differ on their algorithm. The descriptor must have the canonical
// digest of the content and the digest algorithm must match the
// annotators canonical algorithm.
//
// Such a facility can be used to map blobs between digest domains, with
// the restriction that the algorithm of the descriptor must match the
// canonical algorithm (ie sha256) of the annotator.
SetDescriptor(ctx context.Context, dgst digest.Digest, desc Descriptor) error
// Clear enables descriptors to be unlinked
Clear(ctx context.Context, dgst digest.Digest) error
}
// ReadSeekCloser is the primary reader type for blob data, combining
// io.ReadSeeker with io.Closer.
type ReadSeekCloser interface {
io.ReadSeeker
io.Closer
}
// BlobProvider describes operations for getting blob data.
type BlobProvider interface {
// Get returns the entire blob identified by digest along with the descriptor.
Get(ctx context.Context, dgst digest.Digest) ([]byte, error)
// Open provides a ReadSeekCloser to the blob identified by the provided
// descriptor. If the blob is not known to the service, an error will be
// returned.
Open(ctx context.Context, dgst digest.Digest) (ReadSeekCloser, error)
}
// BlobServer can serve blobs via http.
type BlobServer interface {
// ServeBlob attempts to serve the blob, identifed by dgst, via http. The
// service may decide to redirect the client elsewhere or serve the data
// directly.
//
// This handler only issues successful responses, such as 2xx or 3xx,
// meaning it serves data or issues a redirect. If the blob is not
// available, an error will be returned and the caller may still issue a
// response.
//
// The implementation may serve the same blob from a different digest
// domain. The appropriate headers will be set for the blob, unless they
// have already been set by the caller.
ServeBlob(ctx context.Context, w http.ResponseWriter, r *http.Request, dgst digest.Digest) error
}
// BlobIngester ingests blob data.
type BlobIngester interface {
// Put inserts the content p into the blob service, returning a descriptor
// or an error.
Put(ctx context.Context, mediaType string, p []byte) (Descriptor, error)
// Create allocates a new blob writer to add a blob to this service. The
// returned handle can be written to and later resumed using an opaque
// identifier. With this approach, one can Close and Resume a BlobWriter
// multiple times until the BlobWriter is committed or cancelled.
Create(ctx context.Context, options ...BlobCreateOption) (BlobWriter, error)
// Resume attempts to resume a write to a blob, identified by an id.
Resume(ctx context.Context, id string) (BlobWriter, error)
}
// BlobCreateOption is a general extensible function argument for blob creation
// methods. A BlobIngester may choose to honor any or none of the given
// BlobCreateOptions, which can be specific to the implementation of the
// BlobIngester receiving them.
// TODO (brianbland): unify this with ManifestServiceOption in the future
type BlobCreateOption interface {
Apply(interface{}) error
}
// BlobWriter provides a handle for inserting data into a blob store.
// Instances should be obtained from BlobWriteService.Writer and
// BlobWriteService.Resume. If supported by the store, a writer can be
// recovered with the id.
type BlobWriter interface {
io.WriteSeeker
io.ReaderFrom
io.Closer
// ID returns the identifier for this writer. The ID can be used with the
// Blob service to later resume the write.
ID() string
// StartedAt returns the time this blob write was started.
StartedAt() time.Time
// Commit completes the blob writer process. The content is verified
// against the provided provisional descriptor, which may result in an
// error. Depending on the implementation, written data may be validated
// against the provisional descriptor fields. If MediaType is not present,
// the implementation may reject the commit or assign "application/octet-
// stream" to the blob. The returned descriptor may have a different
// digest depending on the blob store, referred to as the canonical
// descriptor.
Commit(ctx context.Context, provisional Descriptor) (canonical Descriptor, err error)
// Cancel ends the blob write without storing any data and frees any
// associated resources. Any data written thus far will be lost. Cancel
// implementations should allow multiple calls even after a commit that
// result in a no-op. This allows use of Cancel in a defer statement,
// increasing the assurance that it is correctly called.
Cancel(ctx context.Context) error
// Get a reader to the blob being written by this BlobWriter
Reader() (io.ReadCloser, error)
}
// BlobService combines the operations to access, read and write blobs. This
// can be used to describe remote blob services.
type BlobService interface {
BlobStatter
BlobProvider
BlobIngester
}
// BlobStore represent the entire suite of blob related operations. Such an
// implementation can access, read, write, delete and serve blobs.
type BlobStore interface {
BlobService
BlobServer
BlobDeleter
}

View File

@@ -1,90 +0,0 @@
# Pony-up!
machine:
pre:
# Install gvm
- bash < <(curl -s -S -L https://raw.githubusercontent.com/moovweb/gvm/1.0.22/binscripts/gvm-installer)
# Install ceph to test rados driver & create pool
- sudo -i ~/distribution/contrib/ceph/ci-setup.sh
- ceph osd pool create docker-distribution 1
# Install codecov for coverage
- pip install --user codecov
post:
# go
- gvm install go1.5.3 --prefer-binary --name=stable
environment:
# Convenient shortcuts to "common" locations
CHECKOUT: /home/ubuntu/$CIRCLE_PROJECT_REPONAME
BASE_DIR: src/github.com/$CIRCLE_PROJECT_USERNAME/$CIRCLE_PROJECT_REPONAME
# Trick circle brainflat "no absolute path" behavior
BASE_STABLE: ../../../$HOME/.gvm/pkgsets/stable/global/$BASE_DIR
DOCKER_BUILDTAGS: "include_rados include_oss include_gcs"
# Workaround Circle parsing dumb bugs and/or YAML wonkyness
CIRCLE_PAIN: "mode: set"
# Ceph config
RADOS_POOL: "docker-distribution"
hosts:
# Not used yet
fancy: 127.0.0.1
dependencies:
pre:
# Copy the code to the gopath of all go versions
- >
gvm use stable &&
mkdir -p "$(dirname $BASE_STABLE)" &&
cp -R "$CHECKOUT" "$BASE_STABLE"
override:
# Install dependencies for every copied clone/go version
- gvm use stable && go get github.com/tools/godep:
pwd: $BASE_STABLE
post:
# For the stable go version, additionally install linting tools
- >
gvm use stable &&
go get github.com/axw/gocov/gocov github.com/golang/lint/golint
test:
pre:
# Output the go versions we are going to test
# - gvm use old && go version
- gvm use stable && go version
# First thing: build everything. This will catch compile errors, and it's
# also necessary for go vet to work properly (see #807).
- gvm use stable && godep go install ./...:
pwd: $BASE_STABLE
# FMT
- gvm use stable && test -z "$(gofmt -s -l . | grep -v Godeps/_workspace/src/ | tee /dev/stderr)":
pwd: $BASE_STABLE
# VET
- gvm use stable && go vet ./...:
pwd: $BASE_STABLE
# LINT
- gvm use stable && test -z "$(golint ./... | grep -v Godeps/_workspace/src/ | tee /dev/stderr)":
pwd: $BASE_STABLE
override:
# Test stable, and report
- gvm use stable; export ROOT_PACKAGE=$(go list .); go list -tags "$DOCKER_BUILDTAGS" ./... | xargs -L 1 -I{} bash -c 'export PACKAGE={}; godep go test -tags "$DOCKER_BUILDTAGS" -test.short -coverprofile=$GOPATH/src/$PACKAGE/coverage.out -coverpkg=$(./coverpkg.sh $PACKAGE $ROOT_PACKAGE) $PACKAGE':
timeout: 600
pwd: $BASE_STABLE
post:
# Report to codecov
- bash <(curl -s https://codecov.io/bash):
pwd: $BASE_STABLE
## Notes
# Disabled the -race detector due to massive memory usage.
# Do we want these as well?
# - go get code.google.com/p/go.tools/cmd/goimports
# - test -z "$(goimports -l -w ./... | tee /dev/stderr)"
# http://labix.org/gocheck

View File

@@ -1,97 +0,0 @@
package main
import (
"flag"
"fmt"
"io"
"log"
"os"
"github.com/docker/distribution/digest"
"github.com/docker/distribution/version"
)
var (
algorithm = digest.Canonical
showVersion bool
)
type job struct {
name string
reader io.Reader
}
func init() {
flag.Var(&algorithm, "a", "select the digest algorithm (shorthand)")
flag.Var(&algorithm, "algorithm", "select the digest algorithm")
flag.BoolVar(&showVersion, "version", false, "show the version and exit")
log.SetFlags(0)
log.SetPrefix(os.Args[0] + ": ")
}
func usage() {
fmt.Fprintf(os.Stderr, "usage: %s [files...]\n", os.Args[0])
fmt.Fprintf(os.Stderr, `
Calculate the digest of one or more input files, emitting the result
to standard out. If no files are provided, the digest of stdin will
be calculated.
`)
flag.PrintDefaults()
}
func unsupported() {
log.Fatalf("unsupported digest algorithm: %v", algorithm)
}
func main() {
var jobs []job
flag.Usage = usage
flag.Parse()
if showVersion {
version.PrintVersion()
return
}
var fail bool // if we fail on one item, foul the exit code
if flag.NArg() > 0 {
for _, path := range flag.Args() {
fp, err := os.Open(path)
if err != nil {
log.Printf("%s: %v", path, err)
fail = true
continue
}
defer fp.Close()
jobs = append(jobs, job{name: path, reader: fp})
}
} else {
// just read stdin
jobs = append(jobs, job{name: "-", reader: os.Stdin})
}
digestFn := algorithm.FromReader
if !algorithm.Available() {
unsupported()
}
for _, job := range jobs {
dgst, err := digestFn(job.reader)
if err != nil {
log.Printf("%s: %v", job.name, err)
fail = true
continue
}
fmt.Printf("%v\t%s\n", dgst, job.name)
}
if fail {
os.Exit(1)
}
}

View File

@@ -1,131 +0,0 @@
// registry-api-descriptor-template uses the APIDescriptor defined in the
// api/v2 package to execute templates passed to the command line.
//
// For example, to generate a new API specification, one would execute the
// following command from the repo root:
//
// $ registry-api-descriptor-template docs/spec/api.md.tmpl > docs/spec/api.md
//
// The templates are passed in the api/v2.APIDescriptor object. Please see the
// package documentation for fields available on that object. The template
// syntax is from Go's standard library text/template package. For information
// on Go's template syntax, please see golang.org/pkg/text/template.
package main
import (
"log"
"net/http"
"os"
"path/filepath"
"regexp"
"text/template"
"github.com/docker/distribution/registry/api/errcode"
"github.com/docker/distribution/registry/api/v2"
)
var spaceRegex = regexp.MustCompile(`\n\s*`)
func main() {
if len(os.Args) != 2 {
log.Fatalln("please specify a template to execute.")
}
path := os.Args[1]
filename := filepath.Base(path)
funcMap := template.FuncMap{
"removenewlines": func(s string) string {
return spaceRegex.ReplaceAllString(s, " ")
},
"statustext": http.StatusText,
"prettygorilla": prettyGorillaMuxPath,
}
tmpl := template.Must(template.New(filename).Funcs(funcMap).ParseFiles(path))
data := struct {
RouteDescriptors []v2.RouteDescriptor
ErrorDescriptors []errcode.ErrorDescriptor
}{
RouteDescriptors: v2.APIDescriptor.RouteDescriptors,
ErrorDescriptors: append(errcode.GetErrorCodeGroup("registry.api.v2"),
// The following are part of the specification but provided by errcode default.
errcode.ErrorCodeUnauthorized.Descriptor(),
errcode.ErrorCodeDenied.Descriptor(),
errcode.ErrorCodeUnsupported.Descriptor()),
}
if err := tmpl.Execute(os.Stdout, data); err != nil {
log.Fatalln(err)
}
}
// prettyGorillaMuxPath removes the regular expressions from a gorilla/mux
// route string, making it suitable for documentation.
func prettyGorillaMuxPath(s string) string {
// Stateful parser that removes regular expressions from gorilla
// routes. It correctly handles balanced bracket pairs.
var output string
var label string
var level int
start:
if s[0] == '{' {
s = s[1:]
level++
goto capture
}
output += string(s[0])
s = s[1:]
goto end
capture:
switch s[0] {
case '{':
level++
case '}':
level--
if level == 0 {
s = s[1:]
goto label
}
case ':':
s = s[1:]
goto skip
default:
label += string(s[0])
}
s = s[1:]
goto capture
skip:
switch s[0] {
case '{':
level++
case '}':
level--
}
s = s[1:]
if level == 0 {
goto label
}
goto skip
label:
if label != "" {
output += "<" + label + ">"
label = ""
}
end:
if s != "" {
goto start
}
return output
}

View File

@@ -1,55 +0,0 @@
version: 0.1
log:
level: debug
fields:
service: registry
environment: development
storage:
cache:
blobdescriptor: redis
filesystem:
rootdirectory: /var/lib/registry-cache
maintenance:
uploadpurging:
enabled: false
http:
addr: :5000
secret: asecretforlocaldevelopment
debug:
addr: localhost:5001
headers:
X-Content-Type-Options: [nosniff]
redis:
addr: localhost:6379
pool:
maxidle: 16
maxactive: 64
idletimeout: 300s
dialtimeout: 10ms
readtimeout: 10ms
writetimeout: 10ms
notifications:
endpoints:
- name: local-8082
url: http://localhost:5003/callback
headers:
Authorization: [Bearer <an example token>]
timeout: 1s
threshold: 10
backoff: 1s
disabled: true
- name: local-8083
url: http://localhost:8083/callback
timeout: 1s
threshold: 10
backoff: 1s
disabled: true
proxy:
remoteurl: https://registry-1.docker.io
username: username
password: password
health:
storagedriver:
enabled: true
interval: 10s
threshold: 3

View File

@@ -1,66 +0,0 @@
version: 0.1
log:
level: debug
fields:
service: registry
environment: development
hooks:
- type: mail
disabled: true
levels:
- panic
options:
smtp:
addr: mail.example.com:25
username: mailuser
password: password
insecure: true
from: sender@example.com
to:
- errors@example.com
storage:
delete:
enabled: true
cache:
blobdescriptor: redis
filesystem:
rootdirectory: /var/lib/registry
maintenance:
uploadpurging:
enabled: false
http:
addr: :5000
debug:
addr: localhost:5001
headers:
X-Content-Type-Options: [nosniff]
redis:
addr: localhost:6379
pool:
maxidle: 16
maxactive: 64
idletimeout: 300s
dialtimeout: 10ms
readtimeout: 10ms
writetimeout: 10ms
notifications:
endpoints:
- name: local-5003
url: http://localhost:5003/callback
headers:
Authorization: [Bearer <an example token>]
timeout: 1s
threshold: 10
backoff: 1s
disabled: true
- name: local-8083
url: http://localhost:8083/callback
timeout: 1s
threshold: 10
backoff: 1s
disabled: true
health:
storagedriver:
enabled: true
interval: 10s
threshold: 3

View File

@@ -1,18 +0,0 @@
version: 0.1
log:
fields:
service: registry
storage:
cache:
blobdescriptor: inmemory
filesystem:
rootdirectory: /var/lib/registry
http:
addr: :5000
headers:
X-Content-Type-Options: [nosniff]
health:
storagedriver:
enabled: true
interval: 10s
threshold: 3

View File

@@ -1,23 +0,0 @@
package main
import (
_ "net/http/pprof"
"github.com/docker/distribution/registry"
_ "github.com/docker/distribution/registry/auth/htpasswd"
_ "github.com/docker/distribution/registry/auth/silly"
_ "github.com/docker/distribution/registry/auth/token"
_ "github.com/docker/distribution/registry/proxy"
_ "github.com/docker/distribution/registry/storage/driver/azure"
_ "github.com/docker/distribution/registry/storage/driver/filesystem"
_ "github.com/docker/distribution/registry/storage/driver/gcs"
_ "github.com/docker/distribution/registry/storage/driver/inmemory"
_ "github.com/docker/distribution/registry/storage/driver/middleware/cloudfront"
_ "github.com/docker/distribution/registry/storage/driver/oss"
_ "github.com/docker/distribution/registry/storage/driver/s3"
_ "github.com/docker/distribution/registry/storage/driver/swift"
)
func main() {
registry.Cmd.Execute()
}

View File

@@ -1,5 +0,0 @@
// +build include_rados
package main
import _ "github.com/docker/distribution/registry/storage/driver/rados"

View File

@@ -1,571 +0,0 @@
package configuration
import (
"fmt"
"io"
"io/ioutil"
"net/http"
"reflect"
"strings"
"time"
)
// Configuration is a versioned registry configuration, intended to be provided by a yaml file, and
// optionally modified by environment variables.
//
// Note that yaml field names should never include _ characters, since this is the separator used
// in environment variable names.
type Configuration struct {
// Version is the version which defines the format of the rest of the configuration
Version Version `yaml:"version"`
// Log supports setting various parameters related to the logging
// subsystem.
Log struct {
// Level is the granularity at which registry operations are logged.
Level Loglevel `yaml:"level"`
// Formatter overrides the default formatter with another. Options
// include "text", "json" and "logstash".
Formatter string `yaml:"formatter,omitempty"`
// Fields allows users to specify static string fields to include in
// the logger context.
Fields map[string]interface{} `yaml:"fields,omitempty"`
// Hooks allows users to configurate the log hooks, to enabling the
// sequent handling behavior, when defined levels of log message emit.
Hooks []LogHook `yaml:"hooks,omitempty"`
}
// Loglevel is the level at which registry operations are logged. This is
// deprecated. Please use Log.Level in the future.
Loglevel Loglevel `yaml:"loglevel,omitempty"`
// Storage is the configuration for the registry's storage driver
Storage Storage `yaml:"storage"`
// Auth allows configuration of various authorization methods that may be
// used to gate requests.
Auth Auth `yaml:"auth,omitempty"`
// Middleware lists all middlewares to be used by the registry.
Middleware map[string][]Middleware `yaml:"middleware,omitempty"`
// Reporting is the configuration for error reporting
Reporting Reporting `yaml:"reporting,omitempty"`
// HTTP contains configuration parameters for the registry's http
// interface.
HTTP struct {
// Addr specifies the bind address for the registry instance.
Addr string `yaml:"addr,omitempty"`
// Net specifies the net portion of the bind address. A default empty value means tcp.
Net string `yaml:"net,omitempty"`
// Host specifies an externally-reachable address for the registry, as a fully
// qualified URL.
Host string `yaml:"host,omitempty"`
Prefix string `yaml:"prefix,omitempty"`
// Secret specifies the secret key which HMAC tokens are created with.
Secret string `yaml:"secret,omitempty"`
// TLS instructs the http server to listen with a TLS configuration.
// This only support simple tls configuration with a cert and key.
// Mostly, this is useful for testing situations or simple deployments
// that require tls. If more complex configurations are required, use
// a proxy or make a proposal to add support here.
TLS struct {
// Certificate specifies the path to an x509 certificate file to
// be used for TLS.
Certificate string `yaml:"certificate,omitempty"`
// Key specifies the path to the x509 key file, which should
// contain the private portion for the file specified in
// Certificate.
Key string `yaml:"key,omitempty"`
// Specifies the CA certs for client authentication
// A file may contain multiple CA certificates encoded as PEM
ClientCAs []string `yaml:"clientcas,omitempty"`
} `yaml:"tls,omitempty"`
// Headers is a set of headers to include in HTTP responses. A common
// use case for this would be security headers such as
// Strict-Transport-Security. The map keys are the header names, and
// the values are the associated header payloads.
Headers http.Header `yaml:"headers,omitempty"`
// Debug configures the http debug interface, if specified. This can
// include services such as pprof, expvar and other data that should
// not be exposed externally. Left disabled by default.
Debug struct {
// Addr specifies the bind address for the debug server.
Addr string `yaml:"addr,omitempty"`
} `yaml:"debug,omitempty"`
} `yaml:"http,omitempty"`
// Notifications specifies configuration about various endpoint to which
// registry events are dispatched.
Notifications Notifications `yaml:"notifications,omitempty"`
// Redis configures the redis pool available to the registry webapp.
Redis struct {
// Addr specifies the the redis instance available to the application.
Addr string `yaml:"addr,omitempty"`
// Password string to use when making a connection.
Password string `yaml:"password,omitempty"`
// DB specifies the database to connect to on the redis instance.
DB int `yaml:"db,omitempty"`
DialTimeout time.Duration `yaml:"dialtimeout,omitempty"` // timeout for connect
ReadTimeout time.Duration `yaml:"readtimeout,omitempty"` // timeout for reads of data
WriteTimeout time.Duration `yaml:"writetimeout,omitempty"` // timeout for writes of data
// Pool configures the behavior of the redis connection pool.
Pool struct {
// MaxIdle sets the maximum number of idle connections.
MaxIdle int `yaml:"maxidle,omitempty"`
// MaxActive sets the maximum number of connections that should be
// opened before blocking a connection request.
MaxActive int `yaml:"maxactive,omitempty"`
// IdleTimeout sets the amount time to wait before closing
// inactive connections.
IdleTimeout time.Duration `yaml:"idletimeout,omitempty"`
} `yaml:"pool,omitempty"`
} `yaml:"redis,omitempty"`
Health Health `yaml:"health,omitempty"`
Proxy Proxy `yaml:"proxy,omitempty"`
}
// LogHook is composed of hook Level and Type.
// After hooks configuration, it can execute the next handling automatically,
// when defined levels of log message emitted.
// Example: hook can sending an email notification when error log happens in app.
type LogHook struct {
// Disable lets user select to enable hook or not.
Disabled bool `yaml:"disabled,omitempty"`
// Type allows user to select which type of hook handler they want.
Type string `yaml:"type,omitempty"`
// Levels set which levels of log message will let hook executed.
Levels []string `yaml:"levels,omitempty"`
// MailOptions allows user to configurate email parameters.
MailOptions MailOptions `yaml:"options,omitempty"`
}
// MailOptions provides the configuration sections to user, for specific handler.
type MailOptions struct {
SMTP struct {
// Addr defines smtp host address
Addr string `yaml:"addr,omitempty"`
// Username defines user name to smtp host
Username string `yaml:"username,omitempty"`
// Password defines password of login user
Password string `yaml:"password,omitempty"`
// Insecure defines if smtp login skips the secure cerification.
Insecure bool `yaml:"insecure,omitempty"`
} `yaml:"smtp,omitempty"`
// From defines mail sending address
From string `yaml:"from,omitempty"`
// To defines mail receiving address
To []string `yaml:"to,omitempty"`
}
// FileChecker is a type of entry in the health section for checking files.
type FileChecker struct {
// Interval is the duration in between checks
Interval time.Duration `yaml:"interval,omitempty"`
// File is the path to check
File string `yaml:"file,omitempty"`
// Threshold is the number of times a check must fail to trigger an
// unhealthy state
Threshold int `yaml:"threshold,omitempty"`
}
// HTTPChecker is a type of entry in the health section for checking HTTP URIs.
type HTTPChecker struct {
// Timeout is the duration to wait before timing out the HTTP request
Timeout time.Duration `yaml:"interval,omitempty"`
// StatusCode is the expected status code
StatusCode int
// Interval is the duration in between checks
Interval time.Duration `yaml:"interval,omitempty"`
// URI is the HTTP URI to check
URI string `yaml:"uri,omitempty"`
// Headers lists static headers that should be added to all requests
Headers http.Header `yaml:"headers"`
// Threshold is the number of times a check must fail to trigger an
// unhealthy state
Threshold int `yaml:"threshold,omitempty"`
}
// TCPChecker is a type of entry in the health section for checking TCP servers.
type TCPChecker struct {
// Timeout is the duration to wait before timing out the TCP connection
Timeout time.Duration `yaml:"interval,omitempty"`
// Interval is the duration in between checks
Interval time.Duration `yaml:"interval,omitempty"`
// Addr is the TCP address to check
Addr string `yaml:"addr,omitempty"`
// Threshold is the number of times a check must fail to trigger an
// unhealthy state
Threshold int `yaml:"threshold,omitempty"`
}
// Health provides the configuration section for health checks.
type Health struct {
// FileCheckers is a list of paths to check
FileCheckers []FileChecker `yaml:"file,omitempty"`
// HTTPCheckers is a list of URIs to check
HTTPCheckers []HTTPChecker `yaml:"http,omitempty"`
// TCPCheckers is a list of URIs to check
TCPCheckers []TCPChecker `yaml:"tcp,omitempty"`
// StorageDriver configures a health check on the configured storage
// driver
StorageDriver struct {
// Enabled turns on the health check for the storage driver
Enabled bool `yaml:"enabled,omitempty"`
// Interval is the duration in between checks
Interval time.Duration `yaml:"interval,omitempty"`
// Threshold is the number of times a check must fail to trigger an
// unhealthy state
Threshold int `yaml:"threshold,omitempty"`
} `yaml:"storagedriver,omitempty"`
}
// v0_1Configuration is a Version 0.1 Configuration struct
// This is currently aliased to Configuration, as it is the current version
type v0_1Configuration Configuration
// UnmarshalYAML implements the yaml.Unmarshaler interface
// Unmarshals a string of the form X.Y into a Version, validating that X and Y can represent uints
func (version *Version) UnmarshalYAML(unmarshal func(interface{}) error) error {
var versionString string
err := unmarshal(&versionString)
if err != nil {
return err
}
newVersion := Version(versionString)
if _, err := newVersion.major(); err != nil {
return err
}
if _, err := newVersion.minor(); err != nil {
return err
}
*version = newVersion
return nil
}
// CurrentVersion is the most recent Version that can be parsed
var CurrentVersion = MajorMinorVersion(0, 1)
// Loglevel is the level at which operations are logged
// This can be error, warn, info, or debug
type Loglevel string
// UnmarshalYAML implements the yaml.Umarshaler interface
// Unmarshals a string into a Loglevel, lowercasing the string and validating that it represents a
// valid loglevel
func (loglevel *Loglevel) UnmarshalYAML(unmarshal func(interface{}) error) error {
var loglevelString string
err := unmarshal(&loglevelString)
if err != nil {
return err
}
loglevelString = strings.ToLower(loglevelString)
switch loglevelString {
case "error", "warn", "info", "debug":
default:
return fmt.Errorf("Invalid loglevel %s Must be one of [error, warn, info, debug]", loglevelString)
}
*loglevel = Loglevel(loglevelString)
return nil
}
// Parameters defines a key-value parameters mapping
type Parameters map[string]interface{}
// Storage defines the configuration for registry object storage
type Storage map[string]Parameters
// Type returns the storage driver type, such as filesystem or s3
func (storage Storage) Type() string {
var storageType []string
// Return only key in this map
for k := range storage {
switch k {
case "maintenance":
// allow configuration of maintenance
case "cache":
// allow configuration of caching
case "delete":
// allow configuration of delete
case "redirect":
// allow configuration of redirect
default:
storageType = append(storageType, k)
}
}
if len(storageType) > 1 {
panic("multiple storage drivers specified in configuration or environment: " + strings.Join(storageType, ", "))
}
if len(storageType) == 1 {
return storageType[0]
}
return ""
}
// Parameters returns the Parameters map for a Storage configuration
func (storage Storage) Parameters() Parameters {
return storage[storage.Type()]
}
// setParameter changes the parameter at the provided key to the new value
func (storage Storage) setParameter(key string, value interface{}) {
storage[storage.Type()][key] = value
}
// UnmarshalYAML implements the yaml.Unmarshaler interface
// Unmarshals a single item map into a Storage or a string into a Storage type with no parameters
func (storage *Storage) UnmarshalYAML(unmarshal func(interface{}) error) error {
var storageMap map[string]Parameters
err := unmarshal(&storageMap)
if err == nil {
if len(storageMap) > 1 {
types := make([]string, 0, len(storageMap))
for k := range storageMap {
switch k {
case "maintenance":
// allow for configuration of maintenance
case "cache":
// allow configuration of caching
case "delete":
// allow configuration of delete
case "redirect":
// allow configuration of redirect
default:
types = append(types, k)
}
}
if len(types) > 1 {
return fmt.Errorf("Must provide exactly one storage type. Provided: %v", types)
}
}
*storage = storageMap
return nil
}
var storageType string
err = unmarshal(&storageType)
if err == nil {
*storage = Storage{storageType: Parameters{}}
return nil
}
return err
}
// MarshalYAML implements the yaml.Marshaler interface
func (storage Storage) MarshalYAML() (interface{}, error) {
if storage.Parameters() == nil {
return storage.Type(), nil
}
return map[string]Parameters(storage), nil
}
// Auth defines the configuration for registry authorization.
type Auth map[string]Parameters
// Type returns the storage driver type, such as filesystem or s3
func (auth Auth) Type() string {
// Return only key in this map
for k := range auth {
return k
}
return ""
}
// Parameters returns the Parameters map for an Auth configuration
func (auth Auth) Parameters() Parameters {
return auth[auth.Type()]
}
// setParameter changes the parameter at the provided key to the new value
func (auth Auth) setParameter(key string, value interface{}) {
auth[auth.Type()][key] = value
}
// UnmarshalYAML implements the yaml.Unmarshaler interface
// Unmarshals a single item map into a Storage or a string into a Storage type with no parameters
func (auth *Auth) UnmarshalYAML(unmarshal func(interface{}) error) error {
var m map[string]Parameters
err := unmarshal(&m)
if err == nil {
if len(m) > 1 {
types := make([]string, 0, len(m))
for k := range m {
types = append(types, k)
}
// TODO(stevvooe): May want to change this slightly for
// authorization to allow multiple challenges.
return fmt.Errorf("must provide exactly one type. Provided: %v", types)
}
*auth = m
return nil
}
var authType string
err = unmarshal(&authType)
if err == nil {
*auth = Auth{authType: Parameters{}}
return nil
}
return err
}
// MarshalYAML implements the yaml.Marshaler interface
func (auth Auth) MarshalYAML() (interface{}, error) {
if auth.Parameters() == nil {
return auth.Type(), nil
}
return map[string]Parameters(auth), nil
}
// Notifications configures multiple http endpoints.
type Notifications struct {
// Endpoints is a list of http configurations for endpoints that
// respond to webhook notifications. In the future, we may allow other
// kinds of endpoints, such as external queues.
Endpoints []Endpoint `yaml:"endpoints,omitempty"`
}
// Endpoint describes the configuration of an http webhook notification
// endpoint.
type Endpoint struct {
Name string `yaml:"name"` // identifies the endpoint in the registry instance.
Disabled bool `yaml:"disabled"` // disables the endpoint
URL string `yaml:"url"` // post url for the endpoint.
Headers http.Header `yaml:"headers"` // static headers that should be added to all requests
Timeout time.Duration `yaml:"timeout"` // HTTP timeout
Threshold int `yaml:"threshold"` // circuit breaker threshold before backing off on failure
Backoff time.Duration `yaml:"backoff"` // backoff duration
}
// Reporting defines error reporting methods.
type Reporting struct {
// Bugsnag configures error reporting for Bugsnag (bugsnag.com).
Bugsnag BugsnagReporting `yaml:"bugsnag,omitempty"`
// NewRelic configures error reporting for NewRelic (newrelic.com)
NewRelic NewRelicReporting `yaml:"newrelic,omitempty"`
}
// BugsnagReporting configures error reporting for Bugsnag (bugsnag.com).
type BugsnagReporting struct {
// APIKey is the Bugsnag api key.
APIKey string `yaml:"apikey,omitempty"`
// ReleaseStage tracks where the registry is deployed.
// Examples: production, staging, development
ReleaseStage string `yaml:"releasestage,omitempty"`
// Endpoint is used for specifying an enterprise Bugsnag endpoint.
Endpoint string `yaml:"endpoint,omitempty"`
}
// NewRelicReporting configures error reporting for NewRelic (newrelic.com)
type NewRelicReporting struct {
// LicenseKey is the NewRelic user license key
LicenseKey string `yaml:"licensekey,omitempty"`
// Name is the component name of the registry in NewRelic
Name string `yaml:"name,omitempty"`
// Verbose configures debug output to STDOUT
Verbose bool `yaml:"verbose,omitempty"`
}
// Middleware configures named middlewares to be applied at injection points.
type Middleware struct {
// Name the middleware registers itself as
Name string `yaml:"name"`
// Flag to disable middleware easily
Disabled bool `yaml:"disabled,omitempty"`
// Map of parameters that will be passed to the middleware's initialization function
Options Parameters `yaml:"options"`
}
// Proxy configures the registry as a pull through cache
type Proxy struct {
// RemoteURL is the URL of the remote registry
RemoteURL string `yaml:"remoteurl"`
// Username of the hub user
Username string `yaml:"username"`
// Password of the hub user
Password string `yaml:"password"`
}
// Parse parses an input configuration yaml document into a Configuration struct
// This should generally be capable of handling old configuration format versions
//
// Environment variables may be used to override configuration parameters other than version,
// following the scheme below:
// Configuration.Abc may be replaced by the value of REGISTRY_ABC,
// Configuration.Abc.Xyz may be replaced by the value of REGISTRY_ABC_XYZ, and so forth
func Parse(rd io.Reader) (*Configuration, error) {
in, err := ioutil.ReadAll(rd)
if err != nil {
return nil, err
}
p := NewParser("registry", []VersionedParseInfo{
{
Version: MajorMinorVersion(0, 1),
ParseAs: reflect.TypeOf(v0_1Configuration{}),
ConversionFunc: func(c interface{}) (interface{}, error) {
if v0_1, ok := c.(*v0_1Configuration); ok {
if v0_1.Loglevel == Loglevel("") {
v0_1.Loglevel = Loglevel("info")
}
if v0_1.Storage.Type() == "" {
return nil, fmt.Errorf("No storage configuration provided")
}
return (*Configuration)(v0_1), nil
}
return nil, fmt.Errorf("Expected *v0_1Configuration, received %#v", c)
},
},
})
config := new(Configuration)
err = p.Parse(in, config)
if err != nil {
return nil, err
}
return config, nil
}

View File

@@ -1,283 +0,0 @@
package configuration
import (
"fmt"
"os"
"reflect"
"sort"
"strconv"
"strings"
"github.com/Sirupsen/logrus"
"gopkg.in/yaml.v2"
)
// Version is a major/minor version pair of the form Major.Minor
// Major version upgrades indicate structure or type changes
// Minor version upgrades should be strictly additive
type Version string
// MajorMinorVersion constructs a Version from its Major and Minor components
func MajorMinorVersion(major, minor uint) Version {
return Version(fmt.Sprintf("%d.%d", major, minor))
}
func (version Version) major() (uint, error) {
majorPart := strings.Split(string(version), ".")[0]
major, err := strconv.ParseUint(majorPart, 10, 0)
return uint(major), err
}
// Major returns the major version portion of a Version
func (version Version) Major() uint {
major, _ := version.major()
return major
}
func (version Version) minor() (uint, error) {
minorPart := strings.Split(string(version), ".")[1]
minor, err := strconv.ParseUint(minorPart, 10, 0)
return uint(minor), err
}
// Minor returns the minor version portion of a Version
func (version Version) Minor() uint {
minor, _ := version.minor()
return minor
}
// VersionedParseInfo defines how a specific version of a configuration should
// be parsed into the current version
type VersionedParseInfo struct {
// Version is the version which this parsing information relates to
Version Version
// ParseAs defines the type which a configuration file of this version
// should be parsed into
ParseAs reflect.Type
// ConversionFunc defines a method for converting the parsed configuration
// (of type ParseAs) into the current configuration version
// Note: this method signature is very unclear with the absence of generics
ConversionFunc func(interface{}) (interface{}, error)
}
type envVar struct {
name string
value string
}
type envVars []envVar
func (a envVars) Len() int { return len(a) }
func (a envVars) Swap(i, j int) { a[i], a[j] = a[j], a[i] }
func (a envVars) Less(i, j int) bool { return a[i].name < a[j].name }
// Parser can be used to parse a configuration file and environment of a defined
// version into a unified output structure
type Parser struct {
prefix string
mapping map[Version]VersionedParseInfo
env envVars
}
// NewParser returns a *Parser with the given environment prefix which handles
// versioned configurations which match the given parseInfos
func NewParser(prefix string, parseInfos []VersionedParseInfo) *Parser {
p := Parser{prefix: prefix, mapping: make(map[Version]VersionedParseInfo)}
for _, parseInfo := range parseInfos {
p.mapping[parseInfo.Version] = parseInfo
}
for _, env := range os.Environ() {
envParts := strings.SplitN(env, "=", 2)
p.env = append(p.env, envVar{envParts[0], envParts[1]})
}
// We must sort the environment variables lexically by name so that
// more specific variables are applied before less specific ones
// (i.e. REGISTRY_STORAGE before
// REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY). This sucks, but it's a
// lot simpler and easier to get right than unmarshalling map entries
// into temporaries and merging with the existing entry.
sort.Sort(p.env)
return &p
}
// Parse reads in the given []byte and environment and writes the resulting
// configuration into the input v
//
// Environment variables may be used to override configuration parameters other
// than version, following the scheme below:
// v.Abc may be replaced by the value of PREFIX_ABC,
// v.Abc.Xyz may be replaced by the value of PREFIX_ABC_XYZ, and so forth
func (p *Parser) Parse(in []byte, v interface{}) error {
var versionedStruct struct {
Version Version
}
if err := yaml.Unmarshal(in, &versionedStruct); err != nil {
return err
}
parseInfo, ok := p.mapping[versionedStruct.Version]
if !ok {
return fmt.Errorf("Unsupported version: %q", versionedStruct.Version)
}
parseAs := reflect.New(parseInfo.ParseAs)
err := yaml.Unmarshal(in, parseAs.Interface())
if err != nil {
return err
}
for _, envVar := range p.env {
pathStr := envVar.name
if strings.HasPrefix(pathStr, strings.ToUpper(p.prefix)+"_") {
path := strings.Split(pathStr, "_")
err = p.overwriteFields(parseAs, pathStr, path[1:], envVar.value)
if err != nil {
return err
}
}
}
c, err := parseInfo.ConversionFunc(parseAs.Interface())
if err != nil {
return err
}
reflect.ValueOf(v).Elem().Set(reflect.Indirect(reflect.ValueOf(c)))
return nil
}
// overwriteFields replaces configuration values with alternate values specified
// through the environment. Precondition: an empty path slice must never be
// passed in.
func (p *Parser) overwriteFields(v reflect.Value, fullpath string, path []string, payload string) error {
for v.Kind() == reflect.Ptr {
if v.IsNil() {
panic("encountered nil pointer while handling environment variable " + fullpath)
}
v = reflect.Indirect(v)
}
switch v.Kind() {
case reflect.Struct:
return p.overwriteStruct(v, fullpath, path, payload)
case reflect.Map:
return p.overwriteMap(v, fullpath, path, payload)
case reflect.Interface:
if v.NumMethod() == 0 {
if !v.IsNil() {
return p.overwriteFields(v.Elem(), fullpath, path, payload)
}
// Interface was empty; create an implicit map
var template map[string]interface{}
wrappedV := reflect.MakeMap(reflect.TypeOf(template))
v.Set(wrappedV)
return p.overwriteMap(wrappedV, fullpath, path, payload)
}
}
return nil
}
func (p *Parser) overwriteStruct(v reflect.Value, fullpath string, path []string, payload string) error {
// Generate case-insensitive map of struct fields
byUpperCase := make(map[string]int)
for i := 0; i < v.NumField(); i++ {
sf := v.Type().Field(i)
upper := strings.ToUpper(sf.Name)
if _, present := byUpperCase[upper]; present {
panic(fmt.Sprintf("field name collision in configuration object: %s", sf.Name))
}
byUpperCase[upper] = i
}
fieldIndex, present := byUpperCase[path[0]]
if !present {
logrus.Warnf("Ignoring unrecognized environment variable %s", fullpath)
return nil
}
field := v.Field(fieldIndex)
sf := v.Type().Field(fieldIndex)
if len(path) == 1 {
// Env var specifies this field directly
fieldVal := reflect.New(sf.Type)
err := yaml.Unmarshal([]byte(payload), fieldVal.Interface())
if err != nil {
return err
}
field.Set(reflect.Indirect(fieldVal))
return nil
}
// If the field is nil, must create an object
switch sf.Type.Kind() {
case reflect.Map:
if field.IsNil() {
field.Set(reflect.MakeMap(sf.Type))
}
case reflect.Ptr:
if field.IsNil() {
field.Set(reflect.New(sf.Type))
}
}
err := p.overwriteFields(field, fullpath, path[1:], payload)
if err != nil {
return err
}
return nil
}
func (p *Parser) overwriteMap(m reflect.Value, fullpath string, path []string, payload string) error {
if m.Type().Key().Kind() != reflect.String {
// non-string keys unsupported
logrus.Warnf("Ignoring environment variable %s involving map with non-string keys", fullpath)
return nil
}
if len(path) > 1 {
// If a matching key exists, get its value and continue the
// overwriting process.
for _, k := range m.MapKeys() {
if strings.ToUpper(k.String()) == path[0] {
mapValue := m.MapIndex(k)
// If the existing value is nil, we want to
// recreate it instead of using this value.
if (mapValue.Kind() == reflect.Ptr ||
mapValue.Kind() == reflect.Interface ||
mapValue.Kind() == reflect.Map) &&
mapValue.IsNil() {
break
}
return p.overwriteFields(mapValue, fullpath, path[1:], payload)
}
}
}
// (Re)create this key
var mapValue reflect.Value
if m.Type().Elem().Kind() == reflect.Map {
mapValue = reflect.MakeMap(m.Type().Elem())
} else {
mapValue = reflect.New(m.Type().Elem())
}
if len(path) > 1 {
err := p.overwriteFields(mapValue, fullpath, path[1:], payload)
if err != nil {
return err
}
} else {
err := yaml.Unmarshal([]byte(payload), mapValue.Interface())
if err != nil {
return err
}
}
m.SetMapIndex(reflect.ValueOf(strings.ToLower(path[0])), reflect.Indirect(mapValue))
return nil
}

View File

@@ -1,85 +0,0 @@
package context
import (
"sync"
"github.com/docker/distribution/uuid"
"golang.org/x/net/context"
)
// Context is a copy of Context from the golang.org/x/net/context package.
type Context interface {
context.Context
}
// instanceContext is a context that provides only an instance id. It is
// provided as the main background context.
type instanceContext struct {
Context
id string // id of context, logged as "instance.id"
once sync.Once // once protect generation of the id
}
func (ic *instanceContext) Value(key interface{}) interface{} {
if key == "instance.id" {
ic.once.Do(func() {
// We want to lazy initialize the UUID such that we don't
// call a random generator from the package initialization
// code. For various reasons random could not be available
// https://github.com/docker/distribution/issues/782
ic.id = uuid.Generate().String()
})
return ic.id
}
return ic.Context.Value(key)
}
var background = &instanceContext{
Context: context.Background(),
}
// Background returns a non-nil, empty Context. The background context
// provides a single key, "instance.id" that is globally unique to the
// process.
func Background() Context {
return background
}
// WithValue returns a copy of parent in which the value associated with key is
// val. Use context Values only for request-scoped data that transits processes
// and APIs, not for passing optional parameters to functions.
func WithValue(parent Context, key, val interface{}) Context {
return context.WithValue(parent, key, val)
}
// stringMapContext is a simple context implementation that checks a map for a
// key, falling back to a parent if not present.
type stringMapContext struct {
context.Context
m map[string]interface{}
}
// WithValues returns a context that proxies lookups through a map. Only
// supports string keys.
func WithValues(ctx context.Context, m map[string]interface{}) context.Context {
mo := make(map[string]interface{}, len(m)) // make our own copy.
for k, v := range m {
mo[k] = v
}
return stringMapContext{
Context: ctx,
m: mo,
}
}
func (smc stringMapContext) Value(key interface{}) interface{} {
if ks, ok := key.(string); ok {
if v, ok := smc.m[ks]; ok {
return v
}
}
return smc.Context.Value(key)
}

View File

@@ -1,89 +0,0 @@
// Package context provides several utilities for working with
// golang.org/x/net/context in http requests. Primarily, the focus is on
// logging relevent request information but this package is not limited to
// that purpose.
//
// The easiest way to get started is to get the background context:
//
// ctx := context.Background()
//
// The returned context should be passed around your application and be the
// root of all other context instances. If the application has a version, this
// line should be called before anything else:
//
// ctx := context.WithVersion(context.Background(), version)
//
// The above will store the version in the context and will be available to
// the logger.
//
// Logging
//
// The most useful aspect of this package is GetLogger. This function takes
// any context.Context interface and returns the current logger from the
// context. Canonical usage looks like this:
//
// GetLogger(ctx).Infof("something interesting happened")
//
// GetLogger also takes optional key arguments. The keys will be looked up in
// the context and reported with the logger. The following example would
// return a logger that prints the version with each log message:
//
// ctx := context.Context(context.Background(), "version", version)
// GetLogger(ctx, "version").Infof("this log message has a version field")
//
// The above would print out a log message like this:
//
// INFO[0000] this log message has a version field version=v2.0.0-alpha.2.m
//
// When used with WithLogger, we gain the ability to decorate the context with
// loggers that have information from disparate parts of the call stack.
// Following from the version example, we can build a new context with the
// configured logger such that we always print the version field:
//
// ctx = WithLogger(ctx, GetLogger(ctx, "version"))
//
// Since the logger has been pushed to the context, we can now get the version
// field for free with our log messages. Future calls to GetLogger on the new
// context will have the version field:
//
// GetLogger(ctx).Infof("this log message has a version field")
//
// This becomes more powerful when we start stacking loggers. Let's say we
// have the version logger from above but also want a request id. Using the
// context above, in our request scoped function, we place another logger in
// the context:
//
// ctx = context.WithValue(ctx, "http.request.id", "unique id") // called when building request context
// ctx = WithLogger(ctx, GetLogger(ctx, "http.request.id"))
//
// When GetLogger is called on the new context, "http.request.id" will be
// included as a logger field, along with the original "version" field:
//
// INFO[0000] this log message has a version field http.request.id=unique id version=v2.0.0-alpha.2.m
//
// Note that this only affects the new context, the previous context, with the
// version field, can be used independently. Put another way, the new logger,
// added to the request context, is unique to that context and can have
// request scoped varaibles.
//
// HTTP Requests
//
// This package also contains several methods for working with http requests.
// The concepts are very similar to those described above. We simply place the
// request in the context using WithRequest. This makes the request variables
// available. GetRequestLogger can then be called to get request specific
// variables in a log line:
//
// ctx = WithRequest(ctx, req)
// GetRequestLogger(ctx).Infof("request variables")
//
// Like above, if we want to include the request data in all log messages in
// the context, we push the logger to a new context and use that one:
//
// ctx = WithLogger(ctx, GetRequestLogger(ctx))
//
// The concept is fairly powerful and ensures that calls throughout the stack
// can be traced in log messages. Using the fields like "http.request.id", one
// can analyze call flow for a particular request with a simple grep of the
// logs.
package context

View File

@@ -1,364 +0,0 @@
package context
import (
"errors"
"net"
"net/http"
"strings"
"sync"
"time"
log "github.com/Sirupsen/logrus"
"github.com/docker/distribution/uuid"
"github.com/gorilla/mux"
)
// Common errors used with this package.
var (
ErrNoRequestContext = errors.New("no http request in context")
ErrNoResponseWriterContext = errors.New("no http response in context")
)
func parseIP(ipStr string) net.IP {
ip := net.ParseIP(ipStr)
if ip == nil {
log.Warnf("invalid remote IP address: %q", ipStr)
}
return ip
}
// RemoteAddr extracts the remote address of the request, taking into
// account proxy headers.
func RemoteAddr(r *http.Request) string {
if prior := r.Header.Get("X-Forwarded-For"); prior != "" {
proxies := strings.Split(prior, ",")
if len(proxies) > 0 {
remoteAddr := strings.Trim(proxies[0], " ")
if parseIP(remoteAddr) != nil {
return remoteAddr
}
}
}
// X-Real-Ip is less supported, but worth checking in the
// absence of X-Forwarded-For
if realIP := r.Header.Get("X-Real-Ip"); realIP != "" {
if parseIP(realIP) != nil {
return realIP
}
}
return r.RemoteAddr
}
// RemoteIP extracts the remote IP of the request, taking into
// account proxy headers.
func RemoteIP(r *http.Request) string {
addr := RemoteAddr(r)
// Try parsing it as "IP:port"
if ip, _, err := net.SplitHostPort(addr); err == nil {
return ip
}
return addr
}
// WithRequest places the request on the context. The context of the request
// is assigned a unique id, available at "http.request.id". The request itself
// is available at "http.request". Other common attributes are available under
// the prefix "http.request.". If a request is already present on the context,
// this method will panic.
func WithRequest(ctx Context, r *http.Request) Context {
if ctx.Value("http.request") != nil {
// NOTE(stevvooe): This needs to be considered a programming error. It
// is unlikely that we'd want to have more than one request in
// context.
panic("only one request per context")
}
return &httpRequestContext{
Context: ctx,
startedAt: time.Now(),
id: uuid.Generate().String(),
r: r,
}
}
// GetRequest returns the http request in the given context. Returns
// ErrNoRequestContext if the context does not have an http request associated
// with it.
func GetRequest(ctx Context) (*http.Request, error) {
if r, ok := ctx.Value("http.request").(*http.Request); r != nil && ok {
return r, nil
}
return nil, ErrNoRequestContext
}
// GetRequestID attempts to resolve the current request id, if possible. An
// error is return if it is not available on the context.
func GetRequestID(ctx Context) string {
return GetStringValue(ctx, "http.request.id")
}
// WithResponseWriter returns a new context and response writer that makes
// interesting response statistics available within the context.
func WithResponseWriter(ctx Context, w http.ResponseWriter) (Context, http.ResponseWriter) {
irw := instrumentedResponseWriter{
ResponseWriter: w,
Context: ctx,
}
if closeNotifier, ok := w.(http.CloseNotifier); ok {
irwCN := &instrumentedResponseWriterCN{
instrumentedResponseWriter: irw,
CloseNotifier: closeNotifier,
}
return irwCN, irwCN
}
return &irw, &irw
}
// GetResponseWriter returns the http.ResponseWriter from the provided
// context. If not present, ErrNoResponseWriterContext is returned. The
// returned instance provides instrumentation in the context.
func GetResponseWriter(ctx Context) (http.ResponseWriter, error) {
v := ctx.Value("http.response")
rw, ok := v.(http.ResponseWriter)
if !ok || rw == nil {
return nil, ErrNoResponseWriterContext
}
return rw, nil
}
// getVarsFromRequest let's us change request vars implementation for testing
// and maybe future changes.
var getVarsFromRequest = mux.Vars
// WithVars extracts gorilla/mux vars and makes them available on the returned
// context. Variables are available at keys with the prefix "vars.". For
// example, if looking for the variable "name", it can be accessed as
// "vars.name". Implementations that are accessing values need not know that
// the underlying context is implemented with gorilla/mux vars.
func WithVars(ctx Context, r *http.Request) Context {
return &muxVarsContext{
Context: ctx,
vars: getVarsFromRequest(r),
}
}
// GetRequestLogger returns a logger that contains fields from the request in
// the current context. If the request is not available in the context, no
// fields will display. Request loggers can safely be pushed onto the context.
func GetRequestLogger(ctx Context) Logger {
return GetLogger(ctx,
"http.request.id",
"http.request.method",
"http.request.host",
"http.request.uri",
"http.request.referer",
"http.request.useragent",
"http.request.remoteaddr",
"http.request.contenttype")
}
// GetResponseLogger reads the current response stats and builds a logger.
// Because the values are read at call time, pushing a logger returned from
// this function on the context will lead to missing or invalid data. Only
// call this at the end of a request, after the response has been written.
func GetResponseLogger(ctx Context) Logger {
l := getLogrusLogger(ctx,
"http.response.written",
"http.response.status",
"http.response.contenttype")
duration := Since(ctx, "http.request.startedat")
if duration > 0 {
l = l.WithField("http.response.duration", duration.String())
}
return l
}
// httpRequestContext makes information about a request available to context.
type httpRequestContext struct {
Context
startedAt time.Time
id string
r *http.Request
}
// Value returns a keyed element of the request for use in the context. To get
// the request itself, query "request". For other components, access them as
// "request.<component>". For example, r.RequestURI
func (ctx *httpRequestContext) Value(key interface{}) interface{} {
if keyStr, ok := key.(string); ok {
if keyStr == "http.request" {
return ctx.r
}
if !strings.HasPrefix(keyStr, "http.request.") {
goto fallback
}
parts := strings.Split(keyStr, ".")
if len(parts) != 3 {
goto fallback
}
switch parts[2] {
case "uri":
return ctx.r.RequestURI
case "remoteaddr":
return RemoteAddr(ctx.r)
case "method":
return ctx.r.Method
case "host":
return ctx.r.Host
case "referer":
referer := ctx.r.Referer()
if referer != "" {
return referer
}
case "useragent":
return ctx.r.UserAgent()
case "id":
return ctx.id
case "startedat":
return ctx.startedAt
case "contenttype":
ct := ctx.r.Header.Get("Content-Type")
if ct != "" {
return ct
}
}
}
fallback:
return ctx.Context.Value(key)
}
type muxVarsContext struct {
Context
vars map[string]string
}
func (ctx *muxVarsContext) Value(key interface{}) interface{} {
if keyStr, ok := key.(string); ok {
if keyStr == "vars" {
return ctx.vars
}
if strings.HasPrefix(keyStr, "vars.") {
keyStr = strings.TrimPrefix(keyStr, "vars.")
}
if v, ok := ctx.vars[keyStr]; ok {
return v
}
}
return ctx.Context.Value(key)
}
// instrumentedResponseWriterCN provides response writer information in a
// context. It implements http.CloseNotifier so that users can detect
// early disconnects.
type instrumentedResponseWriterCN struct {
instrumentedResponseWriter
http.CloseNotifier
}
// instrumentedResponseWriter provides response writer information in a
// context. This variant is only used in the case where CloseNotifier is not
// implemented by the parent ResponseWriter.
type instrumentedResponseWriter struct {
http.ResponseWriter
Context
mu sync.Mutex
status int
written int64
}
func (irw *instrumentedResponseWriter) Write(p []byte) (n int, err error) {
n, err = irw.ResponseWriter.Write(p)
irw.mu.Lock()
irw.written += int64(n)
// Guess the likely status if not set.
if irw.status == 0 {
irw.status = http.StatusOK
}
irw.mu.Unlock()
return
}
func (irw *instrumentedResponseWriter) WriteHeader(status int) {
irw.ResponseWriter.WriteHeader(status)
irw.mu.Lock()
irw.status = status
irw.mu.Unlock()
}
func (irw *instrumentedResponseWriter) Flush() {
if flusher, ok := irw.ResponseWriter.(http.Flusher); ok {
flusher.Flush()
}
}
func (irw *instrumentedResponseWriter) Value(key interface{}) interface{} {
if keyStr, ok := key.(string); ok {
if keyStr == "http.response" {
return irw
}
if !strings.HasPrefix(keyStr, "http.response.") {
goto fallback
}
parts := strings.Split(keyStr, ".")
if len(parts) != 3 {
goto fallback
}
irw.mu.Lock()
defer irw.mu.Unlock()
switch parts[2] {
case "written":
return irw.written
case "status":
return irw.status
case "contenttype":
contentType := irw.Header().Get("Content-Type")
if contentType != "" {
return contentType
}
}
}
fallback:
return irw.Context.Value(key)
}
func (irw *instrumentedResponseWriterCN) Value(key interface{}) interface{} {
if keyStr, ok := key.(string); ok {
if keyStr == "http.response" {
return irw
}
}
return irw.instrumentedResponseWriter.Value(key)
}

View File

@@ -1,116 +0,0 @@
package context
import (
"fmt"
"github.com/Sirupsen/logrus"
"runtime"
)
// Logger provides a leveled-logging interface.
type Logger interface {
// standard logger methods
Print(args ...interface{})
Printf(format string, args ...interface{})
Println(args ...interface{})
Fatal(args ...interface{})
Fatalf(format string, args ...interface{})
Fatalln(args ...interface{})
Panic(args ...interface{})
Panicf(format string, args ...interface{})
Panicln(args ...interface{})
// Leveled methods, from logrus
Debug(args ...interface{})
Debugf(format string, args ...interface{})
Debugln(args ...interface{})
Error(args ...interface{})
Errorf(format string, args ...interface{})
Errorln(args ...interface{})
Info(args ...interface{})
Infof(format string, args ...interface{})
Infoln(args ...interface{})
Warn(args ...interface{})
Warnf(format string, args ...interface{})
Warnln(args ...interface{})
}
// WithLogger creates a new context with provided logger.
func WithLogger(ctx Context, logger Logger) Context {
return WithValue(ctx, "logger", logger)
}
// GetLoggerWithField returns a logger instance with the specified field key
// and value without affecting the context. Extra specified keys will be
// resolved from the context.
func GetLoggerWithField(ctx Context, key, value interface{}, keys ...interface{}) Logger {
return getLogrusLogger(ctx, keys...).WithField(fmt.Sprint(key), value)
}
// GetLoggerWithFields returns a logger instance with the specified fields
// without affecting the context. Extra specified keys will be resolved from
// the context.
func GetLoggerWithFields(ctx Context, fields map[interface{}]interface{}, keys ...interface{}) Logger {
// must convert from interface{} -> interface{} to string -> interface{} for logrus.
lfields := make(logrus.Fields, len(fields))
for key, value := range fields {
lfields[fmt.Sprint(key)] = value
}
return getLogrusLogger(ctx, keys...).WithFields(lfields)
}
// GetLogger returns the logger from the current context, if present. If one
// or more keys are provided, they will be resolved on the context and
// included in the logger. While context.Value takes an interface, any key
// argument passed to GetLogger will be passed to fmt.Sprint when expanded as
// a logging key field. If context keys are integer constants, for example,
// its recommended that a String method is implemented.
func GetLogger(ctx Context, keys ...interface{}) Logger {
return getLogrusLogger(ctx, keys...)
}
// GetLogrusLogger returns the logrus logger for the context. If one more keys
// are provided, they will be resolved on the context and included in the
// logger. Only use this function if specific logrus functionality is
// required.
func getLogrusLogger(ctx Context, keys ...interface{}) *logrus.Entry {
var logger *logrus.Entry
// Get a logger, if it is present.
loggerInterface := ctx.Value("logger")
if loggerInterface != nil {
if lgr, ok := loggerInterface.(*logrus.Entry); ok {
logger = lgr
}
}
if logger == nil {
fields := logrus.Fields{}
// Fill in the instance id, if we have it.
instanceID := ctx.Value("instance.id")
if instanceID != nil {
fields["instance.id"] = instanceID
}
fields["go.version"] = runtime.Version()
// If no logger is found, just return the standard logger.
logger = logrus.StandardLogger().WithFields(fields)
}
fields := logrus.Fields{}
for _, key := range keys {
v := ctx.Value(key)
if v != nil {
fields[fmt.Sprint(key)] = v
}
}
return logger.WithFields(fields)
}

View File

@@ -1,104 +0,0 @@
package context
import (
"runtime"
"time"
"github.com/docker/distribution/uuid"
)
// WithTrace allocates a traced timing span in a new context. This allows a
// caller to track the time between calling WithTrace and the returned done
// function. When the done function is called, a log message is emitted with a
// "trace.duration" field, corresponding to the elapased time and a
// "trace.func" field, corresponding to the function that called WithTrace.
//
// The logging keys "trace.id" and "trace.parent.id" are provided to implement
// dapper-like tracing. This function should be complemented with a WithSpan
// method that could be used for tracing distributed RPC calls.
//
// The main benefit of this function is to post-process log messages or
// intercept them in a hook to provide timing data. Trace ids and parent ids
// can also be linked to provide call tracing, if so required.
//
// Here is an example of the usage:
//
// func timedOperation(ctx Context) {
// ctx, done := WithTrace(ctx)
// defer done("this will be the log message")
// // ... function body ...
// }
//
// If the function ran for roughly 1s, such a usage would emit a log message
// as follows:
//
// INFO[0001] this will be the log message trace.duration=1.004575763s trace.func=github.com/docker/distribution/context.traceOperation trace.id=<id> ...
//
// Notice that the function name is automatically resolved, along with the
// package and a trace id is emitted that can be linked with parent ids.
func WithTrace(ctx Context) (Context, func(format string, a ...interface{})) {
if ctx == nil {
ctx = Background()
}
pc, file, line, _ := runtime.Caller(1)
f := runtime.FuncForPC(pc)
ctx = &traced{
Context: ctx,
id: uuid.Generate().String(),
start: time.Now(),
parent: GetStringValue(ctx, "trace.id"),
fnname: f.Name(),
file: file,
line: line,
}
return ctx, func(format string, a ...interface{}) {
GetLogger(ctx,
"trace.duration",
"trace.id",
"trace.parent.id",
"trace.func",
"trace.file",
"trace.line").
Debugf(format, a...)
}
}
// traced represents a context that is traced for function call timing. It
// also provides fast lookup for the various attributes that are available on
// the trace.
type traced struct {
Context
id string
parent string
start time.Time
fnname string
file string
line int
}
func (ts *traced) Value(key interface{}) interface{} {
switch key {
case "trace.start":
return ts.start
case "trace.duration":
return time.Since(ts.start)
case "trace.id":
return ts.id
case "trace.parent.id":
if ts.parent == "" {
return nil // must return nil to signal no parent.
}
return ts.parent
case "trace.func":
return ts.fnname
case "trace.file":
return ts.file
case "trace.line":
return ts.line
}
return ts.Context.Value(key)
}

View File

@@ -1,32 +0,0 @@
package context
import (
"time"
)
// Since looks up key, which should be a time.Time, and returns the duration
// since that time. If the key is not found, the value returned will be zero.
// This is helpful when inferring metrics related to context execution times.
func Since(ctx Context, key interface{}) time.Duration {
startedAtI := ctx.Value(key)
if startedAtI != nil {
if startedAt, ok := startedAtI.(time.Time); ok {
return time.Since(startedAt)
}
}
return 0
}
// GetStringValue returns a string value from the context. The empty string
// will be returned if not found.
func GetStringValue(ctx Context, key interface{}) (value string) {
stringi := ctx.Value(key)
if stringi != nil {
if valuev, ok := stringi.(string); ok {
value = valuev
}
}
return value
}

View File

@@ -1,16 +0,0 @@
package context
// WithVersion stores the application version in the context. The new context
// gets a logger to ensure log messages are marked with the application
// version.
func WithVersion(ctx Context, version string) Context {
ctx = WithValue(ctx, "version", version)
// push a new logger onto the stack
return WithLogger(ctx, GetLogger(ctx, "version"))
}
// GetVersion returns the application version from the context. An empty
// string may returned if the version was not set on the context.
func GetVersion(ctx Context) string {
return GetStringValue(ctx, "version")
}

View File

@@ -1,36 +0,0 @@
# Apache HTTPd sample for Registry v1, v2 and mirror
3 containers involved
* Docker Registry v1 (registry 0.9.1)
* Docker Registry v2 (registry 2.0.0)
* Docker Registry v1 in mirror mode
HTTP for mirror and HTTPS for v1 & v2
* http://registry.example.com proxify Docker Registry 1.0 in Mirror mode
* https://registry.example.com proxify Docker Registry 1.0 or 2.0 in Hosting mode
## 3 Docker containers should be started
* Docker Registry 1.0 in Mirror mode : port 5001
* Docker Registry 1.0 in Hosting mode : port 5000
* Docker Registry 2.0 in Hosting mode : port 5002
### Registry v1
docker run -d -e SETTINGS_FLAVOR=dev -v /var/lib/docker-registry/storage/hosting-v1:/tmp -p 5000:5000 registry:0.9.1"
### Mirror
docker run -d -e SETTINGS_FLAVOR=dev -e STANDALONE=false -e MIRROR_SOURCE=https://registry-1.docker.io -e MIRROR_SOURCE_INDEX=https://index.docker.io \
-e MIRROR_TAGS_CACHE_TTL=172800 -v /var/lib/docker-registry/storage/mirror:/tmp -p 5001:5000 registry:0.9.1"
### Registry v2
docker run -d -e SETTINGS_FLAVOR=dev -v /var/lib/axway/docker-registry/storage/hosting2-v2:/tmp -p 5002:5000 registry:2"
# For Hosting mode access
* users should have account (valid-user) to be able to fetch images
* only users using account docker-deployer will be allowed to push images

View File

@@ -1,127 +0,0 @@
#
# Sample Apache 2.x configuration where :
#
<VirtualHost *:80>
ServerName registry.example.com
ServerAlias www.registry.example.com
ProxyRequests off
ProxyPreserveHost on
# no proxy for /error/ (Apache HTTPd errors messages)
ProxyPass /error/ !
ProxyPass /_ping http://localhost:5001/_ping
ProxyPassReverse /_ping http://localhost:5001/_ping
ProxyPass /v1 http://localhost:5001/v1
ProxyPassReverse /v1 http://localhost:5001/v1
# Logs
ErrorLog ${APACHE_LOG_DIR}/mirror_error_log
CustomLog ${APACHE_LOG_DIR}/mirror_access_log combined env=!dontlog
</VirtualHost>
<VirtualHost *:443>
ServerName registry.example.com
ServerAlias www.registry.example.com
SSLEngine on
SSLCertificateFile /etc/apache2/ssl/registry.example.com.crt
SSLCertificateKeyFile /etc/apache2/ssl/registry.example.com.key
# Higher Strength SSL Ciphers
SSLProtocol all -SSLv2 -SSLv3 -TLSv1
SSLCipherSuite RC4-SHA:HIGH
SSLHonorCipherOrder on
# Logs
ErrorLog ${APACHE_LOG_DIR}/registry_error_ssl_log
CustomLog ${APACHE_LOG_DIR}/registry_access_ssl_log combined env=!dontlog
Header always set "Docker-Distribution-Api-Version" "registry/2.0"
Header onsuccess set "Docker-Distribution-Api-Version" "registry/2.0"
RequestHeader set X-Forwarded-Proto "https"
ProxyRequests off
ProxyPreserveHost on
# no proxy for /error/ (Apache HTTPd errors messages)
ProxyPass /error/ !
#
# Registry v1
#
ProxyPass /v1 http://localhost:5000/v1
ProxyPassReverse /v1 http://localhost:5000/v1
ProxyPass /_ping http://localhost:5000/_ping
ProxyPassReverse /_ping http://localhost:5000/_ping
# Authentication require for push
<Location /v1>
Order deny,allow
Allow from all
AuthName "Registry Authentication"
AuthType basic
AuthUserFile "/etc/apache2/htpasswd/registry-htpasswd"
# Read access to authentified users
<Limit GET HEAD>
Require valid-user
</Limit>
# Write access to docker-deployer account only
<Limit POST PUT DELETE>
Require user docker-deployer
</Limit>
</Location>
# Allow ping to run unauthenticated.
<Location /v1/_ping>
Satisfy any
Allow from all
</Location>
# Allow ping to run unauthenticated.
<Location /_ping>
Satisfy any
Allow from all
</Location>
#
# Registry v2
#
ProxyPass /v2 http://localhost:5002/v2
ProxyPassReverse /v2 http://localhost:5002/v2
<Location /v2>
Order deny,allow
Allow from all
AuthName "Registry Authentication"
AuthType basic
AuthUserFile "/etc/apache2/htpasswd/registry-htpasswd"
# Read access to authentified users
<Limit GET HEAD>
Require valid-user
</Limit>
# Write access to docker-deployer only
<Limit POST PUT DELETE>
Require user docker-deployer
</Limit>
</Location>
</VirtualHost>

View File

@@ -1,119 +0,0 @@
#! /bin/bash
#
# Ceph cluster setup in Circle CI
#
set -x
set -e
set -u
NODE=$(hostname)
CEPHDIR=/tmp/ceph
mkdir cluster
pushd cluster
# Install
retries=0
until [ $retries -ge 5 ]; do
pip install ceph-deploy && break
retries=$[$retries+1]
sleep 30
done
retries=0
until [ $retries -ge 5 ]; do
ceph-deploy install --release hammer $NODE && break
retries=$[$retries+1]
sleep 30
done
retries=0
until [ $retries -ge 5 ]; do
ceph-deploy pkg --install librados-dev $NODE && break
retries=$[$retries+1]
sleep 30
done
echo $(ip route get 1 | awk '{print $NF;exit}') $(hostname) >> /etc/hosts
ssh-keygen -t rsa -f ~/.ssh/id_rsa -q -N ""
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
ssh-keyscan $NODE >> ~/.ssh/known_hosts
ceph-deploy new $NODE
cat >> ceph.conf <<EOF
osd objectstore = memstore
memstore device bytes = 2147483648
osd data = $CEPHDIR
osd journal = $CEPHDIR/journal
osd crush chooseleaf type = 0
osd pool default size = 1
osd pool default min size = 1
osd scrub load threshold = 1000
debug_lockdep = 0/0
debug_context = 0/0
debug_crush = 0/0
debug_buffer = 0/0
debug_timer = 0/0
debug_filer = 0/0
debug_objecter = 0/0
debug_rados = 0/0
debug_rbd = 0/0
debug_journaler = 0/0
debug_objectcatcher = 0/0
debug_client = 0/0
debug_osd = 0/0
debug_optracker = 0/0
debug_objclass = 0/0
debug_filestore = 0/0
debug_journal = 0/0
debug_ms = 0/0
debug_monc = 0/0
debug_tp = 0/0
debug_auth = 0/0
debug_finisher = 0/0
debug_heartbeatmap = 0/0
debug_perfcounter = 0/0
debug_asok = 0/0
debug_throttle = 0/0
debug_mon = 0/0
debug_paxos = 0/0
debug_rgw = 0/0
osd_op_num_threads_per_shard = 1 //You may want to try with 1 as well
osd_op_num_shards = 5 //Depends on your cpu util
ms_nocrc = true
cephx_sign_messages = false
cephx_require_signatures = false
ms_dispatch_throttle_bytes = 0
throttler_perf_counter = false
[osd]
osd_client_message_size_cap = 0
osd_client_message_cap = 0
osd_enable_op_tracker = false
EOF
sed -i -r 's/mon_host =.*/mon_host = 127.0.0.1/' ceph.conf
sed -i -r 's/auth_cluster_required =.*/auth_cluster_required = none/' ceph.conf
sed -i -r 's/auth_service_required =.*/auth_service_required = none/' ceph.conf
sed -i -r 's/auth_client_required =.*/auth_client_required = none/' ceph.conf
# Setup monitor and keyrings
ceph-deploy mon create-initial
ceph-deploy admin $NODE
sudo chmod a+r /etc/ceph/ceph.client.admin.keyring
# Setup OSD
mkdir -p $CEPHDIR
OSD=$(ceph osd create)
ceph osd crush add osd.${OSD} 1 root=default host=$NODE
ceph-osd --id ${OSD} --mkjournal --mkfs
ceph-osd --id ${OSD}
# Status
ceph status
ceph health detail
ceph osd tree
popd

View File

@@ -1,147 +0,0 @@
# Docker Compose V1 + V2 registry
This compose configuration configures a `v1` and `v2` registry behind an `nginx`
proxy. By default, you can access the combined registry at `localhost:5000`.
The configuration does not support pushing images to `v2` and pulling from `v1`.
If a `docker` client has a version less than 1.6, Nginx will route its requests
to the 1.0 registry. Requests from newer clients will route to the 2.0 registry.
### Install Docker Compose
1. Open a new terminal on the host with your `distribution` source.
2. Get the `docker-compose` binary.
$ sudo wget https://github.com/docker/compose/releases/download/1.1.0/docker-compose-`uname -s`-`uname -m` -O /usr/local/bin/docker-compose
This command installs the binary in the `/usr/local/bin` directory.
3. Add executable permissions to the binary.
$ sudo chmod +x /usr/local/bin/docker-compose
## Build and run with Compose
1. In your terminal, navigate to the `distribution/contrib/compose` directory
This directory includes a single `docker-compose.yml` configuration.
nginx:
build: "nginx"
ports:
- "5000:5000"
links:
- registryv1:registryv1
- registryv2:registryv2
registryv1:
image: registry
ports:
- "5000"
registryv2:
build: "../../"
ports:
- "5000"
This configuration builds a new `nginx` image as specified by the
`nginx/Dockerfile` file. The 1.0 registry comes from Docker's official
public image. Finally, the registry 2.0 image is built from the
`distribution/Dockerfile` you've used previously.
2. Get a registry 1.0 image.
$ docker pull registry:0.9.1
The Compose configuration looks for this image locally. If you don't do this
step, later steps can fail.
3. Build `nginx`, the registry 2.0 image, and
$ docker-compose build
registryv1 uses an image, skipping
Building registryv2...
Step 0 : FROM golang:1.4
...
Removing intermediate container 9f5f5068c3f3
Step 4 : COPY docker-registry-v2.conf /etc/nginx/docker-registry-v2.conf
---> 74acc70fa106
Removing intermediate container edb84c2b40cb
Successfully built 74acc70fa106
The commmand outputs its progress until it completes.
4. Start your configuration with compose.
$ docker-compose up
Recreating compose_registryv1_1...
Recreating compose_registryv2_1...
Recreating compose_nginx_1...
Attaching to compose_registryv1_1, compose_registryv2_1, compose_nginx_1
...
5. In another terminal, display the running configuration.
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a81ad2557702 compose_nginx:latest "nginx -g 'daemon of 8 minutes ago Up 8 minutes 80/tcp, 443/tcp, 0.0.0.0:5000->5000/tcp compose_nginx_1
0618437450dd compose_registryv2:latest "registry cmd/regist 8 minutes ago Up 8 minutes 0.0.0.0:32777->5000/tcp compose_registryv2_1
aa82b1ed8e61 registry:latest "docker-registry" 8 minutes ago Up 8 minutes 0.0.0.0:32776->5000/tcp compose_registryv1_1
### Explore a bit
1. Check for TLS on your `nginx` server.
$ curl -v https://localhost:5000
* Rebuilt URL to: https://localhost:5000/
* Hostname was NOT found in DNS cache
* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 5000 (#0)
* successfully set certificate verify locations:
* CAfile: none
CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS alert, Server hello (2):
* SSL certificate problem: self signed certificate
* Closing connection 0
curl: (60) SSL certificate problem: self signed certificate
More details here: http://curl.haxx.se/docs/sslcerts.html
2. Tag the `v1` registry image.
$ docker tag registry:latest localhost:5000/registry_one:latest
2. Push it to the localhost.
$ docker push localhost:5000/registry_one:latest
If you are using the 1.6 Docker client, this pushes the image the `v2 `registry.
4. Use `curl` to list the image in the registry.
$ curl -v -X GET http://localhost:32777/v2/registry1/tags/list
* Hostname was NOT found in DNS cache
* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 32777 (#0)
> GET /v2/registry1/tags/list HTTP/1.1
> User-Agent: curl/7.36.0
> Host: localhost:32777
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Type: application/json; charset=utf-8
< Docker-Distribution-Api-Version: registry/2.0
< Date: Tue, 14 Apr 2015 22:34:13 GMT
< Content-Length: 39
<
{"name":"registry1","tags":["latest"]}
* Connection #0 to host localhost left intact
This example refers to the specific port assigned to the 2.0 registry. You saw
this port earlier, when you used `docker ps` to show your running containers.

View File

@@ -1,15 +0,0 @@
nginx:
build: "nginx"
ports:
- "5000:5000"
links:
- registryv1:registryv1
- registryv2:registryv2
registryv1:
image: registry
ports:
- "5000"
registryv2:
build: "../../"
ports:
- "5000"

View File

@@ -1,6 +0,0 @@
FROM nginx:1.7
COPY nginx.conf /etc/nginx/nginx.conf
COPY registry.conf /etc/nginx/conf.d/registry.conf
COPY docker-registry.conf /etc/nginx/docker-registry.conf
COPY docker-registry-v2.conf /etc/nginx/docker-registry-v2.conf

View File

@@ -1,6 +0,0 @@
proxy_pass http://docker-registry-v2;
proxy_set_header Host $http_host; # required for docker client's sake
proxy_set_header X-Real-IP $remote_addr; # pass on real client's IP
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_read_timeout 900;

View File

@@ -1,7 +0,0 @@
proxy_pass http://docker-registry;
proxy_set_header Host $http_host; # required for docker client's sake
proxy_set_header X-Real-IP $remote_addr; # pass on real client's IP
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Authorization ""; # For basic auth through nginx in v1 to work, please comment this line
proxy_read_timeout 900;

View File

@@ -1,27 +0,0 @@
user nginx;
worker_processes 1;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
keepalive_timeout 65;
include /etc/nginx/conf.d/*.conf;
}

View File

@@ -1,41 +0,0 @@
# Docker registry proxy for api versions 1 and 2
upstream docker-registry {
server registryv1:5000;
}
upstream docker-registry-v2 {
server registryv2:5000;
}
# No client auth or TLS
server {
listen 5000;
server_name localhost;
# disable any limits to avoid HTTP 413 for large image uploads
client_max_body_size 0;
# required to avoid HTTP 411: see Issue #1486 (https://github.com/docker/docker/issues/1486)
chunked_transfer_encoding on;
location /v2/ {
# Do not allow connections from docker 1.5 and earlier
# docker pre-1.6.0 did not properly set the user agent on ping, catch "Go *" user agents
if ($http_user_agent ~ "^(docker\/1\.(3|4|5(?!\.[0-9]-dev))|Go ).*$" ) {
return 404;
}
# To add basic authentication to v2 use auth_basic setting plus add_header
# auth_basic "registry.localhost";
# auth_basic_user_file test.password;
# add_header 'Docker-Distribution-Api-Version' 'registry/2.0' always;
include docker-registry-v2.conf;
}
location / {
include docker-registry.conf;
}
}

View File

@@ -1,46 +0,0 @@
FROM debian:jessie
MAINTAINER Docker Distribution Team <distribution@docker.com>
# compile and runtime deps
# https://github.com/docker/docker/blob/master/project/PACKAGERS.md#runtime-dependencies
RUN apt-get update && apt-get install -y --no-install-recommends \
# For DIND
ca-certificates \
curl \
iptables \
procps \
e2fsprogs \
xz-utils \
# For build
build-essential \
file \
git \
net-tools \
&& apt-get clean && rm -rf /var/lib/apt/lists/*
# Install Docker
ENV VERSION 1.7.1
RUN curl -L -o /usr/local/bin/docker https://test.docker.com/builds/Linux/x86_64/docker-${VERSION} \
&& chmod +x /usr/local/bin/docker
# Install DIND
RUN curl -L -o /dind https://raw.githubusercontent.com/docker/docker/v1.8.1/hack/dind \
&& chmod +x /dind
# Install bats
RUN cd /usr/local/src/ \
&& git clone https://github.com/sstephenson/bats.git \
&& cd bats \
&& ./install.sh /usr/local
# Install docker-compose
RUN curl -L https://github.com/docker/compose/releases/download/1.3.3/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose \
&& chmod +x /usr/local/bin/docker-compose
RUN mkdir -p /go/src/github.com/docker/distribution
WORKDIR /go/src/github.com/docker/distribution/contrib/docker-integration
VOLUME /var/lib/docker
ENTRYPOINT ["/dind"]

View File

@@ -1,138 +0,0 @@
# Docker Registry Integration Testing
These integration tests cover interactions between the Docker daemon and the
registry server. All tests are run using the docker cli.
The compose configuration is intended to setup a testing environment for Docker
using multiple registry configurations. These configurations include different
combinations of a v1 and v2 registry as well as TLS configurations.
## Running inside of Docker
### Get integration container
The container image to run the integation tests will need to be pulled or built
locally.
*Building locally*
```
$ docker build -t distribution/docker-integration .
```
### Run script
Invoke the tests within Docker through the `run.sh` script.
```
$ ./run.sh
```
Run with aufs driver and tmp volume
**NOTE: Using a volume will prevent multiple runs from needing to
re-pull images**
```
$ DOCKER_GRAPHDRIVER=aufs DOCKER_VOLUME=/tmp/volume ./run.sh
```
### Example developer flow
These tests are useful for developing both as a registry and docker
core developer. The following setup may be used to do integration
testing between development versions
Insert into your `.zshrc` or `.bashrc`
```
# /usr/lib/docker for Docker-in-Docker
# Set this directory to make each invocation run much faster, without
# the need to repull images.
export DOCKER_VOLUME=$HOME/.docker-test-volume
# Use overlay for all Docker testing, try aufs if overlay not supported
export DOCKER_GRAPHDRIVER=overlay
# Name this according to personal preference
function rdtest() {
if [ "$1" != "" ]; then
DOCKER_BINARY=$GOPATH/src/github.com/docker/docker/bundles/$1/binary/docker
if [ ! -f $DOCKER_BINARY ]; then
current_version=`cat $GOPATH/src/github.com/docker/docker/VERSION`
echo "$DOCKER_BINARY does not exist"
echo "Current checked out docker version: $current_version"
echo "Checkout desired version and run 'make binary' from $GOPATH/src/github.com/docker/docker"
return 1
fi
fi
$GOPATH/src/github.com/docker/distribution/contrib/docker-integration/run.sh
}
```
Run with Docker release version
```
$ rdtest
```
Run using local development version of docker
```
$ cd $GOPATH/src/github.com/docker/docker
$ make binary
$ rdtest `cat VERSION`
```
## Running manually outside of Docker
### Install Docker Compose
[Docker Compose Installation Guide](https://docs.docker.com/compose/install/)
### Start compose setup
```
docker-compose up
```
### Install Certificates
The certificates must be installed in /etc/docker/cert.d in order to use TLS
client auth and use the CA certificate.
```
sudo sh ./install_certs.sh
```
### Test with Docker
Tag an image as with any other private registry. Attempt to push the image.
```
docker pull hello-world
docker tag hello-world localhost:5440/hello-world
docker push localhost:5440/hello-world
docker tag hello-world localhost:5441/hello-world
docker push localhost:5441/hello-world
# Perform login using user `testuser` and password `passpassword`
```
### Set /etc/hosts entry
Find the non-localhost ip address of local machine
### Run bats
Run the bats tests after updating /etc/hosts, installing the certificates, and
running the `docker-compose` script.
```
bats -p .
```
## Configurations
Port | V2 | V1 | TLS | Authentication
--- | --- | --- | --- | ---
5000 | yes | yes | no | none
5001 | no | yes | no | none
5002 | yes | no | no | none
5011 | no | yes | yes | none
5440 | yes | yes | yes | none
5441 | yes | yes | yes | basic (testuser/passpassword)
5442 | yes | yes | yes | TLS client
5443 | yes | yes | yes | TLS client (no CA)
5444 | yes | yes | yes | TLS client + basic (testuser/passpassword)
5445 | yes | yes | yes (no CA) | none
5446 | yes | yes | yes (no CA) | basic (testuser/passpassword)
5447 | yes | yes | yes (no CA) | TLS client
5448 | yes | yes | yes (SSLv3) | none

View File

@@ -1,27 +0,0 @@
nginx:
build: "nginx"
ports:
- "5000:5000"
- "5001:5001"
- "5002:5002"
- "5011:5011"
- "5440:5440"
- "5441:5441"
- "5442:5442"
- "5443:5443"
- "5444:5444"
- "5445:5445"
- "5446:5446"
- "5447:5447"
- "5448:5448"
links:
- registryv1:registryv1
- registryv2:registryv2
registryv1:
image: registry:0.9.1
ports:
- "5000"
registryv2:
build: "../../"
ports:
- "5000"

View File

@@ -1,21 +0,0 @@
# Start docker daemon
function start_daemon() {
# Drivers to use for Docker engines the tests are going to create.
STORAGE_DRIVER=${STORAGE_DRIVER:-overlay}
EXEC_DRIVER=${EXEC_DRIVER:-native}
docker --daemon --log-level=panic \
--storage-driver="$STORAGE_DRIVER" --exec-driver="$EXEC_DRIVER" &
DOCKER_PID=$!
# Wait for it to become reachable.
tries=10
until docker version &> /dev/null; do
(( tries-- ))
if [ $tries -le 0 ]; then
echo >&2 "error: daemon failed to start"
exit 1
fi
sleep 1
done
}

View File

@@ -1,38 +0,0 @@
#!/bin/sh
set -e
hostname=$1
if [ "$hostname" = "" ]; then
hostname="localhost"
fi
mkdir -p /etc/docker/certs.d/$hostname:5011
cp ./nginx/ssl/registry-ca+ca.pem /etc/docker/certs.d/$hostname:5011/ca.crt
mkdir -p /etc/docker/certs.d/$hostname:5440
cp ./nginx/ssl/registry-ca+ca.pem /etc/docker/certs.d/$hostname:5440/ca.crt
mkdir -p /etc/docker/certs.d/$hostname:5441
cp ./nginx/ssl/registry-ca+ca.pem /etc/docker/certs.d/$hostname:5441/ca.crt
mkdir -p /etc/docker/certs.d/$hostname:5442
cp ./nginx/ssl/registry-ca+ca.pem /etc/docker/certs.d/$hostname:5442/ca.crt
cp ./nginx/ssl/registry-ca+client-cert.pem /etc/docker/certs.d/$hostname:5442/client.cert
cp ./nginx/ssl/registry-ca+client-key.pem /etc/docker/certs.d/$hostname:5442/client.key
mkdir -p /etc/docker/certs.d/$hostname:5443
cp ./nginx/ssl/registry-ca+ca.pem /etc/docker/certs.d/$hostname:5443/ca.crt
cp ./nginx/ssl/registry-noca+client-cert.pem /etc/docker/certs.d/$hostname:5443/client.cert
cp ./nginx/ssl/registry-noca+client-key.pem /etc/docker/certs.d/$hostname:5443/client.key
mkdir -p /etc/docker/certs.d/$hostname:5444
cp ./nginx/ssl/registry-ca+ca.pem /etc/docker/certs.d/$hostname:5444/ca.crt
cp ./nginx/ssl/registry-ca+client-cert.pem /etc/docker/certs.d/$hostname:5444/client.cert
cp ./nginx/ssl/registry-ca+client-key.pem /etc/docker/certs.d/$hostname:5444/client.key
mkdir -p /etc/docker/certs.d/$hostname:5447
cp ./nginx/ssl/registry-ca+client-cert.pem /etc/docker/certs.d/$hostname:5447/client.cert
cp ./nginx/ssl/registry-ca+client-key.pem /etc/docker/certs.d/$hostname:5447/client.key
mkdir -p /etc/docker/certs.d/$hostname:5448
cp ./nginx/ssl/registry-ca+ca.pem /etc/docker/certs.d/$hostname:5448/ca.crt

View File

@@ -1,10 +0,0 @@
FROM nginx:1.9
COPY nginx.conf /etc/nginx/nginx.conf
COPY registry.conf /etc/nginx/conf.d/registry.conf
COPY docker-registry.conf /etc/nginx/docker-registry.conf
COPY docker-registry-v2.conf /etc/nginx/docker-registry-v2.conf
COPY registry-noauth.conf /etc/nginx/registry-noauth.conf
COPY registry-basic.conf /etc/nginx/registry-basic.conf
COPY test.passwd /etc/nginx/test.passwd
COPY ssl /etc/nginx/ssl

View File

@@ -1,6 +0,0 @@
proxy_pass http://docker-registry-v2;
proxy_set_header Host $http_host; # required for docker client's sake
proxy_set_header X-Real-IP $remote_addr; # pass on real client's IP
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_read_timeout 900;

View File

@@ -1,7 +0,0 @@
proxy_pass http://docker-registry;
proxy_set_header Host $http_host; # required for docker client's sake
proxy_set_header X-Real-IP $remote_addr; # pass on real client's IP
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Authorization ""; # see https://github.com/docker/docker-registry/issues/170
proxy_read_timeout 900;

View File

@@ -1,27 +0,0 @@
user nginx;
worker_processes 1;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
keepalive_timeout 65;
include /etc/nginx/conf.d/*.conf;
}

View File

@@ -1,13 +0,0 @@
client_max_body_size 0;
chunked_transfer_encoding on;
location /v2/ {
auth_basic "registry.localhost";
auth_basic_user_file test.passwd;
add_header 'Docker-Distribution-Api-Version' 'registry/2.0' always;
include docker-registry-v2.conf;
}
location / {
auth_basic "registry.localhost";
auth_basic_user_file test.passwd;
include docker-registry.conf;
}

View File

@@ -1,8 +0,0 @@
client_max_body_size 0;
chunked_transfer_encoding on;
location /v2/ {
include docker-registry-v2.conf;
}
location / {
include docker-registry.conf;
}

View File

@@ -1,277 +0,0 @@
# Docker registry proxy for api versions 1 and 2
upstream docker-registry {
server registryv1:5000;
}
upstream docker-registry-v2 {
server registryv2:5000;
}
# No client auth or TLS
server {
listen 5000;
server_name localhost;
# disable any limits to avoid HTTP 413 for large image uploads
client_max_body_size 0;
# required to avoid HTTP 411: see Issue #1486 (https://github.com/docker/docker/issues/1486)
chunked_transfer_encoding on;
location /v2/ {
# Do not allow connections from docker 1.5 and earlier
# docker pre-1.6.0 did not properly set the user agent on ping, catch "Go *" user agents
if ($http_user_agent ~ "^(docker\/1\.(3|4|5(?!\.[0-9]-dev))|Go ).*$" ) {
return 404;
}
include docker-registry-v2.conf;
}
location / {
include docker-registry.conf;
}
}
# No client auth or TLS (V1 Only)
server {
listen 5001;
server_name localhost;
# disable any limits to avoid HTTP 413 for large image uploads
client_max_body_size 0;
# required to avoid HTTP 411: see Issue #1486 (https://github.com/docker/docker/issues/1486)
chunked_transfer_encoding on;
location / {
include docker-registry.conf;
}
}
# No client auth or TLS (V2 Only)
server {
listen 5002;
server_name localhost;
# disable any limits to avoid HTTP 413 for large image uploads
client_max_body_size 0;
# required to avoid HTTP 411: see Issue #1486 (https://github.com/docker/docker/issues/1486)
chunked_transfer_encoding on;
location / {
include docker-registry-v2.conf;
}
}
# TLS localhost (V1 Only)
server {
listen 5011;
server_name localhost;
ssl on;
ssl_certificate /etc/nginx/ssl/registry-ca+localhost-cert.pem;
ssl_certificate_key /etc/nginx/ssl/registry-ca+localhost-key.pem;
client_max_body_size 0;
chunked_transfer_encoding on;
location / {
include docker-registry.conf;
}
}
# TLS localregistry (V1 Only)
server {
listen 5011;
server_name localregistry;
ssl on;
ssl_certificate /etc/nginx/ssl/registry-ca+localregistry-cert.pem;
ssl_certificate_key /etc/nginx/ssl/registry-ca+localregistry-key.pem;
client_max_body_size 0;
chunked_transfer_encoding on;
location / {
include docker-registry.conf;
}
}
# TLS Configuration chart
# Username/Password: testuser/passpassword
# | ca | client | basic | notes
# 5440 | yes | no | no | Tests CA certificate
# 5441 | yes | no | yes | Tests basic auth over TLS
# 5442 | yes | yes | no | Tests client auth with client CA
# 5443 | yes | yes | no | Tests client auth without client CA
# 5444 | yes | yes | yes | Tests using basic auth + tls auth
# 5445 | no | no | no | Tests insecure using TLS
# 5446 | no | no | yes | Tests sending credentials to server with insecure TLS
# 5447 | no | yes | no | Tests client auth to insecure
# 5448 | yes | no | no | Bad SSL version
server {
listen 5440;
server_name localhost;
ssl on;
ssl_certificate /etc/nginx/ssl/registry-ca+localhost-cert.pem;
ssl_certificate_key /etc/nginx/ssl/registry-ca+localhost-key.pem;
include registry-noauth.conf;
}
server {
listen 5441;
server_name localhost;
ssl on;
ssl_certificate /etc/nginx/ssl/registry-ca+localhost-cert.pem;
ssl_certificate_key /etc/nginx/ssl/registry-ca+localhost-key.pem;
include registry-basic.conf;
}
server {
listen 5442;
listen 5443;
server_name localhost;
ssl on;
ssl_certificate /etc/nginx/ssl/registry-ca+localhost-cert.pem;
ssl_certificate_key /etc/nginx/ssl/registry-ca+localhost-key.pem;
ssl_client_certificate /etc/nginx/ssl/registry-ca+ca.pem;
ssl_verify_client on;
include registry-noauth.conf;
}
server {
listen 5444;
server_name localhost;
ssl on;
ssl_certificate /etc/nginx/ssl/registry-ca+localhost-cert.pem;
ssl_certificate_key /etc/nginx/ssl/registry-ca+localhost-key.pem;
ssl_client_certificate /etc/nginx/ssl/registry-ca+ca.pem;
ssl_verify_client on;
include registry-basic.conf;
}
server {
listen 5445;
server_name localhost;
ssl on;
ssl_certificate /etc/nginx/ssl/registry-noca+localhost-cert.pem;
ssl_certificate_key /etc/nginx/ssl/registry-noca+localhost-key.pem;
include registry-noauth.conf;
}
server {
listen 5446;
server_name localhost;
ssl on;
ssl_certificate /etc/nginx/ssl/registry-noca+localhost-cert.pem;
ssl_certificate_key /etc/nginx/ssl/registry-noca+localhost-key.pem;
include registry-basic.conf;
}
server {
listen 5447;
server_name localhost;
ssl on;
ssl_certificate /etc/nginx/ssl/registry-noca+localhost-cert.pem;
ssl_certificate_key /etc/nginx/ssl/registry-noca+localhost-key.pem;
ssl_client_certificate /etc/nginx/ssl/registry-ca+ca.pem;
ssl_verify_client on;
include registry-noauth.conf;
}
server {
listen 5448;
server_name localhost;
ssl on;
ssl_certificate /etc/nginx/ssl/registry-ca+localhost-cert.pem;
ssl_certificate_key /etc/nginx/ssl/registry-ca+localhost-key.pem;
ssl_protocols SSLv3;
include registry-noauth.conf;
}
# Add configuration for localregistry server_name
# Requires configuring /etc/hosts to use
# Set /etc/hosts entry to external IP, not 127.0.0.1 for testing
# Docker secure/insecure registry features
server {
listen 5440;
server_name localregistry;
ssl on;
ssl_certificate /etc/nginx/ssl/registry-ca+localregistry-cert.pem;
ssl_certificate_key /etc/nginx/ssl/registry-ca+localregistry-key.pem;
include registry-noauth.conf;
}
server {
listen 5441;
server_name localregistry;
ssl on;
ssl_certificate /etc/nginx/ssl/registry-ca+localregistry-cert.pem;
ssl_certificate_key /etc/nginx/ssl/registry-ca+localregistry-key.pem;
include registry-basic.conf;
}
server {
listen 5442;
listen 5443;
server_name localregistry;
ssl on;
ssl_certificate /etc/nginx/ssl/registry-ca+localregistry-cert.pem;
ssl_certificate_key /etc/nginx/ssl/registry-ca+localregistry-key.pem;
ssl_client_certificate /etc/nginx/ssl/registry-ca+ca.pem;
ssl_verify_client on;
include registry-noauth.conf;
}
server {
listen 5444;
server_name localregistry;
ssl on;
ssl_certificate /etc/nginx/ssl/registry-ca+localregistry-cert.pem;
ssl_certificate_key /etc/nginx/ssl/registry-ca+localregistry-key.pem;
ssl_client_certificate /etc/nginx/ssl/registry-ca+ca.pem;
ssl_verify_client on;
include registry-basic.conf;
}
server {
listen 5445;
server_name localregistry;
ssl on;
ssl_certificate /etc/nginx/ssl/registry-noca+localregistry-cert.pem;
ssl_certificate_key /etc/nginx/ssl/registry-noca+localregistry-key.pem;
include registry-noauth.conf;
}
server {
listen 5446;
server_name localregistry;
ssl on;
ssl_certificate /etc/nginx/ssl/registry-noca+localregistry-cert.pem;
ssl_certificate_key /etc/nginx/ssl/registry-noca+localregistry-key.pem;
include registry-basic.conf;
}
server {
listen 5447;
server_name localregistry;
ssl on;
ssl_certificate /etc/nginx/ssl/registry-noca+localregistry-cert.pem;
ssl_certificate_key /etc/nginx/ssl/registry-noca+localregistry-key.pem;
ssl_client_certificate /etc/nginx/ssl/registry-ca+ca.pem;
ssl_verify_client on;
include registry-noauth.conf;
}
server {
listen 5448;
server_name localregistry;
ssl on;
ssl_certificate /etc/nginx/ssl/registry-ca+localregistry-cert.pem;
ssl_certificate_key /etc/nginx/ssl/registry-ca+localregistry-key.pem;
ssl_protocols SSLv3;
include registry-noauth.conf;
}

View File

@@ -1 +0,0 @@
testuser:$apr1$YmLhHjm6$AjP4z8J1WgcUNxU8J4ue5.

View File

@@ -1,75 +0,0 @@
#!/usr/bin/env bash
set -e
set -x
cd "$(dirname "$(readlink -f "$BASH_SOURCE")")"
source helpers.bash
# Root directory of Distribution
DISTRIBUTION_ROOT=$(cd ../..; pwd -P)
volumeMount=""
if [ "$DOCKER_VOLUME" != "" ]; then
volumeMount="-v ${DOCKER_VOLUME}:/var/lib/docker"
fi
dockerMount=""
if [ "$DOCKER_BINARY" != "" ]; then
dockerMount="-v ${DOCKER_BINARY}:/usr/local/bin/docker"
else
DOCKER_BINARY=docker
fi
# Image containing the integration tests environment.
INTEGRATION_IMAGE=${INTEGRATION_IMAGE:-distribution/docker-integration}
if [ "$1" == "-d" ]; then
start_daemon
shift
fi
TESTS=${@:-.}
# Make sure we upgrade the integration environment.
docker pull $INTEGRATION_IMAGE
# Start a Docker engine inside a docker container
ID=$(docker run -d -it --privileged $volumeMount $dockerMount \
-v ${DISTRIBUTION_ROOT}:/go/src/github.com/docker/distribution \
-e "DOCKER_GRAPHDRIVER=$DOCKER_GRAPHDRIVER" \
${INTEGRATION_IMAGE} \
./run_engine.sh)
# Stop container on exit
trap "docker rm -f -v $ID" EXIT
# Wait for it to become reachable.
tries=10
until docker exec "$ID" docker version &> /dev/null; do
(( tries-- ))
if [ $tries -le 0 ]; then
echo >&2 "error: daemon failed to start"
exit 1
fi
sleep 1
done
# If no volume is specified, transfer images into the container from
# the outer docker instance
if [ "$DOCKER_VOLUME" == "" ]; then
# Make sure we have images outside the container, to transfer to the container.
# Not much will happen here if the images are already present.
docker-compose pull
docker-compose build
# Transfer images to the inner container.
for image in "$INTEGRATION_IMAGE" registry:0.9.1 dockerintegration_nginx dockerintegration_registryv2; do
docker save "$image" | docker exec -i "$ID" docker load
done
fi
# Run the tests.
docker exec -it "$ID" sh -c "./test_runner.sh $TESTS"

View File

@@ -1,23 +0,0 @@
#!/bin/sh
set -e
set -x
DOCKER_GRAPHDRIVER=${DOCKER_GRAPHDRIVER:-overlay}
EXEC_DRIVER=${EXEC_DRIVER:-native}
# Set IP address in /etc/hosts for localregistry
IP=$(ifconfig eth0|grep "inet addr:"| cut -d: -f2 | awk '{ print $1}')
echo "$IP localregistry" >> /etc/hosts
sh install_certs.sh localregistry
DOCKER_VERSION=$(docker --version | cut -d ' ' -f3 | cut -d ',' -f1)
major=$(echo "$DOCKER_VERSION"| cut -d '.' -f1)
minor=$(echo "$DOCKER_VERSION"| cut -d '.' -f2)
daemonOpts="daemon"
if [ $major -le 1 ] && [ $minor -lt 9 ]; then
daemonOpts="--daemon"
fi
docker $daemonOpts --log-level=debug --storage-driver="$DOCKER_GRAPHDRIVER"

View File

@@ -1,53 +0,0 @@
#!/usr/bin/env bash
# Run the integration tests with multiple versions of the Docker engine
set -e
set -x
source helpers.bash
if [ `uname` = "Linux" ]; then
tmpdir_template="$TMPDIR/docker-versions.XXXXX"
else
# /tmp isn't available for mounting in boot2docker
tmpdir_template="`pwd`/../../../docker-versions.XXXXX"
fi
tmpdir=`mktemp -d "$tmpdir_template"`
trap "rm -rf $tmpdir" EXIT
if [ "$1" == "-d" ]; then
start_daemon
fi
# Released versions
versions="1.6.1 1.7.1 1.8.3 1.9.1"
for v in $versions; do
echo "Extracting Docker $v from dind image"
binpath="$tmpdir/docker-$v/docker"
ID=$(docker create dockerswarm/dind:$v)
docker cp "$ID:/usr/local/bin/docker" "$tmpdir/docker-$v"
echo "Running tests with Docker $v"
DOCKER_BINARY="$binpath" DOCKER_VOLUME="$DOCKER_VOLUME" DOCKER_GRAPHDRIVER="$DOCKER_GRAPHDRIVER" ./run.sh
# Cleanup.
docker rm -f "$ID"
done
# Latest experimental version
echo "Extracting Docker master from dind image"
binpath="$tmpdir/docker-master/docker"
docker pull dockerswarm/dind-master
ID=$(docker create dockerswarm/dind-master)
docker cp "$ID:/usr/local/bin/docker" "$tmpdir/docker-master"
echo "Running tests with Docker master"
DOCKER_BINARY="$binpath" DOCKER_VOLUME="$DOCKER_VOLUME" ./run.sh
# Cleanup.
docker rm -f "$ID"

View File

@@ -1,18 +0,0 @@
#!/usr/bin/env bash
set -e
cd "$(dirname "$(readlink -f "$BASH_SOURCE")")"
TESTS=${@:-.}
function execute() {
>&2 echo "++ $@"
eval "$@"
}
execute time docker-compose build
execute docker-compose up -d
# Run the tests.
execute time bats -p $TESTS

View File

@@ -1,102 +0,0 @@
# Registry host name, should be set to non-localhost address and match
# DNS name in nginx/ssl certificates and what is installed in /etc/docker/cert.d
hostname="localregistry"
image="hello-world:latest"
# Login information, should match values in nginx/test.passwd
user="testuser"
password="passpassword"
email="distribution@docker.com"
function setup() {
docker pull $image
}
# skip basic auth tests with Docker 1.6, where they don't pass due to
# certificate issues
function basic_auth_version_check() {
run sh -c 'docker version | fgrep -q "Client version: 1.6."'
if [ "$status" -eq 0 ]; then
skip "Basic auth tests don't support 1.6.x"
fi
}
# has_digest enforces the last output line is "Digest: sha256:..."
# the input is the name of the array containing the output lines
function has_digest() {
filtered=$(echo "$1" |sed -rn '/[dD]igest\: sha(256|384|512)/ p')
[ "$filtered" != "" ]
}
function login() {
run docker login -u $user -p $password -e $email $1
[ "$status" -eq 0 ]
# First line is WARNING about credential save
[ "${lines[1]}" = "Login Succeeded" ]
}
@test "Test valid certificates" {
docker tag -f $image $hostname:5440/$image
run docker push $hostname:5440/$image
[ "$status" -eq 0 ]
has_digest "$output"
}
@test "Test basic auth" {
basic_auth_version_check
login $hostname:5441
docker tag -f $image $hostname:5441/$image
run docker push $hostname:5441/$image
[ "$status" -eq 0 ]
has_digest "$output"
}
@test "Test TLS client auth" {
docker tag -f $image $hostname:5442/$image
run docker push $hostname:5442/$image
[ "$status" -eq 0 ]
has_digest "$output"
}
@test "Test TLS client with invalid certificate authority fails" {
docker tag -f $image $hostname:5443/$image
run docker push $hostname:5443/$image
[ "$status" -ne 0 ]
}
@test "Test basic auth with TLS client auth" {
basic_auth_version_check
login $hostname:5444
docker tag -f $image $hostname:5444/$image
run docker push $hostname:5444/$image
[ "$status" -eq 0 ]
has_digest "$output"
}
@test "Test unknown certificate authority fails" {
docker tag -f $image $hostname:5445/$image
run docker push $hostname:5445/$image
[ "$status" -ne 0 ]
}
@test "Test basic auth with unknown certificate authority fails" {
run login $hostname:5446
[ "$status" -ne 0 ]
docker tag -f $image $hostname:5446/$image
run docker push $hostname:5446/$image
[ "$status" -ne 0 ]
}
@test "Test TLS client auth to server with unknown certificate authority fails" {
docker tag -f $image $hostname:5447/$image
run docker push $hostname:5447/$image
[ "$status" -ne 0 ]
}
@test "Test failure to connect to server fails to fallback to SSLv3" {
docker tag -f $image $hostname:5448/$image
run docker push $hostname:5448/$image
[ "$status" -ne 0 ]
}

View File

@@ -1,7 +0,0 @@
#!/usr/bin/env bash
# Given a subpackage and the containing package, figures out which packages
# need to be passed to `go test -coverpkg`: this includes all of the
# subpackage's dependencies within the containing package, as well as the
# subpackage itself.
DEPENDENCIES="$(go list -f $'{{range $f := .Deps}}{{$f}}\n{{end}}' ${1} | grep ${2})"
echo "${1} ${DEPENDENCIES}" | xargs echo -n | tr ' ' ','

View File

@@ -1,7 +0,0 @@
// Package distribution will define the interfaces for the components of
// docker distribution. The goal is to allow users to reliably package, ship
// and store content related to docker images.
//
// This is currently a work in progress. More details are available in the
// README.md.
package distribution

View File

@@ -1,17 +0,0 @@
FROM docs/base:latest
MAINTAINER Mary Anthony <mary@docker.com> (@moxiegirl)
RUN svn checkout https://github.com/docker/docker/trunk/docs /docs/content/engine
RUN svn checkout https://github.com/docker/compose/trunk/docs /docs/content/compose
RUN svn checkout https://github.com/docker/machine/trunk/docs /docs/content/machine
RUN svn checkout https://github.com/docker/distribution/trunk/docs /docs/content/registry
RUN svn checkout https://github.com/kitematic/kitematic/trunk/docs /docs/content/kitematic
RUN svn checkout https://github.com/docker/tutorials/trunk/docs /docs/content/tutorials
RUN svn checkout https://github.com/docker/opensource/trunk/docs /docs/content/opensource
ENV PROJECT=registry
# To get the git info for this repo
COPY . /src
COPY . /docs/content/$PROJECT/

View File

@@ -1,55 +0,0 @@
.PHONY: all binary build cross default docs docs-build docs-shell shell test test-unit test-integration test-integration-cli test-docker-py validate
# env vars passed through directly to Docker's build scripts
# to allow things like `make DOCKER_CLIENTONLY=1 binary` easily
# `docs/sources/contributing/devenvironment.md ` and `project/PACKAGERS.md` have some limited documentation of some of these
DOCKER_ENVS := \
-e BUILDFLAGS \
-e DOCKER_CLIENTONLY \
-e DOCKER_EXECDRIVER \
-e DOCKER_GRAPHDRIVER \
-e TESTDIRS \
-e TESTFLAGS \
-e TIMEOUT
# note: we _cannot_ add "-e DOCKER_BUILDTAGS" here because even if it's unset in the shell, that would shadow the "ENV DOCKER_BUILDTAGS" set in our Dockerfile, which is very important for our official builds
# to allow `make DOCSDIR=docs docs-shell` (to create a bind mount in docs)
DOCS_MOUNT := $(if $(DOCSDIR),-v $(CURDIR)/$(DOCSDIR):/$(DOCSDIR))
# to allow `make DOCSPORT=9000 docs`
DOCSPORT := 8000
# Get the IP ADDRESS
DOCKER_IP=$(shell python -c "import urlparse ; print urlparse.urlparse('$(DOCKER_HOST)').hostname or ''")
HUGO_BASE_URL=$(shell test -z "$(DOCKER_IP)" && echo localhost || echo "$(DOCKER_IP)")
HUGO_BIND_IP=0.0.0.0
GIT_BRANCH := $(shell git rev-parse --abbrev-ref HEAD 2>/dev/null)
DOCKER_IMAGE := docker$(if $(GIT_BRANCH),:$(GIT_BRANCH))
DOCKER_DOCS_IMAGE := docs-base$(if $(GIT_BRANCH),:$(GIT_BRANCH))
DOCKER_RUN_DOCS := docker run --rm -it $(DOCS_MOUNT) -e AWS_S3_BUCKET -e NOCACHE
# for some docs workarounds (see below in "docs-build" target)
GITCOMMIT := $(shell git rev-parse --short HEAD 2>/dev/null)
default: docs
docs: docs-build
$(DOCKER_RUN_DOCS) -p $(if $(DOCSPORT),$(DOCSPORT):)8000 -e DOCKERHOST "$(DOCKER_DOCS_IMAGE)" hugo server --port=$(DOCSPORT) --baseUrl=$(HUGO_BASE_URL) --bind=$(HUGO_BIND_IP)
docs-draft: docs-build
$(DOCKER_RUN_DOCS) -p $(if $(DOCSPORT),$(DOCSPORT):)8000 -e DOCKERHOST "$(DOCKER_DOCS_IMAGE)" hugo server --buildDrafts="true" --port=$(DOCSPORT) --baseUrl=$(HUGO_BASE_URL) --bind=$(HUGO_BIND_IP)
docs-shell: docs-build
$(DOCKER_RUN_DOCS) -p $(if $(DOCSPORT),$(DOCSPORT):)8000 "$(DOCKER_DOCS_IMAGE)" bash
docs-build:
# ( git remote | grep -v upstream ) || git diff --name-status upstream/release..upstream/docs ./ > ./changed-files
# echo "$(GIT_BRANCH)" > GIT_BRANCH
# echo "$(AWS_S3_BUCKET)" > AWS_S3_BUCKET
# echo "$(GITCOMMIT)" > GITCOMMIT
docker build -t "$(DOCKER_DOCS_IMAGE)" .

View File

@@ -1,213 +0,0 @@
<!--[metadata]>
+++
title = "Authenticating proxy with apache"
description = "Restricting access to your registry using an apache proxy"
keywords = ["registry, on-prem, images, tags, repository, distribution, authentication, proxy, apache, httpd, TLS, recipe, advanced"]
+++
<![end-metadata]-->
# Authenticating proxy with apache
## Use-case
People already relying on an apache proxy to authenticate their users to other services might want to leverage it and have Registry communications tunneled through the same pipeline.
Usually, that includes enterprise setups using LDAP/AD on the backend and a SSO mechanism fronting their internal http portal.
### Alternatives
If you just want authentication for your registry, and are happy maintaining users access separately, you should really consider sticking with the native [basic auth registry feature](deploying.md#native-basic-auth).
### Solution
With the method presented here, you implement basic authentication for docker engines in a reverse proxy that sits in front of your registry.
While we use a simple htpasswd file as an example, any other apache authentication backend should be fairly easy to implement once you are done with the exemple.
We also implement push restriction (to a limited user group) for the sake of the exemple. Again, you should modify this to fit your mileage.
### Gotchas
While this model gives you the ability to use whatever authentication backend you want through the secondary authentication mechanism implemented inside your proxy, it also requires that you move TLS termination from the Registry to the proxy itself.
Furthermore, introducing an extra http layer in your communication pipeline will make it more complex to deploy, maintain, and debug, and will possibly create issues.
## Setting things up
Read again [the requirements](recipes.md#requirements).
Ready?
Run the following script:
```
mkdir -p auth
mkdir -p data
# This is the main apache configuration you will use
cat <<EOF > auth/httpd.conf
LoadModule headers_module modules/mod_headers.so
LoadModule authn_file_module modules/mod_authn_file.so
LoadModule authn_core_module modules/mod_authn_core.so
LoadModule authz_groupfile_module modules/mod_authz_groupfile.so
LoadModule authz_user_module modules/mod_authz_user.so
LoadModule authz_core_module modules/mod_authz_core.so
LoadModule auth_basic_module modules/mod_auth_basic.so
LoadModule access_compat_module modules/mod_access_compat.so
LoadModule log_config_module modules/mod_log_config.so
LoadModule ssl_module modules/mod_ssl.so
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule unixd_module modules/mod_unixd.so
<IfModule ssl_module>
SSLRandomSeed startup builtin
SSLRandomSeed connect builtin
</IfModule>
<IfModule unixd_module>
User daemon
Group daemon
</IfModule>
ServerAdmin you@example.com
ErrorLog /proc/self/fd/2
LogLevel warn
<IfModule log_config_module>
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %b" common
<IfModule logio_module>
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio
</IfModule>
CustomLog /proc/self/fd/1 common
</IfModule>
ServerRoot "/usr/local/apache2"
Listen 5043
<Directory />
AllowOverride none
Require all denied
</Directory>
<VirtualHost *:5043>
ServerName myregistrydomain.com
SSLEngine on
SSLCertificateFile /usr/local/apache2/conf/domain.crt
SSLCertificateKeyFile /usr/local/apache2/conf/domain.key
## SSL settings recommandation from: https://raymii.org/s/tutorials/Strong_SSL_Security_On_Apache2.html
# Anti CRIME
SSLCompression off
# POODLE and other stuff
SSLProtocol all -SSLv2 -SSLv3 -TLSv1
# Secure cypher suites
SSLCipherSuite EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH
SSLHonorCipherOrder on
Header always set "Docker-Distribution-Api-Version" "registry/2.0"
Header onsuccess set "Docker-Distribution-Api-Version" "registry/2.0"
RequestHeader set X-Forwarded-Proto "https"
ProxyRequests off
ProxyPreserveHost on
# no proxy for /error/ (Apache HTTPd errors messages)
ProxyPass /error/ !
ProxyPass /v2 http://registry:5000/v2
ProxyPassReverse /v2 http://registry:5000/v2
<Location /v2>
Order deny,allow
Allow from all
AuthName "Registry Authentication"
AuthType basic
AuthUserFile "/usr/local/apache2/conf/httpd.htpasswd"
AuthGroupFile "/usr/local/apache2/conf/httpd.groups"
# Read access to authentified users
<Limit GET HEAD>
Require valid-user
</Limit>
# Write access to docker-deployer only
<Limit POST PUT DELETE PATCH>
Require group pusher
</Limit>
</Location>
</VirtualHost>
EOF
# Now, create a password file for "testuser" and "testpassword"
docker run --entrypoint htpasswd httpd:2.4 -Bbn testuser testpassword > auth/httpd.htpasswd
# Create another one for "testuserpush" and "testpasswordpush"
docker run --entrypoint htpasswd httpd:2.4 -Bbn testuserpush testpasswordpush >> auth/httpd.htpasswd
# Create your group file
echo "pusher: testuserpush" > auth/httpd.groups
# Copy over your certificate files
cp domain.crt auth
cp domain.key auth
# Now create your compose file
cat <<EOF > docker-compose.yml
apache:
image: "httpd:2.4"
hostname: myregistrydomain.com
ports:
- 5043:5043
links:
- registry:registry
volumes:
- `pwd`/auth:/usr/local/apache2/conf
registry:
image: registry:2
ports:
- 127.0.0.1:5000:5000
volumes:
- `pwd`/data:/var/lib/registry
EOF
```
## Starting and stopping
Now, start your stack:
docker-compose up -d
Login with a "push" authorized user (using `testuserpush` and `testpasswordpush`), then tag and push your first image:
docker login myregistrydomain.com:5043
docker tag ubuntu myregistrydomain.com:5043/test
docker push myregistrydomain.com:5043/test
Now, login with a "pull-only" user (using `testuser` and `testpassword`), then pull back the image:
docker login myregistrydomain.com:5043
docker pull myregistrydomain.com:5043/test
Verify that the "pull-only" can NOT push:
docker push myregistrydomain.com:5043/test

View File

@@ -1,54 +0,0 @@
<!--[metadata]>
+++
draft = true
+++
<![end-metadata]-->
# Architecture
## Design
**TODO(stevvooe):** Discuss the architecture of the registry, internally and externally, in a few different deployment scenarios.
### Eventual Consistency
> **NOTE:** This section belongs somewhere, perhaps in a design document. We
> are leaving this here so the information is not lost.
Running the registry on eventually consistent backends has been part of the
design from the beginning. This section covers some of the approaches to
dealing with this reality.
There are a few classes of issues that we need to worry about when
implementing something on top of the storage drivers:
1. Read-After-Write consistency (see this [article on
s3](http://shlomoswidler.com/2009/12/read-after-write-consistency-in-amazon.html)).
2. [Write-Write Conflicts](http://en.wikipedia.org/wiki/Write%E2%80%93write_conflict).
In reality, the registry must worry about these kinds of errors when doing the
following:
1. Accepting data into a temporary upload file may not have latest data block
yet (read-after-write).
2. Moving uploaded data into its blob location (write-write race).
3. Modifying the "current" manifest for given tag (write-write race).
4. A whole slew of operations around deletes (read-after-write, delete-write
races, garbage collection, etc.).
The backend path layout employs a few techniques to avoid these problems:
1. Large writes are done to private upload directories. This alleviates most
of the corruption potential under multiple writers by avoiding multiple
writers.
2. Constraints in storage driver implementations, such as support for writing
after the end of a file to extend it.
3. Digest verification to avoid data corruption.
4. Manifest files are stored by digest and cannot change.
5. All other non-content files (links, hashes, etc.) are written as an atomic
unit. Anything that requires additions and deletions is broken out into
separate "files". Last writer still wins.
Unfortunately, one must play this game when trying to build something like
this on top of eventually consistent storage systems. If we run into serious
problems, we can wrap the storagedrivers in a shared consistency layer but
that would increase complexity and hinder registry cluster performance.

View File

@@ -1,158 +0,0 @@
<!--[metadata]>
+++
title = "Build instructions"
description = "Explains how to build & hack on the registry"
keywords = ["registry, on-prem, images, tags, repository, distribution, build, recipe, advanced"]
+++
<![end-metadata]-->
# Building the registry source
## Use-case
This is useful if you intend to actively work on the registry.
### Alternatives
Most people should use the [official Registry docker image](https://hub.docker.com/r/library/registry/).
People looking for advanced operational use cases might consider rolling their own image with a custom Dockerfile inheriting `FROM registry:2`.
OS X users who want to run natively can do so following [the instructions here](osx-setup-guide.md).
### Gotchas
You are expected to know your way around with go & git.
If you are a casual user with no development experience, and no preliminary knowledge of go, building from source is probably not a good solution for you.
## Build the development environment
The first prerequisite of properly building distribution targets is to have a Go
development environment setup. Please follow [How to Write Go Code](https://golang.org/doc/code.html)
for proper setup. If done correctly, you should have a GOROOT and GOPATH set in the
environment.
If a Go development environment is setup, one can use `go get` to install the
`registry` command from the current latest:
go get github.com/docker/distribution/cmd/registry
The above will install the source repository into the `GOPATH`.
Now create the directory for the registry data (this might require you to set permissions properly)
mkdir -p /var/lib/registry
... or alternatively `export REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY=/somewhere` if you want to store data into another location.
The `registry`
binary can then be run with the following:
$ $GOPATH/bin/registry --version
$GOPATH/bin/registry github.com/docker/distribution v2.0.0-alpha.1+unknown
> __NOTE:__ While you do not need to use `go get` to checkout the distribution
> project, for these build instructions to work, the project must be checked
> out in the correct location in the `GOPATH`. This should almost always be
> `$GOPATH/src/github.com/docker/distribution`.
The registry can be run with the default config using the following
incantation:
$ $GOPATH/bin/registry $GOPATH/src/github.com/docker/distribution/cmd/registry/config-example.yml
INFO[0000] endpoint local-5003 disabled, skipping app.id=34bbec38-a91a-494a-9a3f-b72f9010081f version=v2.0.0-alpha.1+unknown
INFO[0000] endpoint local-8083 disabled, skipping app.id=34bbec38-a91a-494a-9a3f-b72f9010081f version=v2.0.0-alpha.1+unknown
INFO[0000] listening on :5000 app.id=34bbec38-a91a-494a-9a3f-b72f9010081f version=v2.0.0-alpha.1+unknown
INFO[0000] debug server listening localhost:5001
If it is working, one should see the above log messages.
### Repeatable Builds
For the full development experience, one should `cd` into
`$GOPATH/src/github.com/docker/distribution`. From there, the regular `go`
commands, such as `go test`, should work per package (please see
[Developing](#developing) if they don't work).
A `Makefile` has been provided as a convenience to support repeatable builds.
Please install the following into `GOPATH` for it to work:
go get github.com/tools/godep github.com/golang/lint/golint
**TODO(stevvooe):** Add a `make setup` command to Makefile to run this. Have to think about how to interact with Godeps properly.
Once these commands are available in the `GOPATH`, run `make` to get a full
build:
$ GOPATH=`godep path`:$GOPATH make
+ clean
+ fmt
+ vet
+ lint
+ build
github.com/docker/docker/vendor/src/code.google.com/p/go/src/pkg/archive/tar
github.com/Sirupsen/logrus
github.com/docker/libtrust
...
github.com/yvasiyarov/gorelic
github.com/docker/distribution/registry/handlers
github.com/docker/distribution/cmd/registry
+ test
...
ok github.com/docker/distribution/digest 7.875s
ok github.com/docker/distribution/manifest 0.028s
ok github.com/docker/distribution/notifications 17.322s
? github.com/docker/distribution/registry [no test files]
ok github.com/docker/distribution/registry/api/v2 0.101s
? github.com/docker/distribution/registry/auth [no test files]
ok github.com/docker/distribution/registry/auth/silly 0.011s
...
+ /Users/sday/go/src/github.com/docker/distribution/bin/registry
+ /Users/sday/go/src/github.com/docker/distribution/bin/registry-api-descriptor-template
+ binaries
The above provides a repeatable build using the contents of the vendored
Godeps directory. This includes formatting, vetting, linting, building,
testing and generating tagged binaries. We can verify this worked by running
the registry binary generated in the "./bin" directory:
$ ./bin/registry -version
./bin/registry github.com/docker/distribution v2.0.0-alpha.2-80-g16d8b2c.m
### Developing
The above approaches are helpful for small experimentation. If more complex
tasks are at hand, it is recommended to employ the full power of `godep`.
The Makefile is designed to have its `GOPATH` defined externally. This allows
one to experiment with various development environment setups. This is
primarily useful when testing upstream bugfixes, by modifying local code. This
can be demonstrated using `godep` to migrate the `GOPATH` to use the specified
dependencies. The `GOPATH` can be migrated to the current package versions
declared in `Godeps` with the following command:
godep restore
> **WARNING:** This command will checkout versions of the code specified in
> Godeps/Godeps.json, modifying the contents of `GOPATH`. If this is
> undesired, it is recommended to create a workspace devoted to work on the
> _Distribution_ project.
With a successful run of the above command, one can now use `make` without
specifying the `GOPATH`:
make
If that is successful, standard `go` commands, such as `go test` should work,
per package, without issue.
### Optional build tags
Optional [build tags](http://golang.org/pkg/go/build/) can be provided using
the environment variable `DOCKER_BUILDTAGS`.
To enable the [Ceph RADOS storage driver](storage-drivers/rados.md)
(librados-dev and librbd-dev will be required to build the bindings):
export DOCKER_BUILDTAGS='include_rados'

File diff suppressed because it is too large Load Diff

View File

@@ -1,231 +0,0 @@
<!--[metadata]>
+++
title = "Deploying a registry server"
description = "Explains how to deploy a registry"
keywords = ["registry, on-prem, images, tags, repository, distribution, deployment"]
[menu.main]
parent="smn_registry"
weight=3
+++
<![end-metadata]-->
# Deploying a registry server
You need to [install Docker version 1.6.0 or newer](https://docs.docker.com/installation/).
## Running on localhost
Start your registry:
docker run -d -p 5000:5000 --restart=always --name registry registry:2
You can now use it with docker.
Get any image from the hub and tag it to point to your registry:
docker pull ubuntu && docker tag ubuntu localhost:5000/ubuntu
... then push it to your registry:
docker push localhost:5000/ubuntu
... then pull it back from your registry:
docker pull localhost:5000/ubuntu
To stop your registry, you would:
docker stop registry && docker rm -v registry
## Storage
By default, your registry data is persisted as a [docker volume](https://docs.docker.com/userguide/dockervolumes/) on the host filesystem. Properly understanding volumes is essential if you want to stick with a local filesystem storage.
Specifically, you might want to point your volume location to a specific place in order to more easily access your registry data. To do so you can:
docker run -d -p 5000:5000 --restart=always --name registry \
-v `pwd`/data:/var/lib/registry \
registry:2
### Alternatives
You should usually consider using [another storage backend](https://github.com/docker/distribution/blob/master/docs/storagedrivers.md) instead of the local filesystem. Use the [storage configuration options](https://github.com/docker/distribution/blob/master/docs/configuration.md#storage) to configure an alternate storage backend.
Using one of these will allow you to more easily scale your registry, and leverage your storage redundancy and availability features.
## Running a domain registry
While running on `localhost` has its uses, most people want their registry to be more widely available. To do so, the Docker engine requires you to secure it using TLS, which is conceptually very similar to configuring your web server with SSL.
### Get a certificate
Assuming that you own the domain `myregistrydomain.com`, and that its DNS record points to the host where you are running your registry, you first need to get a certificate from a CA.
Create a `certs` directory:
mkdir -p certs
Then move and/or rename your crt file to: `certs/domain.crt`, and your key file to: `certs/domain.key`.
Make sure you stopped your registry from the previous steps, then start your registry again with TLS enabled:
docker run -d -p 5000:5000 --restart=always --name registry \
-v `pwd`/certs:/certs \
-e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/domain.crt \
-e REGISTRY_HTTP_TLS_KEY=/certs/domain.key \
registry:2
You should now be able to access your registry from another docker host:
docker pull ubuntu
docker tag ubuntu myregistrydomain.com:5000/ubuntu
docker push myregistrydomain.com:5000/ubuntu
docker pull myregistrydomain.com:5000/ubuntu
#### Gotcha
A certificate issuer may supply you with an *intermediate* certificate. In this case, you must combine your certificate with the intermediate's to form a *certificate bundle*. You can do this using the `cat` command:
cat domain.crt intermediate-certificates.pem > certs/domain.crt
### Alternatives
While rarely advisable, you may want to use self-signed certificates instead, or use your registry in an insecure fashion. You will find instructions [here](insecure.md).
## Load Balancing Considerations
One may want to use a load balancer to distribute load, terminate TLS or
provide high availability. While a full load balancing setup is outside the
scope of this document, there are a few considerations that can make the process
smoother.
The most important aspect is that a load balanced cluster of registries must
share the same resources. For the current version of the registry, this means
the following must be the same:
- Storage Driver
- HTTP Secret
- Redis Cache (if configured)
If any of these are different, the registry will have trouble serving requests.
As an example, if you're using the filesystem driver, all registry instances
must have access to the same filesystem root, which means they should be in
the same machine. For other drivers, such as s3 or azure, they should be
accessing the same resource, and will likely share an identical configuration.
The _HTTP Secret_ coordinates uploads, so also must be the same across
instances. Configuring different redis instances will work (at the time
of writing), but will not be optimal if the instances are not shared, causing
more requests to be directed to the backend.
Getting the headers correct is very important. For all responses to any
request under the "/v2/" url space, the `Docker-Distribution-API-Version`
header should be set to the value "registry/2.0", even for a 4xx response.
This header allows the docker engine to quickly resolve authentication realms
and fallback to version 1 registries, if necessary. Confirming this is setup
correctly can help avoid problems with fallback.
In the same train of thought, you must make sure you are properly sending the
`X-Forwarded-Proto`, `X-Forwarded-For` and `Host` headers to their "client-side"
values. Failure to do so usually makes the registry issue redirects to internal
hostnames or downgrading from https to http.
A properly secured registry should return 401 when the "/v2/" endpoint is hit
without credentials. The response should include a `WWW-Authenticate`
challenge, providing guidance on how to authenticate, such as with basic auth
or a token service. If the load balancer has health checks, it is recommended
to configure it to consider a 401 response as healthy and any other as down.
This will secure your registry by ensuring that configuration problems with
authentication don't accidentally expose an unprotected registry. If you're
using a less sophisticated load balancer, such as Amazon's Elastic Load
Balancer, that doesn't allow one to change the healthy response code, health
checks can be directed at "/", which will always return a `200 OK` response.
## Restricting access
Except for registries running on secure local networks, registries should always implement access restrictions.
### Native basic auth
The simplest way to achieve access restriction is through basic authentication (this is very similar to other web servers' basic authentication mechanism).
> **Warning**: You **cannot** use authentication with an insecure registry. You have to [configure TLS first](#running-a-domain-registry) for this to work.
First create a password file with one entry for the user "testuser", with password "testpassword":
mkdir auth
docker run --entrypoint htpasswd registry:2 -Bbn testuser testpassword > auth/htpasswd
Make sure you stopped your registry from the previous step, then start it again:
docker run -d -p 5000:5000 --restart=always --name registry \
-v `pwd`/auth:/auth \
-e "REGISTRY_AUTH=htpasswd" \
-e "REGISTRY_AUTH_HTPASSWD_REALM=Registry Realm" \
-e REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd \
-v `pwd`/certs:/certs \
-e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/domain.crt \
-e REGISTRY_HTTP_TLS_KEY=/certs/domain.key \
registry:2
You should now be able to:
docker login myregistrydomain.com:5000
And then push and pull images as an authenticated user.
#### Gotcha
Seeing X509 errors is usually a sign you are trying to use self-signed certificates, and failed to [configure your docker daemon properly](insecure.md).
### Alternatives
1. You may want to leverage more advanced basic auth implementations through a proxy design, in front of the registry. You will find examples of such patterns in the [recipes list](recipes.md).
2. Alternatively, the Registry also supports delegated authentication, redirecting users to a specific, trusted token server. That approach requires significantly more investment, and only makes sense if you want to fully configure ACLs and more control over the Registry integration into your global authorization and authentication systems.
You will find [background information here](spec/auth/token.md), and [configuration information here](configuration.md#auth).
Beware that you will have to implement your own authentication service for this to work, or leverage a third-party implementation.
## Managing with Compose
As your registry configuration grows more complex, dealing with it can quickly become tedious.
It's highly recommended to use [Docker Compose](https://docs.docker.com/compose/) to facilitate operating your registry.
Here is a simple `docker-compose.yml` example that condenses everything explained so far:
```
registry:
restart: always
image: registry:2
ports:
- 5000:5000
environment:
REGISTRY_HTTP_TLS_CERTIFICATE: /certs/domain.crt
REGISTRY_HTTP_TLS_KEY: /certs/domain.key
REGISTRY_AUTH: htpasswd
REGISTRY_AUTH_HTPASSWD_PATH: /auth/htpasswd
REGISTRY_AUTH_HTPASSWD_REALM: Registry Realm
volumes:
- /path/data:/var/lib/registry
- /path/certs:/certs
- /path/auth:/auth
```
> **Warning**: replace `/path` by whatever directory that holds your `certs` and `auth` folder from above.
You can then start your registry with a simple
docker-compose up -d
## Next
You will find more specific and advanced informations in the following sections:
- [Configuration reference](configuration.md)
- [Working with notifications](notifications.md)
- [Advanced "recipes"](recipes.md)
- [Registry API](spec/api.md)
- [Storage driver model](storagedrivers.md)
- [Token authentication](spec/auth/token.md)

View File

@@ -1,70 +0,0 @@
<!--[metadata]>
+++
draft = true
+++
<![end-metadata]-->
# Glossary
This page contains definitions for distribution related terms.
<dl>
<dt id="blob"><h4>Blob</h4></dt>
<dd>
<blockquote>A blob is any kind of content that is stored by a Registry under a content-addressable identifier (a "digest").</blockquote>
<p>
<a href="#layer">Layers</a> are a good example of "blobs".
</p>
</dd>
<dt id="image"><h4>Image</h4></dt>
<dd>
<blockquote>An image is a named set of immutable data from which a Docker container can be created.</blockquote>
<p>
An image is represented by a json file called a <a href="#manifest">manifest</a>, and is conceptually a set of <a hred="#layer">layers</a>.
Image names indicate the location where they can be pulled from and pushed to, as they usually start with a <a href="#registry">registry</a> domain name and port.
</p>
</dd>
<dt id="layer"><h4>Layer</h4></dt>
<dd>
<blockquote>A layer is a tar archive bundling partial content from a filesystem.</blockquote>
<p>
Layers from an <a href="#image">image</a> are usually extracted in order on top of each other to make up a root filesystem from which containers run out.
</p>
</dd>
<dt id="manifest"><h4>Manifest</h4></dt>
<dd><blockquote>A manifest is the JSON representation of an image.</blockquote></dd>
<dt id="namespace"><h4>Namespace</h4></dt>
<dd><blockquote>A namespace is a collection of repositories with a common name prefix.</blockquote>
<p>
The namespace with an empty prefix is considered the Global Namespace.
</p>
</dd>
<dt id="registry"><h4>Registry</h4></dt>
<dd><blockquote>A registry is a service that let you store and deliver <a href="#images">images</a>.</blockquote>
</dd>
<dt id="registry"><h4>Repository</h4></dt>
<dd>
<blockquote>A repository is a set of data containing all versions of a given image.</blockquote>
</dd>
<dt id="scope"><h4>Scope</h4></dt>
<dd><blockquote>A scope is the portion of a namespace onto which a given authorization token is granted.</blockquote></dd>
<dt id="tag"><h4>Tag</h4></dt>
<dd><blockquote>A tag is conceptually a "version" of a <a href="#image">named image</a>.</blockquote>
<p>
Example: `docker pull myimage:latest` instructs docker to pull the image "myimage" in version "latest".
</p>
</dd>
</dl>

View File

@@ -1,24 +0,0 @@
<!--[metadata]>
+++
title = "Getting help"
description = "Getting help with the Registry"
keywords = ["registry, on-prem, images, tags, repository, distribution, help, 101, TL;DR"]
[menu.main]
parent="smn_registry"
weight=9
+++
<![end-metadata]-->
# Getting help
If you need help, or just want to chat, you can reach us:
- on irc: `#docker-distribution` on freenode
- on the [mailing list](https://groups.google.com/a/dockerproject.org/forum/#!forum/distribution) (mail at <distribution@dockerproject.org>)
If you want to report a bug:
- be sure to first read about [how to contribute](https://github.com/docker/distribution/blob/master/CONTRIBUTING.md)
- you can then do so on the [GitHub project bugtracker](https://github.com/docker/distribution/issues)
You can also find out more about the Docker's project [Getting Help resources](https://docs.docker.com/opensource/get-help/).

File diff suppressed because one or more lines are too long

Binary file not shown.

Before

Width:  |  Height:  |  Size: 37 KiB

File diff suppressed because one or more lines are too long

Before

Width:  |  Height:  |  Size: 31 KiB

View File

@@ -1,66 +0,0 @@
<!--[metadata]>
+++
title = "Docker Registry"
description = "High-level overview of the Registry"
keywords = ["registry, on-prem, images, tags, repository, distribution"]
[menu.main]
parent="smn_registry"
+++
<![end-metadata]-->
# Docker Registry
## What it is
The Registry is a stateless, highly scalable server side application that stores and lets you distribute Docker images.
The Registry is open-source, under the permissive [Apache license](http://en.wikipedia.org/wiki/Apache_License).
## Why use it
You should use the Registry if you want to:
* tightly control where your images are being stored
* fully own your images distribution pipeline
* integrate image storage and distribution tightly into your in-house development workflow
## Alternatives
Users looking for a zero maintenance, ready-to-go solution are encouraged to head-over to the [Docker Hub](https://hub.docker.com), which provides a free-to-use, hosted Registry, plus additional features (organization accounts, automated builds, and more).
Users looking for a commercially supported version of the Registry should look into [Docker Trusted Registry](https://docs.docker.com/docker-trusted-registry/).
## Requirements
The Registry is compatible with Docker engine **version 1.6.0 or higher**.
If you really need to work with older Docker versions, you should look into the [old python registry](https://github.com/docker/docker-registry).
## TL;DR
Start your registry
docker run -d -p 5000:5000 --name registry registry:2
Pull (or build) some image from the hub
docker pull ubuntu
Tag the image so that it points to your registry
docker tag ubuntu localhost:5000/myfirstimage
Push it
docker push localhost:5000/myfirstimage
Pull it back
docker pull localhost:5000/myfirstimage
Now stop your registry and remove all data
docker stop registry && docker rm -v registry
## Next
You should now read the [detailed introduction about the registry](introduction.md), or jump directly to [deployment instructions](deploying.md).

View File

@@ -1,90 +0,0 @@
<!--[metadata]>
+++
title = "Testing an insecure registry"
description = "Deploying a Registry in an insecure fashion"
keywords = ["registry, on-prem, images, tags, repository, distribution, insecure"]
+++
<![end-metadata]-->
# Insecure Registry
While it's highly recommended to secure your registry using a TLS certificate issued by a known CA, you may alternatively decide to use self-signed certificates, or even use your registry over plain http.
You have to understand the downsides in doing so, and the extra burden in configuration.
## Deploying a plain HTTP registry
> **Warning**: it's not possible to use an insecure registry with basic authentication
This basically tells Docker to entirely disregard security for your registry.
1. edit the file `/etc/default/docker` so that there is a line that reads: `DOCKER_OPTS="--insecure-registry myregistrydomain.com:5000"` (or add that to existing `DOCKER_OPTS`)
2. restart your Docker daemon: on ubuntu, this is usually `service docker stop && service docker start`
**Pros:**
- relatively easy to configure
**Cons:**
- this is **very** insecure: you are basically exposing yourself to trivial MITM, and this solution should only be used for isolated testing or in a tightly controlled, air-gapped environment
- you have to configure every docker daemon that wants to access your registry
## Using self-signed certificates
> **Warning**: using this along with basic authentication requires to **also** trust the certificate into the OS cert store for some versions of docker (see below)
Generate your own certificate:
mkdir -p certs && openssl req \
-newkey rsa:4096 -nodes -sha256 -keyout certs/domain.key \
-x509 -days 365 -out certs/domain.crt
Be sure to use the name `myregistrydomain.com` as a CN.
Use the result to [start your registry with TLS enabled](https://github.com/docker/distribution/blob/master/docs/deploying.md#get-a-certificate)
Then you have to instruct every docker daemon to trust that certificate. This is done by copying the `domain.crt` file to `/etc/docker/certs.d/myregistrydomain.com:5000/ca.crt`.
Don't forget to restart docker after doing so.
**Pros:**
- more secure than the insecure registry solution
**Cons:**
- you have to configure every docker daemon that wants to access your registry
## Failing...
Failing to configure docker and trying to pull from a registry that is not using TLS will result in the following message:
```
FATA[0000] Error response from daemon: v1 ping attempt failed with error:
Get https://myregistrydomain.com:5000/v1/_ping: tls: oversized record received with length 20527.
If this private registry supports only HTTP or HTTPS with an unknown CA certificate,please add
`--insecure-registry myregistrydomain.com:5000` to the daemon's arguments.
In the case of HTTPS, if you have access to the registry's CA certificate, no need for the flag;
simply place the CA certificate at /etc/docker/certs.d/myregistrydomain.com:5000/ca.crt
```
## Docker still complains about the certificate when using authentication?
When using authentication, some versions of docker also require you to trust the certificate at the OS level.
Usually, on Ubuntu this is done with:
cp auth/domain.crt /usr/local/share/ca-certificates/myregistrydomain.com.crt
update-ca-certificates
... and on Red Hat (and its derivatives) with:
cp auth/domain.crt /etc/pki/ca-trust/source/anchors/myregistrydomain.com.crt
update-ca-trust
... On some distributions, e.g. Oracle Linux 6, the Shared System Certificates feature needs to be manually enabled:
update-ca-trust enable
Now restart docker (`service docker stop && service docker start`, or any other way you use to restart docker).

View File

@@ -1,55 +0,0 @@
<!--[metadata]>
+++
title = "Understanding the Registry"
description = "Explains what the Registry is, basic use cases and requirements"
keywords = ["registry, on-prem, images, tags, repository, distribution, use cases, requirements"]
[menu.main]
parent="smn_registry"
weight=2
+++
<![end-metadata]-->
# Understanding the Registry
A registry is a storage and content delivery system, holding named Docker images, available in different tagged versions.
> Example: the image `distribution/registry`, with tags `2.0` and `2.1`.
Users interact with a registry by using docker push and pull commands.
> Example: `docker pull registry-1.docker.io/distribution/registry:2.1`.
Storage itself is delegated to drivers. The default storage driver is the local posix filesystem, which is suitable for development or small deployments. Additional cloud-based storage drivers like S3, Microsoft Azure, Ceph Rados, OpenStack Swift and Aliyun OSS are also supported. People looking into using other storage backends may do so by writing their own driver implementing the [Storage API](storagedrivers.md).
Since securing access to your hosted images is paramount, the Registry natively supports TLS and basic authentication.
The Registry GitHub repository includes additional information about advanced authentication and authorization methods. Only very large or public deployments are expected to extend the Registry in this way.
Finally, the Registry ships with a robust [notification system](notifications.md), calling webhooks in response to activity, and both extensive logging and reporting, mostly useful for large installations that want to collect metrics.
## Understanding image naming
Image names as used in typical docker commands reflect their origin:
* `docker pull ubuntu` instructs docker to pull an image named `ubuntu` from the official Docker Hub. This is simply a shortcut for the longer `docker pull docker.io/library/ubuntu` command
* `docker pull myregistrydomain:port/foo/bar` instructs docker to contact the registry located at `myregistrydomain:port` to find the image `foo/bar`
You can find out more about the various Docker commands dealing with images in the [official Docker engine documentation](https://docs.docker.com/reference/commandline/cli/).
## Use cases
Running your own Registry is a great solution to integrate with and complement your CI/CD system. In a typical workflow, a commit to your source revision control system would trigger a build on your CI system, which would then push a new image to your Registry if the build is successful. A notification from the Registry would then trigger a deployment on a staging environment, or notify other systems that a new image is available.
It's also an essential component if you want to quickly deploy a new image over a large cluster of machines.
Finally, it's the best way to distribute images inside an isolated network.
## Requirements
You absolutely need to be familiar with Docker, specifically with regard to pushing and pulling images. You must understand the difference between the daemon and the cli, and at least grasp basic concepts about networking.
Also, while just starting a registry is fairly easy, operating it in a production environment requires operational skills, just like any other service. You are expected to be familiar with systems availability and scalability, logging and log processing, systems monitoring, and security 101. Strong understanding of http and overall network communications, plus familiarity with golang are certainly useful as well for advanced operations or hacking.
## Next
Dive into [deploying your registry](deploying.md)

View File

@@ -1,30 +0,0 @@
<!--[metadata]>
+++
draft = true
+++
<![end-metadata]-->
# Migrating a 1.0 registry to 2.0
TODO: This needs to be revised in light of Olivier's work
A few thoughts here:
There was no "1.0". There was an implementation of the Registry API V1 but only a version 0.9 of the service was released.
The image formats are not compatible in any way. One must convert v1 images to v2 images using a docker client or other tool.
One can migrate images from one version to the other by pulling images from the old registry and pushing them to the v2 registry.
-----
The Docker Registry 2.0 is backward compatible with images created by the earlier specification. If you are migrating a private registry to version 2.0, you should use the following process:
1. Configure and test a 2.0 registry image in a sandbox environment.
2. Back up up your production image storage.
Your production image storage should reside on a volume or storage backend.
Make sure you have a backup of its contents.
3. Stop your existing registry service.
4. Restart your registry with your tested 2.0 image.

View File

@@ -1,72 +0,0 @@
<!--[metadata]>
+++
title = "Mirroring Docker Hub"
description = "Setting-up a local mirror for Docker Hub images"
keywords = ["registry, on-prem, images, tags, repository, distribution, mirror, Hub, recipe, advanced"]
+++
<![end-metadata]-->
# Registry as a pull through cache
## Use-case
If you have multiple instances of Docker running in your environment (e.g., multiple physical or virtual machines, all running the Docker daemon), each time one of them requires an image that it doesnt have it will go out to the internet and fetch it from the public Docker registry. By running a local registry mirror, you can keep most of the redundant image fetch traffic on your local network.
### Alternatives
Alternatively, if the set of images you are using is well delimited, you can simply pull them manually and push them to a simple, local, private registry.
Furthermore, if your images are all built in-house, not using the Hub at all and relying entirely on your local registry is the simplest scenario.
### Gotcha
It's currently not possible to mirror another private registry. Only the central Hub can be mirrored.
### Solution
The Registry can be configured as a pull through cache. In this mode a Registry responds to all normal docker pull requests but stores all content locally.
## How does it work?
The first time you request an image from your local registry mirror, it pulls the image from the public Docker registry and stores it locally before handing it back to you. On subsequent requests, the local registry mirror is able to serve the image from its own storage.
### What if the content changes on the Hub?
When a pull is attempted with a tag, the Registry will check the remote to ensure if it has the latest version of the requested content. If it doesn't it will fetch the latest content and cache it.
### What about my disk?
In environments with high churn rates, stale data can build up in the cache. When running as a pull through cache the Registry will periodically remove old content to save disk space. Subsequent requests for removed content will cause a remote fetch and local re-caching.
To ensure best performance and guarantee correctness the Registry cache should be configured to use the `filesystem` driver for storage.
## Running a Registry as a pull through cache
The easiest way to run a registry as a pull through cache is to run the official Registry image.
Multiple registry caches can be deployed over the same back-end. A single registry cache will ensure that concurrent requests do not pull duplicate data, but this property will not hold true for a registry cache cluster.
### Configuring the cache
To configure a Registry to run as a pull through cache, the addition of a `proxy` section is required to the config file.
In order to access private images on the Docker Hub, a username and password can be supplied.
proxy:
remoteurl: https://registry-1.docker.io
username: [username]
password: [password]
> :warn: if you specify a username and password, it's very important to understand that private resources that this user has access to on the Hub will be made available on your mirror. It's thus paramount that you secure your mirror by implementing authentication if you expect these resources to stay private!
### Configuring the Docker daemon
You will need to pass the `--registry-mirror` option to your Docker daemon on startup:
docker --registry-mirror=https://<my-docker-mirror-host> daemon
For example, if your mirror is serving on http://10.0.0.2:5000, you would run:
docker --registry-mirror=https://10.0.0.2:5000 daemon
NOTE: Depending on your local host setup, you may be able to add the `--registry-mirror` option to the `DOCKER_OPTS` variable in `/etc/default/docker`.

View File

@@ -1,188 +0,0 @@
<!--[metadata]>
+++
title = "Authenticating proxy with nginx"
description = "Restricting access to your registry using a nginx proxy"
keywords = ["registry, on-prem, images, tags, repository, distribution, nginx, proxy, authentication, TLS, recipe, advanced"]
+++
<![end-metadata]-->
# Authenticating proxy with nginx
## Use-case
People already relying on a nginx proxy to authenticate their users to other services might want to leverage it and have Registry communications tunneled through the same pipeline.
Usually, that includes enterprise setups using LDAP/AD on the backend and a SSO mechanism fronting their internal http portal.
### Alternatives
If you just want authentication for your registry, and are happy maintaining users access separately, you should really consider sticking with the native [basic auth registry feature](deploying.md#native-basic-auth).
### Solution
With the method presented here, you implement basic authentication for docker engines in a reverse proxy that sits in front of your registry.
While we use a simple htpasswd file as an example, any other nginx authentication backend should be fairly easy to implement once you are done with the example.
We also implement push restriction (to a limited user group) for the sake of the example. Again, you should modify this to fit your mileage.
### Gotchas
While this model gives you the ability to use whatever authentication backend you want through the secondary authentication mechanism implemented inside your proxy, it also requires that you move TLS termination from the Registry to the proxy itself.
Furthermore, introducing an extra http layer in your communication pipeline will make it more complex to deploy, maintain, and debug, and will possibly create issues. Make sure the extra complexity is required.
For instance, Amazon's Elastic Load Balancer (ELB) in HTTPS mode already sets the following client header:
```
X-Real-IP
X-Forwarded-For
X-Forwarded-Proto
```
So if you have an nginx sitting behind it, should remove these lines from the example config below:
```
X-Real-IP $remote_addr; # pass on real client's IP
X-Forwarded-For $proxy_add_x_forwarded_for;
X-Forwarded-Proto $scheme;
```
Otherwise nginx will reset the ELB's values, and the requests will not be routed properly. For more information, see [#970](https://github.com/docker/distribution/issues/970).
## Setting things up
Read again [the requirements](recipes.md#requirements).
Ready?
--
Create the required directories
```
mkdir -p auth
mkdir -p data
```
Create the main nginx configuration you will use.
```
cat <<EOF > auth/nginx.conf
events {
worker_connections 1024;
}
http {
upstream docker-registry {
server registry:5000;
}
## Set a variable to help us decide if we need to add the
## 'Docker-Distribution-Api-Version' header.
## The registry always sets this header.
## In the case of nginx performing auth, the header will be unset
## since nginx is auth-ing before proxying.
map \$upstream_http_docker_distribution_api_version \$docker_distribution_api_version {
'registry/2.0' '';
default registry/2.0;
}
server {
listen 443 ssl;
server_name myregistrydomain.com;
# SSL
ssl_certificate /etc/nginx/conf.d/domain.crt;
ssl_certificate_key /etc/nginx/conf.d/domain.key;
# Recommendations from https://raymii.org/s/tutorials/Strong_SSL_Security_On_nginx.html
ssl_protocols TLSv1.1 TLSv1.2;
ssl_ciphers 'EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH';
ssl_prefer_server_ciphers on;
ssl_session_cache shared:SSL:10m;
# disable any limits to avoid HTTP 413 for large image uploads
client_max_body_size 0;
# required to avoid HTTP 411: see Issue #1486 (https://github.com/docker/docker/issues/1486)
chunked_transfer_encoding on;
location /v2/ {
# Do not allow connections from docker 1.5 and earlier
# docker pre-1.6.0 did not properly set the user agent on ping, catch "Go *" user agents
if (\$http_user_agent ~ "^(docker\/1\.(3|4|5(?!\.[0-9]-dev))|Go ).*\$" ) {
return 404;
}
# To add basic authentication to v2 use auth_basic setting.
auth_basic "Registry realm";
auth_basic_user_file /etc/nginx/conf.d/nginx.htpasswd;
## If $docker_distribution_api_version is empty, the header will not be added.
## See the map directive above where this variable is defined.
add_header 'Docker-Distribution-Api-Version' \$docker_distribution_api_version always;
proxy_pass http://docker-registry;
proxy_set_header Host \$http_host; # required for docker client's sake
proxy_set_header X-Real-IP \$remote_addr; # pass on real client's IP
proxy_set_header X-Forwarded-For \$proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto \$scheme;
proxy_read_timeout 900;
}
}
}
EOF
```
Now create a password file for "testuser" and "testpassword"
```
docker run --rm --entrypoint htpasswd registry:2 -bn testuser testpassword > auth/nginx.htpasswd
```
Copy over your certificate files
```
cp domain.crt auth
cp domain.key auth
```
Now create your compose file
```
cat <<EOF > docker-compose.yml
nginx:
image: "nginx:1.9"
ports:
- 5043:443
links:
- registry:registry
volumes:
- ./auth:/etc/nginx/conf.d
- ./auth/nginx.conf:/etc/nginx/nginx.conf:ro
registry:
image: registry:2
ports:
- 127.0.0.1:5000:5000
volumes:
- `pwd`./data:/var/lib/registry
EOF
```
## Starting and stopping
Now, start your stack:
docker-compose up -d
Login with a "push" authorized user (using `testuser` and `testpassword`), then tag and push your first image:
docker login -p=testuser -u=testpassword -e=root@example.ch myregistrydomain.com:5043
docker tag ubuntu myregistrydomain.com:5043/test
docker push myregistrydomain.com:5043/test
docker pull myregistrydomain.com:5043/test

View File

@@ -1,320 +0,0 @@
<!--[metadata]>
+++
title = "Working with notifications"
description = "Explains how to work with registry notifications"
keywords = ["registry, on-prem, images, tags, repository, distribution, notifications, advanced"]
[menu.main]
parent="smn_registry"
weight=5
+++
<![end-metadata]-->
# Notifications
The Registry supports sending webhook notifications in response to events
happening within the registry. Notifications are sent in response to manifest
pushes and pulls and layer pushes and pulls. These actions are serialized into
events. The events are queued into a registry-internal broadcast system which
queues and dispatches events to [_Endpoints_](#endpoints).
![](images/notifications.png)
## Endpoints
Notifications are sent to _endpoints_ via HTTP requests. Each configured
endpoint has isolated queues, retry configuration and http targets within each
instance of a registry. When an action happens within the registry, it is
converted into an event which is dropped into an inmemory queue. When the
event reaches the end of the queue, an http request is made to the endpoint
until the request succeeds. The events are sent serially to each endpoint but
order is not guaranteed.
## Configuration
To setup a registry instance to send notifications to endpoints, one must add
them to the configuration. A simple example follows:
notifications:
endpoints:
- name: alistener
url: https://mylistener.example.com/event
headers:
Authorization: [Bearer <your token, if needed>]
timeout: 500ms
threshold: 5
backoff: 1s
The above would configure the registry with an endpoint to send events to
`https://mylistener.example.com/event`, with the header "Authorization: Bearer
<your token, if needed>". The request would timeout after 500 milliseconds. If
5 failures happen consecutively, the registry will backoff for 1 second before
trying again.
For details on the fields, please see the [configuration documentation](configuration.md#notifications).
A properly configured endpoint should lead to a log message from the registry
upon startup:
```
INFO[0000] configuring endpoint alistener (https://mylistener.example.com/event), timeout=500ms, headers=map[Authorization:[Bearer <your token if needed>]] app.id=812bfeb2-62d6-43cf-b0c6-152f541618a3 environment=development service=registry
```
## Events
Events have a well-defined JSON structure and are sent as the body of
notification requests. One or more events are sent in a structure called an
envelope. Each event has a unique id that can be used to uniquely identify incoming
requests, if required. Along with that, an _action_ is provided with a
_target, identifying the object mutated during the event.
The fields available in an event are described in detail in the
[godoc](http://godoc.org/github.com/docker/distribution/notifications#Event).
**TODO:** Let's break out the fields here rather than rely on the godoc.
The following is an example of a JSON event, sent in response to the push of a
manifest:
```json
{
"id": "asdf-asdf-asdf-asdf-0",
"timestamp": "2006-01-02T15:04:05Z",
"action": "push",
"target": {
"mediaType": "application/vnd.docker.distribution.manifest.v1+json",
"size": 1,
"digest": "sha256:0123456789abcdef0",
"length": 1,
"repository": "library/test",
"url": "http://example.com/v2/library/test/manifests/latest"
},
"request": {
"id": "asdfasdf",
"addr": "client.local",
"host": "registrycluster.local",
"method": "PUT",
"useragent": "test/0.1"
},
"actor": {
"name": "test-actor"
},
"source": {
"addr": "hostname.local:port"
}
}
```
> __NOTE:__ As of version 2.1, the `length` field for event targets
> is being deprecated for the `size` field, bringing the target in line with
> common nomenclature. Both will continue to be set for the foreseeable
> future. Newer code should favor `size` but accept either.
## Envelope
The envelope contains one or more events, with the following json structure:
```json
{
"events": [ ... ],
}
```
While events may be sent in the same envelope, the set of events within that
envelope have no implied relationship. For example, the registry may choose to
group unrelated events and send them in the same envelope to reduce the total
number of requests.
The full package has the mediatype
"application/vnd.docker.distribution.events.v1+json", which will be set on the
request coming to an endpoint.
An example of a full event may look as follows:
```json
GET /callback
Host: application/vnd.docker.distribution.events.v1+json
Authorization: Bearer <your token, if needed>
Content-Type: application/vnd.docker.distribution.events.v1+json
{
"events": [
{
"id": "asdf-asdf-asdf-asdf-0",
"timestamp": "2006-01-02T15:04:05Z",
"action": "push",
"target": {
"mediaType": "application/vnd.docker.distribution.manifest.v1+json",
"length": 1,
"digest": "sha256:0123456789abcdef0",
"repository": "library/test",
"url": "http://example.com/v2/library/test/manifests/latest"
},
"request": {
"id": "asdfasdf",
"addr": "client.local",
"host": "registrycluster.local",
"method": "PUT",
"useragent": "test/0.1"
},
"actor": {
"name": "test-actor"
},
"source": {
"addr": "hostname.local:port"
}
},
{
"id": "asdf-asdf-asdf-asdf-1",
"timestamp": "2006-01-02T15:04:05Z",
"action": "push",
"target": {
"mediaType": "application/vnd.docker.container.image.rootfs.diff+x-gtar",
"length": 2,
"digest": "sha256:3b3692957d439ac1928219a83fac91e7bf96c153725526874673ae1f2023f8d5",
"repository": "library/test",
"url": "http://example.com/v2/library/test/manifests/latest"
},
"request": {
"id": "asdfasdf",
"addr": "client.local",
"host": "registrycluster.local",
"method": "PUT",
"useragent": "test/0.1"
},
"actor": {
"name": "test-actor"
},
"source": {
"addr": "hostname.local:port"
}
},
{
"id": "asdf-asdf-asdf-asdf-2",
"timestamp": "2006-01-02T15:04:05Z",
"action": "push",
"target": {
"mediaType": "application/vnd.docker.container.image.rootfs.diff+x-gtar",
"length": 3,
"digest": "sha256:3b3692957d439ac1928219a83fac91e7bf96c153725526874673ae1f2023f8d6",
"repository": "library/test",
"url": "http://example.com/v2/library/test/manifests/latest"
},
"request": {
"id": "asdfasdf",
"addr": "client.local",
"host": "registrycluster.local",
"method": "PUT",
"useragent": "test/0.1"
},
"actor": {
"name": "test-actor"
},
"source": {
"addr": "hostname.local:port"
}
}
]
}
```
## Responses
The registry is fairly accepting of the response codes from endpoints. If an
endpoint responds with any 2xx or 3xx response code (after following
redirects), the message will be considered delivered and discarded.
In turn, it is recommended that endpoints are accepting of incoming responses,
as well. While the format of event envelopes are standardized by media type,
any "pickyness" about validation may cause the queue to backup on the
registry.
## Monitoring
The state of the endpoints are reported via the debug/vars http interface,
usually configured to `http://localhost:5001/debug/vars`. Information such as
configuration and metrics are available by endpoint.
The following provides an example of a few endpoints that have experienced
several failures and have since recovered:
```json
"notifications":{
"endpoints":[
{
"name":"local-5003",
"url":"http://localhost:5003/callback",
"Headers":{
"Authorization":[
"Bearer \u003can example token\u003e"
]
},
"Timeout":1000000000,
"Threshold":10,
"Backoff":1000000000,
"Metrics":{
"Pending":76,
"Events":76,
"Successes":0,
"Failures":0,
"Errors":46,
"Statuses":{
}
}
},
{
"name":"local-8083",
"url":"http://localhost:8083/callback",
"Headers":null,
"Timeout":1000000000,
"Threshold":10,
"Backoff":1000000000,
"Metrics":{
"Pending":0,
"Events":76,
"Successes":76,
"Failures":0,
"Errors":28,
"Statuses":{
"202 Accepted":76
}
}
}
]
}
```
If using notification as part of a larger application, it is _critical_ to
monitor the size ("Pending" above) of the endpoint queues. If failures or
queue sizes are increasing, it can indicate a larger problem.
The logs are also a valuable resource for monitoring problems. A failing
endpoint will lead to messages similar to the following:
```
ERRO[0340] retryingsink: error writing events: httpSink{http://localhost:5003/callback}: error posting: Post http://localhost:5003/callback: dial tcp 127.0.0.1:5003: connection refused, retrying
WARN[0340] httpSink{http://localhost:5003/callback} encountered too many errors, backing off
```
The above indicates that several errors have led to a backoff and the registry
will wait before retrying.
## Considerations
Currently, the queues are inmemory, so endpoints should be _reasonably
reliable_. They are designed to make a best-effort to send the messages but if
an instance is lost, messages may be dropped. If an endpoint goes down, care
should be taken to ensure that the registry instance is not terminated before
the endpoint comes back up or messages will be lost.
This can be mitigated by running endpoints in close proximity to the registry
instances. One could run an endpoint that pages to disk and then forwards a
request to provide better durability.
The notification system is designed around a series of interchangeable _sinks_
which can be wired up to achieve interesting behavior. If this system doesn't
provide acceptable guarantees, adding a transactional `Sink` to the registry
is a possibility, although it may have an effect on request service time.
Please see the
[godoc](http://godoc.org/github.com/docker/distribution/notifications#Sink)
for more information.

View File

@@ -1,79 +0,0 @@
<!--[metadata]>
+++
title = "Running on OS X"
description = "Explains how to run a registry on OS X"
keywords = ["registry, on-prem, images, tags, repository, distribution, OS X, recipe, advanced"]
+++
<![end-metadata]-->
# OS X Setup Guide
## Use-case
This is useful if you intend to run a registry server natively on OS X.
### Alternatives
You can start a VM on OS X, and deploy your registry normally as a container using Docker inside that VM.
The simplest road to get there is traditionally to use the [docker Toolbox](https://www.docker.com/toolbox), or [docker-machine](https://docs.docker.com/machine/), which usually relies on the [boot2docker](http://boot2docker.io/) iso inside a VirtualBox VM.
### Solution
Using the method described here, you install and compile your own from the git repository and run it as an OS X agent.
### Gotchas
Production services operation on OS X is out of scope of this document. Be sure you understand well these aspects before considering going to production with this.
## Setup golang on your machine
If you know, safely skip to the next section.
If you don't, the TLDR is:
bash < <(curl -s -S -L https://raw.githubusercontent.com/moovweb/gvm/master/binscripts/gvm-installer)
source ~/.gvm/scripts/gvm
gvm install go1.4.2
gvm use go1.4.2
If you want to understand, you should read [How to Write Go Code](https://golang.org/doc/code.html).
## Checkout the Docker Distribution source tree
mkdir -p $GOPATH/src/github.com/docker
git clone https://github.com/docker/distribution.git $GOPATH/src/github.com/docker/distribution
cd $GOPATH/src/github.com/docker/distribution
## Build the binary
GOPATH=$(PWD)/Godeps/_workspace:$GOPATH make binaries
sudo cp bin/registry /usr/local/libexec/registry
## Setup
Copy the registry configuration file in place:
mkdir /Users/Shared/Registry
cp docs/osx/config.yml /Users/Shared/Registry/config.yml
## Running the Docker Registry under launchd
Copy the Docker registry plist into place:
plutil -lint docs/osx/com.docker.registry.plist
cp docs/osx/com.docker.registry.plist ~/Library/LaunchAgents/
chmod 644 ~/Library/LaunchAgents/com.docker.registry.plist
Start the Docker registry:
launchctl load ~/Library/LaunchAgents/com.docker.registry.plist
### Restarting the docker registry service
launchctl stop com.docker.registry
launchctl start com.docker.registry
### Unloading the docker registry service
launchctl unload ~/Library/LaunchAgents/com.docker.registry.plist

View File

@@ -1,42 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key>
<string>com.docker.registry</string>
<key>KeepAlive</key>
<true/>
<key>StandardErrorPath</key>
<string>/Users/Shared/Registry/registry.log</string>
<key>StandardOutPath</key>
<string>/Users/Shared/Registry/registry.log</string>
<key>Program</key>
<string>/usr/local/libexec/registry</string>
<key>ProgramArguments</key>
<array>
<string>/usr/local/libexec/registry</string>
<string>/Users/Shared/Registry/config.yml</string>
</array>
<key>Sockets</key>
<dict>
<key>http-listen-address</key>
<dict>
<key>SockServiceName</key>
<string>5000</string>
<key>SockType</key>
<string>dgram</string>
<key>SockFamily</key>
<string>IPv4</string>
</dict>
<key>http-debug-address</key>
<dict>
<key>SockServiceName</key>
<string>5001</string>
<key>SockType</key>
<string>dgram</string>
<key>SockFamily</key>
<string>IPv4</string>
</dict>
</dict>
</dict>
</plist>

View File

@@ -1,16 +0,0 @@
version: 0.1
log:
level: info
fields:
service: registry
environment: macbook-air
storage:
cache:
blobdescriptor: inmemory
filesystem:
rootdirectory: /Users/Shared/Registry
http:
addr: 0.0.0.0:5000
secret: mytokensecret
debug:
addr: localhost:5001

View File

@@ -1,35 +0,0 @@
<!--[metadata]>
+++
title = "Recipes"
description = "Fun stuff to do with your registry"
keywords = ["registry, on-prem, images, tags, repository, distribution, recipes, advanced"]
+++
<![end-metadata]-->
# Recipes
You will find here a list of "recipes", end-to-end scenarios for exotic or otherwise advanced use-cases.
Most users are not expected to have a use for these.
## Requirements
You should have followed entirely the basic [deployment guide](deploying.md).
If you have not, please take the time to do so.
At this point, it's assumed that:
* you understand Docker security requirements, and how to configure your docker engines properly
* you have installed Docker Compose
* it's HIGHLY recommended that you get a certificate from a known CA instead of self-signed certificates
* inside the current directory, you have a X509 `domain.crt` and `domain.key`, for the CN `myregistrydomain.com`
* be sure you have stopped and removed any previously running registry (typically `docker stop registry && docker rm -v registry`)
## The List
* [using Apache as an authenticating proxy](apache.md)
* [using Nginx as an authenticating proxy](nginx.md)
* [running a Registry on OS X](osx-setup-guide.md)
* [hacking the registry: build instructions](building.md)
* [mirror the Docker Hub](mirror.md)

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

Some files were not shown because too many files have changed in this diff Show More