github/distribution
1
0
Fork 0
mirror of https://github.com/distribution/distribution.git synced 2025-09-24 12:37:00 +00:00
Code Issues Projects Releases Wiki Activity

docs: add hugo site

Browse Source 
Signed-off-by: David Karlsson <35727626+dvdksn@users.noreply.github.com>
This commit is contained in:
David Karlsson
2023-10-11 11:59:20 +02:00
parent f7b3869062
commit e2ae76f1f2
291 changed files with 2833 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

70
docs/content/storage-drivers/_index.md Normal file
View File

@@ -0,0 +1,70 @@
---
description: Explains how to use storage drivers
keywords: registry, on-prem, images, tags, repository, distribution, storage drivers, advanced
redirect_from:
- /registry/storagedrivers/
title: Registry storage driver
---
This document describes the registry storage driver model, implementation, and explains how to contribute new storage drivers.
## Provided drivers
This storage driver package comes bundled with several drivers:
- [inmemory](inmemory.md): A temporary storage driver using a local inmemory map. This exists solely for reference and testing.
- [filesystem](filesystem.md): A local storage driver configured to use a directory tree in the local filesystem.
- [s3](s3.md): A driver storing objects in an Amazon Simple Storage Service (S3) bucket.
- [azure](azure.md): A driver storing objects in [Microsoft Azure Blob Storage](https://azure.microsoft.com/en-us/services/storage/).
- [gcs](gcs.md): A driver storing objects in a [Google Cloud Storage](https://cloud.google.com/storage/) bucket.
- oss: *NO LONGER SUPPORTED*
- swift: *NO LONGER SUPPORTED*
## Storage driver API
The storage driver API is designed to model a filesystem-like key/value storage in a manner abstract enough to support a range of drivers from the local filesystem to Amazon S3 or other distributed object storage systems.
Storage drivers are required to implement the `storagedriver.StorageDriver` interface provided in `storagedriver.go`, which includes methods for reading, writing, and deleting content, as well as listing child objects of a specified prefix key.
Storage drivers are intended to be written in Go, providing compile-time
validation of the `storagedriver.StorageDriver` interface.
## Driver selection and configuration
The preferred method of selecting a storage driver is using the `StorageDriverFactory` interface in the `storagedriver/factory` package. These factories provide a common interface for constructing storage drivers with a parameters map. The factory model is based on the [Register](https://golang.org/pkg/database/sql/#Register) and [Open](https://golang.org/pkg/database/sql/#Open) methods in the builtin [database/sql](https://golang.org/pkg/database/sql) package.
Storage driver factories may be registered by name using the
`factory.Register` method, and then later invoked by calling `factory.Create`
with a driver name and parameters map. If no such storage driver can be found,
`factory.Create` returns an `InvalidStorageDriverError`.
## Driver contribution
New storage drivers are not currently being accepted.
See https://github.com/distribution/distribution/issues/3988 for discussion.
There are forks of this repo that implement custom storage drivers.
These are not supported by the OCI distribution project.
The known forks are:
- Storj DCS: https://github.com/storj/docker-registry
- HuaweiCloud OBS: https://github.com/setoru/distribution/tree/obs
- us3: https://github.com/lambertxiao/distribution/tree/main
- Baidu BOS: https://github.com/dolfly/distribution/tree/bos
- HDFS: https://github.com/haosdent/distribution/tree/master
### Writing new storage drivers
To create a valid storage driver, one must implement the
`storagedriver.StorageDriver` interface and make sure to expose this driver
via the factory system.
#### Registering
Storage drivers should call `factory.Register` with their driver name in an `init` method, allowing callers of `factory.New` to construct instances of this driver without requiring modification of imports throughout the codebase.
## Testing
Storage driver test suites are provided in
`storagedriver/testsuites/testsuites.go` and may be used for any storage
driver written in Go. Tests can be registered using the `RegisterSuite`
function, which run the same set of tests for any registered drivers.

26
docs/content/storage-drivers/azure.md Normal file
View File

@@ -0,0 +1,26 @@
---
description: Explains how to use the Azure storage drivers
keywords: registry, service, driver, images, storage, azure
title: Microsoft Azure storage driver
---
An implementation of the `storagedriver.StorageDriver` interface which uses [Microsoft Azure Blob Storage](https://azure.microsoft.com/en-us/services/storage/) for object storage.
## Parameters
| Parameter | Required | Description |
|:-----------------------------------|:---------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `accountname` | yes | Name of the Azure Storage Account. |
| `accountkey` | yes | Primary or Secondary Key for the Storage Account. |
| `container` | yes | Name of the Azure root storage container in which all registry data is stored. Must comply the storage container name [requirements](https://docs.microsoft.com/rest/api/storageservices/fileservices/naming-and-referencing-containers--blobs--and-metadata). For example, if your url is `https://myaccount.blob.core.windows.net/myblob` use the container value of `myblob`.|
| `realm` | no | Domain name suffix for the Storage Service API endpoint. For example realm for "Azure in China" would be `core.chinacloudapi.cn` and realm for "Azure Government" would be `core.usgovcloudapi.net`. By default, this is `core.windows.net`. |
| `copy_status_poll_max_retry` | no | Max retry number for polling of copy operation status. Retries use a simple backoff algorithm where each retry number is multiplied by `copy_status_poll_delay`, and this number is used as the delay. Set to -1 to disable retries and abort if the copy does not complete immediately. Defaults to 5. |
| `copy_status_poll_delay` | no | Time to wait between retries for polling of copy operation status. This time is multiplied by N on each retry, where N is the retry number. Defaults to 100ms |
## Related information
* To get information about
[azure-blob-storage](https://azure.microsoft.com/en-us/services/storage/), visit
the Microsoft website.
* You can use Microsoft's [Blob Service REST API](https://docs.microsoft.com/en-us/rest/api/storageservices/Blob-Service-REST-API) to [create a storage container](https://docs.microsoft.com/en-us/rest/api/storageservices/Create-Container).

19
docs/content/storage-drivers/filesystem.md Normal file
View File

@@ -0,0 +1,19 @@
---
description: Explains how to use the filesystem storage drivers
keywords: registry, service, driver, images, storage, filesystem
title: Filesystem storage driver
---
An implementation of the `storagedriver.StorageDriver` interface which uses the local filesystem.
## Parameters
* `rootdirectory`: (optional) The absolute path to a root directory tree in which
to store all registry files. The registry stores all its data here so make sure
there is adequate space available. Defaults to `/var/lib/registry`. If the directory
does not exist, it will be created honoring [`umask`](https://man7.org/linux/man-pages/man2/umask.2.html)
bits. If `umask` bits are not set, the resulting permission will be `0777`.
* `maxthreads`: (optional) The maximum number of simultaneous blocking filesystem
operations permitted within the registry. Each operation spawns a new thread and
may cause thread exhaustion issues if many are done in parallel. Defaults to
`100`, and cannot be lower than `25`.

19
docs/content/storage-drivers/gcs.md Normal file
View File

@@ -0,0 +1,19 @@
---
description: Explains how to use the Google Cloud Storage drivers
keywords: registry, service, driver, images, storage, gcs, google, cloud
title: Google Cloud Storage driver
---
An implementation of the `storagedriver.StorageDriver` interface which uses Google Cloud for object storage.
## Parameters
| Parameter | Required | Description |
|:--------------|:---------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `bucket` | yes | The name of your Google Cloud Storage bucket where you wish to store objects (needs to already be created prior to driver initialization). |
| `keyfile` | no | A private service account key file in JSON format used for [Service Account Authentication](https://cloud.google.com/storage/docs/authentication#service_accounts). |
| `rootdirectory` | no | The root directory tree in which all registry files are stored. Defaults to the empty string (bucket root). If a prefix is used, the path `bucketname/<prefix>` has to be pre-created before starting the registry. The prefix is applied to all Google Cloud Storage keys to allow you to segment data in your bucket if necessary.|
| `chunksize` | no (default 5242880) | This is the chunk size used for uploading large blobs, must be a multiple of 256*1024. |
**Note:** Instead of a key file you can use [Google Application Default Credentials](https://developers.google.com/identity/protocols/application-default-credentials).

16
docs/content/storage-drivers/inmemory.md Normal file
View File

@@ -0,0 +1,16 @@
---
description: Explains how to use the in-memory storage drivers
keywords: registry, service, driver, images, storage, in-memory
title: In-memory storage driver (testing only)
---
For purely tests purposes, you can use the `inmemory` storage driver. This
driver is an implementation of the `storagedriver.StorageDriver` interface which
uses local memory for object storage. If you would like to run a registry from
volatile memory, use the [`filesystem` driver](filesystem.md) on a ramdisk.
**IMPORTANT**: This storage driver *does not* persist data across runs. This is why it is only suitable for testing. *Never* use this driver in production.
## Parameters
None

165
docs/content/storage-drivers/s3.md Normal file
View File

@@ -0,0 +1,165 @@
---
description: Explains how to use the S3 storage drivers
keywords: registry, service, driver, images, storage, S3
title: S3 storage driver
---
An implementation of the `storagedriver.StorageDriver` interface which uses
Amazon S3 or S3 compatible services for object storage.
## Parameters
| Parameter | Required | Description |
|:--------------|:---------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `accesskey` | no | Your AWS Access Key. If you use [IAM roles](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html), omit to fetch temporary credentials from IAM. |
| `secretkey` | no | Your AWS Secret Key. If you use [IAM roles](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html), omit to fetch temporary credentials from IAM. |
| `region` | yes | The AWS region in which your bucket exists. |
| `regionendpoint` | no | Endpoint for S3 compatible storage services (Minio, etc). |
| `forcepathstyle` | no | To enable path-style addressing when the value is set to `true`. The default is `true`. |
| `bucket` | yes | The bucket name in which you want to store the registry's data. |
| `encrypt` | no | Specifies whether the registry stores the image in encrypted format or not. A boolean value. The default is `false`. |
| `keyid` | no | Optional KMS key ID to use for encryption (encrypt must be true, or this parameter is ignored). The default is `none`. |
| `secure` | no | Indicates whether to use HTTPS instead of HTTP. A boolean value. The default is `true`. |
| `skipverify` | no | Skips TLS verification when the value is set to `true`. The default is `false`. |
| `v4auth` | no | Indicates whether the registry uses Version 4 of AWS's authentication. The default is `true`. |
| `chunksize` | no | The S3 API requires multipart upload chunks to be at least 5MB. This value should be a number that is larger than 5 * 1024 * 1024.|
| `rootdirectory` | no | This is a prefix that is applied to all S3 keys to allow you to segment data in your bucket if necessary. |
| `storageclass` | no | The S3 storage class applied to each registry file. The default is `STANDARD`. |
| `objectacl` | no | The S3 Canned ACL for objects. The default value is "private". |
| `loglevel` | no | The log level for the S3 client. The default value is `off`. |
> **Note** You can provide empty strings for your access and secret keys to run the driver
> on an ec2 instance and handles authentication with the instance's credentials. If you
> use [IAM roles](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html),
> omit these keys to fetch temporary credentials from IAM.
`region`: The name of the aws region in which you would like to store objects (for example `us-east-1`). For a list of regions, see [Regions, Availability Zones, and Local Zones](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html).
`regionendpoint`: (optional) Endpoint URL for S3 compatible APIs. This should not be provided when using Amazon S3.
`forcepathstyle`: (optional) The force path style for S3 compatible APIs. Some manufacturers only support force path style, while others only support DNS based bucket routing. Amazon S3 supports both.
`bucket`: The name of your S3 bucket where you wish to store objects. The bucket must exist prior to the driver initialization.
`encrypt`: (optional) Whether you would like your data encrypted on the server side (defaults to false if not specified).
`keyid`: (optional) Whether you would like your data encrypted with this KMS key ID (defaults to none if not specified, is ignored if encrypt is not true).
`secure`: (optional) Whether you would like to transfer data to the bucket over ssl or not. Defaults to true (meaning transferring over ssl) if not specified. While setting this to false improves performance, it is not recommended due to security concerns.
`v4auth`: (optional) Whether you would like to use aws signature version 4 with your requests. This defaults to `false` if not specified. The `eu-central-1` region does not work with version 2 signatures, so the driver errors out if initialized with this region and v4auth set to `false`.
`chunksize`: (optional) The default part size for multipart uploads (performed by WriteStream) to S3. The default is 10 MB. Keep in mind that the minimum part size for S3 is 5MB. Depending on the speed of your connection to S3, a larger chunk size may result in better performance; faster connections benefit from larger chunk sizes.
`rootdirectory`: (optional) The root directory tree in which all registry files are stored. Defaults to the empty string (bucket root).
`storageclass`: (optional) The storage class applied to each registry file. Defaults to STANDARD. Valid options are STANDARD and REDUCED_REDUNDANCY.
`objectacl`: (optional) The canned object ACL to be applied to each registry object. Defaults to `private`. If you are using a bucket owned by another AWS account, it is recommended that you set this to `bucket-owner-full-control` so that the bucket owner can access your objects. Other valid options are available in the [AWS S3 documentation](http://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl).
`loglevel`: (optional) Valid values are: `off` (default), `debug`, `debugwithsigning`, `debugwithhttpbody`, `debugwithrequestretries`, `debugwithrequesterrors` and `debugwitheventstreambody`. See the [AWS SDK for Go API reference](https://docs.aws.amazon.com/sdk-for-go/api/aws/#LogLevelType) for details.
## S3 permission scopes
The following AWS policy is required by the registry for push and pull. Make sure to replace `S3_BUCKET_NAME` with the name of your bucket.
```
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:GetBucketLocation",
"s3:ListBucketMultipartUploads"
],
"Resource": "arn:aws:s3:::S3_BUCKET_NAME"
},
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject",
"s3:ListMultipartUploadParts",
"s3:AbortMultipartUpload"
],
"Resource": "arn:aws:s3:::S3_BUCKET_NAME/*"
}
]
}
```
See [the S3 policy documentation](http://docs.aws.amazon.com/AmazonS3/latest/dev/mpuAndPermissions.html) for more details.
# CloudFront as Middleware with S3 backend
## Use Case
Adding CloudFront as a middleware for your S3 backed registry can dramatically
improve pull times. Your registry can retrieve your images
from edge servers, rather than the geographically limited location of your S3
bucket. The farther your registry is from your bucket, the more improvements are
possible. See [Amazon CloudFront](https://aws.amazon.com/cloudfront/details/).
An alternative method for CloudFront that requires less configuration and will use
the same edge servers is [S3 Transfer Acceleration](https://docs.aws.amazon.com/AmazonS3/latest/dev/transfer-acceleration.html).
Please check acceleration [Requirements](https://docs.aws.amazon.com/AmazonS3/latest/dev/transfer-acceleration.html#transfer-acceleration-requirements)
to see whether you need CloudFront or S3 Transfer Acceleration.
## Configuring CloudFront for Distribution
If you are unfamiliar with creating a CloudFront distribution, see [Getting
Started with
Cloudfront](http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/GettingStarted.html).
Defaults can be kept in most areas except:
### Origin:
- The CloudFront distribution must be created such that the `Origin Path` is set
to the directory level of the root "docker" key in S3. If your registry exists
on the root of the bucket, this path should be left blank.
- For private S3 buckets, you must set `Restrict Bucket Access` to `Yes`. See
the [CloudFront documentation](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/PrivateContent.html).
### Behaviors:
- Viewer Protocol Policy: HTTPS Only
- Allowed HTTP Methods: GET, HEAD, OPTIONS, PUT, POST, PATCH, DELETE
- Cached HTTP Methods: OPTIONS (checked)
- Restrict Viewer Access (Use Signed URLs or Signed Cookies): Yes
- Trusted Signers: Self (Can add other accounts as long as you have access to CloudFront Key Pairs for those additional accounts)
## Registry configuration
Here the `middleware` option is used. It is still important to keep the
`storage` option, because CloudFront only handles `pull` actions; `push` actions
are still directly written to S3.
The following example shows a minimum configuration:
```yaml
...
storage:
s3:
region: us-east-1
bucket: docker.myregistry.com
middleware:
storage:
- name: cloudfront
options:
baseurl: https://abcdefghijklmn.cloudfront.net/
privatekey: /etc/docker/cloudfront/pk-ABCEDFGHIJKLMNOPQRST.pem
keypairid: ABCEDFGHIJKLMNOPQRST
...
```
## CloudFront Key-Pair
A CloudFront key-pair is required for all AWS accounts needing access to your
CloudFront distribution. You must have access to your AWS account's root credentials to create the required Cloudfront keypair. For information, see [Creating CloudFront Key
Pairs](http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/private-content-trusted-signers.html#private-content-creating-cloudfront-key-pairs).