github/distribution
1
0
Fork 0
mirror of https://github.com/distribution/distribution.git synced 2025-08-20 15:55:27 +00:00
Code Issues Projects Releases Wiki Activity
2,880 Commits 24 Branches 6 Tags 149 MiB
05a2d10b76
Commit Graph

2880 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Collin Shoop
05a2d10b76 storagedriver/s3: Cleaning up tests. 2021-06-30 08:56:01 -04:00
Collin Shoop
b2bdea24aa storagedriver/s3: Forgoing any interface conformance tests and moved it all into S3 tests. 2021-06-30 08:56:01 -04:00
Collin Shoop
70573f92c7 storagedriver: Adding an interface conformance test for Walk, cont. 2021-06-30 08:56:01 -04:00
Collin Shoop
fbb45bd8a0 storagedriver: Adding an interface conformance test for Walk 2021-06-30 08:56:01 -04:00
Collin Shoop
847738fd5e storagedriver/s3: fixed a bug in s3 Walk impl preventing ErrSkipDir from stopping gracefully 2021-06-30 08:56:01 -04:00
Collin Shoop
aea873cc92 storagedriver/s3: test refactor. fixed a bug in WalkFallback preventing ErrSkipDir from stopping gracefully 2021-06-30 08:56:01 -04:00
Collin Shoop
8b726cc377 storagedriver/s3: Reverting a few changes from previous WalkFiles impl that was left in. 2021-06-30 08:56:01 -04:00
Collin Shoop
e20be1ead5 storagedriver/s3: Reverting WalkFiles method. Instead, optimizing the Walk method. 2021-06-30 08:56:01 -04:00
Collin Shoop
1c3ee66061 storagedriver/s3: continue refining tests 2021-06-30 08:56:01 -04:00
Collin Shoop
c98c694d51 storagedriver/s3: refining tests 2021-06-30 08:56:01 -04:00
Collin Shoop
a08cc38c05 storagedriver/s3: additional tests 2021-06-30 08:56:01 -04:00
Collin Shoop
88420f2c7c Added unit testing for storage driver WalkFallback and WalkFilesFallback 2021-06-30 08:56:01 -04:00
Collin Shoop
cff975b29f Fixed the fallback implementation of WalkFilesFallback that wasn't working & added a Files Removed test for WalkFilesFallback. 2021-06-30 08:56:01 -04:00
Collin Shoop
48e2373d5b storagedriver/s3: Simplified conditional in Walk impl 2021-06-30 08:56:01 -04:00
Collin Shoop
c83e9ae10a Replaced usage of storagedriver.Walk with storagedriver.WalkFiles for linked/blobstore 2021-06-30 08:56:01 -04:00
Collin Shoop
f31195b2e8 storagedriver: Added a new WalkFiles method (later removed) 2021-06-30 08:56:01 -04:00
wayne
e465862185
Merge pull request #14 from digitalocean/wwarren/set-ContentLength-on-HeadObjectOutput 
s3: work around buggy s3 api client implementation
 2020-11-04 15:33:55 -06:00
Wayne Warren
bdbf05f6f8 flyby: set ContentLength on GetObjectOutput also 2020-11-04 15:03:52 -06:00
Wayne Warren
a8c352063e s3: nit - ignore HeadObjectOutput if it doesn't have ContentLength 2020-11-04 00:06:41 -06:00
Wayne Warren
b0fc5ddc8b s3: use a request handler to set ContentLength on HeadObjectOutput 2020-11-04 00:05:55 -06:00
Adam Wolfe Gordon
78f71c18b4
Merge pull request #13 from digitalocean/awg/quotas 
Handle S3 QuotaExceeded errors appropriately
 2020-10-26 11:12:58 -06:00
Adam Wolfe Gordon
c1574b87be handlers: Handle QuotaExceededError and return a DENIED to the client 
If we get a QuotaExceededError back from the storage driver when doing a write
for either manifests or blobs, return a DENIED to the client with the message
"quota exceeded".

Signed-off-by: Adam Wolfe Gordon <awg@digitalocean.com>
 2020-10-23 15:46:04 -06:00
Adam Wolfe Gordon
d192a974fb s3: Handle QuotaExceeded errors 
Add a new storagedriver error to be used when quotas are exceeded, and return it
from the S3 driver.

Signed-off-by: Adam Wolfe Gordon <awg@digitalocean.com>
 2020-10-23 15:29:39 -06:00
Adam Wolfe Gordon
f2331f9d03 s3: Parse S3 errors returned during multipart operations 
The S3 implementation of `Writer` creates a multipart upload and then uploads to
it as data is written. Previously errors were returned verbatim, but in order to
handle things like quotas we should parse the S3 errors so we can return more
meaningful errors to callers of `Write`.

Signed-off-by: Adam Wolfe Gordon <awg@digitalocean.com>
 2020-10-23 14:25:36 -06:00
wayne
122552b40b
Merge pull request #12 from digitalocean/wwarren/s3-driver-dont-use-KeyCount 
s3: don't use s3.ListObjectsV2Output.KeyCount
 2020-10-22 00:04:25 +00:00
Wayne Warren
a10b3a9ab4 s3: don't use s3.ListObjectsV2Output.KeyCount 2020-10-20 16:51:33 -05:00
wayne
6255476b3b
Merge pull request #11 from waynr/fix-azure-autorest-dependencies 
vendor: Update github.com/Azure/go-autorest
 2020-09-03 12:41:29 -05:00
Wayne Warren
37be5b31d2 vendor: Update github.com/Azure/go-autorest 2020-09-03 16:27:22 +00:00
wayne
0e6ff0e95e
Merge pull request #1 from waynr/add-vacuum-removelayerlink-func 
Add Vacuum.RemoveLayerLink function
 2020-09-03 09:05:16 -05:00
Wayne Warren
28166f7383 simplify RemoveLayerLink function 2020-09-02 21:54:36 +00:00
Wayne Warren
ec64b15a77 fix signature on RemoveLayerLink 2020-09-02 21:53:48 +00:00
Wayne Warren
5fcc8f55e2 need schema2 import 2020-09-02 21:53:48 +00:00
Wayne Warren
3b7741805d Remove layer links along with manifest 2020-09-02 21:53:48 +00:00
Wayne Warren
4366eee39a add Vacuum.RemoveLayerLink function 2020-09-02 21:53:48 +00:00
Adam Wolfe Gordon
1e1c67e13f
Merge pull request #10 from digitalocean/awg/resume-commit-race 
Return a sensible error in races between blob upload completions
 2020-08-24 14:59:01 -06:00
Adam Wolfe Gordon
611f55e79a Return a sensible error in races between blob upload completions 
Completing a blob upload is a multi-step process, especially on a storage
backend like S3 which lacks a "move" primitive. If a client times out (or
disconnects) during a commit, then retries immediately, we'll end up invoking
the "resume" logic for the blob upload while the completion from the previous
request is still in progress. The resume tries to find the upload's data, but
the commit has already moved it to its new location, resulting in a PathNotFound
error from the storage driver.

Previously, this error was passed back to the HTTP handler and translated into a
500 Internal Server Error. Return ErrBlobUploadUnknown instead, so that the HTTP
handler will return a 404.
 2020-08-24 13:49:09 -06:00
Adam Wolfe Gordon
d9adf97319
Merge pull request #9 from digitalocean/awg/fix-unlinked-blobs 
Revert "OCI: GET request to deleted blob URL should yield 404 response"
 2020-08-17 11:02:58 -06:00
Adam Wolfe Gordon
37ab304645 Revert "OCI: GET request to deleted blob URL should yield 404 response" 
Using the linked blob store for blob requests fixes a conformance issue and
works correctly for newly-pushed images. However, any images that were pushed
before the change could be missing layer links (since previously it was possible
to fetch a blob without the layer link existing) and it's not possible to pull
these iamges after the change.

Revert the change to use the linked blob store, for now, until we can figure out
how to fix this without breaking existing repositories.

This reverts commit 71f1c25220.
 2020-08-17 10:37:50 -06:00
wayne
3796bf3c1a
Merge pull request #8 from digitalocean/wwarren/s3-storage-driver_log-s3-api-requests 
s3: log s3 api requests
 2020-08-13 19:48:06 -05:00
Wayne Warren
f3d61f1000 s3: log S3 api requests 
Adds "Completed" handler for S3 requests that logs critical HTTP request/response fields necessary to understand problems between registry and underlying S3 api.

Signed-off-by: Wayne Warren <wayne.warren.s@gmail.com>
 2020-08-14 00:45:57 +00:00
wayne
04e669847f
Merge pull request #7 from digitalocean/wwarren/merge-upstream-master 
merge upstream/master branch into digitalocean/master
 2020-08-07 10:23:15 -05:00
Wayne Warren
3ebc5127c2 restore upstream merge change that broke log tests 2020-07-31 23:17:45 +00:00
Wayne Warren
a4ac140fd3 Merge remote-tracking branch 'docker/master' into wwarren/merge-upstream-master 2020-07-31 22:47:21 +00:00
wayne
a68e2a8111
Merge pull request #6 from digitalocean/wwarren/oci-compliance-post-request-wtih-digest-and-blob-should-yield-201 
OCI: POST request with digest and blob should yield 201
 2020-07-31 17:38:00 -05:00
Wayne Warren
2ff5f5bf80 OCI: POST request with digest and blob should yield 201 2020-07-31 22:37:31 +00:00
wayne
f747a0f2f7
Merge pull request #5 from digitalocean/wwarren/oci-compliance-get-request-to-deleted-blob-url 
OCI: GET request to deleted blob URL should yield 404 response
 2020-07-31 17:35:57 -05:00
Manish Tomar
1d0ea8ed7b
Merge pull request #3140 from linux-on-ibm-z/travis-s390x 
Add s390x support for travis
 2020-07-30 16:15:55 -07:00
Wayne Warren
71f1c25220 OCI: GET request to deleted blob URL should yield 404 response 2020-07-29 22:36:40 +00:00
Derek McGowan
208d68bd5c
Merge pull request #3209 from zhipengzuo/master 
clean up code because err is always nil
 2020-07-22 09:18:06 -07:00
zhipengzuo
f361d443b7 clean up code because err is always nil 
Signed-off-by: zhipengzuo <zuozhipeng@baidu.com>
 2020-07-22 10:54:46 +08:00