github/distribution
1
0
Fork 0
mirror of https://github.com/distribution/distribution.git synced 2025-09-19 01:16:59 +00:00
Code Issues Projects Releases Wiki Activity

Pass the last paging flag to storage drivers

Browse Source 
Storage drivers may be able to take advantage of the hint to start
their walk more efficiently.

For S3: The API takes a start-after parameter. Registries with many
repositories can drastically reduce calls to s3 by telling s3 to only
list results lexographically after the last parameter.

For the fallback: We can start deeper in the tree and avoid statting
the files and directories before the hint in a walk. For a filesystem
this improves performance a little, but many of the API based drivers
are currently treated like a filesystem, so this drastically improves
the performance of GCP and Azure blob.

Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
This commit is contained in:
James Hewitt
2022-07-10 03:04:50 +01:00
parent 3a44c2e10e
commit e22f7cbc73
13 changed files with 390 additions and 63 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
registry/storage/driver/filesystem/driver.go
View File

@@ -290,8 +290,8 @@ func (d *driver) URLFor(ctx context.Context, path string, options map[string]int
// Walk traverses a filesystem defined within driver, starting
// from the given path, calling f on each file and directory
func (d *driver) Walk(ctx context.Context, path string, f storagedriver.WalkFn) error {
return storagedriver.WalkFallback(ctx, d, path, f)
func (d *driver) Walk(ctx context.Context, path string, f storagedriver.WalkFn, options ...func(*storagedriver.WalkOptions)) error {
return storagedriver.WalkFallback(ctx, d, path, f, options...)
}
// fullPath returns the absolute path of a key within the Driver's storage.