options to split image steps and manifest steps

Signed-off-by: Avi Deitcher <avi@deitcher.net>

docs/packages.md

@@ -7,7 +7,7 @@ packages, as it's very easy. Packages are the unit of customisation
 in a LinuxKit-based project, if you know how to build a container,
 you should be able to build a LinuxKit package.
 
-All LinuxKit packages are:
+All official LinuxKit packages are:
 - Signed with Docker Content Trust.
 - Enabled with multi-arch manifests to work on multiple architectures.
 - Derived from well-known (and signed) sources for repeatable builds.
@@ -15,6 +15,7 @@ All LinuxKit packages are:
 
 
 ## CI and Package Builds
+
 When building and merging packages, it is important to note that our CI process builds packages. The targets `make ci` and `make ci-pr` execute `make -C pkg build`. These in turn execute `linuxkit pkg build` for each package under `pkg/`. This in turn will try to pull the image whose tag matches the tree hash or, failing that, to build it.
 
 We do not want the builds to happen with each CI run for two reasons:
@@ -73,38 +74,89 @@ should also be set up with signing keys for packages and your signing
 key should have a passphrase, which we call `<passphrase>` throughout.
 
 All official LinuxKit packages are multi-arch manifests and most of
-them are available for `amd64`, `arm64`, and `s390x`. Official images
+them are available for the following platforms:
-*must* be build on both architectures and they must be build *in
-sequence*, i.e., they can't be build in parallel.
 
-To build a package on an architecture:
+* `linux/amd64`
+* `linux/arm64`
+* `linux/s390x`
+
+Official images *must* be built on all architectures for which they are available.
+They can be built and pushed in parallel, but the manifest should be pushed once
+when all of the images are done.
+
+Pushing out a package as a maintainer involves two distinct stages:
+
+1. Building and pushing out the platform-specific image
+1. Creating, pushing out and signing the multi-arch manifest, a.k.a. OCI image index
+
+The `linuxkit pkg` command contains automation which performs all of the steps.
+Note that `«path-to-package»` is the path to the package's source directory
+(containing at least `build.yml` and `Dockerfile`). It can be `.` if
+the package is in the current directory.
+
+
+#### Image Only
+
+To build and push out the platform-specific image, on that platform:
+
+```
+linuxkit pkg push --manifest=false «path-to-package»
+```
+
+The options do the following:
+
+* `--manifest=false` means not to push or sign a manifest
+
+Repeat the above on each platform where you need an image.
+
+This will do the following:
+
+1. Determine the name and tag for the image as follows:
+   * The tag is from the hash of the git tree for that package. You can see it by doing `linuxkit pkg show-tag «path-to-package»`.
+   * The name for the image is from `«path-to-package»/build.yml`
+   * The organization for the package is given on the command-line, default to `linuxkit`.
+1. Build the package in the given path using your local docker instance for the local platform. E.g. if you are running on `linux/arm64`, it will build for `linux/arm64`.
+1. Tag the build image as `«image-name»:«hash»-«arch»`
+1. Push the image to the hub
+
+#### Manifest Only
+
+To perform just the manifest steps, do:
+
+```
+DOCKER_CONTENT_TRUST_REPOSITORY_PASSPHRASE="<passphrase>" linuxkit pkg push --image=false --manifest «path-to-package»
+```
+
+The options do the following:
+
+* `--image=false` do not push the image, as you already did it; you can, of course, skip this argument and push the image as well
+* `--manifest` create and push the manifest
+
+This will do the following:
+
+1. Find all of the images on the hub of the format `«image-name»:«hash»-«arch»`
+1. Create a multi-arch manifest called `«image-name»:«hash»` (note no `-«arch»`)
+1. Push the manifest to the hub
+1. Sign the manifest with your key
+
+Each time you perform the manifest steps, it will find all of the images,
+including any that have been added since last time.
+The LinuxKit YAML files should consume the package as the multi-arch manifest:
+`linuxkit/<image>:<hash>`.
+
+#### Everything at once
+
+To perform _all_ of the steps at once - build and push out the image for whatever platform
+you are running on, and create and sign a manifest - do:
 
 ```
 DOCKER_CONTENT_TRUST_REPOSITORY_PASSPHRASE="<passphrase>" linuxkit pkg push «path-to-package»
 ```
 
-`«path-to-package»` is the path to the package's source directory
+#### Prerequisites
-(containing at least `build.yml` and `Dockerfile`). It can be `.` if
-the package is in the current directory.
-
-**Note:** You *must* be logged into hub (`docker login`) and the
-passphrase for the key *must* be supplied as an environment
-variable. The build process has to resort to using `expect` to drive
-`notary` so none of the credentials can be entered interactively.
-
-This will:
-- Build a local images as `linuxkit/<image>:<hash>-<arch>`
-- Push it to hub
-- Sign it with your key
-- Create a manifest called `linuxkit/<image>:<hash>` (note no `-<arch>`)
-- Push the manifest to hub
-- Sign the manifest
-
-If you repeat the same on another architecture, a new manifest will be
-pushed and signed containing the previous and the new
-architecture. The YAML files should consume the package as:
-`linuxkit/<image>:<hash>`.
 
+* For all of the steps, you *must* be logged into hub (`docker login`).
+* For the manifest steps, you must be logged into hub and the passphrase for the key *must* be supplied as an environment variable. The build process has to resort to using `expect` to drive `notary` so none of the credentials can be entered interactively.
 
 Since it is not very good to have your passphrase in the clear (or
 even stashed in your shell history), we recommend using a password
@@ -173,5 +225,5 @@ if you want to use it, you will need to add the following line to the dockerfile
 ARG all_proxy
 ```
 
-Linuxkit does not judge between lower-cased or upper-cased variants of these options, e.g. `http_proxy` vs `HTTP_PROXY`,
+LinuxKit does not judge between lower-cased or upper-cased variants of these options, e.g. `http_proxy` vs `HTTP_PROXY`,
 as `docker build` does not either. It just passes them through "as-is".

src/cmd/linuxkit/pkg_push.go

@@ -22,6 +22,9 @@ func pkgPush(args []string) {
 	force := flags.Bool("force", false, "Force rebuild")
 	release := flags.String("release", "", "Release the given version")
 	nobuild := flags.Bool("nobuild", false, "Skip the build")
+	manifest := flags.Bool("manifest", true, "Create and push multi-arch manifest")
+	image := flags.Bool("image", true, "Build and push image for the current platform")
+	sign := flags.Bool("sign", true, "sign the manifest, if a manifest is created; ignored if --manifest=false")
 
 	p, err := pkglib.NewFromCLI(flags, args...)
 	if err != nil {
@@ -44,6 +47,16 @@ func pkgPush(args []string) {
 	if *release != "" {
 		opts = append(opts, pkglib.WithRelease(*release))
 	}
+	if *manifest {
+		opts = append(opts, pkglib.WithBuildManifest())
+	}
+	if *image {
+		opts = append(opts, pkglib.WithBuildImage())
+	}
+	// only sign manifests; ignore for image only
+	if *sign && *manifest {
+		opts = append(opts, pkglib.WithBuildSign())
+	}
 
 	if *nobuild {
 		fmt.Printf("Pushing %q without building\n", p.Tag())

src/cmd/linuxkit/pkglib/build.go

@@ -18,6 +18,9 @@ type buildOpts struct {
 	force     bool
 	push      bool
 	release   string
+	manifest  bool
+	sign      bool
+	image     bool
 }
 
 // BuildOpt allows callers to specify options to Build
@@ -47,6 +50,30 @@ func WithBuildPush() BuildOpt {
 	}
 }
 
+// WithBuildImage builds the image
+func WithBuildImage() BuildOpt {
+	return func(bo *buildOpts) error {
+		bo.image = true
+		return nil
+	}
+}
+
+// WithBuildManifest creates a multi-arch manifest for the image
+func WithBuildManifest() BuildOpt {
+	return func(bo *buildOpts) error {
+		bo.manifest = true
+		return nil
+	}
+}
+
+// WithBuildSign signs the image and/or the index
+func WithBuildSign() BuildOpt {
+	return func(bo *buildOpts) error {
+		bo.sign = true
+		return nil
+	}
+}
+
 // WithRelease releases as the given version after push
 func WithRelease(r string) BuildOpt {
 	return func(bo *buildOpts) error {
@@ -64,7 +91,7 @@ func (p Pkg) Build(bos ...BuildOpt) error {
 		}
 	}
 
-	if _, ok := os.LookupEnv("DOCKER_CONTENT_TRUST_REPOSITORY_PASSPHRASE"); !ok && p.trust && bo.push {
+	if _, ok := os.LookupEnv("DOCKER_CONTENT_TRUST_REPOSITORY_PASSPHRASE"); !ok && bo.sign && p.trust && bo.push {
 		return fmt.Errorf("Pushing with trust enabled requires $DOCKER_CONTENT_TRUST_REPOSITORY_PASSPHRASE to be set")
 	}
 
@@ -98,7 +125,7 @@ func (p Pkg) Build(bos ...BuildOpt) error {
 		return fmt.Errorf("Cannot release %q if not pushing", bo.release)
 	}
 
-	d := newDockerRunner(p.trust, p.cache)
+	d := newDockerRunner(p.trust, p.cache, bo.sign)
 
 	if !bo.force {
 		ok, err := d.pull(p.Tag())
@@ -111,7 +138,7 @@ func (p Pkg) Build(bos ...BuildOpt) error {
 		fmt.Println("No image pulled, continuing with build")
 	}
 
-	if !bo.skipBuild {
+	if bo.image && !bo.skipBuild {
 		var args []string
 
 		if err := p.dockerDepends.Do(d); err != nil {
@@ -171,7 +198,7 @@ func (p Pkg) Build(bos ...BuildOpt) error {
 	// matters given we do either pull or build above in the
 	// !force case.
 
-	if err := d.pushWithManifest(p.Tag(), suffix); err != nil {
+	if err := d.pushWithManifest(p.Tag(), suffix, bo.image, bo.manifest, bo.sign); err != nil {
 		return err
 	}
 
@@ -189,7 +216,7 @@ func (p Pkg) Build(bos ...BuildOpt) error {
 		return err
 	}
 
-	if err := d.pushWithManifest(relTag, suffix); err != nil {
+	if err := d.pushWithManifest(relTag, suffix, bo.image, bo.manifest, bo.sign); err != nil {
 		return err
 	}
 

src/cmd/linuxkit/pkglib/docker.go

@@ -39,6 +39,7 @@ var platforms = []string{
 type dockerRunner struct {
 	dct   bool
 	cache bool
+	sign  bool
 
 	// Optional build context to use
 	ctx buildContext
@@ -49,8 +50,8 @@ type buildContext interface {
 	Copy(io.WriteCloser) error
 }
 
-func newDockerRunner(dct, cache bool) dockerRunner {
+func newDockerRunner(dct, cache, sign bool) dockerRunner {
-	return dockerRunner{dct: dct, cache: cache}
+	return dockerRunner{dct: dct, cache: cache, sign: sign}
 }
 
 func isExecErrNotFound(err error) bool {
@@ -83,7 +84,10 @@ func (dr dockerRunner) command(args ...string) error {
 	cmd.Env = os.Environ()
 
 	dct := ""
-	if dr.dct {
+
+	// when we are doing a push, we need to disable DCT if not signing
+	isPush := len(args) >= 2 && args[0] == "image" && args[1] == "push"
+	if dr.dct && (!isPush || dr.sign) {
 		cmd.Env = append(cmd.Env, dctEnableEnv)
 		dct = dctEnableEnv + " "
 	}
@@ -147,10 +151,19 @@ func (dr dockerRunner) push(img string) error {
 	return dr.command("image", "push", img)
 }
 
-func (dr dockerRunner) pushWithManifest(img, suffix string) error {
+func (dr dockerRunner) pushWithManifest(img, suffix string, pushImage, pushManifest, sign bool) error {
-	fmt.Printf("Pushing %s\n", img+suffix)
+	var (
-	if err := dr.push(img + suffix); err != nil {
+		digest string
-		return err
+		l      int
+		err    error
+	)
+	if pushImage {
+		fmt.Printf("Pushing %s\n", img+suffix)
+		if err := dr.push(img + suffix); err != nil {
+			return err
+		}
+	} else {
+		fmt.Print("Image push disabled, skipping...\n")
 	}
 
 	auth, err := getDockerAuth()
@@ -158,16 +171,24 @@ func (dr dockerRunner) pushWithManifest(img, suffix string) error {
 		return fmt.Errorf("failed to get auth: %v", err)
 	}
 
-	fmt.Printf("Pushing %s to manifest %s\n", img+suffix, img)
+	if pushManifest {
-	digest, l, err := manifestPush(img, auth)
+		fmt.Printf("Pushing %s to manifest %s\n", img+suffix, img)
-	if err != nil {
+		digest, l, err = manifestPush(img, auth)
-		return err
+		if err != nil {
+			return err
+		}
+	} else {
+		fmt.Print("Manifest push disabled, skipping...\n")
 	}
 	// if trust is not enabled, nothing more to do
 	if !dr.dct {
 		fmt.Println("trust disabled, not signing")
 		return nil
 	}
+	if !sign {
+		fmt.Println("signing disabled, not signing")
+		return nil
+	}
 	fmt.Printf("Signing manifest for %s\n", img)
 	return signManifest(img, digest, l, auth)
 }
@@ -279,7 +300,7 @@ func signManifest(img, digest string, length int, auth dockertypes.AuthConfig) e
 	}
 
 	// report output
-	fmt.Printf("New signed multi-arch image: %s:%s\n", repo, tag)
+	fmt.Printf("Signed manifest index: %s:%s\n", repo, tag)
 
 	return nil
 }