github/linuxkit
1
0
Fork 0
mirror of https://github.com/linuxkit/linuxkit.git synced 2025-09-04 16:30:52 +00:00
Code Issues Projects Releases Wiki Activity

Remove Notary and Content Trust

Browse Source 
This commit removes Notary and Content Trust.
Notary v1 is due to be replaced with Notary v2 soon.
There is no clean migration path from one to the other.
For now, this removes all signing from LinuxKit.
We will look to add this back once a new Notary alternative
becomes available.

Signed-off-by: Dave Tucker <dave@dtucker.co.uk>
This commit is contained in:
Dave Tucker
2021-03-19 15:14:28 +00:00
parent 1f93eab506
commit 561ce6f4be
171 changed files with 126 additions and 29608 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/image-cache.md
View File

@@ -52,7 +52,7 @@ Image names are kept in `index.json` in the [annotation](https://github.com/open
For each image that linuxkit needs to read, it does the following. Note that if the `--pull` option
is provided, it always will pull, independent of what is in the cache.
1. Check in the cache for the image name in the cache `index.json`. If it does not find it, pull it down and store it in cache, using content trust, if enabled.
1. Check in the cache for the image name in the cache `index.json`. If it does not find it, pull it down and store it in cache.
1. Read the root hash from `index.json`.
1. Find the root blob in the `blobs/` directory via the hash and read it.
1. Proceed to read the manifest, config and layers.

4
docs/kernels.md
View File

@@ -149,7 +149,7 @@ appended. Then you can also override the Hub organisation to use the
image elsewhere with (and also disable image signing):
```sh
make ORG=<your hub org> NOTRUST=1
make ORG=<your hub org>
```
The image will be uploaded to Hub and can be use in a YAML file as
@@ -322,7 +322,7 @@ yourself:
```sh
cd kernel
make ORG=<foo> NOTRUST=1 push_zfs_4.9.x # or different kernel version
make ORG=<foo> push_zfs_4.9.x # or different kernel version
```
will build and push a `zfs-kmod-4.9.<version>` image to Docker Hub

282
docs/packages.md
View File

@@ -8,7 +8,6 @@ in a LinuxKit-based project, if you know how to build a container,
you should be able to build a LinuxKit package.
All official LinuxKit packages are:
- Signed with Docker Content Trust.
- Enabled with multi-arch manifests to work on multiple architectures.
- Derived from well-known (and signed) sources for repeatable builds.
- Built with multi-stage builds to minimise their size.
@@ -41,7 +40,6 @@ A package source consists of a directory containing at least two files:
- `extra-sources` _(list of strings)_: Additional sources for the package outside the package directory. The format is `src:dst`, where `src` can be relative to the package directory and `dst` is the destination in the build context. This is useful for sharing files, such as vendored go code, between packages.
- `gitrepo` _(string)_: The git repository where the package source is kept.
- `network` _(bool)_: Allow network access during the package build (default: no)
- `disable-content-trust` _(bool)_: Disable Docker content trust for this package (default: no)
- `disable-cache` _(bool)_: Disable build cache for this package (default: no)
- `config`: _(struct `github.com/moby/tool/src/moby.ImageConfig`)_: Image configuration, marshalled to JSON and added as `org.mobyproject.config` label on image (default: no label)
- `depends`: Contains information on prerequisites which must be satisfied in order to build the package. Has subfields:
@@ -56,7 +54,7 @@ A package source consists of a directory containing at least two files:
Before you can build packages you need:
- Docker version 19.03 or newer, which includes [buildx](https://docs.docker.com/buildx/working-with-buildx/)
- If you are on a Mac you also need `docker-credential-osxkeychain.bin`, which comes with Docker for Mac.
- `make`, `notary`, `base64`, `jq`, and `expect`
- `make`, `base64`, `jq`, and `expect`
- A *recent* version of `manifest-tool` which you can build with `make
bin/manifest-tool`, or `go get github.com:estesp/manifest-tool`, or
via the LinuxKit homebrew tap with `brew install --HEAD
@@ -98,10 +96,6 @@ linuxkit pkg push
### Build packages as a maintainer
If you have write access to the `linuxkit` organisation on hub, you
should also be set up with signing keys for packages and your signing
key should have a passphrase, which we call `<passphrase>` throughout.
All official LinuxKit packages are multi-arch manifests and most of
them are available for the following platforms:
@@ -153,7 +147,7 @@ This will do the following:
To perform just the manifest steps, do:
```
DOCKER_CONTENT_TRUST_REPOSITORY_PASSPHRASE="<passphrase>" linuxkit pkg push --image=false --manifest «path-to-package»
linuxkit pkg push --image=false --manifest «path-to-package»
```
The options do the following:
@@ -179,282 +173,18 @@ To perform _all_ of the steps at once - build and push out the image for whateve
you are running on, and create and sign a manifest - do:
```
DOCKER_CONTENT_TRUST_REPOSITORY_PASSPHRASE="<passphrase>" linuxkit pkg push «path-to-package»
linuxkit pkg push «path-to-package»
```
#### Prerequisites
* For all of the steps, you *must* be logged into hub (`docker login`).
* For the manifest steps, you must be logged into hub and the passphrase for the key *must* be supplied as an environment variable. The build process has to resort to using `expect` to drive `notary` so none of the credentials can be entered interactively.
Since it is not very good to have your passphrase in the clear (or
even stashed in your shell history), we recommend using a password
manager with a CLI interface, such as LastPass or `pass`. You can then
invoke the build like this (for LastPass):
```
DOCKER_CONTENT_TRUST_REPOSITORY_PASSPHRASE=$(lpass show <key> --password) linuxkit pkg push «path-to-package»
```
or alternatively you may add the command to `~/.moby/linuxkit/config.yml` e.g.:
```
pkg:
content-trust-passphrase-command: "lpass show <key> --password"
```
#### Signing Manually
If, for whatever reason, you want to sign an individual tag manually, whether the index (a.k.a. "multi-arch manifest") or the architecture-specific manifest, do the following:
1. Make sure you have ready your credentials:
* docker hub login and passphrase
* docker notary signing key passphrase
1. Get the following information:
* the name of the image repository you want to sign, including the registry host but **not** including the tag, e.g. `linuxkit/containerd`
* the tag of the image you want to sign, e.g. `a4aa19c608556f7d786852557c36136255220c1f` or `v5.0`
* the size of the image you want to sign in bytes, e.g. `1052`. See below for information on how to get this.
* the hash of the manifest or index to which the tag points, **not** including the `sha256:` leader, e.g. `66b3d74aeb855f393ddb85e7371a00d5f7994cc26b425825df2ce910583d74dc`. See below for information on how to get this.
1. Set env vars with the following:
* `IMAGE`: name of the image, e.g. `IMAGE=docker.io/linuxkit/containerd`
* `TAG`: the tag you want to sign. It could be a tag pointing at a multi-arch manifest or tag pointing at an individual architecture's manifest, e.g. `TAG=a4aa19c608556f7d786852557c36136255220c1f` or `TAG=a4aa19c608556f7d786852557c36136255220c1f-s390x`
* `SIZE`: size of the pointed-at manifest or index, e.g. `SIZE=1052`
* `HASH`: sha256 hash of the pointed-at manifest or index, e.g. `HASH=66b3d74aeb855f393ddb85e7371a00d5f7994cc26b425825df2ce910583d74dc`
1. Run the command: `notary -s https://notary.docker.io -d ~/.docker/trust addhash -p $IMAGE $TAG $SIZE --sha256 $HASH -r targets/releases`
For example:
```console
IMAGE=docker.io/linuxkit/containerd
TAG=a4aa19c608556f7d786852557c36136255220c1f
SIZE=1052
HASH=66b3d74aeb855f393ddb85e7371a00d5f7994cc26b425825df2ce910583d74dc
notary -s https://notary.docker.io -d ~/.docker/trust addhash -p $IMAGE $TAG $SIZE --sha256 $HASH -r targets/releases
```
##### Getting Size and Hash
There are several ways to get the size and hash of a particular manifest or index. Remember that you are signing a
tag, so you are looking for the size and hash of whatever the tag points to, manifest or index.
* `docker push`
* script
* `manifest-tool`
* `ocidist`
###### docker push
If you pushed the image tag using `docker push`, the very last line of output will give you the hash and size:
```console
$ docker push linuxkit/containerd:1ae8f054e9fe792d1dbdb9a65f1b5e14491cb106
The push refers to repository [docker.io/linuxkit/containerd]
fce5742422e4: Layer already exists
48a02e7b3096: Layer already exists
4381f8a59bb1: Layer already exists
c0328291406b: Layer already exists
79053b1996f5: Layer already exists
a4aa19c608556f7d786852557c36136255220c1f: digest: sha256:164f6c27410f145b479cdce1ed08e694c9b3d1e3e320c94d0e1ece9755043ea8 size: 1357
```
The first part is the tag you pushed, followed by the keyword `digest`, then the hash, then the size.
##### script
The following script command will provide the output for docker hub. Set the `IMAGE` name and `TAG`
environment variables.
```console
IMAGE=linuxkit/containerd
TAG=v0.8-amd64
jwt=$(curl -sSL "https://auth.docker.io/token?service=registry.docker.io&scope=repository:${IMAGE}:pull" | jq -r .token)
curl https://index.docker.io/v2/linuxkit/containerd/manifests/${TAG} -H "Authorization: Bearer ${jwt}" -H "Accept: application/vnd.docker.distribution.manifest.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, application/vnd.docker.distribution.manifest.list.v2+json" -D /dev/stdout -o /dev/null -s
```
##### manifest-tool
The [manifest-tool](https://github.com/estesp/manifest-tool) allows you to inspect manifests, including
both OCI indexes, a.k.a. multi-arch manifests, and simple manifests.
If you inspect the actual tag, you will get just the hash, not the size.
If you inspect an index that includes a manifest that you want, you will get the hash and size.
For example, inspecting just a single arch manifest gives us the hash on the second line, but not the
size:
```console
$ manifest-tool inspect linuxkit/containerd:1ae8f054e9fe792d1dbdb9a65f1b5e14491cb106-amd64
Name: linuxkit/containerd:1ae8f054e9fe792d1dbdb9a65f1b5e14491cb106-amd64 (Type: application/vnd.docker.distribution.manifest.v2+json)
Digest: sha256:0dc4f37966e23c0dffa6961119f29100c6d181b221e748c4688a280c08ab52a8
OS: linux
Arch: amd64
# Layers: 5
layer 1: digest = sha256:319073c03e01a960e61913b0e05b4e0094061726f6959732371a1496098c0980
layer 2: digest = sha256:85521c11021aed78da3b61193b3e2cd1f316040eb08744f684cb98fa8ba35dc3
layer 3: digest = sha256:f29bf65845868b4b2adccc661040b939e4119ca5b5cb34cb0583b8b4e279bcc9
layer 4: digest = sha256:95c51328f79f6be125241ba10488e8962bdfd807fe93fc5d4d990eea7ac065e2
layer 5: digest = sha256:794ca16dd5d22f1ccb5f58dea0ef9cb0c95d957ed33af5c4ab008cbdd30c359e
```
While inspecting the index that includes the above tag, gives us the hash but not the size of the
index, but finding the right entry, for example the first one is `amd64`, gives us the size as
`Mfst Length: 1357`:
```console
$ manifest-tool inspect linuxkit/containerd:1ae8f054e9fe792d1dbdb9a65f1b5e14491cb106
Name: linuxkit/containerd:1ae8f054e9fe792d1dbdb9a65f1b5e14491cb106 (Type: application/vnd.docker.distribution.manifest.list.v2+json)
Digest: sha256:247e1eb712c2f5e9d80bb1a9ddf9bb5479b3f785a7e0dd4a8844732bbaa96851
* Contains 3 manifest references:
1 Mfst Type: application/vnd.docker.distribution.manifest.v2+json
1 Digest: sha256:0dc4f37966e23c0dffa6961119f29100c6d181b221e748c4688a280c08ab52a8
1 Mfst Length: 1357
1 Platform:
1 - OS: linux
1 - OS Vers:
1 - OS Feat: []
1 - Arch: amd64
1 - Variant:
1 # Layers: 5
layer 1: digest = sha256:319073c03e01a960e61913b0e05b4e0094061726f6959732371a1496098c0980
layer 2: digest = sha256:85521c11021aed78da3b61193b3e2cd1f316040eb08744f684cb98fa8ba35dc3
layer 3: digest = sha256:f29bf65845868b4b2adccc661040b939e4119ca5b5cb34cb0583b8b4e279bcc9
layer 4: digest = sha256:95c51328f79f6be125241ba10488e8962bdfd807fe93fc5d4d990eea7ac065e2
layer 5: digest = sha256:794ca16dd5d22f1ccb5f58dea0ef9cb0c95d957ed33af5c4ab008cbdd30c359e
2 Mfst Type: application/vnd.docker.distribution.manifest.v2+json
2 Digest: sha256:febd923be587826c64db19c429f92a369d6e41d8abb715ff30643250ceafa621
2 Mfst Length: 1357
2 Platform:
2 - OS: linux
2 - OS Vers:
2 - OS Feat: []
2 - Arch: arm64
2 - Variant:
2 # Layers: 5
layer 1: digest = sha256:c35625c316366a48ec51192731e4155191b39fac7848e1b41fa46be1de9d11dc
layer 2: digest = sha256:a73cb03ae4fe7b79bf9ec202ee734a55f962a597b93e9a9625c64e9f2be9e78f
layer 3: digest = sha256:75b2023060fd85e40f4eed9fc5fe60c5b1866d909fc9ea783a21318ec2437e96
layer 4: digest = sha256:413204d4c4ee875fd84dd93799ed1346cfb15e02a508b6306ea7da1a160babc3
layer 5: digest = sha256:cf2293c110f0718e58e01ff4cbafa53eadde280999902fcdcd57269e8ba48339
3 Mfst Type: application/vnd.docker.distribution.manifest.v2+json
3 Digest: sha256:b6adad183487d969059b3badeb5dce032bb449f61607eb024d91cfeabcaf0e57
3 Mfst Length: 1357
3 Platform:
3 - OS: linux
3 - OS Vers:
3 - OS Feat: []
3 - Arch: s390x
3 - Variant:
3 # Layers: 5
layer 1: digest = sha256:16c1054185680ee839fa57dff29f412c179f1739191c12d33ab59bceca28a8ac
layer 2: digest = sha256:e38fe65829ed75127337f18dc2a641e2e9f6c2859a314cf5ac1b7d5022150e26
layer 3: digest = sha256:f2e84a29733f5f17cc860468b94eeeebf378d2a8af9bfc468427b1da430fe927
layer 4: digest = sha256:b38f9350a90499ce01e7704a58b52c90ee28c5562379f7096ce930b5fea160be
layer 5: digest = sha256:cc86a47d79015d074b41a4a3f0918e98dfb13f2fc6ef8def180a81fd36ae2544
```
##### ocidist
[ocidist](https://github.com/deitch/ocidist) is a simple utility to inspect or pull images, manifests,
indexes and individual blobs. If you call `ocidist manifest` and pass it the `--detail` flag, it will
report the hash and size.
For an index:
```console
$ ocidist manifest docker.io/linuxkit/containerd:1ae8f054e9fe792d1dbdb9a65f1b5e14491cb106 --detail
2020/11/12 11:00:03 ref name.Tag{Repository:name.Repository{Registry:name.Registry{insecure:false, registry:"index.docker.io"}, repository:"linuxkit/containerd"}, tag:"v0.8", original:"docker.io/linuxkit/containerd:1ae8f054e9fe792d1dbdb9a65f1b5e14491cb106"}
2020/11/12 11:00:03 advanced API
2020/11/12 11:00:06 referenced manifest hash sha256:247e1eb712c2f5e9d80bb1a9ddf9bb5479b3f785a7e0dd4a8844732bbaa96851 size 1052
{
"schemaVersion": 2,
"mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
"manifests": [
{
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"size": 1357,
"digest": "sha256:0dc4f37966e23c0dffa6961119f29100c6d181b221e748c4688a280c08ab52a8",
"platform": {
"architecture": "amd64",
"os": "linux"
}
},
{
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"size": 1357,
"digest": "sha256:febd923be587826c64db19c429f92a369d6e41d8abb715ff30643250ceafa621",
"platform": {
"architecture": "arm64",
"os": "linux"
}
},
{
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"size": 1357,
"digest": "sha256:b6adad183487d969059b3badeb5dce032bb449f61607eb024d91cfeabcaf0e57",
"platform": {
"architecture": "s390x",
"os": "linux"
}
}
]
}
```
For a single manifest:
```console
$ ocidist manifest docker.io/linuxkit/containerd:1ae8f054e9fe792d1dbdb9a65f1b5e14491cb106-amd64 --detail
2020/11/12 10:59:08 ref name.Tag{Repository:name.Repository{Registry:name.Registry{insecure:false, registry:"index.docker.io"}, repository:"linuxkit/containerd"}, tag:"v0.8-amd64", original:"docker.io/linuxkit/containerd:1ae8f054e9fe792d1dbdb9a65f1b5e14491cb106-amd64"}
2020/11/12 10:59:08 advanced API
2020/11/12 10:59:11 referenced manifest hash sha256:0dc4f37966e23c0dffa6961119f29100c6d181b221e748c4688a280c08ab52a8 size 1357
{
"schemaVersion": 2,
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"config": {
"mediaType": "application/vnd.docker.container.image.v1+json",
"size": 1973,
"digest": "sha256:b11103cf6c84fc3a2968d89e9d6fd7ce9e427380098c17828e3bda27de61ed6a"
},
"layers": [
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 41779632,
"digest": "sha256:319073c03e01a960e61913b0e05b4e0094061726f6959732371a1496098c0980"
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 328,
"digest": "sha256:85521c11021aed78da3b61193b3e2cd1f316040eb08744f684cb98fa8ba35dc3"
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 176,
"digest": "sha256:f29bf65845868b4b2adccc661040b939e4119ca5b5cb34cb0583b8b4e279bcc9"
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 202,
"digest": "sha256:95c51328f79f6be125241ba10488e8962bdfd807fe93fc5d4d990eea7ac065e2"
},
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 300,
"digest": "sha256:794ca16dd5d22f1ccb5f58dea0ef9cb0c95d957ed33af5c4ab008cbdd30c359e"
}
]
}
```
### Build packages as a developer
If you want to develop packages or test them locally, it is best to
override the hub organisation used. You may also want to disable
signing while developing. A typical example would be:
```
linuxkit pkg build -org=wombat -disable-content-trust «path-to-package»
linuxkit pkg build -org=wombat «path-to-package»
```
This will create a local image: `wombat/<image>:<hash>-<arch>` which
@@ -463,7 +193,7 @@ on other systems you can push the image to your hub account and pull
from a different system by issuing:
```
linuxkit pkg build -org=wombat -disable-content-trust push
linuxkit pkg build -org=wombat push
```
This will push both `wombat/<image>:<hash>-<arch>` and
@@ -473,7 +203,7 @@ Finally, if you are tired of the long hashes you can override the hash
with:
```
linuxkit pkg build -org=wombat -disable-content-trust -hash=foo push
linuxkit pkg build -org=wombat -hash=foo push
```
and this will create `wombat/<image>:foo-<arch>` and

2
docs/security.md
View File

@@ -50,8 +50,6 @@ and namespaced separately from the host as appropriate.
LinuxKit's build process heavily leverages Docker images for packaging. Of note, all intermediate build images
are referenced by digest to ensures reproducibility across LinuxKit builds. Tags are mutable, and thus subject to override
(intentionally or maliciously) - referencing by digest mitigates classes of registry poisoning attacks in LinuxKit's buildchain.
Certain images, such as the kernel image, will be signed by LinuxKit maintainers using [Docker Content Trust](https://docs.docker.com/engine/security/trust/content_trust/),
which guarantees authenticity, integrity, and freshness of the image.
Moreover, LinuxKit's build process leverages [Alpine Linux's](https://alpinelinux.org/) hardened userspace tools such as
Musl libc, and compiler options that include `-fstack-protector` and position-independent executable output. Go binaries

49
docs/signing.md
View File

@@ -1,49 +0,0 @@
# Signing LinuxKit Hub Images
We sign and verify LinuxKit component images, such as `linuxkit/kernel`, using [Notary](https://github.com/docker/notary).
This document details the process for setting this up, intended for maintainers.
## Initialize a New Repository
Let's say we're publishing a new `linuxkit/foo` image that we want to sign and verify in LinuxKit.
We first need to initialize the Notary repository:
```
notary -s https://notary.docker.io -d ~/.docker/trust init -p docker.io/linuxkit/foo
```
This command will generate some private keys in `~/.docker/trust` and ask you for passphrases such that they are encrypted at rest.
All linuxkit repositories are currently using the same root key so we can pin trust on key ID `1908a0cf4f55710138e63f65ab2a97e8fa3948e5ca3b8857a29f235a3b61ea1b`.
We'll also let the notary server take control of the snapshot key, for easier delegation collaboration:
```
notary -s https://notary.docker.io -d ~/.docker/trust key rotate docker.io/linuxkit/foo snapshot -r
```
## Add maintainers to delegation roles:
Maintainers are to sign with `delegation` keys, which are adminstered by a non-root key.
Thusly, they are easily rotated without having to bring the root key online.
Additionally, maintainers can be added to separate roles for auditing purposes: the current setup is to add maintainers to both the `targets/releases` role that is intended
for release consumption, as well as an individual `targets/<maintainer_name>` role for auditing.
Docker will automatically sign into both roles when pushing with Docker Content Trust.
Here's what the command looks like to add all maintainers to the `targets/releases` role:
```
notary -s https://notary.docker.io -d ~/.docker/trust delegation add -p docker.io/linuxkit/foo targets/releases alice.crt bob.crt charlie.crt --all-paths
```
Here's what the commands look like to add all maintainers to their individually named roles:
```
notary -s https://notary.docker.io -d ~/.docker/trust delegation add -p docker.io/linuxkit/foo targets/alice alice.crt --all-paths
notary -s https://notary.docker.io -d ~/.docker/trust delegation add -p docker.io/linuxkit/foo targets/bob bob.crt --all-paths
notary -s https://notary.docker.io -d ~/.docker/trust delegation add -p docker.io/linuxkit/foo targets/charlie charlie.crt --all-paths
```
## Maintainers import their private keys
It's important that each maintainer imports their private key into Docker's key storage, so Docker can use it to sign:
```
notary -d ~/.docker/trust key import alice.key -r user
```

13
docs/yaml.md
View File

@@ -125,19 +125,6 @@ file:
Because a `tmpfs` is mounted onto `/var`, `/run`, and `/tmp` by default, the `tmpfs` mounts will shadow anything specified in `files` section for those directories.
## `trust`
The `trust` section specifies which build components are to be cryptographically verified with
[Docker Content Trust](https://docs.docker.com/engine/security/trust/content_trust/) prior to pulling.
Trust is a central concern in any build system, and LinuxKit's is no exception: Docker Content Trust provides authenticity,
integrity, and freshness guarantees for the components it verifies. The LinuxKit maintainers are responsible for signing
`linuxkit` components, though collaborators can sign their own images with Docker Content Trust or [Notary](https://github.com/docker/notary).
- `image` lists which individual images to enforce pulling with Docker Content Trust.
The image name may include tag or digest, but the matching also succeeds if the base image name is the same.
- `org` lists which organizations for which Docker Content Trust is to be enforced across all images,
for example `linuxkit` is the org for `linuxkit/kernel`
## Image specification
Entries in the `onboot` and `services` sections specify an OCI image and