20 Commits

Author SHA1 Message Date
Avi Deitcher
dc8c6d5985 Merge pull request #4089 from deitch/tag-in-build-yml
support --tag in build.yml for packages
2024-12-23 18:00:06 +02:00
Avi Deitcher
4f765b5da0 support --tag in build.yml for packages
Signed-off-by: Avi Deitcher <avi@deitcher.net>
2024-12-23 17:28:49 +02:00
Avi Deitcher
ad95c6fc2e Merge pull request #4085 from deitch/volume-image
additional volume support in building
2024-10-01 15:57:17 +03:00
Avi Deitcher
76f4802ccf additional volume support in building
Signed-off-by: Avi Deitcher <avi@deitcher.net>
2024-10-01 15:27:55 +03:00
Avi Deitcher
e4d41061b6 Merge pull request #4084 from deitch/cache-platform-instead-of-arch
internal restructure to use explicit platform instead of implicit arch in cache
2024-10-01 15:14:21 +03:00
Avi Deitcher
81f0c3eff2 internal restructure to use explicit platform instead of implicit arch in cache
Signed-off-by: Avi Deitcher <avi@deitcher.net>
2024-10-01 14:30:03 +03:00
Avi Deitcher
5e3f7dd9a5 Merge pull request #4083 from deitch/restructure-logging
restructure logging
2024-10-01 14:00:06 +03:00
Avi Deitcher
67e9e22a36 restructure logging
Signed-off-by: Avi Deitcher <avi@deitcher.net>
2024-10-01 12:50:43 +03:00
Avi Deitcher
8556f024ef Merge pull request #4082 from kolyshkin/moby-cap
vendor: switch to moby/sys/capability
2024-10-01 11:07:29 +03:00
Kir Kolyshkin
da3be29998 vendor: switch to moby/sys/capability
github.com/moby/sys/capability is a fork of the (no longer maintained)
github.com/syndtr/gocapability package.

For changes since the fork took place, see
https://github.com/moby/sys/blob/main/capability/CHANGELOG.md

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2024-09-30 18:10:16 -07:00
Avi Deitcher
d7a6bc8899 Merge pull request #4077 from deitch/docker-bump
bump docker deps to v27.2.0
2024-09-08 13:00:19 +03:00
Avi Deitcher
2159aacb09 bump docker deps to v27.2.0
Signed-off-by: Avi Deitcher <avi@deitcher.net>
2024-09-08 12:22:57 +03:00
Avi Deitcher
fa3207c86e Merge pull request #4072 from christoph-zededa/docker_cache_consider_architecture
moby: check architecture for docker image
2024-08-29 22:15:19 +03:00
Avi Deitcher
1d6d5fa612 Merge pull request #4074 from deitch/efi-kernel
remove linuxefi grub EFI handover to normal linux loading
2024-08-29 21:13:48 +03:00
Avi Deitcher
ba25e59640 remove linuxefi grub EFI handover to normal linux loading
Signed-off-by: Avi Deitcher <avi@deitcher.net>
2024-08-29 17:30:57 +03:00
Avi Deitcher
6979859e76 Merge pull request #4073 from deitch/init-debug-no-control
use only stdout/stderr or file for runc output
2024-08-28 15:28:33 +03:00
Avi Deitcher
5848a2856f use only stdout/stderr or file for runc output
Signed-off-by: Avi Deitcher <avi@deitcher.net>
2024-08-28 14:45:26 +03:00
Christoph Ostarek
cb8f36adf3 moby: check architecture for docker image
under certain cases the container image is already in the local docker
registry, but with the wrong architecture; in this case just pretend
it is not there and let the caller decide if they want to build it

Signed-off-by: Christoph Ostarek <christoph@zededa.com>
2024-08-27 15:49:21 +02:00
Avi Deitcher
5f09346e1e Merge pull request #4070 from deitch/verbose-runc
more verbose runc messages
2024-08-22 20:55:44 +03:00
Avi Deitcher
15c808c4ee more verbose runc messages
Signed-off-by: Avi Deitcher <avi@deitcher.net>
2024-08-22 20:23:44 +03:00
195 changed files with 1696 additions and 789 deletions

View File

@@ -122,13 +122,13 @@ jobs:
- name: Build Packages
# Skip s390x as emulation is unreliable
run: |
make OPTIONS="-v --skip-platforms linux/s390x" -C pkg build
make OPTIONS="-v 2 --skip-platforms linux/s390x" -C pkg build
- name: Build Test Packages
# ensures that the test packages are in linuxkit cache when we need them for tests later
# Skip s390x as emulation is unreliable
run: |
make OPTIONS="-v --skip-platforms linux/s390x" -C test/pkg build
make OPTIONS="-v 2 --skip-platforms linux/s390x" -C test/pkg build
- name: Check Kernel Dependencies up to date
# checks that any kernel dependencies are up to date.
@@ -145,7 +145,7 @@ jobs:
# ensures that the kernel packages are in linuxkit cache when we need them for tests later
# no need for excluding s390x, as each build.yml in the kernel explicitly lists archs
run: |
make OPTIONS="-v" -C kernel build
make OPTIONS="-v 2" -C kernel build
- name: list cache contents
run: |

View File

@@ -11,5 +11,9 @@ within linuxkit. Unless standard Linux options exist, these all are prefaced wit
| Option | Description |
|---|---|
| `linuxkit.runc_debug=1` | Start runc for `onboot` and `onshutdown` containers to run with `--debug`. Also sends output to the console, in addition to the normal output to logfiles. If not present or set to 0, default to usual mode. |
| `linuxkit.unified_cgroup_hierarchy=0` | Start up cgroups v1. If not present or set to 1, default to cgroups v1. |
| `linuxkit.runc_debug=1` | Start runc for `onboot` and `onshutdown` containers to run with `--debug`, and add extra logging messages for each stage of starting those containers. If not present or set to 0, default to usual mode. |
| `linuxkit.runc_console=1` | Send logs for runc for `onboot` and `onshutdown` containers, as well as the output of the containers themselves, to the console, instead of the normal output to logfiles. If not present or set to 0, default to usual mode. |
It often is useful to combine both of the `linuxkit.runc_debug` and `linuxkit.runc_console` options to get the most
information about what is happening with `onboot` containers.

View File

@@ -50,6 +50,7 @@ A package source consists of a directory containing at least two files:
- `image` _(string)_: *(mandatory)* The name of the image to build
- `org` _(string)_: The hub/registry organisation to which this package belongs
- `tag` _(string)_: The tag to use for the image, can be fixed string or template (default: `{{.Hash}}`)
- `dockerfile` _(string)_: The dockerfile to use to build this package, must be in this directory or below (default: `Dockerfile`)
- `arches` _(list of string)_: The architectures which this package should be built for (valid entries are `GOARCH` names)
- `extra-sources` _(list of strings)_: Additional sources for the package outside the package directory. The format is `src:dst`, where `src` can be relative to the package directory and `dst` is the destination in the build context. This is useful for sharing files, such as vendored go code, between packages.

View File

@@ -18,8 +18,17 @@ For private registries or private repositories on a registry credentials provide
## Sections
The configuration file is processed in the order `kernel`, `init`, `onboot`, `onshutdown`,
`services`, `files`, `volumes`. Each section adds files to the root file system. Sections may be omitted.
The configuration file is processed in the order:
1. `kernel`
1. `init`
1. `volumes`
1. `onboot`
1. `onshutdown`
1. `services`
1. `files`
Each section adds files to the root file system. Sections may be omitted.
Each container that is specified is allocated a unique `uid` and `gid` that it may use if it
wishes to run as an isolated user (or user namespace). Anywhere you specify a `uid` or `gid`
@@ -100,8 +109,13 @@ including those in `services`, `onboot` and `onshutdown`. The volumes are create
chosen by linuxkit at build-time. The volumes then can be referenced by other containers and
mounted into them.
Volumes normally are blank directories. If an image is provided, the contents of that image
will be used to populate the volume.
Volumes can be in one of several formats:
* Blank directory: This is the default, and is an empty directory that is created at build-time. It is an overlayfs mount, and can be shared among multiple containers.
* Image laid out as filesystem: The contents of the image are used to populate the volume. Default format when an image is provided.
* Image as OCI v1-layout: The image is used as an [OCI v1-layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md). Indicated by `format: oci`.
Examples of each are given later in this section.
The `volumes` section can declare a volume to be read-write or read-only. If the volume is read-write,
a volume that is mounted into a container can be mounted read-only or read-write. If the volume is read-only,
@@ -111,7 +125,36 @@ By default, volumes are created read-write, and are mounted read-write.
Volume names **must** be unique, and must contain only lower-case alphanumeric characters, hyphens, and
underscores.
Sample `volumes` section:
#### Samples of `volumes`
##### Empty directory
Yaml showing both read-only and read-write:
```yml
volumes:
- name: dira
readonly: true
- name: dirb
readonly: true
```
Contents:
```sh
$ cd dir && ls -la
drwxr-xr-x 19 root wheel 608 Sep 30 15:03 .
drwxrwxrwt 130 root wheel 4160 Sep 30 15:03 ..
```
In the above example:
* `dira` is empty and is read-only.
* `volb` is empty and is read-write.
##### Image directory
Yaml showing both read-only and read-write:
```yml
volumes:
@@ -120,8 +163,7 @@ volumes:
readonly: true
- name: volb
image: alpine:latest
readonly: false
- name: volc
format: filesystem # optional, as this is the default format
readonly: false
```
@@ -129,7 +171,56 @@ In the above example:
* `vola` is populated by the contents of `alpine:latest` and is read-only.
* `volb` is populated by the contents of `alpine:latest` and is read-write.
* `volc` is an empty volume and is read-write.
Contents:
```sh
$ cd dir && ls -la
drwxr-xr-x 19 root wheel 608 Sep 30 15:03 .
drwxrwxrwt 130 root wheel 4160 Sep 30 15:03 ..
drwxr-xr-x 84 root wheel 2688 Sep 6 14:34 bin
drwxr-xr-x 2 root wheel 64 Sep 6 14:34 dev
drwxr-xr-x 37 root wheel 1184 Sep 6 14:34 etc
drwxr-xr-x 2 root wheel 64 Sep 6 14:34 home
drwxr-xr-x 13 root wheel 416 Sep 6 14:34 lib
drwxr-xr-x 5 root wheel 160 Sep 6 14:34 media
drwxr-xr-x 2 root wheel 64 Sep 6 14:34 mnt
drwxr-xr-x 2 root wheel 64 Sep 6 14:34 opt
dr-xr-xr-x 2 root wheel 64 Sep 6 14:34 proc
drwx------ 2 root wheel 64 Sep 6 14:34 root
drwxr-xr-x 2 root wheel 64 Sep 6 14:34 run
drwxr-xr-x 63 root wheel 2016 Sep 6 14:34 sbin
drwxr-xr-x 2 root wheel 64 Sep 6 14:34 srv
drwxr-xr-x 2 root wheel 64 Sep 6 14:34 sys
drwxr-xr-x 2 root wheel 64 Sep 6 14:34 tmp
drwxr-xr-x 7 root wheel 224 Sep 6 14:34 usr
drwxr-xr-x 13 root wheel 416 Sep 6 14:34 var
```
##### Image OCI Layout
Yaml showing both read-only and read-write, and both all architectures and a limited subset:
```yml
volumes:
- name: volo
image: alpine:latest
format: oci
readonly: true
- name: volp
image: alpine:latest
readonly: false
format: oci
platforms:
- linux/amd64
```
In the above example:
* `volo` is populated by the contents of `alpine:latest` as an OCI v1-layout for all architectures and is read-only.
* `volb` is populated by the contents of `alpine:latest` as an OCI v1-layout just for linux/amd64 and is read-write.
##### Volumes in `services`
Sample usage of volumes in `services` section:

View File

@@ -2,7 +2,7 @@ kernel:
image: linuxkit/kernel:6.6.13
cmdline: "console=tty0 console=ttyS0 console=ttyAMA0 console=ttysclp0"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435
- linuxkit/ca-certificates:5aaa343474e5ac3ac01f8b917e82efb1063d80ff

View File

@@ -2,7 +2,7 @@ kernel:
image: linuxkit/kernel:6.6.13
cmdline: "console=tty0 console=ttyS0 console=ttyAMA0 console=ttysclp0"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435
- linuxkit/ca-certificates:5aaa343474e5ac3ac01f8b917e82efb1063d80ff

View File

@@ -3,7 +3,7 @@ kernel:
image: linuxkit/kernel:6.6.13
cmdline: "console=tty0 console=ttyS0 console=ttyAMA0"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435
- linuxkit/ca-certificates:5aaa343474e5ac3ac01f8b917e82efb1063d80ff

View File

@@ -2,7 +2,7 @@ kernel:
image: linuxkit/kernel:6.6.13
cmdline: "console=tty0 console=ttyS0"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435
- linuxkit/ca-certificates:5aaa343474e5ac3ac01f8b917e82efb1063d80ff

View File

@@ -2,7 +2,7 @@ kernel:
image: linuxkit/kernel:6.6.13
cmdline: "console=tty0 console=ttyS0"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435
- linuxkit/ca-certificates:5aaa343474e5ac3ac01f8b917e82efb1063d80ff

View File

@@ -4,7 +4,7 @@ kernel:
cmdline: "console=ttyS0 page_poison=1"
init:
- linuxkit/vpnkit-expose-port:77e45e4681c78d59f1d8a48818260948d55f9d05 # install vpnkit-expose-port and vpnkit-iptables-wrapper on host
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435
- linuxkit/ca-certificates:5aaa343474e5ac3ac01f8b917e82efb1063d80ff

View File

@@ -2,7 +2,7 @@ kernel:
image: linuxkit/kernel:6.6.13
cmdline: "console=tty0 console=ttyS0 console=ttyAMA0 console=ttysclp0"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435
- linuxkit/ca-certificates:5aaa343474e5ac3ac01f8b917e82efb1063d80ff

View File

@@ -2,7 +2,7 @@ kernel:
image: linuxkit/kernel:6.6.13
cmdline: "console=tty0 console=ttyS0 console=ttyAMA0 console=ttysclp0"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435
- linuxkit/ca-certificates:5aaa343474e5ac3ac01f8b917e82efb1063d80ff

View File

@@ -2,7 +2,7 @@ kernel:
image: linuxkit/kernel:6.6.13
cmdline: "console=tty0 console=ttyS0 console=ttyAMA0 console=ttysclp0"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435
- linuxkit/ca-certificates:5aaa343474e5ac3ac01f8b917e82efb1063d80ff

View File

@@ -2,7 +2,7 @@ kernel:
image: linuxkit/kernel:6.6.13
cmdline: "console=tty0 console=ttyS0 console=ttyAMA0"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435
- linuxkit/ca-certificates:5aaa343474e5ac3ac01f8b917e82efb1063d80ff

View File

@@ -3,7 +3,7 @@ kernel:
image: linuxkit/kernel:6.6.13
cmdline: "console=tty0 console=ttyS0 console=ttyAMA0"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435
- linuxkit/ca-certificates:5aaa343474e5ac3ac01f8b917e82efb1063d80ff

View File

@@ -2,7 +2,7 @@ kernel:
image: linuxkit/kernel:6.6.13
cmdline: "console=tty0 console=ttyS0 console=ttyAMA0"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435
onboot:

View File

@@ -2,7 +2,7 @@ kernel:
image: linuxkit/kernel:6.6.13
cmdline: "console=tty0 console=ttyS0"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435
services:

View File

@@ -2,7 +2,7 @@ kernel:
image: linuxkit/kernel:6.6.13
cmdline: "console=ttyS0"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435
- linuxkit/ca-certificates:5aaa343474e5ac3ac01f8b917e82efb1063d80ff

View File

@@ -2,7 +2,7 @@ kernel:
image: linuxkit/kernel:6.6.13
cmdline: "console=ttyS0"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435
- linuxkit/ca-certificates:5aaa343474e5ac3ac01f8b917e82efb1063d80ff

View File

@@ -2,7 +2,7 @@ kernel:
image: linuxkit/kernel:6.6.13
cmdline: "console=ttyS0"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435
- linuxkit/ca-certificates:5aaa343474e5ac3ac01f8b917e82efb1063d80ff

View File

@@ -3,7 +3,7 @@ kernel:
cmdline: console=ttyS1
ucode: intel-ucode.cpio
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435
- linuxkit/ca-certificates:5aaa343474e5ac3ac01f8b917e82efb1063d80ff

View File

@@ -2,7 +2,7 @@ kernel:
image: linuxkit/kernel:6.6.13
cmdline: "console=ttyS0"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435
- linuxkit/ca-certificates:5aaa343474e5ac3ac01f8b917e82efb1063d80ff

View File

@@ -3,7 +3,7 @@ kernel:
cmdline: console=ttyS1
ucode: intel-ucode.cpio
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435
- linuxkit/ca-certificates:5aaa343474e5ac3ac01f8b917e82efb1063d80ff

View File

@@ -2,7 +2,7 @@ kernel:
image: linuxkit/kernel:6.6.13-rt
cmdline: "console=tty0"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435
- linuxkit/ca-certificates:5aaa343474e5ac3ac01f8b917e82efb1063d80ff

View File

@@ -2,7 +2,7 @@ kernel:
image: linuxkit/kernel:6.6.13
cmdline: "console=tty0 console=ttyS0 console=ttyAMA0 console=ttysclp0 root=/dev/vda"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435
- linuxkit/ca-certificates:5aaa343474e5ac3ac01f8b917e82efb1063d80ff

View File

@@ -2,7 +2,7 @@ kernel:
image: linuxkit/kernel:6.6.13
cmdline: "console=tty0"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435
- linuxkit/ca-certificates:5aaa343474e5ac3ac01f8b917e82efb1063d80ff

View File

@@ -2,7 +2,7 @@ kernel:
image: linuxkit/kernel:6.6.13
cmdline: "console=ttyS0"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435
- linuxkit/ca-certificates:5aaa343474e5ac3ac01f8b917e82efb1063d80ff

View File

@@ -4,7 +4,7 @@ kernel:
image: linuxkit/kernel:6.6.13
cmdline: "console=tty0 console=ttyS0 console=ttyAMA0 console=ttysclp0"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435
onboot:

View File

@@ -2,7 +2,7 @@ kernel:
image: linuxkit/kernel:6.6.13
cmdline: "console=tty0 console=ttyS0 console=ttyAMA0 console=ttysclp0"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435
- linuxkit/ca-certificates:5aaa343474e5ac3ac01f8b917e82efb1063d80ff

View File

@@ -2,7 +2,7 @@ kernel:
image: linuxkit/kernel:6.6.13
cmdline: "console=tty0 console=ttyS0 console=ttyAMA0"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435
onboot:

View File

@@ -2,7 +2,7 @@ kernel:
image: linuxkit/kernel:6.6.13
cmdline: "console=tty0 console=ttyS0 console=ttyAMA0 console=ttysclp0"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435
- linuxkit/ca-certificates:5aaa343474e5ac3ac01f8b917e82efb1063d80ff

View File

@@ -2,7 +2,7 @@ kernel:
image: linuxkit/kernel:6.6.13
cmdline: "console=tty0 console=ttyS0"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435
- linuxkit/ca-certificates:5aaa343474e5ac3ac01f8b917e82efb1063d80ff

View File

@@ -3,7 +3,7 @@ kernel:
image: linuxkit/kernel:6.6.13
cmdline: "console=tty0 console=ttyS0 console=ttyAMA0"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435
- linuxkit/ca-certificates:5aaa343474e5ac3ac01f8b917e82efb1063d80ff

View File

@@ -2,7 +2,7 @@ kernel:
image: linuxkit/kernel:6.6.13
cmdline: "console=ttyS0"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435
onboot:

View File

@@ -2,7 +2,7 @@ kernel:
image: linuxkit/kernel:6.6.13
cmdline: "console=ttyS0"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435
onboot:

View File

@@ -2,7 +2,7 @@ kernel:
image: linuxkit/kernel:6.6.13
cmdline: "console=tty0 console=ttyS0 console=ttyAMA0"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435
- linuxkit/ca-certificates:5aaa343474e5ac3ac01f8b917e82efb1063d80ff

View File

@@ -2,7 +2,7 @@ kernel:
image: linuxkit/kernel:6.6.13
cmdline: "console=tty0 console=ttyS0 console=ttyAMA0"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435
- linuxkit/ca-certificates:5aaa343474e5ac3ac01f8b917e82efb1063d80ff

View File

@@ -74,21 +74,29 @@ func runcInit(rootPath, serviceType string) int {
log.Printf("Using %s", msg)
// did we choose to run in debug mode? If so, runc will be in debug, and all messages will go to stdout/stderr in addition to the log
var runcDebugMode bool
var runcDebugMode, runcConsoleMode bool
dt, err := os.ReadFile("/proc/cmdline")
if err != nil {
log.Fatalf("error reading /proc/cmdline: %v", err)
}
debugLogger := log.New()
debugLogger.Level = log.InfoLevel
for _, s := range strings.Fields(string(dt)) {
if s == "linuxkit.runc_debug=1" {
runcDebugMode = true
break
debugLogger.Level = log.DebugLevel
}
if s == "linuxkit.runc_console=1" {
runcConsoleMode = true
}
}
for _, file := range files {
name := file.Name()
path := filepath.Join(rootPath, name)
log.Printf("%s %s: from %s", serviceType, name, path)
runtimeConfig := getRuntimeConfig(path)
@@ -97,6 +105,7 @@ func runcInit(rootPath, serviceType string) int {
status = 1
continue
}
debugLogger.Debugf("%s %s: creating", serviceType, name)
pidfile := filepath.Join(tmpdir, name)
cmdArgs := []string{"create", "--bundle", path, "--pid-file", pidfile, name}
if runcDebugMode {
@@ -122,12 +131,16 @@ func runcInit(rootPath, serviceType string) int {
}
defer stderr.Close()
if runcDebugMode {
cmd.Stdout = io.MultiWriter(stdout, os.Stdout)
cmd.Stderr = io.MultiWriter(stderr, os.Stderr)
} else {
cmd.Stdout = stdout
cmd.Stderr = stderr
cmd.Stdout = stdout
cmd.Stderr = stderr
// if in console mode, send output to stdout/stderr instead of the log
// do not try io.MultiWriter(os.Stdout, stdout) as console messages will hang.
// it is not clear why, but since this is all for debugging anyways, it doesn't matter
// much.
if runcConsoleMode {
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
}
if err := cmd.Run(); err != nil {
@@ -149,6 +162,7 @@ func runcInit(rootPath, serviceType string) int {
continue
}
debugLogger.Debugf("%s %s: preparing", serviceType, name)
if err := prepareProcess(pid, runtimeConfig); err != nil {
log.Printf("Cannot prepare process: %v", err)
status = 1
@@ -166,7 +180,12 @@ func runcInit(rootPath, serviceType string) int {
waitFor <- state
}()
cmd = exec.Command(runcBinary, "start", name)
debugLogger.Debugf("%s %s: starting", serviceType, name)
cmdArgs = []string{"start", name}
if runcDebugMode {
cmdArgs = append([]string{"--debug"}, cmdArgs...)
}
cmd = exec.Command(runcBinary, cmdArgs...)
cmd.Stdout = stdout
cmd.Stderr = stderr
@@ -176,8 +195,10 @@ func runcInit(rootPath, serviceType string) int {
continue
}
debugLogger.Debugf("%s %s: waiting for completion", serviceType, name)
_ = <-waitFor
debugLogger.Debugf("%s %s: cleaning up", serviceType, name)
cleanup(path)
_ = os.Remove(pidfile)
@@ -186,6 +207,7 @@ func runcInit(rootPath, serviceType string) int {
// once that is fixed, this can be cleaned up
logger.Dump(stdoutLog)
logger.Dump(stderrLog)
debugLogger.Debugf("%s %s: complete", serviceType, name)
}
_ = os.RemoveAll(tmpdir)

View File

@@ -2,7 +2,7 @@ kernel:
image: linuxkit/kernel-clear-containers:4.9.x
cmdline: "root=/dev/pmem0p1 rootflags=dax,data=ordered,errors=remount-ro rw rootfstype=ext4 tsc=reliable no_timer_check rcupdate.rcu_expedited=1 i8042.direct=1 i8042.dumbkbd=1 i8042.nopnp=1 i8042.noaux=1 noreplace-smp reboot=k panic=1 console=hvc0 console=hvc1 initcall_debug iommu=off quiet cryptomgr.notests page_poison=on"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
onboot:
- name: sysctl
image: mobylinux/sysctl:2cf2f9d5b4d314ba1bfc22b2fe931924af666d8c

View File

@@ -2,7 +2,7 @@ kernel:
image: linuxkit/kernel:6.6.13
cmdline: "console=ttyS0 page_poison=1"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435
- linuxkit/ca-certificates:5aaa343474e5ac3ac01f8b917e82efb1063d80ff

View File

@@ -2,7 +2,7 @@ kernel:
image: linuxkit/kernel:6.6.13
cmdline: "console=ttyS0 page_poison=1"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435
- linuxkit/ca-certificates:5aaa343474e5ac3ac01f8b917e82efb1063d80ff

View File

@@ -2,7 +2,7 @@ kernel:
image: linuxkit/kernel-ima:4.11.1-186dd3605ee7b23214850142f8f02b4679dbd148
cmdline: "console=ttyS0 console=tty0 page_poison=1 ima_appraise=enforce_ns"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435
- linuxkit/ca-certificates:5aaa343474e5ac3ac01f8b917e82efb1063d80ff

View File

@@ -2,7 +2,7 @@ kernel:
image: mobylinux/kernel-landlock:4.9.x
cmdline: "console=ttyS0 page_poison=1"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- mobylinux/runc:b0fb122e10dbb7e4e45115177a61a3f8d68c19a9
- mobylinux/containerd:18eaf72f3f4f9a9f29ca1951f66df701f873060b
- mobylinux/ca-certificates:eabc5a6e59f05aa91529d80e9a595b85b046f935

View File

@@ -2,7 +2,7 @@ kernel:
image: "linuxkitprojects/kernel-memorizer:4.10_dbg"
cmdline: "console=ttyS0 page_poison=1"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435
onboot:

View File

@@ -2,7 +2,7 @@ kernel:
image: linuxkit/kernel:6.6.13
cmdline: "console=ttyS0 page_poison=1"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435
- linuxkit/ca-certificates:5aaa343474e5ac3ac01f8b917e82efb1063d80ff

View File

@@ -2,7 +2,7 @@ kernel:
image: linuxkit/kernel:6.6.13
cmdline: "console=ttyS0 page_poison=1"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435
onboot:

View File

@@ -2,7 +2,7 @@ kernel:
image: okernel:latest
cmdline: "console=tty0 page_poison=1"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435
- linuxkit/ca-certificates:5aaa343474e5ac3ac01f8b917e82efb1063d80ff

View File

@@ -2,7 +2,7 @@ kernel:
image: linuxkitprojects/kernel-shiftfs:4.11.4-881a041fc14bd95814cf140b5e98d97dd65160b5
cmdline: "console=ttyS0 console=tty0 page_poison=1"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435
- linuxkit/ca-certificates:5aaa343474e5ac3ac01f8b917e82efb1063d80ff

View File

@@ -22,12 +22,13 @@ func createPackageResolver(baseDir string) spec.PackageResolver {
pkgValue = pkgTmpl
case strings.HasPrefix(pkgTmpl, templateFlag+templatePkg):
pkgPath := strings.TrimPrefix(pkgTmpl, templateFlag+templatePkg)
piBase := pkglib.NewPkgInfo()
var pkgs []pkglib.Pkg
pkgConfig := pkglib.PkglibConfig{
BuildYML: defaultPkgBuildYML,
HashCommit: defaultPkgCommit,
Tag: defaultPkgTag,
Tag: piBase.Tag,
}
pkgs, err = pkglib.NewFromConfig(pkgConfig, path.Join(baseDir, pkgPath))
if err != nil {

View File

@@ -8,6 +8,7 @@ import (
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/match"
"github.com/google/go-containerregistry/pkg/v1/partial"
imagespec "github.com/opencontainers/image-spec/specs-go/v1"
)
// matchPlatformsOSArch because match.Platforms rejects it if the provided
@@ -46,7 +47,7 @@ func matchAllAnnotations(annotations map[string]string) match.Matcher {
}
}
func (p *Provider) findImage(imageName, architecture string) (v1.Image, error) {
func (p *Provider) findImage(imageName string, platform imagespec.Platform) (v1.Image, error) {
root, err := p.FindRoot(imageName)
if err != nil {
return nil, err
@@ -58,7 +59,7 @@ func (p *Provider) findImage(imageName, architecture string) (v1.Image, error) {
ii, err := root.ImageIndex()
if err == nil {
// we have the index, get the manifest that represents the manifest for the desired architecture
platform := v1.Platform{OS: "linux", Architecture: architecture}
platform := v1.Platform{OS: platform.OS, Architecture: platform.Architecture}
images, err := partial.FindImages(ii, matchPlatformsOSArch(platform))
if err != nil || len(images) < 1 {
return nil, fmt.Errorf("error retrieving image %s for platform %v from cache: %v", imageName, platform, err)

View File

@@ -10,7 +10,6 @@ import (
"github.com/containerd/containerd/reference"
"github.com/google/go-containerregistry/pkg/name"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/empty"
"github.com/google/go-containerregistry/pkg/v1/match"
"github.com/google/go-containerregistry/pkg/v1/mutate"
"github.com/google/go-containerregistry/pkg/v1/partial"
@@ -32,10 +31,10 @@ const (
// ImageSource a source for an image in the OCI distribution cache.
// Implements a spec.ImageSource.
type ImageSource struct {
ref *reference.Spec
provider *Provider
architecture string
descriptor *v1.Descriptor
ref *reference.Spec
provider *Provider
platform *imagespec.Platform
descriptor *v1.Descriptor
}
type spdxStatement struct {
@@ -45,12 +44,12 @@ type spdxStatement struct {
// NewSource return an ImageSource for a specific ref and architecture in the given
// cache directory.
func (p *Provider) NewSource(ref *reference.Spec, architecture string, descriptor *v1.Descriptor) lktspec.ImageSource {
func (p *Provider) NewSource(ref *reference.Spec, platform *imagespec.Platform, descriptor *v1.Descriptor) lktspec.ImageSource {
return ImageSource{
ref: ref,
provider: p,
architecture: architecture,
descriptor: descriptor,
ref: ref,
provider: p,
platform: platform,
descriptor: descriptor,
}
}
@@ -58,7 +57,7 @@ func (p *Provider) NewSource(ref *reference.Spec, architecture string, descripto
// architecture, if necessary.
func (c ImageSource) Config() (imagespec.ImageConfig, error) {
imageName := c.ref.String()
image, err := c.provider.findImage(imageName, c.architecture)
image, err := c.provider.findImage(imageName, *c.platform)
if err != nil {
return imagespec.ImageConfig{}, err
}
@@ -84,7 +83,7 @@ func (c ImageSource) TarReader() (io.ReadCloser, error) {
imageName := c.ref.String()
// get a reference to the image
image, err := c.provider.findImage(imageName, c.architecture)
image, err := c.provider.findImage(imageName, *c.platform)
if err != nil {
return nil, err
}
@@ -104,7 +103,7 @@ func (c ImageSource) V1TarReader(overrideName string) (io.ReadCloser, error) {
return nil, fmt.Errorf("error parsing image name: %v", err)
}
// get a reference to the image
image, err := c.provider.findImage(imageName, c.architecture)
image, err := c.provider.findImage(imageName, *c.platform)
if err != nil {
return nil, err
}
@@ -129,7 +128,7 @@ func (c ImageSource) OCITarReader(overrideName string) (io.ReadCloser, error) {
return nil, fmt.Errorf("error parsing image name: %v", err)
}
// get a reference to the image
image, err := c.provider.findImage(imageName, c.architecture)
image, err := c.provider.findImage(imageName, *c.platform)
if err != nil {
return nil, err
}
@@ -139,160 +138,37 @@ func (c ImageSource) OCITarReader(overrideName string) (io.ReadCloser, error) {
defer w.Close()
tw := tar.NewWriter(w)
defer tw.Close()
// layout file
layoutFileBytes := []byte(layoutFile)
if err := tw.WriteHeader(&tar.Header{
Name: "oci-layout",
Mode: 0644,
Size: int64(len(layoutFileBytes)),
Typeflag: tar.TypeReg,
}); err != nil {
_ = w.CloseWithError(err)
return
}
if _, err := tw.Write(layoutFileBytes); err != nil {
if err := writeLayoutHeader(tw); err != nil {
_ = w.CloseWithError(err)
return
}
// make blobs directory
if err := tw.WriteHeader(&tar.Header{
Name: "blobs/",
Mode: 0755,
Typeflag: tar.TypeDir,
}); err != nil {
if err := writeLayoutImage(tw, image); err != nil {
_ = w.CloseWithError(err)
return
}
// make blobs/sha256 directory
if err := tw.WriteHeader(&tar.Header{
Name: "blobs/sha256/",
Mode: 0755,
Typeflag: tar.TypeDir,
}); err != nil {
_ = w.CloseWithError(err)
return
}
// write config, each layer, manifest, saving the digest for each
config, err := image.RawConfigFile()
imageDigest, err := image.Digest()
if err != nil {
_ = w.CloseWithError(err)
return
}
configDigest, configSize, err := v1.SHA256(bytes.NewReader(config))
imageSize, err := image.Size()
if err != nil {
_ = w.CloseWithError(err)
return
}
if err := tw.WriteHeader(&tar.Header{
Name: fmt.Sprintf("blobs/sha256/%s", configDigest.Hex),
Mode: 0644,
Typeflag: tar.TypeReg,
Size: configSize,
}); err != nil {
_ = w.CloseWithError(err)
return
}
if _, err := tw.Write(config); err != nil {
_ = w.CloseWithError(err)
return
}
layers, err := image.Layers()
if err != nil {
_ = w.CloseWithError(err)
return
}
for _, layer := range layers {
blob, err := layer.Compressed()
if err != nil {
_ = w.CloseWithError(err)
return
}
defer blob.Close()
blobDigest, err := layer.Digest()
if err != nil {
_ = w.CloseWithError(err)
return
}
blobSize, err := layer.Size()
if err != nil {
_ = w.CloseWithError(err)
return
}
if err := tw.WriteHeader(&tar.Header{
Name: fmt.Sprintf("blobs/sha256/%s", blobDigest.Hex),
Mode: 0644,
Size: blobSize,
Typeflag: tar.TypeReg,
}); err != nil {
_ = w.CloseWithError(err)
return
}
if _, err := io.Copy(tw, blob); err != nil {
_ = w.CloseWithError(err)
return
}
}
// write the manifest
manifest, err := image.RawManifest()
if err != nil {
_ = w.CloseWithError(err)
return
}
manifestDigest, manifestSize, err := v1.SHA256(bytes.NewReader(manifest))
if err != nil {
_ = w.CloseWithError(err)
return
}
if err := tw.WriteHeader(&tar.Header{
Name: fmt.Sprintf("blobs/sha256/%s", manifestDigest.Hex),
Mode: 0644,
Size: int64(len(manifest)),
Typeflag: tar.TypeReg,
}); err != nil {
_ = w.CloseWithError(err)
return
}
if _, err := tw.Write(manifest); err != nil {
_ = w.CloseWithError(err)
return
}
// write the index file
desc := v1.Descriptor{
MediaType: types.OCIImageIndex,
Size: manifestSize,
Digest: manifestDigest,
Size: imageSize,
Digest: imageDigest,
Annotations: map[string]string{
imagespec.AnnotationRefName: refName.String(),
},
}
ii := empty.Index
index, err := ii.IndexManifest()
if err != nil {
_ = w.CloseWithError(err)
return
}
index.Manifests = append(index.Manifests, desc)
rawIndex, err := json.MarshalIndent(index, "", " ")
if err != nil {
_ = w.CloseWithError(err)
return
}
// write the index
if err := tw.WriteHeader(&tar.Header{
Name: "index.json",
Mode: 0644,
Size: int64(len(rawIndex)),
}); err != nil {
_ = w.CloseWithError(err)
return
}
if _, err := tw.Write(rawIndex); err != nil {
if err := writeLayoutIndex(tw, desc); err != nil {
_ = w.CloseWithError(err)
return
}
@@ -314,15 +190,15 @@ func (c ImageSource) SBoMs() ([]io.ReadCloser, error) {
}
// get the digest of the manifest that represents our targeted architecture
descs, err := partial.FindManifests(index, matchPlatformsOSArch(v1.Platform{OS: "linux", Architecture: c.architecture}))
descs, err := partial.FindManifests(index, matchPlatformsOSArch(v1.Platform{OS: c.platform.OS, Architecture: c.platform.Architecture}))
if err != nil {
return nil, err
}
if len(descs) < 1 {
return nil, fmt.Errorf("no manifest found for %s arch %s", c.ref.String(), c.architecture)
return nil, fmt.Errorf("no manifest found for %s platform %s", c.ref.String(), c.platform)
}
if len(descs) > 1 {
return nil, fmt.Errorf("multiple manifests found for %s arch %s", c.ref.String(), c.architecture)
return nil, fmt.Errorf("multiple manifests found for %s platform %s", c.ref.String(), c.platform)
}
// get the digest of the manifest that represents our targeted architecture
desc := descs[0]
@@ -336,7 +212,7 @@ func (c ImageSource) SBoMs() ([]io.ReadCloser, error) {
return nil, err
}
if len(descs) > 1 {
return nil, fmt.Errorf("multiple manifests found for %s arch %s", c.ref.String(), c.architecture)
return nil, fmt.Errorf("multiple manifests found for %s platform %s", c.ref.String(), c.platform)
}
if len(descs) < 1 {
return nil, nil
@@ -348,10 +224,10 @@ func (c ImageSource) SBoMs() ([]io.ReadCloser, error) {
return nil, err
}
if len(images) < 1 {
return nil, fmt.Errorf("no attestation image found for %s arch %s, even though the manifest exists", c.ref.String(), c.architecture)
return nil, fmt.Errorf("no attestation image found for %s platform %s, even though the manifest exists", c.ref.String(), c.platform)
}
if len(images) > 1 {
return nil, fmt.Errorf("multiple attestation images found for %s arch %s", c.ref.String(), c.architecture)
return nil, fmt.Errorf("multiple attestation images found for %s platform %s", c.ref.String(), c.platform)
}
image := images[0]
manifest, err := image.Manifest()
@@ -363,7 +239,7 @@ func (c ImageSource) SBoMs() ([]io.ReadCloser, error) {
return nil, err
}
if len(manifest.Layers) != len(layers) {
return nil, fmt.Errorf("manifest layers and image layers do not match for the attestation for %s arch %s", c.ref.String(), c.architecture)
return nil, fmt.Errorf("manifest layers and image layers do not match for the attestation for %s platform %s", c.ref.String(), c.platform)
}
var readers []io.ReadCloser
for i, layer := range manifest.Layers {

162
src/cmd/linuxkit/cache/indexsource.go vendored Normal file
View File

@@ -0,0 +1,162 @@
package cache
import (
"archive/tar"
"bytes"
"fmt"
"io"
"github.com/containerd/containerd/reference"
"github.com/google/go-containerregistry/pkg/name"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/types"
"github.com/linuxkit/linuxkit/src/cmd/linuxkit/spec"
lktspec "github.com/linuxkit/linuxkit/src/cmd/linuxkit/spec"
imagespec "github.com/opencontainers/image-spec/specs-go/v1"
)
// IndexSource a source for an image in the OCI distribution cache.
// Implements a spec.ImageSource.
type IndexSource struct {
ref *reference.Spec
provider *Provider
descriptor *v1.Descriptor
platforms []imagespec.Platform
}
// NewIndexSource return an IndexSource for a specific ref in the given
// cache directory.
func (p *Provider) NewIndexSource(ref *reference.Spec, descriptor *v1.Descriptor, platforms []imagespec.Platform) lktspec.IndexSource {
return IndexSource{
ref: ref,
provider: p,
descriptor: descriptor,
platforms: platforms,
}
}
// Config return the imagespec.ImageConfig for the given source. Resolves to the
// architecture, if necessary.
func (c IndexSource) Image(platform imagespec.Platform) (spec.ImageSource, error) {
imageName := c.ref.String()
index, err := c.provider.findIndex(imageName)
if err != nil {
return nil, err
}
manifests, err := index.IndexManifest()
if err != nil {
return nil, err
}
for _, manifest := range manifests.Manifests {
if manifest.Platform != nil && manifest.Platform.Architecture == platform.Architecture && manifest.Platform.OS == platform.OS {
return c.provider.NewSource(c.ref, &platform, &manifest), nil
}
}
return nil, fmt.Errorf("no manifest found for platform %q", platform)
}
// OCITarReader return an io.ReadCloser to read the image as a v1 tarball whose contents match OCI v1 layout spec
func (c IndexSource) OCITarReader(overrideName string) (io.ReadCloser, error) {
imageName := c.ref.String()
saveName := imageName
if overrideName != "" {
saveName = overrideName
}
refName, err := name.ParseReference(saveName)
if err != nil {
return nil, fmt.Errorf("error parsing image name: %v", err)
}
// get a reference to the image
index, err := c.provider.findIndex(c.ref.String())
if err != nil {
return nil, err
}
// convert the writer to a reader
r, w := io.Pipe()
go func() {
defer w.Close()
tw := tar.NewWriter(w)
defer tw.Close()
if err := writeLayoutHeader(tw); err != nil {
_ = w.CloseWithError(err)
return
}
manifests, err := index.IndexManifest()
if err != nil {
_ = w.CloseWithError(err)
return
}
// for each manifest, write the manifest blob, then go through each manifest and find the image for it
// and write its blobs
for _, manifest := range manifests.Manifests {
// if we restricted this image source to certain platforms, we should only write those
if len(c.platforms) > 0 {
found := false
for _, platform := range c.platforms {
if platform.Architecture == manifest.Platform.Architecture && platform.OS == manifest.Platform.OS &&
(platform.Variant == "" || platform.Variant == manifest.Platform.Variant) {
found = true
break
}
}
if !found {
continue
}
}
switch manifest.MediaType {
case types.OCIManifestSchema1, types.DockerManifestSchema2:
// this is an image manifest
image, err := index.Image(manifest.Digest)
if err != nil {
_ = w.CloseWithError(err)
return
}
if err := writeLayoutImage(tw, image); err != nil {
_ = w.CloseWithError(err)
return
}
}
}
// write the index directly as a blob
indexSize, err := index.Size()
if err != nil {
_ = w.CloseWithError(err)
return
}
indexDigest, err := index.Digest()
if err != nil {
_ = w.CloseWithError(err)
return
}
indexBytes, err := index.RawManifest()
if err != nil {
_ = w.CloseWithError(err)
return
}
if err := writeLayoutBlob(tw, indexDigest.Hex, indexSize, bytes.NewReader(indexBytes)); err != nil {
_ = w.CloseWithError(err)
return
}
desc := v1.Descriptor{
MediaType: types.OCIImageIndex,
Size: indexSize,
Digest: indexDigest,
Annotations: map[string]string{
imagespec.AnnotationRefName: refName.String(),
},
}
if err := writeLayoutIndex(tw, desc); err != nil {
_ = w.CloseWithError(err)
return
}
}()
return r, nil
}
// Descriptor return the descriptor of the index.
func (c IndexSource) Descriptor() *v1.Descriptor {
return c.descriptor
}

145
src/cmd/linuxkit/cache/layout.go vendored Normal file
View File

@@ -0,0 +1,145 @@
package cache
import (
"archive/tar"
"bytes"
"encoding/json"
"fmt"
"io"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/empty"
)
func writeLayoutHeader(tw *tar.Writer) error {
// layout file
layoutFileBytes := []byte(layoutFile)
if err := tw.WriteHeader(&tar.Header{
Name: "oci-layout",
Mode: 0644,
Size: int64(len(layoutFileBytes)),
Typeflag: tar.TypeReg,
}); err != nil {
return err
}
if _, err := tw.Write(layoutFileBytes); err != nil {
return err
}
// make blobs directory
if err := tw.WriteHeader(&tar.Header{
Name: "blobs/",
Mode: 0755,
Typeflag: tar.TypeDir,
}); err != nil {
return err
}
// make blobs/sha256 directory
if err := tw.WriteHeader(&tar.Header{
Name: "blobs/sha256/",
Mode: 0755,
Typeflag: tar.TypeDir,
}); err != nil {
return err
}
return nil
}
func writeLayoutImage(tw *tar.Writer, image v1.Image) error {
// write config, each layer, manifest, saving the digest for each
manifest, err := image.Manifest()
if err != nil {
return err
}
configDesc := manifest.Config
configBytes, err := image.RawConfigFile()
if err != nil {
return err
}
if err := writeLayoutBlob(tw, configDesc.Digest.Hex, configDesc.Size, bytes.NewReader(configBytes)); err != nil {
return err
}
layers, err := image.Layers()
if err != nil {
return err
}
for _, layer := range layers {
blob, err := layer.Compressed()
if err != nil {
return err
}
defer blob.Close()
blobDigest, err := layer.Digest()
if err != nil {
return err
}
blobSize, err := layer.Size()
if err != nil {
return err
}
if err := writeLayoutBlob(tw, blobDigest.Hex, blobSize, blob); err != nil {
return err
}
}
// write the manifest
manifestSize, err := image.Size()
if err != nil {
return err
}
manifestDigest, err := image.Digest()
if err != nil {
return err
}
manifestBytes, err := image.RawManifest()
if err != nil {
return err
}
if err := writeLayoutBlob(tw, manifestDigest.Hex, manifestSize, bytes.NewReader(manifestBytes)); err != nil {
return err
}
return nil
}
func writeLayoutBlob(tw *tar.Writer, digest string, size int64, blob io.Reader) error {
if err := tw.WriteHeader(&tar.Header{
Name: fmt.Sprintf("blobs/sha256/%s", digest),
Mode: 0644,
Size: size,
Typeflag: tar.TypeReg,
}); err != nil {
return err
}
if _, err := io.Copy(tw, blob); err != nil {
return err
}
return nil
}
func writeLayoutIndex(tw *tar.Writer, desc v1.Descriptor) error {
ii := empty.Index
index, err := ii.IndexManifest()
if err != nil {
return err
}
index.Manifests = append(index.Manifests, desc)
rawIndex, err := json.MarshalIndent(index, "", " ")
if err != nil {
return err
}
// write the index
if err := tw.WriteHeader(&tar.Header{
Name: "index.json",
Mode: 0644,
Size: int64(len(rawIndex)),
}); err != nil {
return err
}
if _, err := tw.Write(rawIndex); err != nil {
return err
}
return nil
}

33
src/cmd/linuxkit/cache/platform.go vendored Normal file
View File

@@ -0,0 +1,33 @@
package cache
import (
"fmt"
"strings"
imagespec "github.com/opencontainers/image-spec/specs-go/v1"
)
func platformString(p imagespec.Platform) string {
parts := []string{p.OS, p.Architecture}
if p.Variant != "" {
parts = append(parts, p.Variant)
}
return strings.Join(parts, "/")
}
func platformMessageGenerator(platforms []imagespec.Platform) string {
var platformMessage string
switch {
case len(platforms) == 0:
platformMessage = "all platforms"
case len(platforms) == 1:
platformMessage = fmt.Sprintf("platform %s", platformString(platforms[0]))
default:
var platStrings []string
for _, p := range platforms {
platStrings = append(platStrings, platformString(p))
}
platformMessage = fmt.Sprintf("platforms %s", strings.Join(platStrings, ","))
}
return platformMessage
}

View File

@@ -3,6 +3,7 @@ package cache
import (
"errors"
"fmt"
"strings"
"github.com/containerd/containerd/reference"
"github.com/google/go-containerregistry/pkg/authn"
@@ -30,12 +31,13 @@ const (
// architecture, and any manifests that have no architecture at all. It will ignore manifests
// for other architectures. If no architecture is provided, it will validate all manifests.
// It also calculates the hash of each component.
func (p *Provider) ValidateImage(ref *reference.Spec, architecture string) (lktspec.ImageSource, error) {
func (p *Provider) ValidateImage(ref *reference.Spec, platforms []imagespec.Platform) (lktspec.ImageSource, error) {
var (
imageIndex v1.ImageIndex
image v1.Image
imageName = ref.String()
desc *v1.Descriptor
imageIndex v1.ImageIndex
image v1.Image
imageName = ref.String()
desc *v1.Descriptor
platformMessage = platformMessageGenerator(platforms)
)
// next try the local cache
root, err := p.FindRoot(imageName)
@@ -71,7 +73,15 @@ func (p *Provider) ValidateImage(ref *reference.Spec, architecture string) (lkts
if err != nil {
return ImageSource{}, fmt.Errorf("could not get index manifest: %w", err)
}
var architectures = make(map[string]bool)
var (
targetPlatforms = make(map[string]bool)
foundPlatforms = make(map[string]bool)
)
for _, plat := range platforms {
pString := platformString(plat)
targetPlatforms[pString] = false
foundPlatforms[pString] = false
}
// ignore only other architectures; manifest entries that have no architectures at all
// are going to be additional metadata, so we need to check them
for _, m := range im.Manifests {
@@ -80,29 +90,50 @@ func (p *Provider) ValidateImage(ref *reference.Spec, architecture string) (lkts
return ImageSource{}, fmt.Errorf("invalid image: %w", err)
}
}
if architecture != "" && m.Platform.Architecture == architecture && m.Platform.OS == linux {
if err := validateManifestContents(imageIndex, m.Digest); err != nil {
return ImageSource{}, fmt.Errorf("invalid image: %w", err)
// go through each target platform, and see if this one matched. If it did, mark the target as
for _, plat := range platforms {
if plat.Architecture == m.Platform.Architecture && plat.OS == m.Platform.OS &&
(plat.Variant == "" || plat.Variant == m.Platform.Variant) {
targetPlatforms[platformString(plat)] = true
break
}
architectures[architecture] = true
}
}
if architecture == "" || architectures[architecture] {
if len(platforms) == 0 {
return p.NewSource(
ref,
architecture,
nil,
desc,
), nil
}
return ImageSource{}, fmt.Errorf("index for %s did not contain image for platform linux/%s", imageName, architecture)
// we have cycled through all of the manifests, let's check if we have all of the platforms
var missing []string
for plat, found := range targetPlatforms {
if !found {
missing = append(missing, plat)
}
}
if len(missing) == 0 {
return p.NewSource(
ref,
nil,
desc,
), nil
}
return ImageSource{}, fmt.Errorf("index for %s did not contain image for platforms %s", imageName, strings.Join(missing, ", "))
case image != nil:
if len(platforms) > 1 {
return ImageSource{}, fmt.Errorf("image %s is not a multi-arch image, but asked for %s", imageName, platformMessage)
}
// we found a local image, make sure it is up to date
if err := validate.Image(image); err != nil {
return ImageSource{}, fmt.Errorf("invalid image, %s", err)
}
return p.NewSource(
ref,
architecture,
&platforms[0],
desc,
), nil
}
@@ -164,7 +195,7 @@ func (p *Provider) Pull(name string, withArchReferences bool) error {
if err := p.cache.WriteIndex(ii); err != nil {
return fmt.Errorf("unable to write index: %v", err)
}
if _, err := p.DescriptorWrite(&v1ref, desc.Descriptor); err != nil {
if err := p.DescriptorWrite(&v1ref, desc.Descriptor); err != nil {
return fmt.Errorf("unable to write index descriptor to cache: %v", err)
}
if withArchReferences {
@@ -179,7 +210,7 @@ func (p *Provider) Pull(name string, withArchReferences bool) error {
if err != nil {
return fmt.Errorf("unable to parse arch-specific reference %s: %v", archSpecific, err)
}
if _, err := p.DescriptorWrite(&archRef, m); err != nil {
if err := p.DescriptorWrite(&archRef, m); err != nil {
return fmt.Errorf("unable to write index descriptor to cache: %v", err)
}
}

View File

@@ -19,9 +19,7 @@ import (
"github.com/google/go-containerregistry/pkg/v1/partial"
"github.com/google/go-containerregistry/pkg/v1/remote"
"github.com/google/go-containerregistry/pkg/v1/types"
lktspec "github.com/linuxkit/linuxkit/src/cmd/linuxkit/spec"
"github.com/linuxkit/linuxkit/src/cmd/linuxkit/util"
lktutil "github.com/linuxkit/linuxkit/src/cmd/linuxkit/util"
imagespec "github.com/opencontainers/image-spec/specs-go/v1"
log "github.com/sirupsen/logrus"
)
@@ -41,45 +39,42 @@ const (
// If you just want to check the status of a local ref, use ValidateImage.
// Note that ImagePull does try ValidateImage first, so if the image is already in the cache, it will not
// do any network activity at all.
func (p *Provider) ImagePull(ref *reference.Spec, trustedRef, architecture string, alwaysPull bool) (lktspec.ImageSource, error) {
func (p *Provider) ImagePull(ref *reference.Spec, platforms []imagespec.Platform, alwaysPull bool) error {
imageName := util.ReferenceExpand(ref.String())
canonicalRef, err := reference.Parse(imageName)
if err != nil {
return ImageSource{}, fmt.Errorf("invalid image name %s: %v", imageName, err)
return fmt.Errorf("invalid image name %s: %v", imageName, err)
}
ref = &canonicalRef
image := ref.String()
pullImageName := image
platformMessage := platformMessageGenerator(platforms)
remoteOptions := []remote.Option{remote.WithAuthFromKeychain(authn.DefaultKeychain)}
if trustedRef != "" {
pullImageName = trustedRef
}
log.Debugf("ImagePull to cache %s trusted reference %s", image, pullImageName)
// unless alwaysPull is set to true, check locally first
if alwaysPull {
log.Printf("Instructed always to pull, so pulling image %s arch %s", image, architecture)
log.Debugf("Instructed always to pull, so pulling image %s %s", image, platformMessage)
} else {
imgSrc, err := p.ValidateImage(ref, architecture)
imgSrc, err := p.ValidateImage(ref, platforms)
switch {
case err == nil && imgSrc != nil:
log.Printf("Image %s arch %s found in local cache, not pulling", image, architecture)
return imgSrc, nil
log.Debugf("Image %s %s found in local cache, not pulling", image, platformMessage)
return nil
case err != nil && errors.Is(err, &noReferenceError{}):
log.Printf("Image %s arch %s not found in local cache, pulling", image, architecture)
log.Debugf("Image %s %s not found in local cache, pulling", image, platformMessage)
default:
log.Printf("Image %s arch %s incomplete or invalid in local cache, error %v, pulling", image, architecture, err)
log.Debugf("Image %s %s incomplete or invalid in local cache, error %v, pulling", image, platformMessage, err)
}
// there was an error, so try to pull
}
remoteRef, err := name.ParseReference(pullImageName)
if err != nil {
return ImageSource{}, fmt.Errorf("invalid image name %s: %v", pullImageName, err)
return fmt.Errorf("invalid image name %s: %v", pullImageName, err)
}
desc, err := remote.Get(remoteRef, remoteOptions...)
if err != nil {
return ImageSource{}, fmt.Errorf("error getting manifest for trusted image %s: %v", pullImageName, err)
return fmt.Errorf("error getting manifest for image %s: %v", pullImageName, err)
}
// use the original image name in the annotation
@@ -90,46 +85,57 @@ func (p *Provider) ImagePull(ref *reference.Spec, trustedRef, architecture strin
// first attempt as an index
ii, err := desc.ImageIndex()
if err == nil {
log.Debugf("ImageWrite retrieved %s is index, saving, first checking if it contains target arch %s", pullImageName, architecture)
log.Debugf("ImageWrite retrieved %s is index, saving, first checking if it contains target %s", pullImageName, platformMessage)
im, err := ii.IndexManifest()
if err != nil {
return ImageSource{}, fmt.Errorf("unable to get IndexManifest: %v", err)
return fmt.Errorf("unable to get IndexManifest: %v", err)
}
// only useful if it contains our architecture
var foundArch bool
var foundPlatforms []*v1.Platform
for _, m := range im.Manifests {
if m.MediaType.IsImage() && m.Platform != nil && m.Platform.Architecture == architecture && m.Platform.OS == linux {
foundArch = true
break
if m.MediaType.IsImage() && m.Platform != nil {
foundPlatforms = append(foundPlatforms, m.Platform)
}
}
if !foundArch {
return ImageSource{}, fmt.Errorf("index %s does not contain target architecture %s", pullImageName, architecture)
// now see if we have all of the platforms we need
var missing []string
for _, requiredPlatform := range platforms {
// we did not find it, so maybe one satisfies it
var matchedPlatform bool
for _, p := range foundPlatforms {
if p.OS == requiredPlatform.OS && p.Architecture == requiredPlatform.Architecture && (p.Variant == requiredPlatform.Variant || requiredPlatform.Variant == "") {
// this one satisfies it, so do not count it missing
matchedPlatform = true
break
}
}
if !matchedPlatform {
missing = append(missing, platformString(requiredPlatform))
}
}
if len(missing) > 0 {
return fmt.Errorf("index %s does not contain target platforms %s", pullImageName, strings.Join(missing, ","))
}
if err := p.cache.WriteIndex(ii); err != nil {
return ImageSource{}, fmt.Errorf("unable to write index: %v", err)
return fmt.Errorf("unable to write index: %v", err)
}
if _, err := p.DescriptorWrite(ref, desc.Descriptor); err != nil {
return ImageSource{}, fmt.Errorf("unable to write index descriptor to cache: %v", err)
if err := p.DescriptorWrite(ref, desc.Descriptor); err != nil {
return fmt.Errorf("unable to write index descriptor to cache: %v", err)
}
} else {
var im v1.Image
// try an image
im, err = desc.Image()
if err != nil {
return ImageSource{}, fmt.Errorf("provided image is neither an image nor an index: %s", image)
return fmt.Errorf("provided image is neither an image nor an index: %s", image)
}
log.Debugf("ImageWrite retrieved %s is image, saving", pullImageName)
if err = p.cache.ReplaceImage(im, match.Name(image), layout.WithAnnotations(annotations)); err != nil {
return ImageSource{}, fmt.Errorf("unable to save image to cache: %v", err)
return fmt.Errorf("unable to save image to cache: %v", err)
}
}
return p.NewSource(
ref,
architecture,
&desc.Descriptor,
), nil
return nil
}
// ImageLoad takes an OCI format image tar stream and writes it locally. It should be
@@ -227,27 +233,27 @@ func (p *Provider) ImageLoad(r io.Reader) ([]v1.Descriptor, error) {
// does not pull down any images; entirely assumes that the subjects of the manifests are present.
// If a reference to the provided already exists and it is an index, updates the manifests in the
// existing index.
func (p *Provider) IndexWrite(ref *reference.Spec, descriptors ...v1.Descriptor) (lktspec.ImageSource, error) {
func (p *Provider) IndexWrite(ref *reference.Spec, descriptors ...v1.Descriptor) error {
image := ref.String()
log.Debugf("writing an index for %s", image)
if len(descriptors) < 1 {
return ImageSource{}, errors.New("cannot create index without any manifests")
return errors.New("cannot create index without any manifests")
}
ii, err := p.cache.ImageIndex()
if err != nil {
return ImageSource{}, fmt.Errorf("unable to get root index: %v", err)
return fmt.Errorf("unable to get root index: %v", err)
}
images, err := partial.FindImages(ii, match.Name(image))
if err != nil {
return ImageSource{}, fmt.Errorf("error parsing index: %v", err)
return fmt.Errorf("error parsing index: %v", err)
}
if err == nil && len(images) > 0 {
return ImageSource{}, fmt.Errorf("image named %s already exists in cache and is not an index", image)
return fmt.Errorf("image named %s already exists in cache and is not an index", image)
}
indexes, err := partial.FindIndexes(ii, match.Name(image))
if err != nil {
return ImageSource{}, fmt.Errorf("error parsing index: %v", err)
return fmt.Errorf("error parsing index: %v", err)
}
var im v1.IndexManifest
// do we update an existing one? Or create a new one?
@@ -255,11 +261,11 @@ func (p *Provider) IndexWrite(ref *reference.Spec, descriptors ...v1.Descriptor)
// we already had one, so update just the referenced index and return
manifest, err := indexes[0].IndexManifest()
if err != nil {
return ImageSource{}, fmt.Errorf("unable to convert index for %s into its manifest: %v", image, err)
return fmt.Errorf("unable to convert index for %s into its manifest: %v", image, err)
}
oldhash, err := indexes[0].Digest()
if err != nil {
return ImageSource{}, fmt.Errorf("unable to get hash of existing index: %v", err)
return fmt.Errorf("unable to get hash of existing index: %v", err)
}
// we only care about avoiding duplicate arch/OS/Variant
var (
@@ -318,7 +324,7 @@ func (p *Provider) IndexWrite(ref *reference.Spec, descriptors ...v1.Descriptor)
appliedManifests[m.Digest] = true
continue
}
value, ok := m.Annotations[lktutil.AnnotationDockerReferenceDigest]
value, ok := m.Annotations[util.AnnotationDockerReferenceDigest]
if !ok {
manifest.Manifests = append(manifest.Manifests, m)
appliedManifests[m.Digest] = true
@@ -336,7 +342,7 @@ func (p *Provider) IndexWrite(ref *reference.Spec, descriptors ...v1.Descriptor)
im = *manifest
// remove the old index
if err := p.cache.RemoveBlob(oldhash); err != nil {
return ImageSource{}, fmt.Errorf("unable to remove old index file: %v", err)
return fmt.Errorf("unable to remove old index file: %v", err)
}
} else {
// we did not have one, so create an index, store it, update the root index.json, and return
@@ -350,18 +356,18 @@ func (p *Provider) IndexWrite(ref *reference.Spec, descriptors ...v1.Descriptor)
// write the updated index, remove the old one
b, err := json.Marshal(im)
if err != nil {
return ImageSource{}, fmt.Errorf("unable to marshal new index to json: %v", err)
return fmt.Errorf("unable to marshal new index to json: %v", err)
}
hash, size, err := v1.SHA256(bytes.NewReader(b))
if err != nil {
return ImageSource{}, fmt.Errorf("error calculating hash of index json: %v", err)
return fmt.Errorf("error calculating hash of index json: %v", err)
}
if err := p.cache.WriteBlob(hash, io.NopCloser(bytes.NewReader(b))); err != nil {
return ImageSource{}, fmt.Errorf("error writing new index to json: %v", err)
return fmt.Errorf("error writing new index to json: %v", err)
}
// finally update the descriptor in the root
if err := p.cache.RemoveDescriptors(match.Name(image)); err != nil {
return ImageSource{}, fmt.Errorf("unable to remove old descriptor from index.json: %v", err)
return fmt.Errorf("unable to remove old descriptor from index.json: %v", err)
}
desc := v1.Descriptor{
MediaType: types.OCIImageIndex,
@@ -372,21 +378,17 @@ func (p *Provider) IndexWrite(ref *reference.Spec, descriptors ...v1.Descriptor)
},
}
if err := p.cache.AppendDescriptor(desc); err != nil {
return ImageSource{}, fmt.Errorf("unable to append new descriptor to index.json: %v", err)
return fmt.Errorf("unable to append new descriptor to index.json: %v", err)
}
return p.NewSource(
ref,
"",
&desc,
), nil
return nil
}
// DescriptorWrite writes a descriptor to the cache index; it validates that it has a name
// and replaces any existing one
func (p *Provider) DescriptorWrite(ref *reference.Spec, desc v1.Descriptor) (lktspec.ImageSource, error) {
func (p *Provider) DescriptorWrite(ref *reference.Spec, desc v1.Descriptor) error {
if ref == nil {
return ImageSource{}, errors.New("cannot write descriptor without reference name")
return errors.New("cannot write descriptor without reference name")
}
image := ref.String()
if desc.Annotations == nil {
@@ -397,22 +399,18 @@ func (p *Provider) DescriptorWrite(ref *reference.Spec, desc v1.Descriptor) (lkt
// do we update an existing one? Or create a new one?
if err := p.cache.RemoveDescriptors(match.Name(image)); err != nil {
return ImageSource{}, fmt.Errorf("unable to remove old descriptors for %s: %v", image, err)
return fmt.Errorf("unable to remove old descriptors for %s: %v", image, err)
}
if err := p.cache.AppendDescriptor(desc); err != nil {
return ImageSource{}, fmt.Errorf("unable to append new descriptor for %s: %v", image, err)
return fmt.Errorf("unable to append new descriptor for %s: %v", image, err)
}
return p.NewSource(
ref,
"",
&desc,
), nil
return nil
}
func (p *Provider) ImageInCache(ref *reference.Spec, trustedRef, architecture string) (bool, error) {
img, err := p.findImage(ref.String(), architecture)
img, err := p.findImage(ref.String(), imagespec.Platform{OS: linux, Architecture: architecture})
if err != nil {
return false, err
}

View File

@@ -4,17 +4,20 @@ import (
"io"
"os"
"runtime"
"strings"
"github.com/containerd/containerd/reference"
v1 "github.com/google/go-containerregistry/pkg/v1"
cachepkg "github.com/linuxkit/linuxkit/src/cmd/linuxkit/cache"
"github.com/linuxkit/linuxkit/src/cmd/linuxkit/util"
imagespec "github.com/opencontainers/image-spec/specs-go/v1"
log "github.com/sirupsen/logrus"
"github.com/spf13/cobra"
)
func cacheExportCmd() *cobra.Command {
var (
arch string
platform string
outputFile string
format string
tagName string
@@ -42,7 +45,16 @@ func cacheExportCmd() *cobra.Command {
log.Fatalf("unable to find image named %s: %v", name, err)
}
src := p.NewSource(&ref, arch, desc)
plat, err := v1.ParsePlatform(platform)
if err != nil {
log.Fatalf("invalid platform %s: %v", platform, err)
}
platspec := imagespec.Platform{
Architecture: plat.Architecture,
OS: plat.OS,
Variant: plat.Variant,
}
src := p.NewSource(&ref, &platspec, desc)
var reader io.ReadCloser
switch format {
case "docker":
@@ -88,7 +100,7 @@ func cacheExportCmd() *cobra.Command {
},
}
cmd.Flags().StringVar(&arch, "arch", runtime.GOARCH, "Architecture to resolve an index to an image, if the provided image name is an index")
cmd.Flags().StringVar(&platform, "platform", strings.Join([]string{"linux", runtime.GOARCH}, "/"), "Platform to resolve an index to an image, if the provided image name is an index")
cmd.Flags().StringVar(&outputFile, "outfile", "", "Path to file to save output, '-' for stdout")
cmd.Flags().StringVar(&format, "format", "oci", "export format, one of 'oci' (OCI tar), 'docker' (docker tar), 'filesystem'")
cmd.Flags().StringVar(&tagName, "name", "", "override the provided image name in the exported tar file; useful only for format=oci")

View File

@@ -43,8 +43,9 @@ func readConfig() {
func newCmd() *cobra.Command {
var (
flagQuiet bool
flagVerbose bool
flagQuiet bool
flagVerbose int
flagVerboseName = "verbose"
)
cmd := &cobra.Command{
Use: "linuxkit",
@@ -54,7 +55,7 @@ func newCmd() *cobra.Command {
readConfig()
// Set up logging
return util.SetupLogging(flagQuiet, flagVerbose)
return util.SetupLogging(flagQuiet, flagVerbose, cmd.Flag(flagVerboseName).Changed)
},
}
@@ -69,7 +70,7 @@ func newCmd() *cobra.Command {
cmd.PersistentFlags().StringVar(&cacheDir, "cache", defaultLinuxkitCache(), fmt.Sprintf("Directory for caching and finding cached image, overrides env var %s", envVarCacheDir))
cmd.PersistentFlags().BoolVarP(&flagQuiet, "quiet", "q", false, "Quiet execution")
cmd.PersistentFlags().BoolVarP(&flagVerbose, "verbose", "v", false, "Verbose execution")
cmd.PersistentFlags().IntVarP(&flagVerbose, flagVerboseName, "v", 1, "Verbosity of logging: 0 = quiet, 1 = info, 2 = debug, 3 = trace. Default is info. Setting it explicitly will create structured logging lines.")
return cmd
}

View File

@@ -3,5 +3,4 @@ package main
const (
defaultPkgBuildYML = "build.yml"
defaultPkgCommit = "HEAD"
defaultPkgTag = "{{.Hash}}"
)

View File

@@ -3,6 +3,7 @@ package docker
import (
"context"
"errors"
"fmt"
"io"
"os"
"sync"
@@ -51,13 +52,19 @@ func createClient() (*client.Client, error) {
}
// HasImage check if the provided ref is available in the docker cache.
func HasImage(ref *reference.Spec) error {
func HasImage(ref *reference.Spec, architecture string) error {
log.Debugf("docker inspect image: %s", ref)
cli, err := Client()
if err != nil {
return err
}
_, err = InspectImage(cli, ref)
imageInspect, err := InspectImage(cli, ref)
if err != nil {
return err
}
if imageInspect.Architecture != "" && imageInspect.Architecture != architecture {
return fmt.Errorf("image not found for right architecture (%s != %s)", imageInspect.Architecture, architecture)
}
return err
}

View File

@@ -14,7 +14,7 @@ require (
github.com/containerd/containerd v1.7.15
github.com/docker/buildx v0.14.1
github.com/docker/cli v26.1.3+incompatible
github.com/docker/docker v27.0.3+incompatible
github.com/docker/docker v27.2.0+incompatible
github.com/docker/go-units v0.5.0
github.com/google/go-containerregistry v0.14.0
github.com/google/uuid v1.6.0
@@ -54,9 +54,9 @@ require (
github.com/Code-Hex/vz/v3 v3.0.0
github.com/equinix/equinix-sdk-go v0.42.0
github.com/in-toto/in-toto-golang v0.5.0
github.com/moby/sys/capability v0.3.0
github.com/spdx/tools-golang v0.5.3
github.com/spf13/cobra v1.8.0
github.com/syndtr/gocapability v0.0.0-20200815063812-42c35b437635
gopkg.in/yaml.v3 v3.0.1
)

View File

@@ -103,8 +103,8 @@ github.com/docker/cli v26.1.3+incompatible h1:bUpXT/N0kDE3VUHI2r5VMsYQgi38kYuoC0
github.com/docker/cli v26.1.3+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8=
github.com/docker/distribution v2.8.2+incompatible h1:T3de5rq0dB1j30rp0sA2rER+m322EBzniBPB6ZIzuh8=
github.com/docker/distribution v2.8.2+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w=
github.com/docker/docker v27.0.3+incompatible h1:aBGI9TeQ4MPlhquTQKq9XbK79rKFVwXNUAYz9aXyEBE=
github.com/docker/docker v27.0.3+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/docker v27.2.0+incompatible h1:Rk9nIVdfH3+Vz4cyI/uhbINhEZ/oLmc+CBXmH6fbNk4=
github.com/docker/docker v27.2.0+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/docker-credential-helpers v0.8.0 h1:YQFtbBQb4VrpoPxhFuzEBPQ9E16qz5SpHLS+uswaCp8=
github.com/docker/docker-credential-helpers v0.8.0/go.mod h1:UGFXcuoQ5TxPiB54nHOZ32AWRqQdECoh/Mg0AlEYb40=
github.com/docker/go-connections v0.5.0 h1:USnMq7hx7gwdVZq1L49hLXaFtUdTADjXGp+uj1Br63c=
@@ -245,6 +245,8 @@ github.com/moby/locker v1.0.1 h1:fOXqR41zeveg4fFODix+1Ch4mj/gT0NE1XJbp/epuBg=
github.com/moby/locker v1.0.1/go.mod h1:S7SDdo5zpBK84bzzVlKr2V0hz+7x9hWbYC/kq7oQppc=
github.com/moby/patternmatcher v0.6.0 h1:GmP9lR19aU5GqSSFko+5pRqHi+Ohk1O69aFiKkVGiPk=
github.com/moby/patternmatcher v0.6.0/go.mod h1:hDPoyOpDY7OrrMDLaYoY3hf52gNCR/YOUYxkhApJIxc=
github.com/moby/sys/capability v0.3.0 h1:kEP+y6te0gEXIaeQhIi0s7vKs/w0RPoH1qPa6jROcVg=
github.com/moby/sys/capability v0.3.0/go.mod h1:4g9IK291rVkms3LKCDOoYlnV8xKwoDTpIrNEE35Wq0I=
github.com/moby/sys/mountinfo v0.7.1 h1:/tTvQaSJRr2FshkhXiIpux6fQ2Zvc4j7tAhMTStAG2g=
github.com/moby/sys/mountinfo v0.7.1/go.mod h1:IJb6JQeOklcdMU9F5xQ8ZALD+CUr5VlGpwtX+VE0rpI=
github.com/moby/sys/sequential v0.5.0 h1:OPvI35Lzn9K04PBbCLW0g4LcFAJgHsvXsRyewg5lXtc=
@@ -329,8 +331,6 @@ github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsT
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/surma/gocpio v1.0.2-0.20160926205914-fcb68777e7dc h1:iA3Eg1OVd2o0M4M+0PBsBBssMz98L8CUH7x0xVkuyUA=
github.com/surma/gocpio v1.0.2-0.20160926205914-fcb68777e7dc/go.mod h1:zaLNaN+EDnfSnNdWPJJf9OZxWF817w5dt8JNzF9LCVI=
github.com/syndtr/gocapability v0.0.0-20200815063812-42c35b437635 h1:kdXcSzyDtseVEc4yCz2qF8ZrQvIDBJLl4S1c3GCXmoI=
github.com/syndtr/gocapability v0.0.0-20200815063812-42c35b437635/go.mod h1:hkRG7XYTFWNJGYcbNJQlaLq0fg1yr4J4t/NcTQtrfww=
github.com/tonistiigi/fsutil v0.0.0-20240424095704-91a3fc46842c h1:+6wg/4ORAbnSoGDzg2Q1i3CeMcT/jjhye/ZfnBHy7/M=
github.com/tonistiigi/fsutil v0.0.0-20240424095704-91a3fc46842c/go.mod h1:vbbYqJlnswsbJqWUcJN8fKtBhnEgldDrcagTgnBVKKM=
github.com/tonistiigi/units v0.0.0-20180711220420-6950e57a87ea h1:SXhTLE6pb6eld/v/cCndK0AMpt1wiVFb/YYmqB3/QG0=

View File

@@ -16,6 +16,9 @@ import (
"strings"
"github.com/containerd/containerd/reference"
v1 "github.com/google/go-containerregistry/pkg/v1"
imagespec "github.com/opencontainers/image-spec/specs-go/v1"
// drop-in 100% compatible replacement and 17% faster than compress/gzip.
gzip "github.com/klauspost/pgzip"
"github.com/linuxkit/linuxkit/src/cmd/linuxkit/moby"
@@ -84,6 +87,8 @@ func OutputTypes() []string {
return ts
}
// outputImage given an image and a section, such as onboot, onshutdown or services, lay it out with correct location
// config, etc. in the filesystem, so runc can use it.
func outputImage(image *moby.Image, section string, index int, prefix string, m moby.Moby, idMap map[string]uint32, dupMap map[string]string, iw *tar.Writer, opts BuildOpts) error {
log.Infof(" Create OCI config for %s", image.Image)
imageName := util.ReferenceExpand(image.Image)
@@ -91,7 +96,7 @@ func outputImage(image *moby.Image, section string, index int, prefix string, m
if err != nil {
return fmt.Errorf("could not resolve references for image %s: %v", image.Image, err)
}
src, err := imagePull(&ref, opts.Pull, opts.CacheDir, opts.DockerCache, opts.Arch)
src, err := imageSource(&ref, opts.Pull, opts.CacheDir, opts.DockerCache, imagespec.Platform{OS: "linux", Architecture: opts.Arch})
if err != nil {
return fmt.Errorf("could not pull image %s: %v", image.Image, err)
}
@@ -280,9 +285,30 @@ func Build(m moby.Moby, w io.Writer, opts BuildOpts) error {
lowerPath := strings.TrimPrefix(lower, "/") + "/"
// get volume tarball from container
if err := ImageTar(location, vol.ImageRef(), lowerPath, apkTar, resolvconfSymlink, opts); err != nil {
return fmt.Errorf("failed to build volume tarball from %s: %v", vol.Name, err)
switch {
case vol.ImageRef() == nil || vol.Format == "" || vol.Format == "filesystem":
if err := ImageTar(location, vol.ImageRef(), lowerPath, apkTar, resolvconfSymlink, opts); err != nil {
return fmt.Errorf("failed to build volume filesystem tarball from %s: %v", vol.Name, err)
}
case vol.Format == "oci":
// convert platforms into imagespec platforms
platforms := make([]imagespec.Platform, len(vol.Platforms))
for i, p := range vol.Platforms {
platform, err := v1.ParsePlatform(p)
if err != nil {
return fmt.Errorf("failed to parse platform %s: %v", p, err)
}
platforms[i] = imagespec.Platform{
Architecture: platform.Architecture,
OS: platform.OS,
Variant: platform.Variant,
}
}
if err := ImageOCITar(location, vol.ImageRef(), lowerPath, apkTar, opts, platforms); err != nil {
return fmt.Errorf("failed to build volume OCI v1 layout tarball from %s: %v", vol.Name, err)
}
}
// make upper and merged dirs which will be used for mounting
// no need to make lower dir, as it is made automatically by ImageTar()
tmpPath := strings.TrimPrefix(tmpDir, "/") + "/"

View File

@@ -11,6 +11,7 @@ import (
"github.com/containerd/containerd/reference"
"github.com/linuxkit/linuxkit/src/cmd/linuxkit/moby"
imagespec "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/opencontainers/runtime-spec/specs-go"
log "github.com/sirupsen/logrus"
)
@@ -175,14 +176,15 @@ func tarPrefix(path, location, refName string, tw tarWriter) error {
return nil
}
// ImageTar takes a Docker image and outputs it to a tar stream
// ImageTar takes a Docker image and outputs it to a tar stream as a merged filesystem for a specific architecture
// defined in opts.
// location is where it is in the linuxkit.yaml file
func ImageTar(location string, ref *reference.Spec, prefix string, tw tarWriter, resolv string, opts BuildOpts) (e error) {
refName := "empty"
if ref != nil {
refName = ref.String()
}
log.Debugf("image tar: %s %s", refName, prefix)
log.Debugf("image filesystem tar: %s %s %s", refName, prefix, opts.Arch)
if prefix != "" && prefix[len(prefix)-1] != '/' {
return fmt.Errorf("prefix does not end with /: %s", prefix)
}
@@ -197,9 +199,8 @@ func ImageTar(location string, ref *reference.Spec, prefix string, tw tarWriter,
return nil
}
// pullImage first checks in the cache, then pulls the image.
// If pull==true, then it always tries to pull from registry.
src, err := imagePull(ref, opts.Pull, opts.CacheDir, opts.DockerCache, opts.Arch)
// get a handle on the image, optionally from docker, pulling from registry if necessary.
src, err := imageSource(ref, opts.Pull, opts.CacheDir, opts.DockerCache, imagespec.Platform{OS: "linux", Architecture: opts.Arch})
if err != nil {
return fmt.Errorf("could not pull image %s: %v", ref, err)
}
@@ -237,7 +238,7 @@ func ImageTar(location string, ref *reference.Spec, prefix string, tw tarWriter,
hdr.PAXRecords[moby.PaxRecordLinuxkitSource] = ref.String()
hdr.PAXRecords[moby.PaxRecordLinuxkitLocation] = location
if exclude[hdr.Name] {
log.Debugf("image tar: %s %s exclude %s", ref, prefix, hdr.Name)
log.Tracef("image tar: %s %s exclude %s", ref, prefix, hdr.Name)
_, err = io.Copy(io.Discard, tr)
if err != nil {
return err
@@ -248,7 +249,7 @@ func ImageTar(location string, ref *reference.Spec, prefix string, tw tarWriter,
hdr.Size = int64(len(contents))
hdr.Name = prefix + hdr.Name
hdr.ModTime = defaultModTime
log.Debugf("image tar: %s %s add %s (replaced)", ref, prefix, hdr.Name)
log.Tracef("image tar: %s %s add %s (replaced)", ref, prefix, hdr.Name)
if err := tw.WriteHeader(hdr); err != nil {
return err
}
@@ -263,7 +264,7 @@ func ImageTar(location string, ref *reference.Spec, prefix string, tw tarWriter,
hdr.Typeflag = tar.TypeSymlink
hdr.Linkname = resolv
hdr.ModTime = defaultModTime
log.Debugf("image tar: %s %s add resolv symlink /etc/resolv.conf -> %s", ref, prefix, resolv)
log.Tracef("image tar: %s %s add resolv symlink /etc/resolv.conf -> %s", ref, prefix, resolv)
if err := tw.WriteHeader(hdr); err != nil {
return err
}
@@ -274,12 +275,12 @@ func ImageTar(location string, ref *reference.Spec, prefix string, tw tarWriter,
}
} else {
if found, ok := touch[hdr.Name]; ok {
log.Debugf("image tar: %s %s add %s (touch)", ref, prefix, hdr.Name)
log.Tracef("image tar: %s %s add %s (touch)", ref, prefix, hdr.Name)
hdr.ModTime = found.ModTime
// record that we saw this one
touchFound[hdr.Name] = true
} else {
log.Debugf("image tar: %s %s add %s (original)", ref, prefix, hdr.Name)
log.Tracef("image tar: %s %s add %s (original)", ref, prefix, hdr.Name)
}
hdr.Name = prefix + hdr.Name
if hdr.Typeflag == tar.TypeLink {
@@ -304,7 +305,7 @@ func ImageTar(location string, ref *reference.Spec, prefix string, tw tarWriter,
sort.Strings(touchNames)
for _, name := range touchNames {
if touchFound[name] {
log.Debugf("image tar: %s already found in original image", name)
log.Tracef("image tar: %s already found in original image", name)
continue
}
hdr := touch[name]
@@ -326,9 +327,9 @@ func ImageTar(location string, ref *reference.Spec, prefix string, tw tarWriter,
hdr.Size = 0
hdr.Typeflag = tar.TypeSymlink
hdr.Linkname = resolv
log.Debugf("image tar: %s %s add resolv symlink /etc/resolv.conf -> %s", ref, prefix, resolv)
log.Tracef("image tar: %s %s add resolv symlink /etc/resolv.conf -> %s", ref, prefix, resolv)
}
log.Debugf("image tar: creating %s", name)
log.Tracef("image tar: creating %s", name)
if err := tw.WriteHeader(&hdr); err != nil {
return err
}
@@ -356,6 +357,79 @@ func ImageTar(location string, ref *reference.Spec, prefix string, tw tarWriter,
return nil
}
// ImageOCITar takes an OCI image and outputs it to a tar stream as a v1 layout format.
// Will include all architectures, or, if specific ones provided, then only those.
// location is where it is in the linuxkit.yaml file
func ImageOCITar(location string, ref *reference.Spec, prefix string, tw tarWriter, opts BuildOpts, platforms []imagespec.Platform) (e error) {
refName := "empty"
if ref != nil {
refName = ref.String()
}
log.Debugf("image v1 layout tar: %s %s %s", refName, prefix, opts.Arch)
if prefix != "" && prefix[len(prefix)-1] != '/' {
return fmt.Errorf("prefix does not end with /: %s", prefix)
}
err := tarPrefix(prefix, location, refName, tw)
if err != nil {
return err
}
// if the image is blank, we do not need to do any more
if ref == nil {
return fmt.Errorf("no image reference provided")
}
// indexSource first checks in the cache, then pulls the image.
// If pull==true, then it always tries to pull from registry.
src, err := indexSource(ref, opts.Pull, opts.CacheDir, platforms)
if err != nil {
return fmt.Errorf("could not pull image %s: %v", ref, err)
}
contents, err := src.OCITarReader("")
if err != nil {
return fmt.Errorf("could not unpack image %s: %v", ref, err)
}
defer contents.Close()
tr := tar.NewReader(contents)
for {
hdr, err := tr.Next()
if err == io.EOF {
break
}
if err != nil {
return err
}
// force PAX format, since it allows for unlimited Name/Linkname
// and we move all files below prefix.
hdr.Format = tar.FormatPAX
// ensure we record the source of the file in the PAX header
if hdr.PAXRecords == nil {
hdr.PAXRecords = make(map[string]string)
}
hdr.PAXRecords[moby.PaxRecordLinuxkitSource] = ref.String()
hdr.PAXRecords[moby.PaxRecordLinuxkitLocation] = location
hdr.Name = prefix + hdr.Name
if hdr.Typeflag == tar.TypeLink {
// hard links are referenced by full path so need to be adjusted
hdr.Linkname = prefix + hdr.Linkname
}
if err := tw.WriteHeader(hdr); err != nil {
return err
}
_, err = io.Copy(tw, tr)
if err != nil {
return err
}
}
return nil
}
// ImageBundle produces an OCI bundle at the given path in a tarball, given an image and a config.json
func ImageBundle(prefix, location string, ref *reference.Spec, config []byte, runtime moby.Runtime, tw tarWriter, readonly bool, dupMap map[string]string, opts BuildOpts) error { // nolint: lll
// if read only, just unpack in rootfs/ but otherwise set up for overlay
@@ -489,7 +563,7 @@ func ImageBundle(prefix, location string, ref *reference.Spec, config []byte, ru
return err
}
log.Debugf("image bundle: %s %s cfg: %s runtime: %s", prefix, ref, string(config), string(runtimeConfig))
log.Tracef("image bundle: %s %s cfg: %s runtime: %s", prefix, ref, string(config), string(runtimeConfig))
return nil
}

View File

@@ -5,20 +5,23 @@ import (
"github.com/linuxkit/linuxkit/src/cmd/linuxkit/cache"
"github.com/linuxkit/linuxkit/src/cmd/linuxkit/docker"
lktspec "github.com/linuxkit/linuxkit/src/cmd/linuxkit/spec"
imagespec "github.com/opencontainers/image-spec/specs-go/v1"
)
// imagePull pull an image from the OCI registry to the cache.
// If the image root already is in the cache, use it, unless
// the option pull is set to true.
// if alwaysPull, then do not even bother reading locally
func imagePull(ref *reference.Spec, alwaysPull bool, cacheDir string, dockerCache bool, architecture string) (lktspec.ImageSource, error) {
// imageSource given an image ref, get a handle on the image so it can be used as a source for its configuration
// and layers. If the image root already is in the cache, use it.
// If not in cache, pull it down from the OCI registry.
// Optionally can look in docker image cache first, before falling back to linuxkit cache and OCI registry.
// Optionally can be told to alwaysPull, in which case it always pulls from the OCI registry.
// Always works for a single architecture, as we are referencing a specific image.
func imageSource(ref *reference.Spec, alwaysPull bool, cacheDir string, dockerCache bool, platform imagespec.Platform) (lktspec.ImageSource, error) {
// several possibilities:
// - alwaysPull: try to pull it down from the registry to linuxkit cache, then fail
// - !alwaysPull && dockerCache: try to read it from docker, then try linuxkit cache, then try to pull from registry, then fail
// - !alwaysPull && !dockerCache: try linuxkit cache, then try to pull from registry, then fail
// first, try docker, if that is available
if !alwaysPull && dockerCache {
if err := docker.HasImage(ref); err == nil {
if err := docker.HasImage(ref, platform.Architecture); err == nil {
return docker.NewSource(ref), nil
}
// docker is not required, so any error - image not available, no docker, whatever - just gets ignored
@@ -31,5 +34,44 @@ func imagePull(ref *reference.Spec, alwaysPull bool, cacheDir string, dockerCach
}
// if we made it here, we either did not have the image, or it was incomplete
return c.ImagePull(ref, ref.String(), architecture, alwaysPull)
if err := c.ImagePull(ref, []imagespec.Platform{platform}, alwaysPull); err != nil {
return nil, err
}
desc, err := c.FindDescriptor(ref)
if err != nil {
return nil, err
}
return c.NewSource(
ref,
&platform,
desc,
), nil
}
// indexSource given an image ref, get a handle on the index so it can be used as a source for its underlying images.
// If the index root already is in the cache, use it.
// If not in cache, pull it down from the OCI registry.
// Optionally can look in docker image cache first, before falling back to linuxkit cache and OCI registry.
// Optionally can be told to alwaysPull, in which case it always pulls from the OCI registry.
// Can provide architectures to list which ones to limit, or leave empty for all available.
func indexSource(ref *reference.Spec, alwaysPull bool, cacheDir string, platforms []imagespec.Platform) (lktspec.IndexSource, error) {
// get a reference to the local cache; we either will find the ref there or will pull to it
c, err := cache.NewProvider(cacheDir)
if err != nil {
return nil, err
}
// if we made it here, we either did not have the image, or it was incomplete
if err := c.ImagePull(ref, platforms, alwaysPull); err != nil {
return nil, err
}
desc, err := c.FindDescriptor(ref)
if err != nil {
return nil, err
}
return c.NewIndexSource(
ref,
desc,
platforms,
), nil
}

View File

@@ -1,12 +1,12 @@
iso: linuxkit/mkimage-iso:08d19f8acf285bdce65dd4aea24f01d8adbedfbc
iso-bios: linuxkit/mkimage-iso-bios:96d5dac296345c308b8ad9e6cae7467e76ba8fd1
iso-efi: linuxkit/mkimage-iso-efi:cf2a3ff1dacfcacfb1f165abc210d9e33dc9d161
iso-efi-initrd: linuxkit/mkimage-iso-efi-initrd:a9e61bc810ae9928bab92f41f6e810e5b4f6183a
iso-efi: linuxkit/mkimage-iso-efi:8b538605a581db7523c021bf92a715d2054f609e
iso-efi-initrd: linuxkit/mkimage-iso-efi-initrd:d390030aae1069f3142523e9f433aad946838911
raw-bios: linuxkit/mkimage-raw-bios:4c21d66c81fd3641c62b9e80ddf5494000a1a442
raw-efi: linuxkit/mkimage-raw-efi:df0979572e8d0251a5cf55b6f73131868d036bb4
raw-efi: linuxkit/mkimage-raw-efi:14b66a308b2047c59b0fe7c43996f73c653a9fcd
squashfs: linuxkit/mkimage-squashfs:a61fd76227ab4998d6c1ba17229cd8bd749e8f13
gcp: linuxkit/mkimage-gcp:035c2c2b4b958060c0b6bdd41d9cbc886a335098
qcow2-efi: linuxkit/mkimage-qcow2-efi:98a6e3e7b6eed965f879cd77c009c7c404d4457d
qcow2-efi: linuxkit/mkimage-qcow2-efi:8b76f4118e6640db6f39196dc365529f401e7670
vhd: linuxkit/mkimage-vhd:91bcc7a6475f46a3d5d84cf6161f07c583dd9c21
dynamic-vhd: linuxkit/mkimage-dynamic-vhd:b755f8ff82c8631d18decaebb09867e7b88c2533
vmdk: linuxkit/mkimage-vmdk:20a370a55bd8d58c2ae9d634c297a955bb006efd

View File

@@ -2,7 +2,7 @@ kernel:
image: linuxkit/kernel:6.6.13
cmdline: "console=ttyS0"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
onboot:
- name: mkimage

View File

@@ -13,10 +13,10 @@ import (
"github.com/containerd/containerd/reference"
"github.com/linuxkit/linuxkit/src/cmd/linuxkit/spec"
"github.com/linuxkit/linuxkit/src/cmd/linuxkit/util"
"github.com/moby/sys/capability"
imagespec "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/opencontainers/runtime-spec/specs-go"
log "github.com/sirupsen/logrus"
"github.com/syndtr/gocapability/capability"
"github.com/xeipuuv/gojsonschema"
"gopkg.in/yaml.v3"
)
@@ -76,10 +76,12 @@ type File struct {
// Volume is the type of a volume specification
type Volume struct {
Name string `yaml:"name" json:"name"`
Image string `yaml:"image,omitempty" json:"image,omitempty"`
ReadOnly bool `yaml:"readonly,omitempty" json:"readonly,omitempty"`
ref *reference.Spec
Name string `yaml:"name" json:"name"`
Image string `yaml:"image,omitempty" json:"image,omitempty"`
ReadOnly bool `yaml:"readonly,omitempty" json:"readonly,omitempty"`
Format string `yaml:"format,omitempty" json:"format,omitempty"`
Platforms []string `yaml:"platforms,omitempty" json:"platforms,omitempty"`
ref *reference.Spec
}
func (v Volume) ImageRef() *reference.Spec {
@@ -446,7 +448,7 @@ func AppendConfig(m0, m1 Moby) (Moby, error) {
// NewImage validates an parses yaml or json for a Image
func NewImage(config []byte) (Image, error) {
log.Debugf("Reading label config: %s", string(config))
log.Tracef("Reading label config: %s", string(config))
mi := Image{}
@@ -781,7 +783,7 @@ func assignStringEmpty4(v1, v2, v3, v4 string) string {
func getAllCapabilities() []string {
var caps []string
for _, cap := range capability.List() {
for _, cap := range capability.ListKnown() {
caps = append(caps, "CAP_"+strings.ToUpper(cap.String()))
}
return caps

View File

@@ -43,7 +43,9 @@ var schema = `
"properties": {
"name": {"type": "string"},
"image": {"type": "string"},
"readonly": {"type": "boolean"}
"readonly": {"type": "boolean"},
"format": {"enum": ["oci","filesystem"]},
"platforms": {"$ref": "#/definitions/strings"}
}
},
"volumes": {

View File

@@ -37,7 +37,6 @@ func pkgCmd() *cobra.Command {
HashPath: hashPath,
Dirty: dirty,
Dev: devMode,
Tag: tag,
}
if cmd.Flags().Changed("disable-cache") && cmd.Flags().Changed("enable-cache") {
return errors.New("cannot set but disable-cache and enable-cache")
@@ -65,6 +64,9 @@ func pkgCmd() *cobra.Command {
if cmd.Flags().Changed("org") {
pkglibConfig.Org = &argOrg
}
if cmd.Flags().Changed("tag") {
pkglibConfig.Tag = tag
}
return nil
},
@@ -88,7 +90,7 @@ func pkgCmd() *cobra.Command {
cmd.PersistentFlags().StringVar(&argOrg, "org", piBase.Org, "Override the hub org")
cmd.PersistentFlags().StringVar(&buildYML, "build-yml", defaultPkgBuildYML, "Override the name of the yml file")
cmd.PersistentFlags().StringVar(&hash, "hash", "", "Override the image hash (default is to query git for the package's tree-sh)")
cmd.PersistentFlags().StringVar(&tag, "tag", defaultPkgTag, "Override the tag using fixed strings and/or text templates. Acceptable are .Hash for the hash")
cmd.PersistentFlags().StringVar(&tag, "tag", piBase.Tag, "Override the tag using fixed strings and/or text templates. Acceptable are .Hash for the hash")
cmd.PersistentFlags().StringVar(&hashCommit, "hash-commit", defaultPkgCommit, "Override the git commit to use for the hash")
cmd.PersistentFlags().StringVar(&hashPath, "hash-path", "", "Override the directory to use for the image hash, must be a parent of the package dir (default is to use the package dir)")
cmd.PersistentFlags().BoolVar(&dirty, "force-dirty", false, "Force the pkg(s) to be considered dirty")

View File

@@ -327,7 +327,7 @@ func (p Pkg) Build(bos ...BuildOpt) error {
case bo.pull:
// need to pull the image from the registry, else build
fmt.Fprintf(writer, "%s %s not found in local cache, trying to pull\n", ref, platform.Architecture)
if _, err := c.ImagePull(&ref, "", platform.Architecture, false); err == nil {
if err := c.ImagePull(&ref, []imagespec.Platform{platform}, false); err == nil {
fmt.Fprintf(writer, "%s pulled\n", ref)
// successfully pulled, no need to build, continue with next platform
continue
@@ -470,7 +470,7 @@ func (p Pkg) Build(bos ...BuildOpt) error {
// - potentially create a release, including push and load into docker
// create a multi-arch index
if _, err := c.IndexWrite(&ref, descs...); err != nil {
if err := c.IndexWrite(&ref, descs...); err != nil {
return err
}
}
@@ -490,7 +490,7 @@ func (p Pkg) Build(bos ...BuildOpt) error {
if err != nil {
return err
}
cacheSource := c.NewSource(&ref, platform.Architecture, desc)
cacheSource := c.NewSource(&ref, &platform, desc)
reader, err := cacheSource.V1TarReader(fmt.Sprintf("%s-%s", p.FullTag(), platform.Architecture))
if err != nil {
return fmt.Errorf("unable to get reader from cache: %v", err)
@@ -562,7 +562,7 @@ func (p Pkg) Build(bos ...BuildOpt) error {
if err != nil {
return err
}
if _, err := c.DescriptorWrite(&ref, *desc); err != nil {
if err := c.DescriptorWrite(&ref, *desc); err != nil {
return err
}
if err := c.Push(fullRelTag, "", bo.manifest, true); err != nil {
@@ -617,7 +617,7 @@ func (p Pkg) buildArch(ctx context.Context, d dockerRunner, c lktspec.CacheProvi
if err != nil {
return nil, fmt.Errorf("could not resolve references for image %s: %v", p.Tag(), err)
}
if _, err := c.ImagePull(&ref, "", arch, false); err == nil {
if err := c.ImagePull(&ref, []imagespec.Platform{{Architecture: arch, OS: "linux"}}, false); err == nil {
fmt.Fprintf(writer, "image already found %s for arch %s", ref, arch)
desc, err := c.FindDescriptor(&ref)
if err != nil {

View File

@@ -240,21 +240,24 @@ type cacheMocker struct {
hashes map[string][]byte
}
func (c *cacheMocker) ImagePull(ref *reference.Spec, trustedRef, architecture string, alwaysPull bool) (lktspec.ImageSource, error) {
func (c *cacheMocker) ImagePull(ref *reference.Spec, platforms []imagespec.Platform, alwaysPull bool) error {
if !c.enableImagePull {
return nil, errors.New("ImagePull disabled")
return errors.New("ImagePull disabled")
}
// make some random data for a layer
b := make([]byte, 256)
_, _ = rand.Read(b)
descs, err := c.imageWriteStream(bytes.NewReader(b))
if err != nil {
return nil, err
return err
}
if len(descs) != 1 {
return nil, fmt.Errorf("expected 1 descriptor, got %d", len(descs))
return fmt.Errorf("expected 1 descriptor, got %d", len(descs))
}
return c.NewSource(ref, architecture, &descs[1]), nil
if len(platforms) != 1 {
return fmt.Errorf("cache does not support multiple platforms %s", platforms)
}
return nil
}
func (c *cacheMocker) ImageInCache(ref *reference.Spec, trustedRef, architecture string) (bool, error) {
@@ -359,9 +362,9 @@ func (c *cacheMocker) imageWriteStream(r io.Reader) ([]registry.Descriptor, erro
return []registry.Descriptor{desc}, nil
}
func (c *cacheMocker) IndexWrite(ref *reference.Spec, descriptors ...registry.Descriptor) (lktspec.ImageSource, error) {
func (c *cacheMocker) IndexWrite(ref *reference.Spec, descriptors ...registry.Descriptor) error {
if !c.enableIndexWrite {
return nil, errors.New("disabled")
return errors.New("disabled")
}
image := ref.String()
im := registry.IndexManifest{
@@ -373,11 +376,11 @@ func (c *cacheMocker) IndexWrite(ref *reference.Spec, descriptors ...registry.De
// write the updated index, remove the old one
b, err := json.Marshal(im)
if err != nil {
return nil, fmt.Errorf("unable to marshal new index to json: %v", err)
return fmt.Errorf("unable to marshal new index to json: %v", err)
}
hash, size, err := registry.SHA256(bytes.NewReader(b))
if err != nil {
return nil, fmt.Errorf("error calculating hash of index json: %v", err)
return fmt.Errorf("error calculating hash of index json: %v", err)
}
c.assignHash(hash.String(), b)
desc := registry.Descriptor{
@@ -390,7 +393,7 @@ func (c *cacheMocker) IndexWrite(ref *reference.Spec, descriptors ...registry.De
}
c.appendImage(image, desc)
return c.NewSource(ref, "", &desc), nil
return nil
}
func (c *cacheMocker) Push(name, remoteName string, withManifest, override bool) error {
if !c.enablePush {
@@ -402,9 +405,9 @@ func (c *cacheMocker) Push(name, remoteName string, withManifest, override bool)
return nil
}
func (c *cacheMocker) DescriptorWrite(ref *reference.Spec, desc registry.Descriptor) (lktspec.ImageSource, error) {
func (c *cacheMocker) DescriptorWrite(ref *reference.Spec, desc registry.Descriptor) error {
if !c.enabledDescriptorWrite {
return nil, errors.New("descriptor disabled")
return errors.New("descriptor disabled")
}
var (
image = ref.String()
@@ -417,11 +420,11 @@ func (c *cacheMocker) DescriptorWrite(ref *reference.Spec, desc registry.Descrip
// write the updated index, remove the old one
b, err := json.Marshal(im)
if err != nil {
return nil, fmt.Errorf("unable to marshal new index to json: %v", err)
return fmt.Errorf("unable to marshal new index to json: %v", err)
}
hash, size, err := registry.SHA256(bytes.NewReader(b))
if err != nil {
return nil, fmt.Errorf("error calculating hash of index json: %v", err)
return fmt.Errorf("error calculating hash of index json: %v", err)
}
c.assignHash(hash.String(), b)
root := registry.Descriptor{
@@ -434,7 +437,7 @@ func (c *cacheMocker) DescriptorWrite(ref *reference.Spec, desc registry.Descrip
}
c.appendImage(image, root)
return c.NewSource(ref, "", &root), nil
return nil
}
func (c *cacheMocker) FindDescriptor(ref *reference.Spec) (*registry.Descriptor, error) {
name := ref.String()
@@ -443,8 +446,8 @@ func (c *cacheMocker) FindDescriptor(ref *reference.Spec) (*registry.Descriptor,
}
return nil, fmt.Errorf("not found %s", name)
}
func (c *cacheMocker) NewSource(ref *reference.Spec, architecture string, descriptor *registry.Descriptor) lktspec.ImageSource {
return cacheMockerSource{c, ref, architecture, descriptor}
func (c *cacheMocker) NewSource(ref *reference.Spec, platform *imagespec.Platform, descriptor *registry.Descriptor) lktspec.ImageSource {
return cacheMockerSource{c, ref, platform, descriptor}
}
func (c *cacheMocker) assignHash(hash string, b []byte) {
if c.hashes == nil {
@@ -473,10 +476,10 @@ func (c *cacheMocker) GetContent(hash v1.Hash) (io.ReadCloser, error) {
}
type cacheMockerSource struct {
c *cacheMocker
ref *reference.Spec
architecture string
descriptor *registry.Descriptor
c *cacheMocker
ref *reference.Spec
platform *imagespec.Platform
descriptor *registry.Descriptor
}
func (c cacheMockerSource) Config() (imagespec.ImageConfig, error) {

View File

@@ -20,6 +20,7 @@ import (
type pkgInfo struct {
Image string `yaml:"image"`
Org string `yaml:"org"`
Tag string `yaml:"tag,omitempty"` // default to {{.Hash}}
Dockerfile string `yaml:"dockerfile"`
Arches []string `yaml:"arches"`
ExtraSources []string `yaml:"extra-sources"`
@@ -60,6 +61,7 @@ func NewPkgInfo() pkgInfo {
return pkgInfo{
Org: "linuxkit",
Arches: []string{"amd64", "arm64"},
Tag: "{{.Hash}}",
GitRepo: "https://github.com/linuxkit/linuxkit",
Network: false,
DisableCache: false,
@@ -257,9 +259,16 @@ func NewFromConfig(cfg PkglibConfig, args ...string) ([]Pkg, error) {
}
}
}
tagTmpl := pi.Tag
if cfg.Tag != "" {
tagTmpl = cfg.Tag
}
if tagTmpl == "" {
tagTmpl = "{{.Hash}}"
}
// calculate the tag to use based on the template and the pkgHash
tmpl, err := template.New("tag").Parse(cfg.Tag)
tmpl, err := template.New("tag").Parse(tagTmpl)
if err != nil {
return nil, fmt.Errorf("invalid tag template: %v", err)
}

View File

@@ -6,6 +6,7 @@ import (
"github.com/containerd/containerd/content"
"github.com/containerd/containerd/reference"
v1 "github.com/google/go-containerregistry/pkg/v1"
imagespec "github.com/opencontainers/image-spec/specs-go/v1"
)
// CacheProvider interface for a provide of a cache.
@@ -19,7 +20,7 @@ type CacheProvider interface {
// ImagePull takes an image name and pulls it from a registry to the cache. It should be
// efficient and only write missing blobs, based on their content hash. If the ref already
// exists in the cache, it should not pull anything, unless alwaysPull is set to true.
ImagePull(ref *reference.Spec, trustedRef, architecture string, alwaysPull bool) (ImageSource, error)
ImagePull(ref *reference.Spec, platform []imagespec.Platform, alwaysPull bool) error
// ImageInCache takes an image name and checks if it exists in the cache, including checking that the given
// architecture is complete. Like ImagePull, it should be efficient and only write missing blobs, based on
// their content hash.
@@ -30,20 +31,20 @@ type CacheProvider interface {
// Cache implementation determines whether it should pull missing blobs from a remote registry.
// If the provided reference already exists and it is an index, updates the manifests in the
// existing index.
IndexWrite(ref *reference.Spec, descriptors ...v1.Descriptor) (ImageSource, error)
IndexWrite(ref *reference.Spec, descriptors ...v1.Descriptor) error
// ImageLoad takes an OCI format image tar stream in the io.Reader and writes it to the cache. It should be
// efficient and only write missing blobs, based on their content hash.
ImageLoad(r io.Reader) ([]v1.Descriptor, error)
// DescriptorWrite writes a descriptor to the cache index; it validates that it has a name
// and replaces any existing one
DescriptorWrite(ref *reference.Spec, descriptors v1.Descriptor) (ImageSource, error)
DescriptorWrite(ref *reference.Spec, descriptors v1.Descriptor) error
// Push an image along with a multi-arch index from local cache to remote registry.
// name is the name as referenced in the local cache, remoteName is the name to give it remotely.
// If remoteName is empty, it is the same as name.
// if withManifest defined will push a multi-arch manifest
Push(name, remoteName string, withManifest, override bool) error
// NewSource return an ImageSource for a specific ref and architecture in the cache.
NewSource(ref *reference.Spec, architecture string, descriptor *v1.Descriptor) ImageSource
NewSource(ref *reference.Spec, platform *imagespec.Platform, descriptor *v1.Descriptor) ImageSource
// GetContent returns an io.Reader to the provided content as is, given a specific digest. It is
// up to the caller to validate it.
GetContent(hash v1.Hash) (io.ReadCloser, error)

View File

@@ -10,12 +10,12 @@ import (
// ImageSource interface to an image. It can have its config read, and a its containers
// can be read via an io.ReadCloser tar stream.
type ImageSource interface {
// Descriptor get the v1.Descriptor of the image
Descriptor() *v1.Descriptor
// Config get the config for the image
Config() (imagespec.ImageConfig, error)
// TarReader get the flattened filesystem of the image as a tar stream
TarReader() (io.ReadCloser, error)
// Descriptor get the v1.Descriptor of the image
Descriptor() *v1.Descriptor
// V1TarReader get the image as v1 tarball, also compatible with `docker load`. If name arg is not "", override name of image in tarfile from default of image.
V1TarReader(overrideName string) (io.ReadCloser, error)
// OCITarReader get the image as an OCI tarball, also compatible with `docker load`. If name arg is not "", override name of image in tarfile from default of image.
@@ -23,3 +23,14 @@ type ImageSource interface {
// SBoM get the sbom for the image, if any is available
SBoMs() ([]io.ReadCloser, error)
}
// IndexSource interface to an image. It can have its config read, and a its containers
// can be read via an io.ReadCloser tar stream.
type IndexSource interface {
// Descriptor get the v1.Descriptor of the index
Descriptor() *v1.Descriptor
// Image get image for a specific architecture
Image(platform imagespec.Platform) (ImageSource, error)
// OCITarReader get the image as an OCI tarball, also compatible with `docker load`. If name arg is not "", override name of image in tarfile from default of image.
OCITarReader(overrideName string) (io.ReadCloser, error)
}

View File

@@ -25,23 +25,38 @@ func (f *infoFormatter) Format(entry *log.Entry) ([]byte, error) {
}
// SetupLogging once the flags have been parsed, setup the logging
func SetupLogging(quiet, verbose bool) error {
func SetupLogging(quiet bool, verbose int, verboseSet bool) error {
// Set up logging
log.SetFormatter(new(infoFormatter))
log.SetLevel(log.InfoLevel)
if quiet && verbose {
if quiet && verboseSet && verbose > 0 {
return errors.New("can't set quiet and verbose flag at the same time")
}
if quiet {
switch {
case quiet, verbose == 0:
log.SetLevel(log.ErrorLevel)
}
if verbose {
case verbose == 1:
if verboseSet {
// Switch back to the standard formatter
log.SetFormatter(defaultLogFormatter)
}
log.SetLevel(log.InfoLevel)
case verbose == 2:
// Switch back to the standard formatter
log.SetFormatter(defaultLogFormatter)
log.SetLevel(log.DebugLevel)
// set go-containerregistry logging as well
ggcrlog.Warn = stdlog.New(log.StandardLogger().WriterLevel(log.WarnLevel), "", 0)
ggcrlog.Debug = stdlog.New(log.StandardLogger().WriterLevel(log.DebugLevel), "", 0)
case verbose == 3:
// Switch back to the standard formatter
log.SetFormatter(defaultLogFormatter)
log.SetLevel(log.TraceLevel)
// set go-containerregistry logging as well
ggcrlog.Warn = stdlog.New(log.StandardLogger().WriterLevel(log.WarnLevel), "", 0)
ggcrlog.Debug = stdlog.New(log.StandardLogger().WriterLevel(log.DebugLevel), "", 0)
default:
return errors.New("verbose flag can only be set to 0, 1, 2 or 3")
}
ggcrlog.Progress = stdlog.New(log.StandardLogger().WriterLevel(log.InfoLevel), "", 0)
return nil

View File

@@ -3,7 +3,7 @@ package api // import "github.com/docker/docker/api"
// Common constants for daemon and client.
const (
// DefaultVersion of the current REST API.
DefaultVersion = "1.46"
DefaultVersion = "1.47"
// MinSupportedAPIVersion is the minimum API version that can be supported
// by the API server, specified as "major.minor". Note that the daemon

View File

@@ -19,10 +19,10 @@ produces:
consumes:
- "application/json"
- "text/plain"
basePath: "/v1.46"
basePath: "/v1.47"
info:
title: "Docker Engine API"
version: "1.46"
version: "1.47"
x-logo:
url: "https://docs.docker.com/assets/images/logo-docker-main.png"
description: |
@@ -55,8 +55,8 @@ info:
the URL is not supported by the daemon, a HTTP `400 Bad Request` error message
is returned.
If you omit the version-prefix, the current version of the API (v1.46) is used.
For example, calling `/info` is the same as calling `/v1.46/info`. Using the
If you omit the version-prefix, the current version of the API (v1.47) is used.
For example, calling `/info` is the same as calling `/v1.47/info`. Using the
API without a version-prefix is deprecated and will be removed in a future release.
Engine releases in the near future should support this version of the API,
@@ -2265,6 +2265,19 @@ definitions:
x-nullable: false
type: "integer"
example: 2
Manifests:
description: |
Manifests is a list of manifests available in this image.
It provides a more detailed view of the platform-specific image manifests
or other image-attached data like build attestations.
WARNING: This is experimental and may change at any time without any backward
compatibility.
type: "array"
x-nullable: false
x-omitempty: true
items:
$ref: "#/definitions/ImageManifestSummary"
AuthConfig:
type: "object"
@@ -5318,7 +5331,7 @@ definitions:
description: |
The default (and highest) API version that is supported by the daemon
type: "string"
example: "1.46"
example: "1.47"
MinAPIVersion:
description: |
The minimum API version that is supported by the daemon
@@ -5334,7 +5347,7 @@ definitions:
The version Go used to compile the daemon, and the version of the Go
runtime in use.
type: "string"
example: "go1.21.11"
example: "go1.21.13"
Os:
description: |
The operating system that the daemon is running on ("linux" or "windows")
@@ -5830,13 +5843,13 @@ definitions:
- "/var/run/cdi"
Containerd:
$ref: "#/definitions/ContainerdInfo"
x-nullable: true
ContainerdInfo:
description: |
Information for connecting to the containerd instance that is used by the daemon.
This is included for debugging purposes only.
type: "object"
x-nullable: true
properties:
Address:
description: "The address of the containerd socket."
@@ -6644,6 +6657,120 @@ definitions:
additionalProperties:
type: "string"
ImageManifestSummary:
x-go-name: "ManifestSummary"
description: |
ImageManifestSummary represents a summary of an image manifest.
type: "object"
required: ["ID", "Descriptor", "Available", "Size", "Kind"]
properties:
ID:
description: |
ID is the content-addressable ID of an image and is the same as the
digest of the image manifest.
type: "string"
example: "sha256:95869fbcf224d947ace8d61d0e931d49e31bb7fc67fffbbe9c3198c33aa8e93f"
Descriptor:
$ref: "#/definitions/OCIDescriptor"
Available:
description: Indicates whether all the child content (image config, layers) is fully available locally.
type: "boolean"
example: true
Size:
type: "object"
x-nullable: false
required: ["Content", "Total"]
properties:
Total:
type: "integer"
format: "int64"
example: 8213251
description: |
Total is the total size (in bytes) of all the locally present
data (both distributable and non-distributable) that's related to
this manifest and its children.
This equal to the sum of [Content] size AND all the sizes in the
[Size] struct present in the Kind-specific data struct.
For example, for an image kind (Kind == "image")
this would include the size of the image content and unpacked
image snapshots ([Size.Content] + [ImageData.Size.Unpacked]).
Content:
description: |
Content is the size (in bytes) of all the locally present
content in the content store (e.g. image config, layers)
referenced by this manifest and its children.
This only includes blobs in the content store.
type: "integer"
format: "int64"
example: 3987495
Kind:
type: "string"
example: "image"
enum:
- "image"
- "attestation"
- "unknown"
description: |
The kind of the manifest.
kind | description
-------------|-----------------------------------------------------------
image | Image manifest that can be used to start a container.
attestation | Attestation manifest produced by the Buildkit builder for a specific image manifest.
ImageData:
description: |
The image data for the image manifest.
This field is only populated when Kind is "image".
type: "object"
x-nullable: true
x-omitempty: true
required: ["Platform", "Containers", "Size", "UnpackedSize"]
properties:
Platform:
$ref: "#/definitions/OCIPlatform"
description: |
OCI platform of the image. This will be the platform specified in the
manifest descriptor from the index/manifest list.
If it's not available, it will be obtained from the image config.
Containers:
description: |
The IDs of the containers that are using this image.
type: "array"
items:
type: "string"
example: ["ede54ee1fda366ab42f824e8a5ffd195155d853ceaec74a927f249ea270c7430", "abadbce344c096744d8d6071a90d474d28af8f1034b5ea9fb03c3f4bfc6d005e"]
Size:
type: "object"
x-nullable: false
required: ["Unpacked"]
properties:
Unpacked:
type: "integer"
format: "int64"
example: 3987495
description: |
Unpacked is the size (in bytes) of the locally unpacked
(uncompressed) image content that's directly usable by the containers
running this image.
It's independent of the distributable content - e.g.
the image might still have an unpacked data that's still used by
some container even when the distributable/compressed content is
already gone.
AttestationData:
description: |
The image data for the attestation manifest.
This field is only populated when Kind is "attestation".
type: "object"
x-nullable: true
x-omitempty: true
required: ["For"]
properties:
For:
description: |
The digest of the image manifest that this attestation is for.
type: "string"
example: "sha256:95869fbcf224d947ace8d61d0e931d49e31bb7fc67fffbbe9c3198c33aa8e93f"
paths:
/containers/json:
get:
@@ -8622,6 +8749,11 @@ paths:
description: "Show digest information as a `RepoDigests` field on each image."
type: "boolean"
default: false
- name: "manifests"
in: "query"
description: "Include `Manifests` in the image summary."
type: "boolean"
default: false
tags: ["Image"]
/build:
post:
@@ -9563,7 +9695,7 @@ paths:
Containers report these events: `attach`, `commit`, `copy`, `create`, `destroy`, `detach`, `die`, `exec_create`, `exec_detach`, `exec_start`, `exec_die`, `export`, `health_status`, `kill`, `oom`, `pause`, `rename`, `resize`, `restart`, `start`, `stop`, `top`, `unpause`, `update`, and `prune`
Images report these events: `create, `delete`, `import`, `load`, `pull`, `push`, `save`, `tag`, `untag`, and `prune`
Images report these events: `create`, `delete`, `import`, `load`, `pull`, `push`, `save`, `tag`, `untag`, and `prune`
Volumes report these events: `create`, `mount`, `unmount`, `destroy`, and `prune`

View File

@@ -0,0 +1,99 @@
package image
import (
"github.com/opencontainers/go-digest"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
)
type ManifestKind string
const (
ManifestKindImage ManifestKind = "image"
ManifestKindAttestation ManifestKind = "attestation"
ManifestKindUnknown ManifestKind = "unknown"
)
type ManifestSummary struct {
// ID is the content-addressable ID of an image and is the same as the
// digest of the image manifest.
//
// Required: true
ID string `json:"ID"`
// Descriptor is the OCI descriptor of the image.
//
// Required: true
Descriptor ocispec.Descriptor `json:"Descriptor"`
// Indicates whether all the child content (image config, layers) is
// fully available locally
//
// Required: true
Available bool `json:"Available"`
// Size is the size information of the content related to this manifest.
// Note: These sizes only take the locally available content into account.
//
// Required: true
Size struct {
// Content is the size (in bytes) of all the locally present
// content in the content store (e.g. image config, layers)
// referenced by this manifest and its children.
// This only includes blobs in the content store.
Content int64 `json:"Content"`
// Total is the total size (in bytes) of all the locally present
// data (both distributable and non-distributable) that's related to
// this manifest and its children.
// This equal to the sum of [Content] size AND all the sizes in the
// [Size] struct present in the Kind-specific data struct.
// For example, for an image kind (Kind == ManifestKindImage),
// this would include the size of the image content and unpacked
// image snapshots ([Size.Content] + [ImageData.Size.Unpacked]).
Total int64 `json:"Total"`
} `json:"Size"`
// Kind is the kind of the image manifest.
//
// Required: true
Kind ManifestKind `json:"Kind"`
// Fields below are specific to the kind of the image manifest.
// Present only if Kind == ManifestKindImage.
ImageData *ImageProperties `json:"ImageData,omitempty"`
// Present only if Kind == ManifestKindAttestation.
AttestationData *AttestationProperties `json:"AttestationData,omitempty"`
}
type ImageProperties struct {
// Platform is the OCI platform object describing the platform of the image.
//
// Required: true
Platform ocispec.Platform `json:"Platform"`
Size struct {
// Unpacked is the size (in bytes) of the locally unpacked
// (uncompressed) image content that's directly usable by the containers
// running this image.
// It's independent of the distributable content - e.g.
// the image might still have an unpacked data that's still used by
// some container even when the distributable/compressed content is
// already gone.
//
// Required: true
Unpacked int64 `json:"Unpacked"`
}
// Containers is an array containing the IDs of the containers that are
// using this image.
//
// Required: true
Containers []string `json:"Containers"`
}
type AttestationProperties struct {
// For is the digest of the image manifest that this attestation is for.
For digest.Digest `json:"For"`
}

View File

@@ -76,6 +76,9 @@ type ListOptions struct {
// ContainerCount indicates whether container count should be computed.
ContainerCount bool
// Manifests indicates whether the image manifests should be returned.
Manifests bool
}
// RemoveOptions holds parameters to remove images.

View File

@@ -1,10 +1,5 @@
package image
// This file was generated by the swagger tool.
// Editing this file might prove futile when you re-run the swagger generate command
// Summary summary
// swagger:model Summary
type Summary struct {
// Number of containers using this image. Includes both stopped and running
@@ -47,6 +42,14 @@ type Summary struct {
// Required: true
ParentID string `json:"ParentId"`
// Manifests is a list of image manifests available in this image. It
// provides a more detailed view of the platform-specific image manifests or
// other image-attached data like build attestations.
//
// WARNING: This is experimental and may change at any time without any backward
// compatibility.
Manifests []ManifestSummary `json:"Manifests,omitempty"`
// List of content-addressable digests of locally available image manifests
// that the image is referenced from. Multiple manifests can refer to the
// same image.

View File

@@ -34,10 +34,9 @@ type AuthConfig struct {
}
// EncodeAuthConfig serializes the auth configuration as a base64url encoded
// RFC4648, section 5) JSON string for sending through the X-Registry-Auth header.
// ([RFC4648, section 5]) JSON string for sending through the X-Registry-Auth header.
//
// For details on base64url encoding, see:
// - RFC4648, section 5: https://tools.ietf.org/html/rfc4648#section-5
// [RFC4648, section 5]: https://tools.ietf.org/html/rfc4648#section-5
func EncodeAuthConfig(authConfig AuthConfig) (string, error) {
buf, err := json.Marshal(authConfig)
if err != nil {
@@ -46,15 +45,14 @@ func EncodeAuthConfig(authConfig AuthConfig) (string, error) {
return base64.URLEncoding.EncodeToString(buf), nil
}
// DecodeAuthConfig decodes base64url encoded (RFC4648, section 5) JSON
// DecodeAuthConfig decodes base64url encoded ([RFC4648, section 5]) JSON
// authentication information as sent through the X-Registry-Auth header.
//
// This function always returns an AuthConfig, even if an error occurs. It is up
// This function always returns an [AuthConfig], even if an error occurs. It is up
// to the caller to decide if authentication is required, and if the error can
// be ignored.
//
// For details on base64url encoding, see:
// - RFC4648, section 5: https://tools.ietf.org/html/rfc4648#section-5
// [RFC4648, section 5]: https://tools.ietf.org/html/rfc4648#section-5
func DecodeAuthConfig(authEncoded string) (*AuthConfig, error) {
if authEncoded == "" {
return &AuthConfig{}, nil
@@ -69,7 +67,7 @@ func DecodeAuthConfig(authEncoded string) (*AuthConfig, error) {
// clients and API versions. Current clients and API versions expect authentication
// to be provided through the X-Registry-Auth header.
//
// Like DecodeAuthConfig, this function always returns an AuthConfig, even if an
// Like [DecodeAuthConfig], this function always returns an [AuthConfig], even if an
// error occurs. It is up to the caller to decide if authentication is required,
// and if the error can be ignored.
func DecodeAuthConfigBody(rdr io.ReadCloser) (*AuthConfig, error) {

View File

@@ -77,9 +77,6 @@ type Info struct {
Containerd *ContainerdInfo `json:",omitempty"`
// Legacy API fields for older API versions.
legacyFields
// Warnings contains a slice of warnings that occurred while collecting
// system information. These warnings are intended to be informational
// messages for the user, and are not intended to be parsed / used for
@@ -124,10 +121,6 @@ type ContainerdNamespaces struct {
Plugins string
}
type legacyFields struct {
ExecutionDriver string `json:",omitempty"` // Deprecated: deprecated since API v1.25, but returned for older versions.
}
// PluginsInfo is a temp struct holding Plugins name
// registered with docker daemon. It is used by [Info] struct
type PluginsInfo struct {

View File

@@ -11,6 +11,11 @@ import (
)
// ImageList returns a list of images in the docker host.
//
// Experimental: Setting the [options.Manifest] will populate
// [image.Summary.Manifests] with information about image manifests.
// This is experimental and might change in the future without any backward
// compatibility.
func (cli *Client) ImageList(ctx context.Context, options image.ListOptions) ([]image.Summary, error) {
var images []image.Summary
@@ -47,6 +52,9 @@ func (cli *Client) ImageList(ctx context.Context, options image.ListOptions) ([]
if options.SharedSize && versions.GreaterThanOrEqualTo(cli.version, "1.42") {
query.Set("shared-size", "1")
}
if options.Manifests && versions.GreaterThanOrEqualTo(cli.version, "1.47") {
query.Set("manifests", "1")
}
serverResp, err := cli.get(ctx, "/images/json", query, nil)
defer ensureReaderClosed(serverResp)

View File

@@ -0,0 +1,3 @@
[codespell]
skip = ./.git
ignore-words-list = nd

View File

@@ -0,0 +1,6 @@
linters:
enable:
- unconvert
- unparam
- gofumpt
- errorlint

View File

@@ -0,0 +1,90 @@
# Changelog
This file documents all notable changes made to this project since the initial fork
from https://github.com/syndtr/gocapability/commit/42c35b4376354fd5.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [0.3.0] - 2024-09-25
### Added
* Added [ListKnown] and [ListSupported] functions. (#153)
* [LastCap] is now available on non-Linux platforms (where it returns an error). (#152)
### Changed
* [List] is now deprecated in favor of [ListKnown] and [ListSupported]. (#153)
### Fixed
* Various documentation improvements. (#151)
* Fix "generated code" comment. (#153)
## [0.2.0] - 2024-09-16
This is the first release after the move to a new home in
github.com/moby/sys/capability.
### Fixed
* Fixed URLs in documentation to reflect the new home.
## [0.1.1] - 2024-08-01
This is a maintenance release, fixing a few minor issues.
### Fixed
* Fixed future kernel compatibility, for real this time. [#11]
* Fixed [LastCap] to be a function. [#12]
## [0.1.0] - 2024-07-31
This is an initial release since the fork.
### Breaking changes
* The `CAP_LAST_CAP` variable is removed; users need to modify the code to
use [LastCap] to get the value. [#6]
* The code now requires Go >= 1.21.
### Added
* `go.mod` and `go.sum` files. [#2]
* New [LastCap] function. [#6]
* Basic CI using GHA infra. [#8], [#9]
* README and CHANGELOG. [#10]
### Fixed
* Fixed ambient capabilities error handling in [Apply]. [#3]
* Fixed future kernel compatibility. [#1]
* Fixed various linter warnings. [#4], [#7]
### Changed
* Go build tags changed from old-style (`+build`) to new Go 1.17+ style (`go:build`). [#2]
### Removed
* Removed support for capabilities v1 and v2. [#1]
* Removed init function so programs that use this package start faster. [#6]
* Removed `CAP_LAST_CAP` (use [LastCap] instead). [#6]
<!-- Doc links. -->
[Apply]: https://pkg.go.dev/github.com/moby/sys/capability#Capabilities.Apply
[LastCap]: https://pkg.go.dev/github.com/moby/sys/capability#LastCap
[List]: https://pkg.go.dev/github.com/moby/sys/capability#List
[ListKnown]: https://pkg.go.dev/github.com/moby/sys/capability#ListKnown
[ListSupported]: https://pkg.go.dev/github.com/moby/sys/capability#ListSupported
<!-- Minor releases. -->
[0.3.0]: https://github.com/moby/sys/releases/tag/capability%2Fv0.3.0
[0.2.0]: https://github.com/moby/sys/releases/tag/capability%2Fv0.2.0
[0.1.1]: https://github.com/kolyshkin/capability/compare/v0.1.0...v0.1.1
[0.1.0]: https://github.com/kolyshkin/capability/compare/42c35b4376354fd5...v0.1.0
<!-- PRs in 0.1.x releases. -->
[#1]: https://github.com/kolyshkin/capability/pull/1
[#2]: https://github.com/kolyshkin/capability/pull/2
[#3]: https://github.com/kolyshkin/capability/pull/3
[#4]: https://github.com/kolyshkin/capability/pull/4
[#6]: https://github.com/kolyshkin/capability/pull/6
[#7]: https://github.com/kolyshkin/capability/pull/7
[#8]: https://github.com/kolyshkin/capability/pull/8
[#9]: https://github.com/kolyshkin/capability/pull/9
[#10]: https://github.com/kolyshkin/capability/pull/10
[#11]: https://github.com/kolyshkin/capability/pull/11
[#12]: https://github.com/kolyshkin/capability/pull/12

View File

@@ -1,3 +1,4 @@
Copyright 2023 The Capability Authors.
Copyright 2013 Suryandaru Triandana <syndtr@gmail.com>
All rights reserved.

View File

@@ -0,0 +1,13 @@
This is a fork of (apparently no longer maintained)
https://github.com/syndtr/gocapability package. It provides basic primitives to
work with [Linux capabilities][capabilities(7)].
For changes, see [CHANGELOG.md](./CHANGELOG.md).
[![Go Reference](https://pkg.go.dev/badge/github.com/moby/sys/capability/capability.svg)](https://pkg.go.dev/github.com/moby/sys/capability)
## Alternatives
* https://pkg.go.dev/kernel.org/pub/linux/libs/security/libcap/cap
[capabilities(7)]: https://man7.org/linux/man-pages/man7/capabilities.7.html

View File

@@ -1,8 +1,9 @@
// Copyright (c) 2013, Suryandaru Triandana <syndtr@gmail.com>
// Copyright 2023 The Capability Authors.
// Copyright 2013 Suryandaru Triandana <syndtr@gmail.com>
// All rights reserved.
//
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Package capability provides utilities for manipulating POSIX capabilities.
package capability
@@ -60,26 +61,27 @@ type Capabilities interface {
Apply(kind CapType) error
}
// NewPid initializes a new Capabilities object for given pid when
// NewPid initializes a new [Capabilities] object for given pid when
// it is nonzero, or for the current process if pid is 0.
//
// Deprecated: Replace with NewPid2. For example, replace:
// Deprecated: Replace with [NewPid2] followed by [Capabilities.Load].
// For example, replace:
//
// c, err := NewPid(0)
// if err != nil {
// return err
// }
// c, err := NewPid(0)
// if err != nil {
// return err
// }
//
// with:
//
// c, err := NewPid2(0)
// if err != nil {
// return err
// }
// err = c.Load()
// if err != nil {
// return err
// }
// c, err := NewPid2(0)
// if err != nil {
// return err
// }
// err = c.Load()
// if err != nil {
// return err
// }
func NewPid(pid int) (Capabilities, error) {
c, err := newPid(pid)
if err != nil {
@@ -89,33 +91,34 @@ func NewPid(pid int) (Capabilities, error) {
return c, err
}
// NewPid2 initializes a new Capabilities object for given pid when
// it is nonzero, or for the current process if pid is 0. This
// NewPid2 initializes a new [Capabilities] object for given pid when
// it is nonzero, or for the current process if pid is 0. This
// does not load the process's current capabilities; to do that you
// must call Load explicitly.
// must call [Capabilities.Load] explicitly.
func NewPid2(pid int) (Capabilities, error) {
return newPid(pid)
}
// NewFile initializes a new Capabilities object for given file path.
//
// Deprecated: Replace with NewFile2. For example, replace:
// Deprecated: Replace with [NewFile2] followed by [Capabilities.Load].
// For example, replace:
//
// c, err := NewFile(path)
// if err != nil {
// return err
// }
// c, err := NewFile(path)
// if err != nil {
// return err
// }
//
// with:
//
// c, err := NewFile2(path)
// if err != nil {
// return err
// }
// err = c.Load()
// if err != nil {
// return err
// }
// c, err := NewFile2(path)
// if err != nil {
// return err
// }
// err = c.Load()
// if err != nil {
// return err
// }
func NewFile(path string) (Capabilities, error) {
c, err := newFile(path)
if err != nil {
@@ -125,9 +128,17 @@ func NewFile(path string) (Capabilities, error) {
return c, err
}
// NewFile2 creates a new initialized Capabilities object for given
// file path. This does not load the process's current capabilities;
// to do that you must call Load explicitly.
// NewFile2 creates a new initialized [Capabilities] object for given
// file path. This does not load the process's current capabilities;
// to do that you must call [Capabilities.Load] explicitly.
func NewFile2(path string) (Capabilities, error) {
return newFile(path)
}
// LastCap returns highest valid capability of the running kernel,
// or an error if it can not be obtained.
//
// See also: [ListSupported].
func LastCap() (Cap, error) {
return lastCap()
}

View File

@@ -1,8 +1,9 @@
// Copyright (c) 2013, Suryandaru Triandana <syndtr@gmail.com>
// Copyright 2023 The Capability Authors.
// Copyright 2013 Suryandaru Triandana <syndtr@gmail.com>
// All rights reserved.
//
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package capability
@@ -12,62 +13,53 @@ import (
"fmt"
"io"
"os"
"strconv"
"strings"
"sync"
"syscall"
)
var errUnknownVers = errors.New("unknown capability version")
const (
linuxCapVer1 = 0x19980330
linuxCapVer2 = 0x20071026
linuxCapVer1 = 0x19980330 // No longer supported.
linuxCapVer2 = 0x20071026 // No longer supported.
linuxCapVer3 = 0x20080522
)
var (
capVers uint32
capLastCap Cap
)
func init() {
var hdr capHeader
capget(&hdr, nil)
capVers = hdr.version
if initLastCap() == nil {
CAP_LAST_CAP = capLastCap
if capLastCap > 31 {
capUpperMask = (uint32(1) << (uint(capLastCap) - 31)) - 1
} else {
capUpperMask = 0
}
}
}
func initLastCap() error {
if capLastCap != 0 {
return nil
}
var lastCap = sync.OnceValues(func() (Cap, error) {
f, err := os.Open("/proc/sys/kernel/cap_last_cap")
if err != nil {
return err
return 0, err
}
defer f.Close()
var b []byte = make([]byte, 11)
_, err = f.Read(b)
buf := make([]byte, 11)
l, err := f.Read(buf)
f.Close()
if err != nil {
return err
return 0, err
}
buf = buf[:l]
fmt.Sscanf(string(b), "%d", &capLastCap)
last, err := strconv.Atoi(strings.TrimSpace(string(buf)))
if err != nil {
return 0, err
}
return Cap(last), nil
})
return nil
func capUpperMask() uint32 {
last, err := lastCap()
if err != nil || last < 32 {
return 0
}
return (uint32(1) << (uint(last) - 31)) - 1
}
func mkStringCap(c Capabilities, which CapType) (ret string) {
for i, first := Cap(0), true; i <= CAP_LAST_CAP; i++ {
last, err := lastCap()
if err != nil {
return ""
}
for i, first := Cap(0), true; i <= last; i++ {
if !c.Get(which, i) {
continue
}
@@ -98,138 +90,33 @@ func mkString(c Capabilities, max CapType) (ret string) {
return
}
func newPid(pid int) (c Capabilities, err error) {
switch capVers {
case linuxCapVer1:
p := new(capsV1)
p.hdr.version = capVers
p.hdr.pid = int32(pid)
c = p
case linuxCapVer2, linuxCapVer3:
p := new(capsV3)
p.hdr.version = capVers
p.hdr.pid = int32(pid)
c = p
default:
err = errUnknownVers
var capVersion = sync.OnceValues(func() (uint32, error) {
var hdr capHeader
err := capget(&hdr, nil)
return hdr.version, err
})
func newPid(pid int) (c Capabilities, retErr error) {
ver, err := capVersion()
if err != nil {
retErr = fmt.Errorf("unable to get capability version from the kernel: %w", err)
return
}
return
}
type capsV1 struct {
hdr capHeader
data capData
}
func (c *capsV1) Get(which CapType, what Cap) bool {
if what > 32 {
return false
}
switch which {
case EFFECTIVE:
return (1<<uint(what))&c.data.effective != 0
case PERMITTED:
return (1<<uint(what))&c.data.permitted != 0
case INHERITABLE:
return (1<<uint(what))&c.data.inheritable != 0
}
return false
}
func (c *capsV1) getData(which CapType) (ret uint32) {
switch which {
case EFFECTIVE:
ret = c.data.effective
case PERMITTED:
ret = c.data.permitted
case INHERITABLE:
ret = c.data.inheritable
switch ver {
case linuxCapVer1, linuxCapVer2:
retErr = errors.New("old/unsupported capability version (kernel older than 2.6.26?)")
default:
// Either linuxCapVer3, or an unknown/future version (such as v4).
// In the latter case, we fall back to v3 as the latest version known
// to this package, as kernel should be backward-compatible to v3.
p := new(capsV3)
p.hdr.version = linuxCapVer3
p.hdr.pid = int32(pid)
c = p
}
return
}
func (c *capsV1) Empty(which CapType) bool {
return c.getData(which) == 0
}
func (c *capsV1) Full(which CapType) bool {
return (c.getData(which) & 0x7fffffff) == 0x7fffffff
}
func (c *capsV1) Set(which CapType, caps ...Cap) {
for _, what := range caps {
if what > 32 {
continue
}
if which&EFFECTIVE != 0 {
c.data.effective |= 1 << uint(what)
}
if which&PERMITTED != 0 {
c.data.permitted |= 1 << uint(what)
}
if which&INHERITABLE != 0 {
c.data.inheritable |= 1 << uint(what)
}
}
}
func (c *capsV1) Unset(which CapType, caps ...Cap) {
for _, what := range caps {
if what > 32 {
continue
}
if which&EFFECTIVE != 0 {
c.data.effective &= ^(1 << uint(what))
}
if which&PERMITTED != 0 {
c.data.permitted &= ^(1 << uint(what))
}
if which&INHERITABLE != 0 {
c.data.inheritable &= ^(1 << uint(what))
}
}
}
func (c *capsV1) Fill(kind CapType) {
if kind&CAPS == CAPS {
c.data.effective = 0x7fffffff
c.data.permitted = 0x7fffffff
c.data.inheritable = 0
}
}
func (c *capsV1) Clear(kind CapType) {
if kind&CAPS == CAPS {
c.data.effective = 0
c.data.permitted = 0
c.data.inheritable = 0
}
}
func (c *capsV1) StringCap(which CapType) (ret string) {
return mkStringCap(c, which)
}
func (c *capsV1) String() (ret string) {
return mkString(c, BOUNDING)
}
func (c *capsV1) Load() (err error) {
return capget(&c.hdr, &c.data)
}
func (c *capsV1) Apply(kind CapType) error {
if kind&CAPS == CAPS {
return capset(&c.hdr, &c.data)
}
return nil
}
type capsV3 struct {
hdr capHeader
data [2]capData
@@ -292,7 +179,8 @@ func (c *capsV3) Full(which CapType) bool {
if (data[0] & 0xffffffff) != 0xffffffff {
return false
}
return (data[1] & capUpperMask) == capUpperMask
mask := capUpperMask()
return (data[1] & mask) == mask
}
func (c *capsV3) Set(which CapType, caps ...Cap) {
@@ -401,15 +289,12 @@ func (c *capsV3) Load() (err error) {
return
}
var status_path string
if c.hdr.pid == 0 {
status_path = fmt.Sprintf("/proc/self/status")
} else {
status_path = fmt.Sprintf("/proc/%d/status", c.hdr.pid)
path := "/proc/self/status"
if c.hdr.pid != 0 {
path = fmt.Sprintf("/proc/%d/status", c.hdr.pid)
}
f, err := os.Open(status_path)
f, err := os.Open(path)
if err != nil {
return
}
@@ -423,11 +308,17 @@ func (c *capsV3) Load() (err error) {
break
}
if strings.HasPrefix(line, "CapB") {
fmt.Sscanf(line[4:], "nd: %08x%08x", &c.bounds[1], &c.bounds[0])
_, err = fmt.Sscanf(line[4:], "nd: %08x%08x", &c.bounds[1], &c.bounds[0])
if err != nil {
break
}
continue
}
if strings.HasPrefix(line, "CapA") {
fmt.Sscanf(line[4:], "mb: %08x%08x", &c.ambient[1], &c.ambient[0])
_, err = fmt.Sscanf(line[4:], "mb: %08x%08x", &c.ambient[1], &c.ambient[0])
if err != nil {
break
}
continue
}
}
@@ -437,6 +328,10 @@ func (c *capsV3) Load() (err error) {
}
func (c *capsV3) Apply(kind CapType) (err error) {
last, err := LastCap()
if err != nil {
return err
}
if kind&BOUNDS == BOUNDS {
var data [2]capData
err = capget(&c.hdr, &data[0])
@@ -444,14 +339,14 @@ func (c *capsV3) Apply(kind CapType) (err error) {
return
}
if (1<<uint(CAP_SETPCAP))&data[0].effective != 0 {
for i := Cap(0); i <= CAP_LAST_CAP; i++ {
for i := Cap(0); i <= last; i++ {
if c.Get(BOUNDING, i) {
continue
}
err = prctl(syscall.PR_CAPBSET_DROP, uintptr(i), 0, 0, 0)
if err != nil {
// Ignore EINVAL since the capability may not be supported in this system.
if errno, ok := err.(syscall.Errno); ok && errno == syscall.EINVAL {
if err == syscall.EINVAL { //nolint:errorlint // Errors from syscall are bare.
err = nil
continue
}
@@ -469,16 +364,19 @@ func (c *capsV3) Apply(kind CapType) (err error) {
}
if kind&AMBS == AMBS {
for i := Cap(0); i <= CAP_LAST_CAP; i++ {
for i := Cap(0); i <= last; i++ {
action := pr_CAP_AMBIENT_LOWER
if c.Get(AMBIENT, i) {
action = pr_CAP_AMBIENT_RAISE
}
err := prctl(pr_CAP_AMBIENT, action, uintptr(i), 0, 0)
// Ignore EINVAL as not supported on kernels before 4.3
if errno, ok := err.(syscall.Errno); ok && errno == syscall.EINVAL {
err = nil
continue
err = prctl(pr_CAP_AMBIENT, action, uintptr(i), 0, 0)
if err != nil {
// Ignore EINVAL as not supported on kernels before 4.3
if err == syscall.EINVAL { //nolint:errorlint // Errors from syscall are bare.
err = nil
continue
}
return
}
}
}
@@ -547,7 +445,8 @@ func (c *capsFile) Full(which CapType) bool {
if (data[0] & 0xffffffff) != 0xffffffff {
return false
}
return (data[1] & capUpperMask) == capUpperMask
mask := capUpperMask()
return (data[1] & mask) == mask
}
func (c *capsFile) Set(which CapType, caps ...Cap) {

View File

@@ -0,0 +1,26 @@
// Copyright 2023 The Capability Authors.
// Copyright 2013 Suryandaru Triandana <syndtr@gmail.com>
// All rights reserved.
//
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//go:build !linux
package capability
import "errors"
var errNotSup = errors.New("not supported")
func newPid(_ int) (Capabilities, error) {
return nil, errNotSup
}
func newFile(_ string) (Capabilities, error) {
return nil, errNotSup
}
func lastCap() (Cap, error) {
return -1, errNotSup
}

View File

@@ -1,11 +1,14 @@
// Copyright (c) 2013, Suryandaru Triandana <syndtr@gmail.com>
// Copyright 2024 The Capability Authors.
// Copyright 2013 Suryandaru Triandana <syndtr@gmail.com>
// All rights reserved.
//
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package capability
import "slices"
type CapType uint
func (c CapType) String() string {
@@ -301,9 +304,27 @@ const (
CAP_CHECKPOINT_RESTORE = Cap(40)
)
var (
// Highest valid capability of the running kernel.
CAP_LAST_CAP = Cap(63)
// List returns the list of all capabilities known to the package.
//
// Deprecated: use [ListKnown] or [ListSupported] instead.
func List() []Cap {
return ListKnown()
}
capUpperMask = ^uint32(0)
)
// ListKnown returns the list of all capabilities known to the package.
func ListKnown() []Cap {
return list()
}
// ListSupported retuns the list of all capabilities known to the package,
// except those that are not supported by the currently running Linux kernel.
func ListSupported() ([]Cap, error) {
last, err := LastCap()
if err != nil {
return nil, err
}
return slices.DeleteFunc(list(), func(c Cap) bool {
// Remove caps not supported by the kernel.
return c > last
}), nil
}

View File

@@ -1,4 +1,4 @@
// generated file; DO NOT EDIT - use go generate in directory with source
// Code generated by go generate; DO NOT EDIT.
package capability
@@ -90,8 +90,7 @@ func (c Cap) String() string {
return "unknown"
}
// List returns list of all supported capabilities
func List() []Cap {
func list() []Cap {
return []Cap{
CAP_CHOWN,
CAP_DAC_OVERRIDE,

View File

@@ -1,8 +1,9 @@
// Copyright (c) 2013, Suryandaru Triandana <syndtr@gmail.com>
// Copyright 2024 The Capability Authors.
// Copyright 2013 Suryandaru Triandana <syndtr@gmail.com>
// All rights reserved.
//
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package capability
@@ -79,9 +80,7 @@ type vfscapData struct {
version int8
}
var (
_vfsXattrName *byte
)
var _vfsXattrName *byte
func init() {
_vfsXattrName, _ = syscall.BytePtrFromString(vfsXattrName)

View File

@@ -1,19 +0,0 @@
// Copyright (c) 2013, Suryandaru Triandana <syndtr@gmail.com>
// All rights reserved.
//
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
// +build !linux
package capability
import "errors"
func newPid(pid int) (Capabilities, error) {
return nil, errors.New("not supported")
}
func newFile(path string) (Capabilities, error) {
return nil, errors.New("not supported")
}

View File

@@ -204,7 +204,7 @@ github.com/docker/cli/cli/connhelper/ssh
# github.com/docker/distribution v2.8.2+incompatible
## explicit
github.com/docker/distribution/registry/client/auth/challenge
# github.com/docker/docker v27.0.3+incompatible
# github.com/docker/docker v27.2.0+incompatible
## explicit
github.com/docker/docker/api
github.com/docker/docker/api/types
@@ -511,6 +511,9 @@ github.com/moby/locker
## explicit; go 1.19
github.com/moby/patternmatcher
github.com/moby/patternmatcher/ignorefile
# github.com/moby/sys/capability v0.3.0
## explicit; go 1.21
github.com/moby/sys/capability
# github.com/moby/sys/signal v0.7.0
## explicit; go 1.16
github.com/moby/sys/signal
@@ -622,9 +625,6 @@ github.com/stretchr/testify/require
# github.com/surma/gocpio v1.0.2-0.20160926205914-fcb68777e7dc
## explicit
github.com/surma/gocpio
# github.com/syndtr/gocapability v0.0.0-20200815063812-42c35b437635
## explicit
github.com/syndtr/gocapability/capability
# github.com/tonistiigi/fsutil v0.0.0-20240424095704-91a3fc46842c
## explicit; go 1.20
github.com/tonistiigi/fsutil

View File

@@ -2,7 +2,7 @@ kernel:
image: linuxkit/kernel:6.6.13
cmdline: "console=ttyS0"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
onboot:
- name: dhcpcd

View File

@@ -3,7 +3,7 @@ kernel:
image: linuxkit/kernel:6.6.13
cmdline: "console=ttyS0"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435

View File

@@ -3,7 +3,7 @@ kernel:
image: linuxkit/kernel:6.6.13
cmdline: "console=ttyS0"
init:
- linuxkit/init:9eabfb21879d4a3c9534f0c4e9a764919423e8a5
- linuxkit/init:e120ea2a30d906bd1ee1874973d6e4b1403b5ca3
- linuxkit/runc:6062483d748609d505f2bcde4e52ee64a3329f5f
- linuxkit/containerd:39301e7312f13eedf19bd5d5551af7b37001d435

Some files were not shown because too many files have changed in this diff Show More