Update moby vendoring

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
This commit is contained in:
Justin Cormack 2018-01-17 14:32:26 +00:00
parent 683df33c9a
commit 99fd71dc7f
6 changed files with 148 additions and 7 deletions

View File

@ -26,7 +26,7 @@ github.com/moby/datakit 97b3d230535397a813323902c23751e176481a86
github.com/moby/hyperkit a12cd7250bcd8d689078e3e42ae4a7cf6a0cbaf3
# When updating also:
# curl -fsSL -o src/cmd/linuxkit/build.go https://raw.githubusercontent.com/moby/tool/«hash»/cmd/moby/build.go
github.com/moby/tool 57b6e2ab947104d47fd60e5af0e34d0edeb9421c
github.com/moby/tool caca03c097eafd7aa1e011de59ded3604e1b318f
github.com/moby/vpnkit 0e4293bb1058598c4b0a406ed171f52573ef414c
github.com/opencontainers/go-digest 21dfd564fd89c944783d00d069f33e3e7123c448
github.com/opencontainers/image-spec v1.0.0

View File

@ -0,0 +1,60 @@
## About
This directory contains a collection of scripts used to build and manage this
repository. If there are any issues regarding the intention of a particular
script (or even part of a certain script), please reach out to us.
It may help us either refine our current scripts, or add on new ones
that are appropriate for a given use case.
## DinD (dind.sh)
DinD is a wrapper script which allows Docker to be run inside a Docker
container. DinD requires the container to
be run with privileged mode enabled.
## Generate Authors (generate-authors.sh)
Generates AUTHORS; a file with all the names and corresponding emails of
individual contributors. AUTHORS can be found in the home directory of
this repository.
## Make
There are two make files, each with different extensions. Neither are supposed
to be called directly; only invoke `make`. Both scripts run inside a Docker
container.
### make.ps1
- The Windows native build script that uses PowerShell semantics; it is limited
unlike `hack\make.sh` since it does not provide support for the full set of
operations provided by the Linux counterpart, `make.sh`. However, `make.ps1`
does provide support for local Windows development and Windows to Windows CI.
More information is found within `make.ps1` by the author, @jhowardmsft
### make.sh
- Referenced via `make test` when running tests on a local machine,
or directly referenced when running tests inside a Docker development container.
- When running on a local machine, `make test` to run all tests found in
`test`, `test-unit`, `test-integration`, and `test-docker-py` on
your local machine. The default timeout is set in `make.sh` to 60 minutes
(`${TIMEOUT:=60m}`), since it currently takes up to an hour to run
all of the tests.
- When running inside a Docker development container, `hack/make.sh` does
not have a single target that runs all the tests. You need to provide a
single command line with multiple targets that performs the same thing.
An example referenced from [Run targets inside a development container](https://docs.docker.com/opensource/project/test-and-docs/#run-targets-inside-a-development-container): `root@5f8630b873fe:/go/src/github.com/moby/moby# hack/make.sh dynbinary binary cross test-unit test-integration test-docker-py`
- For more information related to testing outside the scope of this README,
refer to
[Run tests and test documentation](https://docs.docker.com/opensource/project/test-and-docs/)
## Release (release.sh)
Releases any bundles built by `make` on a public AWS S3 bucket.
For information regarding configuration, please view `release.sh`.
## Vendor (vendor.sh)
A shell script that is a wrapper around Vndr. For information on how to use
this, please refer to [vndr's README](https://github.com/LK4D4/vndr/blob/master/README.md)

View File

@ -0,0 +1,69 @@
# Integration Testing on Swarm
IT on Swarm allows you to execute integration test in parallel across a Docker Swarm cluster
## Architecture
### Master service
- Works as a funker caller
- Calls a worker funker (`-worker-service`) with a chunk of `-check.f` filter strings (passed as a file via `-input` flag, typically `/mnt/input`)
### Worker service
- Works as a funker callee
- Executes an equivalent of `TESTFLAGS=-check.f TestFoo|TestBar|TestBaz ... make test-integration-cli` using the bind-mounted API socket (`docker.sock`)
### Client
- Controls master and workers via `docker stack`
- No need to have a local daemon
Typically, the master and workers are supposed to be running on a cloud environment,
while the client is supposed to be running on a laptop, e.g. Docker for Mac/Windows.
## Requirement
- Docker daemon 1.13 or later
- Private registry for distributed execution with multiple nodes
## Usage
### Step 1: Prepare images
$ make build-integration-cli-on-swarm
Following environment variables are known to work in this step:
- `BUILDFLAGS`
- `DOCKER_INCREMENTAL_BINARY`
Note: during the transition into Moby Project, you might need to create a symbolic link `$GOPATH/src/github.com/docker/docker` to `$GOPATH/src/github.com/moby/moby`.
### Step 2: Execute tests
$ ./hack/integration-cli-on-swarm/integration-cli-on-swarm -replicas 40 -push-worker-image YOUR_REGISTRY.EXAMPLE.COM/integration-cli-worker:latest
Following environment variables are known to work in this step:
- `DOCKER_GRAPHDRIVER`
- `DOCKER_EXPERIMENTAL`
#### Flags
Basic flags:
- `-replicas N`: the number of worker service replicas. i.e. degree of parallelism.
- `-chunks N`: the number of chunks. By default, `chunks` == `replicas`.
- `-push-worker-image REGISTRY/IMAGE:TAG`: push the worker image to the registry. Note that if you have only single node and hence you do not need a private registry, you do not need to specify `-push-worker-image`.
Experimental flags for mitigating makespan nonuniformity:
- `-shuffle`: Shuffle the test filter strings
Flags for debugging IT on Swarm itself:
- `-rand-seed N`: the random seed. This flag is useful for deterministic replaying. By default(0), the timestamp is used.
- `-filters-file FILE`: the file contains `-check.f` strings. By default, the file is automatically generated.
- `-dry-run`: skip the actual workload
- `keep-executor`: do not auto-remove executor containers, which is used for running privileged programs on Swarm

View File

@ -0,0 +1,2 @@
# dependencies specific to worker (i.e. github.com/docker/docker/...) are not vendored here
github.com/bfirsh/funker-go eaa0a2e06f30e72c9a0b7f858951e581e26ef773

View File

@ -70,6 +70,7 @@ type Image struct {
// ImageConfig is the configuration part of Image, it is the subset
// which is valid in a "org.mobyproject.config" label on an image.
// Everything except Runtime and ref is used to build the OCI spec
type ImageConfig struct {
Capabilities *[]string `yaml:"capabilities,omitempty" json:"capabilities,omitempty"`
Ambient *[]string `yaml:"ambient,omitempty" json:"ambient,omitempty"`
@ -100,7 +101,9 @@ type ImageConfig struct {
Rlimits *[]string `yaml:"rlimits,omitempty" json:"rlimits,omitempty"`
UIDMappings *[]specs.LinuxIDMapping `yaml:"uidMappings,omitempty" json:"uidMappings,omitempty"`
GIDMappings *[]specs.LinuxIDMapping `yaml:"gidMappings,omitempty" json:"gidMappings,omitempty"`
Runtime *Runtime `yaml:"runtime,omitempty" json:"runtime,omitempty"`
Annotations *map[string]string `yaml:"annotations,omitempty" json:"annotations,omitempty"`
Runtime *Runtime `yaml:"runtime,omitempty" json:"runtime,omitempty"`
ref *reference.Spec
}
@ -112,6 +115,7 @@ type Runtime struct {
Mkdir *[]string `yaml:"mkdir,omitempty" json:"mkdir,omitempty"`
Interfaces *[]Interface `yaml:"interfaces,omitempty,omitempty" json:"interfaces,omitempty"`
BindNS Namespaces `yaml:"bindNS,omitempty" json:"bindNS,omitempty"`
Namespace *string `yaml:"namespace,omitempty" json:"namespace,omitempty"`
}
// Namespaces is the type for configuring paths to bind namespaces
@ -585,6 +589,7 @@ func assignRuntime(v1, v2 *Runtime) Runtime {
runtimeMounts := assignBinds(v1.Mounts, v2.Mounts)
runtimeMkdir := assignStrings(v1.Mkdir, v2.Mkdir)
runtimeInterfaces := assignRuntimeInterfaceArray(v1.Interfaces, v2.Interfaces)
runtimeNamespace := assignString(v1.Namespace, v2.Namespace)
runtime := Runtime{
Cgroups: &runtimeCgroups,
Mounts: &runtimeMounts,
@ -599,6 +604,7 @@ func assignRuntime(v1, v2 *Runtime) Runtime {
User: assignStringPtr(v1.BindNS.User, v2.BindNS.User),
Uts: assignStringPtr(v1.BindNS.Uts, v2.BindNS.Uts),
},
Namespace: &runtimeNamespace,
}
return runtime
}
@ -1025,6 +1031,7 @@ func ConfigInspectToOCI(yaml *Image, inspect types.ImageInspect, idMap map[strin
oci.Hostname = assignStringEmpty(label.Hostname, yaml.Hostname)
oci.Mounts = mountList
oci.Annotations = assignMaps(label.Annotations, yaml.Annotations)
resources := assignResources(label.Resources, yaml.Resources)

View File

@ -49,6 +49,10 @@ var schema = string(`
"type": "array",
"items": {"type": "string"}
},
"mapstring": {
"type": "object",
"additionalProperties": {"type": "string"}
},
"mount": {
"type": "object",
"additionalProperties": false,
@ -244,7 +248,8 @@ var schema = string(`
"mounts": {"$ref": "#/definitions/mounts"},
"mkdir": {"$ref": "#/definitions/strings"},
"interfaces": {"$ref": "#/definitions/interfaces"},
"bindNS": {"$ref": "#/definitions/namespaces"}
"bindNS": {"$ref": "#/definitions/namespaces"},
"namespace": {"type": "string"}
}
},
"image": {
@ -282,13 +287,11 @@ var schema = string(`
"rootfsPropagation": {"type": "string"},
"cgroupsPath": {"type": "string"},
"resources": {"$ref": "#/definitions/resources"},
"sysctl": {
"type": "array",
"items": { "$ref": "#/definitions/strings" }
},
"sysctl": { "$ref": "#/definitions/mapstring" },
"rlimits": { "$ref": "#/definitions/strings" },
"uidMappings": { "$ref": "#/definitions/idmappings" },
"gidMappings": { "$ref": "#/definitions/idmappings" },
"annotations": { "$ref": "#/definitions/mapstring" },
"runtime": {"$ref": "#/definitions/runtime"}
}
},