Commit Graph

858 Commits

Author SHA1 Message Date
Robb Kistler
b2b585ac99 Allow docker engine to be pulled from s3 bucket
Use aws cli to pull docker engine binaries from s3 bucket.  Conditional and
happens if DOCKER_BIN_URL begins with 's3://'

Example usage:
```
  export AWS_ACCESS_KEY_ID=key
  export AWS_SECRET_ACCESS_KEY=secret
  make DOCKER_BIN_URL=s3://path-to-engine-binaries
```
Signed-off-by: Robb Kistler <robb.kistler@docker.com>

Signed-off-by: Robb Kistler <robb.kistler@docker.com>
2017-02-23 20:32:00 -08:00
French Ben
a118ce4f05 Added quotes for curling
Signed-off-by: French Ben <frenchben@docker.com>
2017-02-23 15:15:28 -08:00
Justin Cormack
457c173849 Update Docker to 17.03.0-ce-rc1
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-02-20 10:52:41 +00:00
Justin Cormack
fe64819020 Update to Go 1.8
This includes most things except the containerd and runc builds
which I will do next time I update them.

Note that all golang:alpine images now for 1.8 are ALpine 3.5 based.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-02-17 16:31:19 +00:00
Justin Cormack
bece837dec Merge pull request #1190 from justincormack/iburst
Stop using dhcp ntp and fix the options used on OSX
2017-02-16 08:50:31 +00:00
Robb Kistler
56059aba3b Re-run mdev -s if /dev/sda1 isn't populated
Fixes #1139

Signed-off-by: Robb Kistler <robb.kistler@docker.com>
2017-02-15 17:47:38 -08:00
Justin Cormack
084291469c Stop using dhcp ntp and fix the options used on OSX
The options added in de6e7f9cbe
were not applied for ntp configured servers.

As the address is always the same, stop using dhcp and just hard code this.
This should make sync work how it used to, ie quite well.

Also trust the GCP ntp server, as it is local and provided by infrastructure.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-02-15 17:54:41 +00:00
Justin Cormack
97b5b71224 Use ntp pool not single server for AWS
Some of the servers are off by a lot, eg 7s in one case I saw.

SHould help with #1186

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-02-14 13:20:18 +00:00
Justin Cormack
0a43565d9a Merge pull request #1124 from riyazdf/apk-audit
Use apk audit to check system binaries
2017-02-13 11:41:14 +00:00
Riyaz Faizullabhoy
1d548cfa1a Merge pull request #1173 from nathanleclaire/azure_bump_vhd
Bump Azure digest and version for 1.13.1
2017-02-09 16:24:02 -08:00
Nathan LeClaire
578f40ac47 Bump Azure digest and version for 1.13.1
Signed-off-by: Nathan LeClaire <nathan.leclaire@gmail.com>
2017-02-09 15:59:25 -08:00
Nathan LeClaire
a067dfe6d3 Merge pull request #1169 from nathanleclaire/system_con_fix
Fix system container bug
2017-02-09 15:38:16 -08:00
Justin Cormack
155676ee21 Do not try to run metrics proxy when not configured
Failure test case was not correct; printed a (harmless) error message
that was confusing.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-02-09 13:58:03 +00:00
Nathan LeClaire
7459a4bfad Fix system container bug
Signed-off-by: Nathan LeClaire <nathan.leclaire@gmail.com>
2017-02-08 17:02:14 -08:00
Justin Cormack
5256eb0263 Update Docker to 1.13.1
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-02-08 09:34:57 +00:00
Riyaz Faizullabhoy
83b9afce02 Merge pull request #1144 from riyazdf/1.13.1-rc2-bump
1.13.1-rc2 docker bump
2017-02-06 21:50:35 -08:00
Justin Cormack
42b75bdeb7 Merge pull request #1137 from justincormack/prometheus
Forward Prometheus metrics to host if enabled
2017-02-04 09:19:05 +01:00
Riyaz Faizullabhoy
a29b6c1391 Revert to non-development containerd and runc until nats port is sorted out
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-02-02 11:57:28 -08:00
Riyaz Faizullabhoy
4fe7784ad1 1.13.1-rc2 docker bump
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-02-02 09:33:46 -08:00
David Gageot
e2a4c42457 Wait for /dev/to be populated by mdev -s
Fixes #1139

Signed-off-by: David Gageot <david@gageot.net>
2017-02-02 16:20:47 +01:00
Justin Cormack
7327d374ec Forward Prometheus metrics to host if enabled
Set your daemon.json to something like:
```
{
	"experimental": true,
	"metrics-addr": "0.0.0.0:4999"
}
```

and your metrics will be at `localhost:4999/metrics`

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-02-02 11:51:24 +00:00
David Sheets
2e39ad92e2 transfused perfstat: fix bug with running multiple perfstats in a row (#1129)
After freeing the head of the perfstat block list, we must set it to NULL
to indicate that new blocks must be allocated. Previously, we risked
segfaults by trying to use a freed list.

Signed-off-by: David Sheets <dsheets@docker.com>
2017-02-01 20:24:21 +00:00
Justin Cormack
ad528b4bea Merge pull request #1123 from dsheets/transfused-perfstat
transfused perfstat
2017-02-01 11:04:14 +00:00
Riyaz Faizullabhoy
2cdefa184e Use apk audit to check system binaries
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-01-31 11:33:11 -08:00
David Sheets
dc7069db1c transfused: add perfstat machinery
This adds two new control channel requests, START_PERFSTAT and
STOP_PERFSTAT, that begin and end performance statistics collection
respectively. START_PERFSTAT takes the name of the mountpoint on which
to enable statistics collection and returns the current monotonic time.
STOP_PERFSTAT takes the name of the mountpoint on which to disable
statistics collection and returns the current monotonic time and the
message timings that have been collected.

Performance statistics are collected in a linked list of blocks. pthread
locks are used for synchronization.

Signed-off-by: David Sheets <dsheets@docker.com>
2017-01-31 18:07:31 +00:00
Justin Cormack
a6416a1214 Merge pull request #1121 from justincormack/fsck-y
Try to fsck harder
2017-01-31 17:50:16 +00:00
Riyaz Faizullabhoy
56899b318d reformat tags before digests
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-01-31 09:33:06 -08:00
Justin Cormack
64bcf1adce Try to fsck harder
This will potentially leave stuff in lost+found but may be able to
recover from more filesystem corruption.

See #736

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-31 12:42:37 +00:00
Justin Cormack
ddb41952ef Update Go to 1.7.5 and remove unnecessary linker flag
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-30 14:10:41 +00:00
David Sheets
85f401ac79 transfused: keep a linked list of open connections
Signed-off-by: David Sheets <dsheets@docker.com>
2017-01-30 10:47:46 +00:00
David Sheets
b10d61a94b transfused: rename some types to end in _t
Signed-off-by: David Sheets <dsheets@docker.com>
2017-01-30 10:37:06 +00:00
Justin Cormack
af88a66bc7 Unmount database after copying
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-28 17:09:01 +00:00
Justin Cormack
9e18a0ae4c Fix mountpoint for mac
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-28 14:45:44 +00:00
Justin Cormack
1674534757 Merge pull request #1108 from justincormack/osx-mountpoint
Mount osx 9p db on /mnt
2017-01-28 12:27:20 +00:00
Justin Cormack
3b6d1ae4f2 Mount osx 9p db on /mnt
Otherwise interferes with transfused startup.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-28 12:03:02 +00:00
Justin Cormack
25c357b375 Update to Docker 1.13.1-rc1
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-28 10:16:15 +00:00
Justin Cormack
fa4f1aaedc Start diagnostics server earlier, before Docker
Allows it to be used to see what the boot state is.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-25 15:17:55 +00:00
Justin Cormack
3d03a8ae9b Merge pull request #1090 from nathanleclaire/chown_azure_proper
Chown azure proper
2017-01-25 00:20:39 +00:00
Nathan LeClaire
9bd4680e09 Modify init script to create+chown proper directory
Signed-off-by: Nathan LeClaire <nathan.leclaire@gmail.com>
2017-01-24 15:53:19 -08:00
Justin Cormack
b53e83d695 Merge pull request #1086 from justincormack/new-containerd
Shift to development track containerd
2017-01-24 17:50:45 +00:00
Justin Cormack
89e98eefa0 Shift to development track containerd
Move to the development track of `containerd` not the legacy 0.2
branch. The commands have changed a bit.

This does increase the image size as we are bundling the Docker
copy and our copy, and the new one is larger as it is growing features.
Hopefully Docker will shrink eventually. Also we may replace `ctr`
with a library.

Fix #1029

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-24 15:39:44 +00:00
Justin Cormack
bb2da98169 Merge pull request #1080 from justincormack/split-database
Split config database setup by platform
2017-01-24 12:08:33 +00:00
Riyaz Faizullabhoy
4c810a27f9 Merge lint.sh into compile.sh, remove unused alpine-build-go
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-01-23 13:25:44 -08:00
Justin Cormack
321c180c60 Split config database setup by platform
Add basic database setup for AWS, GCP although these are not yet used
by the setup code but will be useful later.

Currently each is gated by `mobyplatform` but this can be removed once we
construct Moby per platform, and once these are containerised so they
are not run from `openrc`.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-23 18:08:26 +00:00
Justin Cormack
4278e0d78e Merge pull request #1079 from justincormack/swapdiskname
Clean up swap partition mounting logic
2017-01-23 17:04:12 +00:00
Justin Cormack
aaa2e30a75 Merge pull request #1077 from djs55/benchmark-ipc
tap-vsockd: add buffering
2017-01-23 16:22:12 +00:00
Justin Cormack
df2c4818ed Clean up swap partition mounting logic
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-23 16:20:10 +00:00
David Scott
370a43de83 tap-vsockd: allocate payload separately for better alignment
Previously we allocated `sizeof(struct ring) + size`. This patch
allocates `sizeof(struct ring)` and then `size` for the payload
separately. Hopefully the payload will be better aligned.

Signed-off-by: David Scott <dave.scott@docker.com>
2017-01-23 14:44:34 +00:00
David Scott
3fc0d994b5 tap-vsockd: add buffering
This patch adds a simple ring buffer implementation and uses it to buffer
the reads and writes to/from the AF_HYPERV socket and tap file descriptor.

This removes the need to perform small reads and writes for the per-packet
headers and allows a read on the Hyper-V socket to block at the same time
as a write to the tap device (and vice-versa)

The configuration in the init.d script is:

- a max message size (individual read or write) of 8192. Experimentally
  this seems to be the largest completely reliable size across the Windows
  versions we can support. Messages of length 16384 sometimes fail.
- a buffer size of 256KiB in each direction.

Single stream TCP throughput as measured by iperf increases modestly, by
another 100Mbit/sec.

Signed-off-by: David Scott <dave.scott@docker.com>
2017-01-23 12:17:52 +00:00
Justin Cormack
02bd52b5c1 Fix up direct database accesses after move to /Database
In #1057 removed the 9p paths from the database mount
but that broke some users accessing them directly. Fix these.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-21 14:32:34 +00:00