Commit Graph

58 Commits

Author SHA1 Message Date
Riyaz Faizullabhoy
4c810a27f9 Merge lint.sh into compile.sh, remove unused alpine-build-go
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2017-01-23 13:25:44 -08:00
Justin Cormack
74d3bce70e Use buildmode pie for Go code
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-17 16:03:50 +00:00
Justin Cormack
6a315fab59 Clean up go-compile
- remove unused `--docker` option
- neater output for stages of check, build

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-15 19:43:03 +00:00
Justin Cormack
5cfb149637 Update to golang:1.7-alpine3.5 images
Now there is an Alpine 3.5 variant of the Go 1.7 images, use this.

fix #972

Note updated the containers/binfmt image as this will be converted
to go-compile shortly, at which point alpine-build-go can be removed.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-14 09:41:12 +00:00
Justin Cormack
b184b3f0e1 Compile Go code with docker run not docker build
Go code is really fast to compile so we do not really need to use the
cache features of `docker build`. So make a compile container instead.
This can also output a build context and Dockerfile if you want to do
a build.

For reference, an uncached `docker build` of our Go code takes about
7s, a cached one 1.2s, and this takes 1.7s, so the best case is a little
worse, but we save a lot of images, and the worst case is better.

This is mainly designed to make the nested builds for containerd
containers simpler too. Will add a variant for the C code as well.

Also add `-static` to the flags so we always make static executables,
which was omitted previously.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-13 15:49:51 +00:00
Justin Cormack
81a7af5299 Update Go build packages to Alpine 3.5
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-01-05 15:58:16 +00:00
Riyaz Faizullabhoy
c8e7a6dd43 Move linting to Go build
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2016-12-21 11:26:33 -08:00
Riyaz Faizullabhoy
b73d2a29a7 Add lint docker image for golang linting and testing to individual package subdirs
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2016-12-20 15:00:08 -08:00
Riyaz Faizullabhoy
51088ee491 Add golint, make linting top-level
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2016-12-19 16:13:02 -08:00
Riyaz Faizullabhoy
27cb33f7b9 Add gofmt, govet
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2016-12-19 15:17:37 -08:00
Simon Ferquel
5c096a83c1 Remove vsudd output from the console.
vsudd is quite verbose, and we are confident enough about its stability.
This will clean Pinata logs as well indirectly

Signed-off-by: Simon Ferquel <simon.ferquel@docker.com>
2016-12-16 15:53:20 +01:00
Justin Cormack
c6163f7ffd Merge pull request #762 from simonferquel/vsudd_dontretry_on_dockerd
[vsudd] Don't retry when dockerd is not running
2016-12-02 08:23:23 -08:00
Justin Cormack
1ecdeeed44 Update to Go 1.7.4
Security update.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-12-02 09:51:36 +00:00
Riyaz Faizullabhoy
4011d4842a Use digests instead of tags where possible
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2016-12-01 09:39:02 -08:00
Simon Ferquel
641669cafb Redirect vsudd stdout/stderr to console
Signed-off-by: Simon Ferquel <simon.ferquel@docker.com>
2016-11-30 10:31:11 +01:00
Simon Ferquel
a12a833b20 [vsudd] Don't retry when dockerd is not running
Signed-off-by: Simon Ferquel <simon.ferquel@docker.com>
2016-11-30 10:31:11 +01:00
Justin Cormack
32020f01f6 always wait for vsudd pidfile
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-11-16 13:07:32 +00:00
Justin Cormack
49511ba5ba vsudd needs bootmisc too for /run symlink
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-11-15 00:19:21 +00:00
Justin Cormack
9d81e2efef Always start vsudd before syslog
If we are using vsudd to forward syslog to the host, as on osx,
we need to start it before syslog starts, and make sure it has
created its socket.

Add a pidfile to vsudd to make startup more reliable.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-11-14 17:08:37 +00:00
Justin Cormack
d435d5115b Update to Go 1.7.3
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-10-20 13:17:16 +01:00
Justin Cormack
db0376cb88 add git to Go build package
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-10-16 13:49:28 +01:00
Justin Cormack
cfbdc7d067 do not remove build containers
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-10-11 16:12:40 +01:00
Justin Cormack
0ac6beff82 Do not create latest tags for base images
As we never use these, do not create them.

Also update images where applicable.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-10-10 14:55:48 +01:00
Justin Cormack
fa19d3259b Use build container id rather than tagging builds
This means that multiple builds will not conflict, so we can
remove the lock from the CI. Also quieter when no errors.

Some still left to do, only done the ones used in build and CI
initially. Some of the others will be cleaned up anyway later.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-09-29 20:21:17 +01:00
Justin Cormack
206dfe0950 simplify vendor tar
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-09-12 16:36:01 +01:00
Justin Cormack
43ab6520cf update to latest alpine base images
Signed-off-by: Justin Cormack <justin@specialbusservice.com>
2016-09-12 16:02:07 +01:00
Justin Cormack
5c634808df simplify tar for vendor directory
Signed-off-by: Justin Cormack <justin@specialbusservice.com>
2016-09-12 16:02:07 +01:00
Justin Cormack
276157a769 Move all output to the directories it will be in eventually
Signed-off-by: Justin Cormack <justin@specialbusservice.com>
2016-09-12 16:02:07 +01:00
Justin Cormack
d00ed9ce6e run with no network
Signed-off-by: Justin Cormack <justin@specialbusservice.com>
2016-09-12 15:59:19 +01:00
Justin Cormack
b4d9394f50 clean up vsudd build like proxy
Signed-off-by: Justin Cormack <justin@specialbusservice.com>
2016-09-12 15:59:19 +01:00
Justin Cormack
7a3dda4995 remove unused include, it is all in the Go code
Signed-off-by: Justin Cormack <justin@specialbusservice.com>
2016-09-12 15:59:19 +01:00
Justin Cormack
28afc72a97 move vendor directory directly under packages
Signed-off-by: Justin Cormack <justin@specialbusservice.com>
2016-09-12 15:59:19 +01:00
Justin Cormack
089c592873 Include the Go binary in the hash of the go build base
As this is not installed via apk it was not being included in the
checksum.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-09-09 15:46:23 +01:00
Justin Cormack
d4b5fc07c9 Closer to reproducible builds
Make an alpine base image for C and Go builds.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-09-06 13:18:11 +01:00
Justin Cormack
30d4cf9675 more specific golang alpine base package
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-09-05 15:05:52 +01:00
Justin Cormack
5d933baed1 Use platform name not vmbus
See #301

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-07-20 14:44:43 +01:00
Justin Cormack
67b2a00853 Add conditional services based on mobyplatform
Replaces https://github.com/docker/moby/pull/282

Use mobyplatform=xxx to specify platform in boot command

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-07-18 12:39:14 +01:00
Ian Campbell
43e2030e31 vsudd: Use RFC5425 scheme for syslog forwarding
This means an ASCII MSG-LEN and a space, rather than a binary message length.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2016-07-04 14:49:46 +01:00
Ian Campbell
ad7f4c74b2 vsudd: Log over syslog
This means that with the previous patches normal vsudd logging will be logged
on the console. The exceptional case of error logging during syslog forwarding
established in the previous patch remains in place.

Prior to this the vsudd.log was actually in /run/vsudd.log and not in /var/log/
(exported to the host) as expected. Prior to c5940b3479 ("Bind the original
/var/log onto /run/log") the log was simply shadowed under the fuse mount over
/var/log.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2016-07-04 14:47:02 +01:00
Ian Campbell
7dd7b0c0da vsudd: Forward syslog from /var/run/syslog.vsock to vsock 514
This is mac only (for now) and will not actually do anything until syslogd is
told to forward to /var/run/syslog.vsock.

syslog uses a SOCK_DGRAM connection to /var/run/syslog.vsock, however vsock
today is SOCK_STREAM only, so we need to "packetise" the stream. Do so by
writing the datagram length as a (little-endian) uint32 before the data itself.
This is slightly modelled after rfc6587 (syslog over TCP) but simplified by
using a 4-byte binary value rather than ASCII digits.

Arrange for vsudd to start before the logger so it is ready and waiting.

Note that the code in vsyslog.go needs to be rather careful about its own
logging, in particular logging forwarding failures over syslog seems likely to
make things worse. Instead this file logs to the console when errors occur,
this will be captured by the logging of the hyperkit VM console.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2016-07-04 14:47:02 +01:00
Ian Campbell
b61451047d vsudd: Make incoming socket forwarding more generic
Rather than hardcoding a single vsock<->docker.sock mapping allow arbitrary
incoming connection forwarding between vsocks and unix domain sockets.

The intention was to subsequently extend this further to support arbitrary
forwarding of outgoing connections too and to use that to forward the syslog
socket out to a vsock.

This turned out not to be a good plan, partly since the syslog socket needs to
be SOCK_DATAGRAM but vsocks only does SOCK_STREAM today (meaning we need some
additional framing here) and partly because handling syslog forwarding in
common code makes error logging in the common code somewhat trickier (logging
syslog errors over syslog).

So instead syslog will be handled as a special case in a following patch.
However some vestiges of the original plan remain, e.g. the inForwards name and
the net field in the forwards which could be unixgram but currently is only
supporting unix(stream).

In principal this patch could be dropped, but it adds some flexibility which
might be useful in the future.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2016-07-04 14:47:02 +01:00
Ian Campbell
98174da08f vsudd: Correctly format go Error's with %s
%#v is not correct (includes opaque pointers etc).

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2016-07-04 14:47:02 +01:00
Ian Campbell
5165a6c795 Add || true to the docker rmi invocations in the clean target
Linux xargs calls the command with no arguments if it gets no inputs, which
`docker rmi` complains about. It provides -r / --no-run-if-empty to prevent
this but unfortunately this isn't supported on OSX.

Ignore errors from `docker rmi` so that `make clean` will keep going and clean
up later stuff.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2016-06-08 13:59:47 +01:00
Rolf Neugebauer
26b2dc0dd7 vsudd: use vendor'ed hvsock/vsock modules
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2016-05-19 13:54:48 +01:00
Justin Cormack
a740d022ff use build-base not alpine-sdk as smaller
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-05-17 10:00:58 -05:00
Ian Campbell
b4722c2533 vsudd: Reduce logging
Previously the logs for a single connection would be something like:

2016/05/04 12:44:41 171 Accepted connection on fd 5 from 00000002.00010006
2016/05/04 12:44:41 171 Connected to docker &{{0xc82008a5b0}}
2016/05/04 12:44:44 171 copying from vsock to docker:  4465 bytes done
2016/05/04 12:44:44 171 copying from docker to vsock:  1324 bytes done
2016/05/04 12:44:44 171 Done. read: 4465 written: 1324
2016/05/04 12:44:44 171 Closing docker &{{0xc82008a5b0}}
2016/05/04 12:44:44 171 Closing vsock &{0xc820086840}

The "Connected" and "Closing" lines are not useful now that it is debugged and
working well. The "copying..." lines are redundant with the "Done" line. Reduce
to just:

2016/05/04 14:00:41 4 Accepted connection on fd 10 from 00000002.00010003
2016/05/04 14:00:41 4 Done. read: 90 written: 145

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2016-05-04 15:01:47 +01:00
Rolf Neugebauer
d93ef30ce7 9pudc: remove binary, which probably was accidentally checked in
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2016-04-21 14:02:13 +01:00
Rolf Neugebauer
f279329456 vsudd: start vsudd on Windows against a GUID
Using 23a432c2-537a-4291-bcb5-d62504644739 as the GUID (randomly generated).
The Windows host side will uses this as service ID, once written.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2016-04-21 14:02:13 +01:00
Rolf Neugebauer
89e8ac2612 vsudd: Add support for Hyper-V sockets
- If a GUID is passed in use AF_HYPERV else AFVSOCK

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2016-04-21 14:02:13 +01:00
Rolf Neugebauer
0ef55d5da1 vsudd: restructure to make it easier to add support for HyperV sockets
- Move VSOCK listen code into separate functions
- Remove VSOCK soecific code from handleOne()
- Turn 'port' argument into a string

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2016-04-21 14:02:13 +01:00