Commit Graph

466 Commits

Author SHA1 Message Date
Justin Cormack
f349515701 Add some more docker tests, including running image
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-07-08 16:54:18 +01:00
Justin Cormack
e6f4364f07 Merge pull request #251 from dsheets/transfused-export-suitability
transfused: distinguish export requests from mount requests
2016-07-08 16:30:04 +01:00
Justin Cormack
ede5a0ea6f Initial test hooks
Beginning of a proper test suite, using qemu. Test just runs docker
for now, will add further integration tests.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-07-08 15:44:04 +01:00
Justin Cormack
95f6ca9f70 Merge pull request #250 from djs55/vsock-ulimit
proxy-vsockd: bump the max number of file descriptors
2016-07-06 14:52:01 +01:00
Ian Campbell
3547fb21cf Merge pull request #241 from ijc25/logging-over-vsock
mac: Redirect syslog over vsock
2016-07-05 17:54:30 +01:00
Ian Campbell
125e3de03b Merge pull request #207 from ijc25/ntp-from-dhcp
[Post Dockercon] Use $ntpsrv from DHCP to configure chrony
2016-07-05 14:23:49 +01:00
David Sheets
fbcba85f0e transfused: distinguish export requests from mount requests
We distinguish export suitability requests from bind mount suitability
requests in the transfuse control protocol. This distinction allows us to
permit both bind mounts of empty directories and export mounts onto empty
directories. Addresses docker/pinata#4213.

Signed-off-by: David Sheets <dsheets@docker.com>
2016-07-04 18:53:05 -07:00
David Scott
e32a516406 proxy-vsockd: bump the max number of file descriptors
Signed-off-by: David Scott <dave.scott@docker.com>
2016-07-04 21:49:17 +01:00
Justin Cormack
18b933e81b use cat not sysctl for reading sysctl values
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-07-04 15:47:46 +01:00
Ian Campbell
b6e7ccd046 vsudd: Use RFC5425 scheme for syslog forwarding
This means an ASCII MSG-LEN and a space, rather than a binary message length.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2016-07-04 14:49:46 +01:00
Ian Campbell
74a94948dc vsudd: Log over syslog
This means that with the previous patches normal vsudd logging will be logged
on the console. The exceptional case of error logging during syslog forwarding
established in the previous patch remains in place.

Prior to this the vsudd.log was actually in /run/vsudd.log and not in /var/log/
(exported to the host) as expected. Prior to c5940b3479 ("Bind the original
/var/log onto /run/log") the log was simply shadowed under the fuse mount over
/var/log.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2016-07-04 14:47:02 +01:00
Ian Campbell
010cc0a55f vsudd: Forward syslog from /var/run/syslog.vsock to vsock 514
This is mac only (for now) and will not actually do anything until syslogd is
told to forward to /var/run/syslog.vsock.

syslog uses a SOCK_DGRAM connection to /var/run/syslog.vsock, however vsock
today is SOCK_STREAM only, so we need to "packetise" the stream. Do so by
writing the datagram length as a (little-endian) uint32 before the data itself.
This is slightly modelled after rfc6587 (syslog over TCP) but simplified by
using a 4-byte binary value rather than ASCII digits.

Arrange for vsudd to start before the logger so it is ready and waiting.

Note that the code in vsyslog.go needs to be rather careful about its own
logging, in particular logging forwarding failures over syslog seems likely to
make things worse. Instead this file logs to the console when errors occur,
this will be captured by the logging of the hyperkit VM console.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2016-07-04 14:47:02 +01:00
Ian Campbell
a49ba65802 vsudd: Make incoming socket forwarding more generic
Rather than hardcoding a single vsock<->docker.sock mapping allow arbitrary
incoming connection forwarding between vsocks and unix domain sockets.

The intention was to subsequently extend this further to support arbitrary
forwarding of outgoing connections too and to use that to forward the syslog
socket out to a vsock.

This turned out not to be a good plan, partly since the syslog socket needs to
be SOCK_DATAGRAM but vsocks only does SOCK_STREAM today (meaning we need some
additional framing here) and partly because handling syslog forwarding in
common code makes error logging in the common code somewhat trickier (logging
syslog errors over syslog).

So instead syslog will be handled as a special case in a following patch.
However some vestiges of the original plan remain, e.g. the inForwards name and
the net field in the forwards which could be unixgram but currently is only
supporting unix(stream).

In principal this patch could be dropped, but it adds some flexibility which
might be useful in the future.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2016-07-04 14:47:02 +01:00
Ian Campbell
aedb514983 Revendor virtsock go library, for vsock.Dial
$ gvt delete github.com/rneugeba/virtsock/go
$ gvt fetch --no-recurse https://github.com/rneugeba/virtsock/go

virtsock.git:

$ git log --oneline 74097e05a883e89c70e6a27b342672c7fe6c846b..650ef8224a0c06b4b20e9bee1600dbf677c8176d -- go/
0e2f0a8 vsock: Implement vsock.Dial
712714a vsock: include the errno when C.bind_sockaddr_vm
03725fe go: make errors public

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2016-07-04 14:47:02 +01:00
Ian Campbell
c00630fe34 vsudd: Correctly format go Error's with %s
%#v is not correct (includes opaque pointers etc).

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2016-07-04 14:47:02 +01:00
Justin Cormack
265ed43f21 Increase ulimits as high as possible
Setting unlimited improves performance.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-07-04 14:37:28 +01:00
Justin Cormack
514bbcf717 Merge pull request #237 from ijc25/diagnostics
Fixup diagnostics logging
2016-07-04 11:13:42 +01:00
Justin Cormack
994f2c965a Merge pull request #243 from djs55/docker.1.12-rc3
Update docker to 1.12.0-rc3
2016-07-04 11:10:38 +01:00
David Scott
269d61a849 Update docker to 1.12.0-rc3
Signed-off-by: David Scott <dave.scott@docker.com>
2016-07-04 11:04:29 +01:00
David Sheets
9d3e8ad516 docker init: remove fixed host export mounts
Signed-off-by: David Sheets <dsheets@docker.com>
(cherry picked from commit 760840ac69)
2016-07-02 12:27:09 -07:00
Ian Campbell
110bdd831b diagnostics: Only log connection failure if not EAFNOTSUPPORT
EAFNOTSUPPORT likely means we are running on a hypervisor which doesn't support
that particular socket type.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2016-07-01 15:35:27 +01:00
Ian Campbell
dde4ab70ba diagnostics: Log over syslog
Instead of spamming the console.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2016-07-01 15:35:27 +01:00
Ian Campbell
a329bbc8e0 diagnostics: Correct error formatting
They should be formatted as %s not %#v (which prints all sorts of opaque
pointers).

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2016-07-01 15:35:26 +01:00
Ian Campbell
f2b432053c diagnostics: run "gofmt -w" over go code
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2016-07-01 15:35:26 +01:00
Rolf Neugebauer
72fd755473 diag: improve diagnostics
- try several wget's on well known IP addresses to check IP
  connectivity (ping in hostnet mode is not good enough)
- try DNS lookup on a well known server
- add service-port-opener logs

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2016-06-22 11:38:51 +01:00
Rolf Neugebauer
082c10d997 win: ensure that the KVP daemon starts early
The Windows start script polls the VM for it's IP address before
initiating things like CIFS mounts.  Getting the IP config is
done via the KVP daemon.  Starting it earlier reduce the start up
delay, in particular if the VM is part of a swarm.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2016-06-21 14:19:59 +01:00
Justin Cormack
2d67ea5cb8 Merge pull request #218 from ijc25/mkdir-p
Use "mkdir -p /run/log" in docker initfile
2016-06-17 14:03:14 +01:00
Ian Campbell
013599c78f Use "mkdir -p /run/log" in docker initfile
For the "no error if existing" rather than "make parents" affect. Otherwise:

moby:~# service docker restart
 * Stopping docker
 * Starting Docker ...
mkdir: can't create directory '/run/log': File exists
 [ ok ]

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2016-06-17 13:57:33 +01:00
Ian Campbell
9cc8e4a106 Avoid brace expansion which is a bashism.
Building on Debian where /bin/sh is dash:

make[1]: Leaving directory '/home/ijc/development/Docker/moby.git/alpine/packages/docker/docker.git'
mkdir -p bin
cp docker.git/bundles/latest/binary-daemon/{dockerd,docker-runc} bin/
cp: cannot stat 'docker.git/bundles/latest/binary-daemon/{dockerd,docker-runc}': No such file or directory
Makefile:9: recipe for target 'bin' failed

Pasting the construct into a file and running checkbashsisms:
possible bashism in foo line 2 (brace expansion):
cp docker.git/bundles/latest/binary-daemon/{dockerd,docker-runc} bin/

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2016-06-17 13:54:14 +01:00
Justin Cormack
d5f0876276 Update to 1.12.0-rc2
Currently the repo points to just the cherrypicks, not yet final tag

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-06-17 13:24:44 +01:00
David Scott
45bb435b51 proxy: add interactive mode with -i
This allows the proxy to be run easily from a terminal or other script
without requiring fd 3 to be open and writable.

Signed-off-by: David Scott <dave.scott@docker.com>
2016-06-17 11:11:10 +01:00
Justin Cormack
a6819a65ba fix #212 do not let chronyd listen on a port for commands
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-06-15 15:28:05 +01:00
Justin Cormack
b70b51af18 reverse on Windows check for diagnostics
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-06-15 10:42:21 +01:00
Justin Cormack
baaa38bc4c default to Docker experimental; later we will need to do multiple builds
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-06-15 10:28:26 +01:00
Justin Cormack
4544bbac58 update docker to 1.12.0-rc1
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-06-15 10:19:31 +01:00
David Scott
abe75e4e50 tap-vsockd: make sure the service daemonizes properly
This tells start-stop-daemon to run tap-vsockd as a daemon, and tells
tap-vsockd not to daemonize itself. This seems to work more reliably
than when tap-vsockd self-daemonizes.

Signed-off-by: David Scott <dave.scott@docker.com>
2016-06-14 15:16:56 +01:00
Justin Cormack
759ad55f03 Update to docker 1.12-dev plus proxy patches, close to what will be rc1
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-06-14 14:13:36 +01:00
Ian Campbell
3196e668e6 Use $ntpsrv from DHCP to configure chrony
The existing `ntp=gateway` cmdline support overrides this, but is largely
redundant now.

A DHCP derived server will not be given the "trust" directive in the
configuration file, since it is harder to argue that we know we should trust a
server which arrives this way since it depends on the context in which Moby is
running. It's not clear to me that it makes much difference anyway.

This only uses the first server in $ntpsrv, since Piñata will only provide one
and supporting more makes the sed quite a bit more complex.

If `ntp=gateway` is given then we now use $router instead of parsing IP output.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2016-06-14 14:06:53 +01:00
Justin Cormack
7d73182c45 Merge pull request #196 from rneugeba/win-proxy-fix
proxy: fix handling of CloseRead/CloseWrite for TCP port proxy
2016-06-13 14:22:30 +01:00
Justin Cormack
b99befab17 Merge pull request #201 from ijc25/bind-aside-original-var-log
Bind aside original var log
2016-06-13 14:22:22 +01:00
Justin Cormack
96ab15bc4b Revert "start automount after syslog as it tries to restart it"
This reverts commit 8276becaf1.
2016-06-13 14:18:05 +01:00
Ian Campbell
c5940b3479 Bind the original /var/log onto /run/log
So that it can be inspected while debugging.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2016-06-13 12:07:26 +01:00
Ian Campbell
a2ba682878 Use /proc/mounts in docker init script
Switch one instance of /etc/mtab.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2016-06-13 12:07:26 +01:00
Justin Cormack
8276becaf1 start automount after syslog as it tries to restart it
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-06-13 11:51:03 +01:00
Justin Cormack
41f39f7efe missing /proc.mounts
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-06-13 10:40:10 +01:00
Rolf Neugebauer
ebf7c5fede proxy: fix handling of CloseRead/CloseWrite for TCP port proxy
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2016-06-10 16:05:14 +01:00
Justin Cormack
2efda11b65 remove transfused dependency from hostsettings by putting log mounts in docker startup
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-06-10 11:26:32 +01:00
David Sheets
a84e4ecfdd transfused: use find rather than rm+glob to clean up /tmp
This avoids annoying error messages about . and ..

Signed-off-by: David Sheets <dsheets@docker.com>
2016-06-09 11:27:59 +01:00
David Sheets
4090ccf9d0 transfused: replace space indentation in rc script with tabs for consistency
Signed-off-by: David Sheets <dsheets@docker.com>
2016-06-08 15:16:43 +01:00
David Sheets
c92ff1d36c transfused: remove anything in /tmp before starting
Signed-off-by: David Sheets <dsheets@docker.com>
2016-06-08 15:16:43 +01:00