Commit Graph

860 Commits

Author SHA1 Message Date
David Scott
55a2becfb4 iptables: only open host ports if native/port-forwarding=true in the db
Signed-off-by: David Scott <dave.scott@docker.com>
2016-07-12 10:26:42 +01:00
David Scott
d0876fb05e iptables: add a TODO
Signed-off-by: David Scott <dave.scott@docker.com>
2016-07-12 10:26:42 +01:00
David Scott
12fbe114f6 iptables: remove the pid file after sending SIGTERM
Signed-off-by: David Scott <dave.scott@docker.com>
2016-07-12 10:26:42 +01:00
David Scott
3c6ad76461 proxy: add a -no-local-ip option
docker itself seems to bind to the port globally inside Moby, so we
get an EADDRINUSE if we try to do it too.

Signed-off-by: David Scott <dave.scott@docker.com>
2016-07-12 10:26:42 +01:00
David Scott
3183d9c72a iptables: get the kill arguments the right way round
Signed-off-by: David Scott <dave.scott@docker.com>
2016-07-12 10:26:42 +01:00
David Scott
a294b0f9b0 iptables: close all the fds we inherit from docker
Signed-off-by: David Scott <dave.scott@docker.com>
2016-07-12 10:26:42 +01:00
David Scott
dfb97863c9 /etc/init.d/docker: prepend /usr/local/sbin to the $PATH
This is where the iptables wrapper lives.

Signed-off-by: David Scott <dave.scott@docker.com>
2016-07-12 10:26:42 +01:00
David Scott
80b234dd3e Add primitive iptables wrapper which can set up port forwards
Signed-off-by: David Scott <dave.scott@docker.com>
2016-07-12 10:26:42 +01:00
Ian Campbell
0c471bdc09 Simple build system for ocaml-based iptables
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2016-07-12 10:26:39 +01:00
Justin Cormack
bf1a0e0011 Merge pull request #265 from dsheets/transfused-mknod-reg
transfused: add mknod reg file event actuation message
2016-07-12 10:18:29 +01:00
David Sheets
20fc67092a transfused: add mknod reg file event actuation message
Signed-off-by: David Sheets <dsheets@docker.com>
2016-07-11 16:26:00 -07:00
Justin Cormack
99ed8b733e Merge pull request #264 from dsheets/sysctl-inotify-watches
sysctl: set a large fs.inotify.max_user_watches limit
2016-07-11 22:38:47 +01:00
David Sheets
026bfeb17e sysctl: set a large fs.inotify.max_user_watches limit
Needed by Ruby guard and Dropbox. See https://forums.docker.com/t/running-guard-with-docker-compose-fails-due-to-inotify-limit/17096

Signed-off-by: David Sheets <dsheets@docker.com>
2016-07-11 14:37:52 -07:00
Justin Cormack
10d81ecdff Merge pull request #262 from dsheets/linux-module-unload
linux: allow kernel module unloading
2016-07-11 19:53:21 +01:00
David Sheets
bd33169d49 linux: allow kernel module unloading
The ability to unload kernel modules helps with rapid development of kernel
modules or Moby-integrated functionality. It has no negative side effects
as far as I am aware.

Signed-off-by: David Sheets <dsheets@docker.com>
2016-07-11 11:48:46 -07:00
Justin Cormack
50ec41a232 Merge pull request #260 from justincormack/userns
Fix user namespace support
2016-07-11 14:58:55 +01:00
Justin Cormack
7fb90b6af5 Fix user namespace support
fix #153

For now, just create the default remap user, rather than trying
to fix the command emulation. The existing code in docker is not
ideal, as it is GNU specific, try to find a better option for
1.13.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-07-11 14:47:23 +01:00
Justin Cormack
0c9603708c remove syslog fix, as now upstream
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-07-11 14:31:25 +01:00
Justin Cormack
8a84cab8c2 Merge pull request #258 from dsheets/remove-slash-Mac
Remove /Mac in docker init and transfused init
2016-07-11 12:29:06 +01:00
David Sheets
8473eb56f2 Remove /Mac in docker init and transfused init
Replace /Mac with /host_docker_app and replace driverDir with just driver.

Signed-off-by: David Sheets <dsheets@docker.com>
2016-07-08 14:36:26 -07:00
Justin Cormack
15e34f9717 Merge pull request #257 from justincormack/sysctl
Add a custom sysctl file
2016-07-08 18:18:29 +01:00
Justin Cormack
51563eb677 Add a custom sysctl file
This increases resource limits that have been generally requested.

See #232

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-07-08 18:16:05 +01:00
Justin Cormack
3e894eb470 Merge pull request #256 from justincormack/tests
Add some more docker tests, including running image
2016-07-08 16:57:34 +01:00
Justin Cormack
09e60eee16 Add some more docker tests, including running image
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-07-08 16:54:18 +01:00
Justin Cormack
839b58e19c Merge pull request #251 from dsheets/transfused-export-suitability
transfused: distinguish export requests from mount requests
2016-07-08 16:30:04 +01:00
Justin Cormack
94d603971c Merge pull request #255 from justincormack/e1000
add e1000 drivers, allows qemu default setup to find an ethernet, and…
2016-07-08 16:28:24 +01:00
Justin Cormack
20ef44b187 Merge pull request #254 from rneugeba/patches-up
linux: properly update kernel patches to v4.4.14
2016-07-08 16:28:16 +01:00
Justin Cormack
c8b5ffcef0 add e1000 drivers, allows qemu default setup to find an ethernet, and used frequently in emulated environments
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-07-08 16:26:16 +01:00
Rolf Neugebauer
f829fef36b linux: properly update kernel patches to v4.4.14
When we updated from 4.4.10 to 4.4.14 we copied the patches over.
This changeset properly updates the patches so that they apply cleanly.
0039-VSOCK-do-not-disconnect-socket-when-peer-has-shutdow.patch was
removed as it made it into 4.4.14 already.

Compile tested only

For reference the patches were generated from:
https://github.com/rneugeba/linux-stable/tree/v4.4.14-moby

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2016-07-08 16:10:51 +01:00
Justin Cormack
92ef135cd3 Merge pull request #253 from justincormack/test
Initial test hooks
2016-07-08 15:47:15 +01:00
Justin Cormack
44d68b268a Initial test hooks
Beginning of a proper test suite, using qemu. Test just runs docker
for now, will add further integration tests.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-07-08 15:44:04 +01:00
Justin Cormack
b3cfed22f4 Merge pull request #250 from djs55/vsock-ulimit
proxy-vsockd: bump the max number of file descriptors
2016-07-06 14:52:01 +01:00
Ian Campbell
120f9d916c Merge pull request #241 from ijc25/logging-over-vsock
mac: Redirect syslog over vsock
2016-07-05 17:54:30 +01:00
Ian Campbell
08d2109c29 Merge pull request #207 from ijc25/ntp-from-dhcp
[Post Dockercon] Use $ntpsrv from DHCP to configure chrony
2016-07-05 14:23:49 +01:00
David Sheets
c75eb93d1c transfused: distinguish export requests from mount requests
We distinguish export suitability requests from bind mount suitability
requests in the transfuse control protocol. This distinction allows us to
permit both bind mounts of empty directories and export mounts onto empty
directories. Addresses docker/pinata#4213.

Signed-off-by: David Sheets <dsheets@docker.com>
2016-07-04 18:53:05 -07:00
Justin Cormack
5ca2fa0006 Merge pull request #249 from justincormack/nomodules
Updated kernel config
2016-07-05 01:19:38 +01:00
David Scott
32ee88c347 proxy-vsockd: bump the max number of file descriptors
Signed-off-by: David Scott <dave.scott@docker.com>
2016-07-04 21:49:17 +01:00
Justin Cormack
880966311c Updated kernel config
Remove one module, Xen ones not removeable. Make oldconfig.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-07-04 17:03:10 +01:00
Justin Cormack
91c7341401 Merge pull request #246 from ijc25/populate-lib-modules
Populate /lib/modules/`uname -r`
2016-07-04 15:48:53 +01:00
Justin Cormack
17628dbbb9 Merge pull request #247 from justincormack/ulimit
use cat not sysctl for reading sysctl values
2016-07-04 15:48:20 +01:00
Justin Cormack
6e25acbe30 use cat not sysctl for reading sysctl values
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-07-04 15:47:46 +01:00
Ian Campbell
7378c6e282 Populate /lib/modules/uname -r
This is less to do with installing modules (which we generally don't expect to
use in Moby) but to populate /lib/modules/`uname -r`/modules.builtin which
turns:

    moby:~# modprobe ip_vs
    modprobe: FATAL: Module ip_vs not found in directory /lib/modules/4.4.14-moby
    moby:~# modprobe nf_nat
    modprobe: FATAL: Module nf_nat not found in directory /lib/modules/4.4.14-moby
    moby:~#

into:

    moby:~# modprobe ip_vs
    moby:~# modprobe nf_nat
    moby:~#

which reduces the amount noise in the logs, e.g. in docker.log:

time="2016-07-04T11:21:58Z" level=warning msg="Running modprobe nf_nat failed with message: `modprobe: WARNING: Module nf_nat not found in directory /lib/modules/4.4.14-moby`, error: exit status 1"

A fair number of these appear in the logs.

This also stops various tools logging about /lib/modules/`uname -r` not
existing (there was one in the boot log until recently I think)

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2016-07-04 15:44:48 +01:00
Ian Campbell
43e2030e31 vsudd: Use RFC5425 scheme for syslog forwarding
This means an ASCII MSG-LEN and a space, rather than a binary message length.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2016-07-04 14:49:46 +01:00
Ian Campbell
916d920bfb Direct syslog to log to /var/run/syslog.vsock on mac
Avoid doing this on non-Docker-for-{Mac,Win} editions (which don't run vsudd)
by checking for vsudd.pid and avoid doing it on Docker-for-Win (for now) by
checking for /sys/bus/vmbus (as /etc/init.d/vsudd does too).

Ideally we would just check for /var/run/syslog.vsock but this may not have
arrived yet (typically vsudd is now started immediately prior to syslog and it
forks via start-stop-daemon and thus before it creates the socket). Since
syslogd will reopen as needed we don't want to delay boot either here or in the
vsudd initscript to await the arrival of the socket.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2016-07-04 14:47:02 +01:00
Ian Campbell
ad7f4c74b2 vsudd: Log over syslog
This means that with the previous patches normal vsudd logging will be logged
on the console. The exceptional case of error logging during syslog forwarding
established in the previous patch remains in place.

Prior to this the vsudd.log was actually in /run/vsudd.log and not in /var/log/
(exported to the host) as expected. Prior to c5940b3479 ("Bind the original
/var/log onto /run/log") the log was simply shadowed under the fuse mount over
/var/log.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2016-07-04 14:47:02 +01:00
Ian Campbell
7dd7b0c0da vsudd: Forward syslog from /var/run/syslog.vsock to vsock 514
This is mac only (for now) and will not actually do anything until syslogd is
told to forward to /var/run/syslog.vsock.

syslog uses a SOCK_DGRAM connection to /var/run/syslog.vsock, however vsock
today is SOCK_STREAM only, so we need to "packetise" the stream. Do so by
writing the datagram length as a (little-endian) uint32 before the data itself.
This is slightly modelled after rfc6587 (syslog over TCP) but simplified by
using a 4-byte binary value rather than ASCII digits.

Arrange for vsudd to start before the logger so it is ready and waiting.

Note that the code in vsyslog.go needs to be rather careful about its own
logging, in particular logging forwarding failures over syslog seems likely to
make things worse. Instead this file logs to the console when errors occur,
this will be captured by the logging of the hyperkit VM console.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2016-07-04 14:47:02 +01:00
Ian Campbell
b61451047d vsudd: Make incoming socket forwarding more generic
Rather than hardcoding a single vsock<->docker.sock mapping allow arbitrary
incoming connection forwarding between vsocks and unix domain sockets.

The intention was to subsequently extend this further to support arbitrary
forwarding of outgoing connections too and to use that to forward the syslog
socket out to a vsock.

This turned out not to be a good plan, partly since the syslog socket needs to
be SOCK_DATAGRAM but vsocks only does SOCK_STREAM today (meaning we need some
additional framing here) and partly because handling syslog forwarding in
common code makes error logging in the common code somewhat trickier (logging
syslog errors over syslog).

So instead syslog will be handled as a special case in a following patch.
However some vestiges of the original plan remain, e.g. the inForwards name and
the net field in the forwards which could be unixgram but currently is only
supporting unix(stream).

In principal this patch could be dropped, but it adds some flexibility which
might be useful in the future.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2016-07-04 14:47:02 +01:00
Ian Campbell
f70ff0aeac Revendor virtsock go library, for vsock.Dial
$ gvt delete github.com/rneugeba/virtsock/go
$ gvt fetch --no-recurse https://github.com/rneugeba/virtsock/go

virtsock.git:

$ git log --oneline 74097e05a883e89c70e6a27b342672c7fe6c846b..650ef8224a0c06b4b20e9bee1600dbf677c8176d -- go/
0e2f0a8 vsock: Implement vsock.Dial
712714a vsock: include the errno when C.bind_sockaddr_vm
03725fe go: make errors public

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2016-07-04 14:47:02 +01:00
Ian Campbell
98174da08f vsudd: Correctly format go Error's with %s
%#v is not correct (includes opaque pointers etc).

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2016-07-04 14:47:02 +01:00
Justin Cormack
78f4d5fa30 Merge pull request #245 from justincormack/ulimit
Increase ulimits as high as possible
2016-07-04 14:45:38 +01:00