Commit Graph

124 Commits

Author SHA1 Message Date
Justin Cormack
20250e5f41 Update Linux to 4.4.24
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-10-08 13:48:29 +02:00
Justin Cormack
457e44dbcc update kernel to 4.4.23
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-10-01 21:03:47 +01:00
Justin Cormack
4e2f887f40 add ncurses to C build for menuconfig
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-10-01 18:26:38 +01:00
Justin Cormack
652d5eddf3 Use build container id rather than tagging builds
This means that multiple builds will not conflict, so we can
remove the lock from the CI. Also quieter when no errors.

Some still left to do, only done the ones used in build and CI
initially. Some of the others will be cleaned up anyway later.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-09-29 20:21:17 +01:00
Justin Cormack
bf29850d3c Use patched aufs-utils so can compile kernel with alpine
This is going to be submitted upstream.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-09-29 16:17:46 +01:00
Justin Cormack
4549970d55 Update Linux kernel to 4.4.22
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-09-26 00:01:53 +01:00
Rolf Neugebauer
de993ec0d9 linux: update kernel to 4.4.21
- removed some Hyper-V patches which were picked up
  between 4.4.20 and 4.4.21
- udate aufs

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2016-09-19 18:20:28 +01:00
Justin Cormack
5c2bdf7b2a Merge pull request #510 from justincormack/nolog
No log driver for large copies
2016-09-14 12:35:25 +01:00
Ian Campbell
927f473b20 kernel: Replace vsock email series with upstream patches
The vsock patches were merged into mainline in v4.8-rc1, replace the patches
based on the email posting with cherry-picks. The patches appear identical so
the only change here is to the annotation of the origin which now references an
upstream commit instead of a message-id.

Add one new upstream patch b226acab2f6a "VSOCK: Use kvfree()"

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2016-09-14 10:03:10 +01:00
Ian Campbell
c41f680f7d kernel: Update vsock patches to RFC v6
Series is at <1469716595-13591-1-git-send-email-stefanha@redhat.com>.

This corresponds to v7 of the spec, posted in
<1470324277-19300-1-git-send-email-stefanha@redhat.com>

Also add a "cherry-picked from" to the "vsock: make listener child lock
ordering explicit" patch and move it to the head of the series with the other
vsock backports.

Finally backport three new upstream fixes:
3fda5d6e5801 vhost/vsock: fix vhost virtio_vsock_pkt use-after-free
28ad55578b8a virtio-vsock: fix include guard typo
21bc54fc0cdc vhost/vsock: drop space available check for TX vq

These were made on top of the version of the vsock patches which were added to
Linux master in v4.8-rc1. This commit is based on the email posting, will
replace with with proper cherry-pick separately.

Requires corresponding backend changes in Hyperkit

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2016-09-14 10:03:10 +01:00
Justin Cormack
c522ca1d31 No log driver for large copies
Takes 75% of the time without logs.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-09-14 09:05:36 +01:00
Justin Cormack
a6fc00e41b fix kernel build missing arch
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-09-13 22:30:26 +01:00
Justin Cormack
99dcbdf322 Rework kernel Makefile
Do not delete build container on `make clean`, as it should be fine
to reuse cached layers.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-09-13 21:49:14 +01:00
Justin Cormack
3665ef0b82 reorder kernel Dockerfile
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-09-13 21:15:34 +01:00
Justin Cormack
ba40e0a33b Use a random tag for Debian kernel build base
To store all the old images they need a different tag, so use a
random one.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-09-12 16:36:01 +01:00
Justin Cormack
3d0cd1deca Use a base image for building kernel
As we do not have a simple way to hash Debian, use the Docker
sha256 until we switch to Alpine.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-09-12 16:36:01 +01:00
Justin Cormack
0a6f4f6bf5 revert to debian for kernel build
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-09-12 16:36:01 +01:00
Justin Cormack
c51a0b831e Simplify kernel output and main Dockerfile
Signed-off-by: Justin Cormack <justin@specialbusservice.com>
2016-09-12 16:02:07 +01:00
Justin Cormack
21a0c10707 Update kernel and aufs
- kernel 4.4.20
- aufs 20160905

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-09-07 22:51:51 +01:00
Justin Cormack
2848c13ed7 remove arm dtb now we no longer build arm kernel
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-08-31 16:51:03 +01:00
Justin Cormack
069930f3a6 quieter kernel build
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-08-31 15:53:16 +01:00
Justin Cormack
c1e2418a16 Remove support for arm kernel build
See #449. Plan is to use upstream Alpine kernel for Arm, as
does not need vsock, hvsock or aufs.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-08-31 14:23:40 +01:00
Justin Cormack
b2086fc693 Unlimit max CPU cores in config
See https://github.com/docker/for-win/issues/59

Even on desktop we have users with more than 8 cores!

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-08-25 09:32:23 +01:00
Justin Cormack
5b527df37c Update build containers to Ubuntu 16.10
Should update to Alpine in future.

Signed-off-by: Justin Cormack <justin@specialbusservice.com>
2016-08-23 19:53:11 +01:00
Justin Cormack
dce8e37a71 Update kernel and aufs
- Linux 4.4.19
- AUFS 20160822

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-08-22 11:05:07 +01:00
David Sheets
1163bf09be kernel: apply debug configuration to all kernels
Signed-off-by: David Sheets <dsheets@docker.com>
2016-08-10 17:08:07 +01:00
David Sheets
9e5c86f006 kernel: add a Makefile DEBUG variable and a kernel_config for debugging
Signed-off-by: David Sheets <dsheets@docker.com>
2016-08-10 17:07:48 +01:00
Justin Cormack
dec4fb790f Update kernel and aufs
- Linux 4.4.17
- AUFS 20160808

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-08-10 15:03:36 +01:00
Justin Cormack
1f4d6c0402 Update kernel to 4.4.16
This contains a lot of fixes for overlay.

Fix #374

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-07-29 16:21:24 +01:00
Natanael Copa
0bb80957bc Move built kernel to arch specific subdir
We want be able to build kernels for different archs without that they
clash with each other so we but the generated files into an $arch subdir.

Signed-off-by: Natanael Copa <natanael.copa@docker.com>
2016-07-25 17:18:02 +02:00
Justin Cormack
8ae5000ae7 Remove some VMWare devices
In particular vsock causes issues with virtio vsock

We are not supporting VMWare platform at present so not relevant..

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-07-25 14:23:24 +01:00
Justin Cormack
054c384230 Add default SCSI device used by VMWare Fusion
As this is the default emulation, nicer to have it work.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-07-22 13:41:45 +01:00
Justin Cormack
bcd922216a Add VMWare kernel features
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-07-22 12:54:20 +01:00
Justin Cormack
7cebdba16b typo
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-07-21 13:41:56 +01:00
Justin Cormack
80cab7bb26 Merge pull request #317 from dsheets/kernel-af_vsock-lockdep
kernel: backport af_vsock lockdep cleanliness patch from v4.7-rc6
2016-07-20 16:27:33 +01:00
David Sheets
97ea6ed818 kernel: backport af_vsock lockdep cleanliness patch from v4.7-rc6
From torvalds/linux@4192f672fa which was
part of torvalds/linux@v4.7-rc6. The change is purely to ensure that vsock
is lockdep-clean.

Signed-off-by: David Sheets <dsheets@docker.com>
2016-07-20 16:22:21 +01:00
David Sheets
a2269275b6 kernel: apply the AUFS patch to integrate with lockdep
AUFS introduces new lockdep relations which are beyond the maximum variants
that lockdep ships with. Without this patch, AUFS triggers lockdep BUG sanity
checks and disables lockdep for the rest of the system.

The present value of the patch is:

```diff
aufs4.4 lockdep patch

diff --git a/include/linux/lockdep.h b/include/linux/lockdep.h
index c57e424..4153563 100644
--- a/include/linux/lockdep.h
+++ b/include/linux/lockdep.h
@@ -29,7 +29,7 @@ extern int lock_stat;
  */
 #define XXX_LOCK_USAGE_STATES		(1+3*4)

-#define MAX_LOCKDEP_SUBCLASSES		8UL
+#define MAX_LOCKDEP_SUBCLASSES		(8UL + 4)

 /*
  * NR_LOCKDEP_CACHING_CLASSES ... Number of classes
@@ -203,7 +203,7 @@ struct lock_chain {
 	u64				chain_key;
 };

-#define MAX_LOCKDEP_KEYS_BITS		13
+#define MAX_LOCKDEP_KEYS_BITS		(13 + 3)
 /*
  * Subtract one because we offset hlock->class_idx by 1 in order
  * to make 0 mean no class. This avoids overflowing the class_idx
diff --git a/kernel/locking/lockdep_internals.h b/kernel/locking/lockdep_internals.h
index 51c4b24..fba7557 100644
--- a/kernel/locking/lockdep_internals.h
+++ b/kernel/locking/lockdep_internals.h
@@ -54,9 +54,9 @@ enum {
  * table (if it's not there yet), and we check it for lock order
  * conflicts and deadlocks.
  */
-#define MAX_LOCKDEP_ENTRIES	32768UL
+#define MAX_LOCKDEP_ENTRIES	(32768UL << 5)

-#define MAX_LOCKDEP_CHAINS_BITS	16
+#define MAX_LOCKDEP_CHAINS_BITS	(16 + 5)
 #define MAX_LOCKDEP_CHAINS	(1UL << MAX_LOCKDEP_CHAINS_BITS)

 #define MAX_LOCKDEP_CHAIN_HLOCKS (MAX_LOCKDEP_CHAINS*5)
@@ -65,7 +65,7 @@ enum {
  * Stack-trace: tightly packed array of stack backtrace
  * addresses. Protected by the hash_lock.
  */
-#define MAX_STACK_TRACE_ENTRIES	524288UL
+#define MAX_STACK_TRACE_ENTRIES	(524288UL << 5)

 extern struct list_head all_lock_classes;
 extern struct lock_chain lock_chains[];
```

Signed-off-by: David Sheets <dsheets@docker.com>
2016-07-20 16:08:00 +01:00
David Sheets
cae79c7ce3 kernel: make sure to gitignore vmlinux
Signed-off-by: David Sheets <dsheets@docker.com>
2016-07-20 15:40:54 +01:00
David Sheets
cdac4fde34 kernel: add vmlinux target to make kernel debugging easier
With this patch, it is easy to get an unstripped kernel object to use with
gdb.

Signed-off-by: David Sheets <dsheets@docker.com>
2016-07-20 15:37:04 +01:00
Natanael Copa
47545dce55 improve kernel build makefile
- create an empty dummy file to indicate that docker image is built
- reuse same make rule to extract the different files from docker image
- make sure that we remove empty files on failure

This makes build more robust and improves parallelism.

Signed-off-by: Natanael Copa <natanael.copa@docker.com>
2016-07-13 18:03:16 +02:00
Rolf Neugebauer
2171a2e471 kernel: Update to 4.4.15
patches from: https://github.com/rneugeba/linux-stable/tree/v4.4.15-moby

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2016-07-12 10:36:57 +01:00
David Sheets
dcfe76be64 linux: allow kernel module unloading
The ability to unload kernel modules helps with rapid development of kernel
modules or Moby-integrated functionality. It has no negative side effects
as far as I am aware.

Signed-off-by: David Sheets <dsheets@docker.com>
2016-07-11 11:48:46 -07:00
Justin Cormack
ab365e38a0 Merge pull request #255 from justincormack/e1000
add e1000 drivers, allows qemu default setup to find an ethernet, and…
2016-07-08 16:28:24 +01:00
Justin Cormack
7d605926f0 add e1000 drivers, allows qemu default setup to find an ethernet, and used frequently in emulated environments
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-07-08 16:26:16 +01:00
Rolf Neugebauer
6125e66e5c linux: properly update kernel patches to v4.4.14
When we updated from 4.4.10 to 4.4.14 we copied the patches over.
This changeset properly updates the patches so that they apply cleanly.
0039-VSOCK-do-not-disconnect-socket-when-peer-has-shutdow.patch was
removed as it made it into 4.4.14 already.

Compile tested only

For reference the patches were generated from:
https://github.com/rneugeba/linux-stable/tree/v4.4.14-moby

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2016-07-08 16:10:51 +01:00
Justin Cormack
cdad79e916 Updated kernel config
Remove one module, Xen ones not removeable. Make oldconfig.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-07-04 17:03:10 +01:00
Ian Campbell
209dd754bc Populate /lib/modules/uname -r
This is less to do with installing modules (which we generally don't expect to
use in Moby) but to populate /lib/modules/`uname -r`/modules.builtin which
turns:

    moby:~# modprobe ip_vs
    modprobe: FATAL: Module ip_vs not found in directory /lib/modules/4.4.14-moby
    moby:~# modprobe nf_nat
    modprobe: FATAL: Module nf_nat not found in directory /lib/modules/4.4.14-moby
    moby:~#

into:

    moby:~# modprobe ip_vs
    moby:~# modprobe nf_nat
    moby:~#

which reduces the amount noise in the logs, e.g. in docker.log:

time="2016-07-04T11:21:58Z" level=warning msg="Running modprobe nf_nat failed with message: `modprobe: WARNING: Module nf_nat not found in directory /lib/modules/4.4.14-moby`, error: exit status 1"

A fair number of these appear in the logs.

This also stops various tools logging about /lib/modules/`uname -r` not
existing (there was one in the boot log until recently I think)

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2016-07-04 15:44:48 +01:00
Justin Cormack
1d22f68e87 update linux to 4.4.14 and update aufs to 20160627
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-06-26 16:16:57 -07:00
Daniel Hiltgen
3479c8b3ac Add SATA disk support
Increase the number of hypervisors where Moby can run and detect
the disks.  With this change, I'm able to boot under KVM and see
the disk detected, formatted and mounted as expected.
2016-06-17 08:11:46 -07:00
Daniel Hiltgen
52b43e9482 Add support for console on tty1
When running moby under other hypervisors, requiring troubleshooting on
the serial port can be painful.  This change enables console support on
tty1 similar to the way prior boot2docker images worked.
2016-06-16 17:17:34 -07:00