Make AUFS optional and ship choice of kernels

This seems the best option, although none are great

- build with `make AUFS=1` to build with AUFS support, currently with 4.8 kernel
- default is to build without AUFS support, with 4.9 kernel

This recognises that AUFS supprot is temporary #620 and only there until
we can phase it out on desktop editions, and allow the other editions that
never shipped with AUFS to ship something very close to mainline.

However we do still apply the patches so that the non AUFS branch runs fine on
all platforms, so it can be tested elsewhere.

We may be able to move the kernel versions back in line when 4.9 aufs support is out.

Plan is to shift CI to build both sets of images, and get the Desktop editions to
pick up the aufs set automatically, once this is merged.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
This commit is contained in:
Justin Cormack 2016-12-13 17:28:32 -08:00
parent 2c55470450
commit 3b774e3657
10 changed files with 1982 additions and 70 deletions

View File

@ -1,6 +1,9 @@
all:
$(MAKE) -C alpine
aufs:
$(MAKE) AUFS=true all
alpine/initrd.img:
$(MAKE) -C alpine initrd.img
@ -42,15 +45,20 @@ test: Dockerfile.test alpine/initrd-test.img alpine/kernel/x86_64/vmlinuz64
TAG=$(shell git rev-parse HEAD)
STATUS=$(shell git status -s)
MOBYLINUX_TAG=alpine/mobylinux.tag
ifdef AUFS
AUFS_PREFIX=aufs-
endif
MEDIA_IMAGE=mobylinux/media:$(MEDIA_PREFIX)$(AUFS_PREFIX)$(TAG)
KERNEL_IMAGE=mobylinux/kernel:$(MEDIA_PREFIX)$(AUFS_PREFIX)$(TAG)
media: Dockerfile.media alpine/initrd.img alpine/kernel/x86_64/vmlinuz64 alpine/mobylinux-efi.iso
ifeq ($(STATUS),)
tar cf - $^ alpine/mobylinux.efi alpine/kernel/x86_64/vmlinux alpine/kernel/x86_64/kernel-headers.tar | docker build -f Dockerfile.media -t mobylinux/media:$(MEDIA_PREFIX)$(TAG) -
docker push mobylinux/media:$(MEDIA_PREFIX)$(TAG)
tar cf - $^ alpine/mobylinux.efi alpine/kernel/x86_64/vmlinux alpine/kernel/x86_64/kernel-headers.tar | docker build -f Dockerfile.media -t $(MEDIA_IMAGE) -
docker push $(MEDIA_IMAGE)
[ -f $(MOBYLINUX_TAG) ]
docker tag $(shell cat $(MOBYLINUX_TAG)) mobylinux/mobylinux:$(MEDIA_PREFIX)$(TAG)
docker push mobylinux/mobylinux:$(MEDIA_PREFIX)$(TAG)
tar cf - Dockerfile.kernel alpine/kernel/x86_64/vmlinuz64 | docker build -f Dockerfile.kernel -t mobylinux/kernel:$(MEDIA_PREFIX)$(TAG) -
docker push mobylinux/kernel:$(MEDIA_PREFIX)$(TAG)
docker tag $(shell cat $(MOBYLINUX_TAG)) $(MEDIA_IMAGE)
docker push $(MEDIA_IMAGE)
tar cf - Dockerfile.kernel alpine/kernel/x86_64/vmlinuz64 | docker build -f Dockerfile.kernel -t $(KERNEL_IMAGE) -
docker push $(KERNEL_IMAGE)
else
$(error "git not clean")
endif

View File

@ -1,7 +1,7 @@
# Tag: 36aecb5cf4738737634140eec9abebe1f6559a39
FROM mobylinux/alpine-build-c@sha256:d66b9625abc831f28f8c584991a9cb6975e85d3bb3d3768474b592f1cf32a3a6
ARG KERNEL_VERSION=4.8.14
ARG KERNEL_VERSION=4.9
ENV KERNEL_SOURCE=https://www.kernel.org/pub/linux/kernel/v4.x/linux-${KERNEL_VERSION}.tar.xz
@ -9,42 +9,6 @@ RUN curl -fsSL -o linux-${KERNEL_VERSION}.tar.xz ${KERNEL_SOURCE}
RUN cat linux-${KERNEL_VERSION}.tar.xz | tar --absolute-names -xJ && mv /linux-${KERNEL_VERSION} /linux
# this is aufs4.8 20161010
ENV AUFS_REPO https://github.com/sfjro/aufs4-standalone
ENV AUFS_BRANCH aufs4.8
ENV AUFS_COMMIT e9fd128dcb16167417683e199a5feb14f3c9eca8
# Download AUFS
RUN git clone -b "$AUFS_BRANCH" "$AUFS_REPO" /aufs && \
cd /aufs && \
git checkout -q "$AUFS_COMMIT"
# aufs-util 20151116
ENV AUFS_TOOLS_REPO https://github.com/ncopa/aufs-util.git
ENV AUFS_TOOLS_COMMIT 3b7c5e262b53598a8204a915e485489c46d4e7a4
# Download aufs tools
RUN git clone ${AUFS_TOOLS_REPO} && \
cd /aufs-util && \
git checkout "$AUFS_TOOLS_COMMIT"
#BUILD
# patch kernel with aufs
RUN cd /linux && \
cp -r /aufs/Documentation /linux && \
cp -r /aufs/fs /linux && \
cp -r /aufs/include/uapi/linux/aufs_type.h /linux/include/uapi/linux/ && \
set -e && for patch in \
/aufs/aufs*-kbuild.patch \
/aufs/aufs*-base.patch \
/aufs/aufs*-mmap.patch \
/aufs/aufs*-standalone.patch \
/aufs/aufs*-loopback.patch \
/aufs/lockdep-debug.patch \
; do \
patch -p1 < "$patch"; \
done
COPY kernel_config /linux/arch/x86/configs/x86_64_defconfig
COPY kernel_config.debug /linux/debug_config
@ -72,12 +36,4 @@ RUN make INSTALL_MOD_PATH=/tmp/kernel-modules modules_install && \
( cd /tmp && tar cf /kernel-headers.tar include ) && \
( cd /tmp/kernel-modules && tar cf /kernel-modules.tar . )
# Build aufs tools, do this here as they need kernel headers and to match aufs
# Fortunately they are built statically linked
RUN cd /aufs-util && \
CPPFLAGS="-I/tmp/include" CFLAGS=$CPPFLAGS LDFLAGS=$CPPFLAGS make && \
DESTDIR=/tmp/aufs-utils make install && \
rm -rf /tmp/aufs-utils/usr/lib /tmp/aufs-utils/usr/share && \
cd /tmp/aufs-utils && rm libau* && tar cf /aufs-utils.tar .
RUN printf "KERNEL_SOURCE=${KERNEL_SOURCE}\nAUFS_REPO=${AUFS_REPO}\nAUFS_BRANCH=${AUFS_BRANCH}\nAUFS_COMMIT=${AUFS_COMMIT}\nAUFS_TOOLS_REPO=${AUFS_TOOLS_REPO}\nAUFS_TOOLS_COMMIT=${AUFS_TOOLS_COMMIT}\n" > /kernel-source-info
RUN printf "KERNEL_SOURCE=${KERNEL_SOURCE}\n" > /kernel-source-info

View File

@ -0,0 +1,85 @@
# Tag: 36aecb5cf4738737634140eec9abebe1f6559a39
FROM mobylinux/alpine-build-c@sha256:d66b9625abc831f28f8c584991a9cb6975e85d3bb3d3768474b592f1cf32a3a6
ARG KERNEL_VERSION=4.8.14
ENV KERNEL_SOURCE=https://www.kernel.org/pub/linux/kernel/v4.x/linux-${KERNEL_VERSION}.tar.xz
RUN curl -fsSL -o linux-${KERNEL_VERSION}.tar.xz ${KERNEL_SOURCE}
RUN cat linux-${KERNEL_VERSION}.tar.xz | tar --absolute-names -xJ && mv /linux-${KERNEL_VERSION} /linux
# this is aufs4.8 20161010
ENV AUFS_REPO https://github.com/sfjro/aufs4-standalone
ENV AUFS_BRANCH aufs4.8
ENV AUFS_COMMIT e9fd128dcb16167417683e199a5feb14f3c9eca8
# Download AUFS
RUN git clone -b "$AUFS_BRANCH" "$AUFS_REPO" /aufs && \
cd /aufs && \
git checkout -q "$AUFS_COMMIT"
# aufs-util 20151116
ENV AUFS_TOOLS_REPO https://github.com/ncopa/aufs-util.git
ENV AUFS_TOOLS_COMMIT 3b7c5e262b53598a8204a915e485489c46d4e7a4
# Download aufs tools
RUN git clone ${AUFS_TOOLS_REPO} && \
cd /aufs-util && \
git checkout "$AUFS_TOOLS_COMMIT"
#BUILD
# patch kernel with aufs
RUN cd /linux && \
cp -r /aufs/Documentation /linux && \
cp -r /aufs/fs /linux && \
cp -r /aufs/include/uapi/linux/aufs_type.h /linux/include/uapi/linux/ && \
set -e && for patch in \
/aufs/aufs*-kbuild.patch \
/aufs/aufs*-base.patch \
/aufs/aufs*-mmap.patch \
/aufs/aufs*-standalone.patch \
/aufs/aufs*-loopback.patch \
/aufs/lockdep-debug.patch \
; do \
patch -p1 < "$patch"; \
done
COPY kernel_config /linux/arch/x86/configs/x86_64_defconfig
COPY kernel_config.debug /linux/debug_config
COPY kernel_config.aufs /linux/aufs_config
RUN cat /linux/aufs_config >> /linux/arch/x86/configs/x86_64_defconfig
ARG DEBUG=0
RUN if [ $DEBUG -ne "0" ]; then \
sed -i 's/CONFIG_PANIC_ON_OOPS=y/# CONFIG_PANIC_ON_OOPS is not set/' /linux/arch/x86/configs/x86_64_defconfig; \
cat /linux/debug_config >> /linux/arch/x86/configs/x86_64_defconfig; \
fi
# Apply local patches
COPY patches-aufs /patches
RUN cd /linux && \
set -e && for patch in /patches/*.patch; do \
echo "Applying $patch"; \
patch -p1 < "$patch"; \
done
WORKDIR /linux
RUN make defconfig && \
make oldconfig && \
make -j "$(getconf _NPROCESSORS_ONLN)" KCFLAGS="-fno-pie"
RUN make INSTALL_MOD_PATH=/tmp/kernel-modules modules_install && \
make INSTALL_HDR_PATH=/tmp headers_install && \
( cd /tmp && tar cf /kernel-headers.tar include ) && \
( cd /tmp/kernel-modules && tar cf /kernel-modules.tar . )
# Build aufs tools, do this here as they need kernel headers and to match aufs
# Fortunately they are built statically linked
RUN cd /aufs-util && \
CPPFLAGS="-I/tmp/include" CFLAGS=$CPPFLAGS LDFLAGS=$CPPFLAGS make && \
DESTDIR=/tmp/aufs-utils make install && \
rm -rf /tmp/aufs-utils/usr/lib /tmp/aufs-utils/usr/share && \
cd /tmp/aufs-utils && rm libau* && tar cf /aufs-utils.tar .
RUN printf "KERNEL_SOURCE=${KERNEL_SOURCE}\nAUFS_REPO=${AUFS_REPO}\nAUFS_BRANCH=${AUFS_BRANCH}\nAUFS_COMMIT=${AUFS_COMMIT}\nAUFS_TOOLS_REPO=${AUFS_TOOLS_REPO}\nAUFS_TOOLS_COMMIT=${AUFS_TOOLS_COMMIT}\n" > /kernel-source-info

View File

@ -2,16 +2,28 @@ DEBUG ?= 0
all: x86_64/vmlinuz64
x86_64/vmlinuz64: Dockerfile kernel_config
ifdef AUFS
x86_64/vmlinuz64: Dockerfile.aufs kernel_config kernel_config.debug kernel_config.aufs
mkdir -p x86_64 etc
BUILD=$$( docker build --build-arg DEBUG=$(DEBUG) -q . ) && [ -n "$$BUILD" ] && echo "Built $$BUILD" && \
BUILD=$$( docker build -f Dockerfile.aufs --build-arg DEBUG=$(DEBUG) -q . ) && [ -n "$$BUILD" ] && echo "Built $$BUILD" && \
docker run --rm --net=none --log-driver=none $$BUILD cat /kernel-modules.tar | tar xf - && \
docker run --rm --net=none --log-driver=none $$BUILD cat /aufs-utils.tar | tar xf - && \
docker run --rm --net=none --log-driver=none $$BUILD cat /kernel-source-info > etc/kernel-source-info && \
docker run --rm --net=none --log-driver=none $$BUILD cat /linux/vmlinux > x86_64/vmlinux && \
docker run --rm --net=none --log-driver=none $$BUILD cat /linux/arch/x86_64/boot/bzImage > $@ && \
docker run --rm --net=none --log-driver=none $$BUILD cat /kernel-headers.tar > x86_64/kernel-headers.tar && \
cp -a patches-aufs etc/kernel-patches
else
x86_64/vmlinuz64: Dockerfile kernel_config kernel_config.debug
mkdir -p x86_64 etc
BUILD=$$( docker build --build-arg DEBUG=$(DEBUG) -q . ) && [ -n "$$BUILD" ] && echo "Built $$BUILD" && \
docker run --rm --net=none --log-driver=none $$BUILD cat /kernel-modules.tar | tar xf - && \
docker run --rm --net=none --log-driver=none $$BUILD cat /kernel-source-info > etc/kernel-source-info && \
docker run --rm --net=none --log-driver=none $$BUILD cat /linux/vmlinux > x86_64/vmlinux && \
docker run --rm --net=none --log-driver=none $$BUILD cat /linux/arch/x86_64/boot/bzImage > $@ && \
docker run --rm --net=none --log-driver=none $$BUILD cat /kernel-headers.tar > x86_64/kernel-headers.tar && \
cp -a patches etc/kernel-patches
endif
clean:
rm -rf x86_64 lib etc usr sbin

View File

@ -2935,22 +2935,6 @@ CONFIG_PSTORE_ZLIB_COMPRESS=y
# CONFIG_PSTORE_RAM is not set
# CONFIG_SYSV_FS is not set
# CONFIG_UFS_FS is not set
CONFIG_AUFS_FS=y
# CONFIG_AUFS_BRANCH_MAX_127 is not set
# CONFIG_AUFS_BRANCH_MAX_511 is not set
# CONFIG_AUFS_BRANCH_MAX_1023 is not set
CONFIG_AUFS_BRANCH_MAX_32767=y
CONFIG_AUFS_SBILIST=y
# CONFIG_AUFS_HNOTIFY is not set
# CONFIG_AUFS_EXPORT is not set
CONFIG_AUFS_XATTR=y
# CONFIG_AUFS_FHSM is not set
# CONFIG_AUFS_RDU is not set
# CONFIG_AUFS_SHWH is not set
# CONFIG_AUFS_BR_RAMFS is not set
# CONFIG_AUFS_BR_FUSE is not set
CONFIG_AUFS_BDEV_LOOP=y
# CONFIG_AUFS_DEBUG is not set
CONFIG_NETWORK_FILESYSTEMS=y
CONFIG_NFS_FS=y
# CONFIG_NFS_V2 is not set

View File

@ -0,0 +1,16 @@
CONFIG_AUFS_FS=y
# CONFIG_AUFS_BRANCH_MAX_127 is not set
# CONFIG_AUFS_BRANCH_MAX_511 is not set
# CONFIG_AUFS_BRANCH_MAX_1023 is not set
CONFIG_AUFS_BRANCH_MAX_32767=y
CONFIG_AUFS_SBILIST=y
# CONFIG_AUFS_HNOTIFY is not set
# CONFIG_AUFS_EXPORT is not set
CONFIG_AUFS_XATTR=y
# CONFIG_AUFS_FHSM is not set
# CONFIG_AUFS_RDU is not set
# CONFIG_AUFS_SHWH is not set
# CONFIG_AUFS_BR_RAMFS is not set
# CONFIG_AUFS_BR_FUSE is not set
CONFIG_AUFS_BDEV_LOOP=y
# CONFIG_AUFS_DEBUG is not set

View File

@ -0,0 +1,30 @@
From afc48615e62910f37b6076f9118c80d2f9613064 Mon Sep 17 00:00:00 2001
From: Ian Campbell <ian.campbell@docker.com>
Date: Mon, 4 Apr 2016 14:50:10 +0100
Subject: [PATCH 1/5] VSOCK: Only allow host network namespace to use AF_VSOCK.
The VSOCK addressing schema does not really lend itself to simply creating an
alternative end point address within a namespace.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
---
net/vmw_vsock/af_vsock.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/net/vmw_vsock/af_vsock.c b/net/vmw_vsock/af_vsock.c
index 8a398b3..0edc54c 100644
--- a/net/vmw_vsock/af_vsock.c
+++ b/net/vmw_vsock/af_vsock.c
@@ -1852,6 +1852,9 @@ static const struct proto_ops vsock_stream_ops = {
static int vsock_create(struct net *net, struct socket *sock,
int protocol, int kern)
{
+ if (!net_eq(net, &init_net))
+ return -EAFNOSUPPORT;
+
if (!sock)
return -EINVAL;
--
2.10.2

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,30 @@
From e8c7a6dee61819c36b77108bc2cddafde26b9876 Mon Sep 17 00:00:00 2001
From: Rolf Neugebauer <rolf.neugebauer@gmail.com>
Date: Mon, 23 May 2016 18:55:45 +0100
Subject: [PATCH 4/5] vmbus: Don't spam the logs with unknown GUIDs
With Hyper-V sockets device types are introduced on the fly. The pr_info()
then prints a message on every connection, which is way too verbose. Since
there doesn't seem to be an easy way to check for registered services,
disable the pr_info() completely.
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
---
drivers/hv/channel_mgmt.c | 1 -
1 file changed, 1 deletion(-)
diff --git a/drivers/hv/channel_mgmt.c b/drivers/hv/channel_mgmt.c
index 8f4e6070..ef4a512 100644
--- a/drivers/hv/channel_mgmt.c
+++ b/drivers/hv/channel_mgmt.c
@@ -147,7 +147,6 @@ static u16 hv_get_dev_type(const uuid_le *guid)
if (!uuid_le_cmp(*guid, vmbus_devs[i].guid))
return i;
}
- pr_info("Unknown GUID: %pUl\n", guid);
return i;
}
--
2.10.2