github/kata-containers
1
0
Fork 1
mirror of https://github.com/kata-containers/kata-containers.git synced 2025-06-30 17:22:33 +00:00
Code Issues Projects Releases Wiki Activity

rootfs-builder: support provisioning existing rootfs

Browse Source 
Add the use case of provisioning an existing rootfs directory with the
components / configurations needed to generate a Kata compatible images.
This supports use cases such as using a rootfs built outside of
osbuilder, and providing a overlay for dracut built initrds.

Signed-off-by: Marco Vedovati <mvedovati@suse.com>
This commit is contained in:
Marco Vedovati 2019-06-12 18:54:55 +02:00
parent 7d38b84203
commit 39370c2aea
2 changed files with 273 additions and 204 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

37
rootfs-builder/README.md
View File

@ -1,9 +1,13 @@
* [Supported base OSs](#supported-base-oss) * [Building a Guest OS rootfs for Kata Containers](#building-a-guest-os-rootfs-for-kata-containers)
* [Rootfs requirements](#rootfs-requirements) * [Supported base OSs](#supported-base-oss)
* [Creating a rootfs](#creating-a-rootfs) * [Extra features](#extra-features)
* [Creating a rootfs with kernel modules](#creating-a-rootfs-with-kernel-modules) * [Supported distributions list](#supported-distributions-list)
* [Build a rootfs using Docker](#build-a-rootfs-using-docker) * [Generate Kata specific files](#generate-kata-specific-files)
* [Adding support for a new guest OS](#adding-support-for-a-new-guest-os) * [Rootfs requirements](#rootfs-requirements)
* [Creating a rootfs](#creating-a-rootfs)
* [Creating a rootfs with kernel modules](#creating-a-rootfs-with-kernel-modules)
* [Build a rootfs using Docker](#build-a-rootfs-using-docker)
* [Adding support for a new guest OS](#adding-support-for-a-new-guest-os)
* [Create template files](#create-template-files) * [Create template files](#create-template-files)
* [Modify template files](#modify-template-files) * [Modify template files](#modify-template-files)
* [Expected rootfs directory content](#expected-rootfs-directory-content) * [Expected rootfs directory content](#expected-rootfs-directory-content)
@ -21,10 +25,25 @@ The `rootfs.sh` script builds a rootfs based on a particular Linux\*
distribution. The script supports multiple distributions and can be extended distribution. The script supports multiple distributions and can be extended
to add further ones. to add further ones.
To list the supported distributions, run: ### Extra features
#### Supported distributions list
Supported distributions can be listed with:
``` ```
$ ./rootfs.sh -h $ ./rootfs.sh -l
```
#### Generate Kata specific files
`rootfs.sh` can be used to only populate a target directory with the set of Kata
specific files and components integrable into a generic Linux rootfs to generate
a Kata guest OS image.
This feature can be used when creating a rootfs with a distribution not officially
supported by osbuilder.
It is also used when building the rootfs using the 'dracut' build method.
To obtain this, simply invoke `rootfs.sh` without specifying a target rootfs, e.g.:
```
mkdir kata-overlay
./rootfs.sh -r `pwd`/kata-overlay
``` ```
## Rootfs requirements ## Rootfs requirements

260
rootfs-builder/rootfs.sh
View File

@ -52,18 +52,31 @@ typeset -r CONFIG_ARCH_SH="config_${ARCH}.sh"
# build_rootfs() function. # build_rootfs() function.
typeset -r LIB_SH="rootfs_lib.sh" typeset -r LIB_SH="rootfs_lib.sh"
# rootfs distro name specified by the user
typeset distro=
# Absolute path to the rootfs root folder
typeset ROOTFS_DIR
# Absolute path in the rootfs to the "init" executable / symlink.
# Typically something like "${ROOTFS_DIR}/init
typeset init=
#$1: Error code if want to exit different to 0 #$1: Error code if want to exit different to 0
usage() usage()
{ {
error="${1:-0}" error="${1:-0}"
cat <<EOT cat <<EOT
Usage: ${script_name} [options] <distro> Usage: ${script_name} [options] [DISTRO]
Build a rootfs based on <distro> OS, to be included in a Kata Containers Build and setup a rootfs directory based on DISTRO OS, used to create
image. Kata Containers images or initramfs.
Supported <distro> values: When no DISTRO is provided, an existing base rootfs at ROOTFS_DIR is provisioned
with the Kata specific components and configuration.
Supported DISTRO values:
$(get_distros | tr "\n" " ") $(get_distros | tr "\n" " ")
Options: Options:
@ -75,7 +88,7 @@ Options:
yaml description. yaml description.
-r <directory> Specify the rootfs base directory. Overrides the ROOTFS_DIR -r <directory> Specify the rootfs base directory. Overrides the ROOTFS_DIR
environment variable. environment variable.
-t Print the test configuration for <distro> and exit -t DISTRO Print the test configuration for DISTRO and exit
immediately. immediately.
Environment Variables: Environment Variables:
@ -100,7 +113,7 @@ DISTRO_REPO Use host repositories to install guest packages.
GO_AGENT_PKG URL of the Git repository hosting the agent package. GO_AGENT_PKG URL of the Git repository hosting the agent package.
Default value: ${GO_AGENT_PKG} Default value: ${GO_AGENT_PKG}
GRACEFUL_EXIT If set, and if the <distro> configuration specifies a GRACEFUL_EXIT If set, and if the DISTRO configuration specifies a
non-empty BUILD_CAN_FAIL variable, do not return with an non-empty BUILD_CAN_FAIL variable, do not return with an
error code in case any of the build step fails. error code in case any of the build step fails.
This is used when running CI jobs, to tolerate failures for This is used when running CI jobs, to tolerate failures for
@ -112,7 +125,7 @@ KERNEL_MODULES_DIR Path to a directory containing kernel modules to include in
Default value: <empty> Default value: <empty>
ROOTFS_DIR Path to the directory that is populated with the rootfs. ROOTFS_DIR Path to the directory that is populated with the rootfs.
Default value: <${script_name} path>/rootfs-<distro-name> Default value: <${script_name} path>/rootfs-<DISTRO-name>
USE_DOCKER If set, build the rootfs inside a container (requires USE_DOCKER If set, build the rootfs inside a container (requires
Docker). Docker).
@ -137,7 +150,9 @@ get_distros() {
} }
get_test_config() { get_test_config() {
local distro="$1" local -r distro="$1"
[ -z "$distro" ] && die "No distro name specified"
local config="${script_dir}/${distro}/config.sh" local config="${script_dir}/${distro}/config.sh"
source ${config} source ${config}
@ -330,82 +345,70 @@ compare_versions()
true true
} }
while getopts a:hlo:r:t: opt check_env_variables()
do {
case $opt in # Fetch the first element from GOPATH as working directory
a) AGENT_VERSION="${OPTARG}" ;; # as go get only works against the first item in the GOPATH
h) usage ;; [ -z "$GOPATH" ] && die "GOPATH not set"
l) get_distros | sort && exit 0;; GOPATH_LOCAL="${GOPATH%%:*}"
o) OSBUILDER_VERSION="${OPTARG}" ;;
r) ROOTFS_DIR="${OPTARG}" ;;
t) get_test_config "${OPTARG}" && exit 0;;
esac
done
shift $(($OPTIND - 1)) [ "$AGENT_INIT" == "yes" -o "$AGENT_INIT" == "no" ] || die "AGENT_INIT($AGENT_INIT) is invalid (must be yes or no)"
# Fetch the first element from GOPATH as working directory [ -n "${KERNEL_MODULES_DIR}" ] && [ ! -d "${KERNEL_MODULES_DIR}" ] && die "KERNEL_MODULES_DIR defined but is not an existing directory"
# as go get only works against the first item in the GOPATH
[ -z "$GOPATH" ] && die "GOPATH not set"
GOPATH_LOCAL="${GOPATH%%:*}"
[ "$AGENT_INIT" == "yes" -o "$AGENT_INIT" == "no" ] || die "AGENT_INIT($AGENT_INIT) is invalid (must be yes or no)" [ -n "${OSBUILDER_VERSION}" ] || die "need osbuilder version"
}
[ -n "${KERNEL_MODULES_DIR}" ] && [ ! -d "${KERNEL_MODULES_DIR}" ] && die "KERNEL_MODULES_DIR defined but is not an existing directory" # Builds a rootfs based on the distro name provided as argument
build_rootfs_distro()
{
[ -n "${distro}" ] || usage 1
distro_config_dir="${script_dir}/${distro}"
[ -z "${OSBUILDER_VERSION}" ] && die "need osbuilder version" # Source config.sh from distro
rootfs_config="${distro_config_dir}/${CONFIG_SH}"
source "${rootfs_config}"
distro="$1" # Source arch-specific config file
rootfs_arch_config="${distro_config_dir}/${CONFIG_ARCH_SH}"
[ -n "${distro}" ] || usage 1 if [ -f "${rootfs_arch_config}" ]; then
distro_config_dir="${script_dir}/${distro}"
# Source config.sh from distro
rootfs_config="${distro_config_dir}/${CONFIG_SH}"
source "${rootfs_config}"
# Source arch-specific config file
rootfs_arch_config="${distro_config_dir}/${CONFIG_ARCH_SH}"
if [ -f "${rootfs_arch_config}" ]; then
source "${rootfs_arch_config}" source "${rootfs_arch_config}"
fi fi
[ -d "${distro_config_dir}" ] || die "Not found configuration directory ${distro_config_dir}" [ -d "${distro_config_dir}" ] || die "Not found configuration directory ${distro_config_dir}"
if [ -z "$ROOTFS_DIR" ]; then if [ -z "$ROOTFS_DIR" ]; then
ROOTFS_DIR="${script_dir}/rootfs-${OS_NAME}" ROOTFS_DIR="${script_dir}/rootfs-${OS_NAME}"
fi fi
init="${ROOTFS_DIR}/sbin/init" if [ -e "${distro_config_dir}/${LIB_SH}" ];then
if [ -e "${distro_config_dir}/${LIB_SH}" ];then
rootfs_lib="${distro_config_dir}/${LIB_SH}" rootfs_lib="${distro_config_dir}/${LIB_SH}"
info "rootfs_lib.sh file found. Loading content" info "rootfs_lib.sh file found. Loading content"
source "${rootfs_lib}" source "${rootfs_lib}"
fi fi
CONFIG_DIR=${distro_config_dir} CONFIG_DIR=${distro_config_dir}
check_function_exist "build_rootfs" check_function_exist "build_rootfs"
if [ -z "$INSIDE_CONTAINER" ] ; then if [ -z "$INSIDE_CONTAINER" ] ; then
# Capture errors, but only outside of the docker container # Capture errors, but only outside of the docker container
trap error_handler ERR trap error_handler ERR
fi fi
mkdir -p ${ROOTFS_DIR} mkdir -p ${ROOTFS_DIR}
detect_go_version || detect_go_version ||
die "Could not detect the required Go version for AGENT_VERSION='${AGENT_VERSION:-master}'." die "Could not detect the required Go version for AGENT_VERSION='${AGENT_VERSION:-master}'."
echo "Required Go version: $GO_VERSION" echo "Required Go version: $GO_VERSION"
if [ -z "${USE_DOCKER}" ] ; then if [ -z "${USE_DOCKER}" ] ; then
#Generate an error if the local Go version is too old #Generate an error if the local Go version is too old
foundVersion=$(go version | sed -E "s/^.+([0-9]+\.[0-9]+\.[0-9]+).*$/\1/g") foundVersion=$(go version | sed -E "s/^.+([0-9]+\.[0-9]+\.[0-9]+).*$/\1/g")
compare_versions "$GO_VERSION" $foundVersion || \ compare_versions "$GO_VERSION" $foundVersion || \
die "Your Go version $foundVersion is older than the minimum expected Go version $GO_VERSION" die "Your Go version $foundVersion is older than the minimum expected Go version $GO_VERSION"
else else
image_name="${distro}-rootfs-osbuilder" image_name="${distro}-rootfs-osbuilder"
generate_dockerfile "${distro_config_dir}" generate_dockerfile "${distro_config_dir}"
@ -468,67 +471,76 @@ else
bash /osbuilder/rootfs.sh "${distro}" bash /osbuilder/rootfs.sh "${distro}"
exit $? exit $?
fi fi
build_rootfs ${ROOTFS_DIR} build_rootfs ${ROOTFS_DIR}
pushd "${ROOTFS_DIR}" >> /dev/null }
if [ "$PWD" != "/" ] ; then
rm -rf ./var/cache/ ./var/lib ./var/log
fi
info "Create symlink to /tmp in /var to create private temporal directories with systemd" # Used to create a minimal directory tree where the agent can be instaleld.
rm -rf ./var/tmp # This is used when a distro is not specified.
ln -s ../tmp ./var/ prepare_overlay()
{
pushd "${ROOTFS_DIR}" >> /dev/null
mkdir -p ./etc ./lib/systemd ./sbin ./var
ln -sf ./usr/lib/systemd/systemd ./init
ln -sf ../../init ./lib/systemd/systemd
ln -sf ../init ./sbin/init
popd >> /dev/null
}
# For some distros tmp.mount may not be installed by default in systemd paths # Setup an existing rootfs directory, based on the OPTIONAL distro name
if ! [ -f "./etc/systemd/system/tmp.mount" ] && \ # provided as argument
setup_rootfs()
{
[ -z "$distro" ] && prepare_overlay
info "Create symlink to /tmp in /var to create private temporal directories with systemd"
pushd "${ROOTFS_DIR}" >> /dev/null
if [ "$PWD" != "/" ] ; then
rm -rf ./var/cache/ ./var/lib ./var/log ./var/tmp
fi
ln -s ../tmp ./var/
# For some distros tmp.mount may not be installed by default in systemd paths
if ! [ -f "./etc/systemd/system/tmp.mount" ] && \
! [ -f "./usr/lib/systemd/system/tmp.mount" ] && ! [ -f "./usr/lib/systemd/system/tmp.mount" ] &&
[ "$AGENT_INIT" != "yes" ]; then [ "$AGENT_INIT" != "yes" ]; then
info "Install tmp.mount in ./etc/systemd/system" info "Install tmp.mount in ./etc/systemd/system"
cp ./usr/share/systemd/tmp.mount ./etc/systemd/system/tmp.mount cp ./usr/share/systemd/tmp.mount ./etc/systemd/system/tmp.mount
fi fi
popd >> /dev/null popd >> /dev/null
[ -n "${KERNEL_MODULES_DIR}" ] && copy_kernel_modules ${KERNEL_MODULES_DIR} ${ROOTFS_DIR} [ -n "${KERNEL_MODULES_DIR}" ] && copy_kernel_modules ${KERNEL_MODULES_DIR} ${ROOTFS_DIR}
chrony_conf_file="${ROOTFS_DIR}/etc/chrony.conf" chrony_conf_file="${ROOTFS_DIR}/etc/chrony.conf"
if [ ${distro} == ubuntu ] || [ ${distro} == debian ] ; then if [ "${distro}" == "ubuntu" ] || [ "${distro}" == "debian" ] ; then
chrony_conf_file="${ROOTFS_DIR}/etc/chrony/chrony.conf" chrony_conf_file="${ROOTFS_DIR}/etc/chrony/chrony.conf"
fi fi
info "Create ${ROOTFS_DIR}/etc" info "Create ${ROOTFS_DIR}/etc"
mkdir -p "${ROOTFS_DIR}/etc" mkdir -p "${ROOTFS_DIR}/etc"
info "Configure chrony file ${chrony_conf_file}" info "Configure chrony file ${chrony_conf_file}"
cat >> "${chrony_conf_file}" <<EOT cat >> "${chrony_conf_file}" <<EOT
refclock PHC /dev/ptp0 poll 3 dpoll -2 offset 0 refclock PHC /dev/ptp0 poll 3 dpoll -2 offset 0
# Step the system clock instead of slewing it if the adjustment is larger than # Step the system clock instead of slewing it if the adjustment is larger than
# one second, at any time # one second, at any time
makestep 1 -1 makestep 1 -1
EOT EOT
# Comment out ntp sources for chrony to be extra careful # Comment out ntp sources for chrony to be extra careful
# Reference: https://chrony.tuxfamily.org/doc/3.4/chrony.conf.html # Reference: https://chrony.tuxfamily.org/doc/3.4/chrony.conf.html
sed -i 's/^\(server \|pool \|peer \)/# &/g' ${chrony_conf_file} sed -i 's/^\(server \|pool \|peer \)/# &/g' ${chrony_conf_file}
chrony_systemd_service="${ROOTFS_DIR}/usr/lib/systemd/system/chronyd.service" # The CC on s390x for fedora needs to be manually set to gcc when the golang is downloaded from the main page.
if [ ${distro} == ubuntu ] || [ ${distro} == debian ] ; then # See issue: https://github.com/kata-containers/osbuilder/issues/217
chrony_systemd_service="${ROOTFS_DIR}/lib/systemd/system/chrony.service" [ "$distro" == "fedora" ] && [ "$ARCH" == "s390x" ] && export CC=gcc
fi
if [ -f "$chrony_systemd_service" ]; then AGENT_DIR="${ROOTFS_DIR}/usr/bin"
sed -i '/^\[Unit\]/a ConditionPathExists=\/dev\/ptp0' ${chrony_systemd_service} AGENT_DEST="${AGENT_DIR}/${AGENT_BIN}"
fi
# The CC on s390x for fedora needs to be manually set to gcc when the golang is downloaded from the main page. if [ -z "${AGENT_SOURCE_BIN}" ] ; then
# See issue: https://github.com/kata-containers/osbuilder/issues/217
[ "$distro" == fedora ] && [ "$ARCH" == "s390x" ] && export CC=gcc
AGENT_DIR="${ROOTFS_DIR}/usr/bin"
AGENT_DEST="${AGENT_DIR}/${AGENT_BIN}"
if [ -z "${AGENT_SOURCE_BIN}" ] ; then
info "Pull Agent source code" info "Pull Agent source code"
go get -d "${GO_AGENT_PKG}" || true go get -d "${GO_AGENT_PKG}" || true
OK "Pull Agent source code" OK "Pull Agent source code"
@ -540,19 +552,57 @@ if [ -z "${AGENT_SOURCE_BIN}" ] ; then
make INIT=${AGENT_INIT} make INIT=${AGENT_INIT}
make install DESTDIR="${ROOTFS_DIR}" INIT=${AGENT_INIT} SECCOMP=${SECCOMP} make install DESTDIR="${ROOTFS_DIR}" INIT=${AGENT_INIT} SECCOMP=${SECCOMP}
popd popd
else else
cp ${AGENT_SOURCE_BIN} ${AGENT_DEST} cp ${AGENT_SOURCE_BIN} ${AGENT_DEST}
OK "cp ${AGENT_SOURCE_BIN} ${AGENT_DEST}" OK "cp ${AGENT_SOURCE_BIN} ${AGENT_DEST}"
fi fi
[ -x "${AGENT_DEST}" ] || die "${AGENT_DEST} is not installed in ${ROOTFS_DIR}" [ -x "${AGENT_DEST}" ] || die "${AGENT_DEST} is not installed in ${ROOTFS_DIR}"
OK "Agent installed" OK "Agent installed"
[ "${AGENT_INIT}" == "yes" ] && setup_agent_init "${AGENT_DEST}" "${init}" [ "${AGENT_INIT}" == "yes" ] && setup_agent_init "${AGENT_DEST}" "${init}"
info "Check init is installed" info "Check init is installed"
[ -x "${init}" ] || [ -L "${init}" ] || die "/sbin/init is not installed in ${ROOTFS_DIR}" [ -x "${init}" ] || [ -L "${init}" ] || die "/sbin/init is not installed in ${ROOTFS_DIR}"
OK "init is installed" OK "init is installed"
info "Creating summary file" info "Creating summary file"
create_summary_file "${ROOTFS_DIR}" create_summary_file "${ROOTFS_DIR}"
}
parse_arguments()
{
while getopts a:hlo:r:t: opt
do
case $opt in
a) AGENT_VERSION="${OPTARG}" ;;
h) usage ;;
l) get_distros | sort && exit 0;;
o) OSBUILDER_VERSION="${OPTARG}" ;;
r) ROOTFS_DIR="${OPTARG}" ;;
t) get_test_config "${OPTARG}" && exit -1;;
*) die "Found an invalid option";;
esac
done
shift $(($OPTIND - 1))
distro="$1"
}
main()
{
parse_arguments $*
check_env_variables
init="${ROOTFS_DIR}/sbin/init"
if [ -n "$distro" ]; then
build_rootfs_distro
else
#Make sure ROOTFS_DIR is set correctly
[ -d "${ROOTFS_DIR}" ] || die "Invalid rootfs directory: '$ROOTFS_DIR'"
fi
setup_rootfs
}
main $*