Commit Graph

4821 Commits

Author SHA1 Message Date
Salvador Fuentes
11c3d81f68 docs: fix markdown check issues
- Removes `CODE_OF_CONDUCT.md` and `CONTRIBUTING.md` from osbuilder
directory.
- Fixes a reference from `image-builder/README.md` to
`rootfs-builder/README.md`
- Updates the main `README.md` making a reference to the local
`tools/osbuilder/README.md`

Signed-off-by: Salvador Fuentes <salvador.fuentes@intel.com>
2020-04-29 17:51:44 -05:00
Salvador Fuentes
21c830c7da repo: Merge osbuilder/1.11.0-rc0 into kata-containers/2.0-dev
Merge osbuilder into kata-containers for 2.0 development cycle.

Signed-off-by: Salvador Fuentes <salvador.fuentes@intel.com>
2020-04-29 16:49:13 -05:00
Salvador Fuentes
715d342519 osbuilder: move code into tools directory
move all osbuilder files into `tools` directory to be able
to merge this into kata-containers repo.

Signed-off-by: Salvador Fuentes <salvador.fuentes@intel.com>
2020-04-29 16:45:00 -05:00
Julio Montes
fb6765973f kernel: boot directly into the uncompressed kernel
QEMU >= 4.0 is able to boot into the uncompressed kernel using the PVH
entry point, but to get this `CONFIG_PVH` must be enabled in the guest
kernel and `pvh.bin` installed in the host.

Booting uncompressed kernels in QEMU 5.0 can reduce the memory footprint,
~17% for KSM and ~15% nonKSM.

fixes #1029

Signed-off-by: Julio Montes <julio.montes@intel.com>
2020-04-29 16:24:19 +00:00
Jose Carlos Venegas Munoz
66fe1d2a69
Merge pull request #199 from bergwolf/runtime-1.11.0-rc0
merge Runtime 1.11.0 rc0
2020-04-29 08:24:20 -05:00
Peng Tao
241d52d4fa travis: enable runtime
Run make and test for both agent and runtime.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2020-04-29 02:44:12 -07:00
Peng Tao
359b78c1c9 runtime: fix cli non-root ci
It turns out kata check needs root privilege to
call modprobe otherwise it fails.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2020-04-29 02:40:33 -07:00
Peng Tao
a5589196ce runtime: fix TestMainBeforeSubCommands
It is simply wrong to test kata-check within before sub commands
as it is NOT before at all. Besides it causes errors if kata is
not installed.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2020-04-29 02:40:33 -07:00
Peng Tao
3b5768e6e0 runtime: remove unneeded test conf
We don't really use them anymore.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2020-04-27 20:52:12 -07:00
Peng Tao
4fceb03d96 markdown: fix markdown checke failures
Pass markdown checker.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2020-04-27 19:50:59 -07:00
Peng Tao
93b72558ad runtime: enable make file
go test needs explicit -mod=vendor argument.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2020-04-27 19:50:45 -07:00
Peng Tao
426a9cab6e runtime: fix agnet ut
We should not use a plain unix socket reader to act as grpc
server. Place a really mock grpc server instead.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2020-04-27 19:50:37 -07:00
Peng Tao
e540648950 vendor: move to go mod
Get rid of go dep finally ;)

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2020-04-27 19:50:29 -07:00
Peng Tao
98e3e99843 Merge remote-tracking branch 'runtime/1.11.0-rc0' into runtime-1.11.0-rc0
Merge runtime 1.11.0-rc0 in.

Fixes: #192
Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2020-04-27 19:49:18 -07:00
Peng Tao
a02a8bda66 runtime: move all code to src/runtime
To prepare for merging into kata-containers repository.

Signed-off-by: Peng Tao <bergwolf@hyper.sh>
2020-04-27 19:39:25 -07:00
Yang Bo
7468750442
Merge pull request #177 from lifupan/upstream_ttrpc
Upstream ttrpc
2020-04-27 20:14:17 +08:00
Salvador Fuentes
c5f0beeacd
Merge pull request #654 from amshinde/add-containerd-annotation-config
annotations: Add config documentation for containerd
2020-04-23 10:46:04 -05:00
James O. D. Hunt
56d7074180
Merge pull request #1006 from Pennyzct/kernel_fragment_on_aarch64
AArch64: Enable kernel fragment on aarch64
2020-04-23 08:42:17 +01:00
GabyCT
01ecb62a48
Merge pull request #652 from GabyCT/topic/updateslesdocum
doc: Fix SLES installation documentation
2020-04-22 09:21:30 -05:00
fupan.lfp
3ea23a9a5d add license headers to proto files
add license headers to proto files

Fixes: #148

Signed-off-by: fupan.lfp <fupan.lfp@antfin.com>
2020-04-22 22:12:38 +08:00
lifupan
a7041c27dc agent: switch from grpc to ttrpc as agent protocol
switch from grpc to ttrpc
Fixes: #148

Signed-off-by: lifupan <lifupan@gmail.com>
2020-04-22 22:12:18 +08:00
Archana Shinde
2baee26ee3 annotations: Add config documentation for containerd
Support for passing sandbox annotations to the OCI layer was added
in containerd 1.3.0. Add this to the docs along with configuration
changes needed.

Fixes #653

Signed-off-by: Archana Shinde <archana.m.shinde@intel.com>
2020-04-21 11:41:56 -07:00
Gabriela Cervantes
9c332adc25 doc: Fix SLES installation documentation
This PR updates the current version of the SLES obs packages that are
being generated.

Fixes #651

Signed-off-by: Gabriela Cervantes <gabriela.cervantes.tellez@intel.com>
2020-04-21 10:43:55 -05:00
Penny Zheng
14a281d646 config: update arm64_kata_kvm_5.4.x
The config file created by kernel fragments scheme is quite different
with the old arm64_kata_kvm_5.4.x.
So I will update arm64_kata_kvm_5.4.x for consistency.

Fixes: #1004

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
2020-04-21 17:51:11 +08:00
Penny Zheng
a2dc502e48 configs: enable kvm-ptp on arm64
kvm-ptp is critical for mitigating time drift between host and guest.
This implementation in kernel side is still one experimental feature on
aarch64, and see https://github.com/kata-containers/packaging/pull/998
for detailed instructions.

Fixes: #1004

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2020-04-21 17:50:37 +08:00
Penny Zheng
5a0e9cc2d3 config: disable pci shpc hotplug for arm64
Since we disable pci shpc hotplug for arm64, see
https://github.com/kata-containers/packaging/pull/498 for detailed
reason.
We need to move CONFIG_HOTPLUG_PCI_SHPC from common conf to
x86_64-specific.

Fixes: #1004

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2020-04-21 14:48:02 +08:00
Penny Zheng
f188ef7f0e config: fix typo error
fix a few typo errors.

Fixes: #1004

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
2020-04-21 14:47:58 +08:00
Penny Zheng
247a84be9c arm64/mm: backport memory hot remove patch series on aarch64
Backport Anshuman Khandual's patch series of Enabling memory hot
remove on aarch64(https://patchwork.kernel.org/cover/11419305/)
to v5.4.x.
XONE_DEVICE is dependent on the implementation of memory hot remove.
This patch series has already been merged, and queued for 5.7.
After backporting this series, we could finally enable nvdimm/dax
on arm64.

Fixes: #1004

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
2020-04-21 14:47:53 +08:00
Penny Zheng
42e56a6309 config: add arm64-specific configs
Add a few arm64-specific configs and classify them into seven new categories
, that is,
1. base architecture-dependent options(base.conf)
It also includes varient-specific features, like CONFIG_ARM64_PMEM is
one ARMv8.2 arichitectural features.
2. crypto-related options(crypto.conf)
ARMv8 adds cryptographic instructions that could significantly improve
performance on tasks such as AES encryption and SHA1 and SHA256 hashing.
3. device tree related options(dt.conf)
The "Open Firmware Device Tree", or simply Device Tree (DT), is a data
structure and language for describing hardware, which is commonly
used in arm architecture.
4. ARM errata workarounds options(errata.conf)
There are many Kconfig entires under "Kernel Features" ->
"ARM errata workarounds via the alternatives framework", which provides
software workarounds to mitigate systems affected by those erratum.
Vendor-specific option will be left to users to decide.
5. pci related options(pci.conf)
a simplified pci host controller for mach-virt.
6. serial devices options(serial.conf)
CONFIG_SERIAL_OF_PLATFORM is used for all 8250 compatible serial ports
that are probed through device tree.
7. rtc related options(rtc.conf)
we don't have KVM’s paravirtualized clock and ptp implementation is
still under experimental mode, so we need rtc on aarch64.
QEMU provides an emulated ARM AMBA PrimeCell PL031 RTC.

Fixes: #1004

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
2020-04-21 14:47:42 +08:00
Penny Zheng
9784bb3e6f configs: add CONFIG_COMPACTION
Compaction is the only memory management component to form high order
(larger physically contiguous) memory blocks reliably.
The page allocator relies on compaction heavily and the lack of the feature
can lead to unexpected OOM killer invocations for high order memory requests.
We shouldn't disable this option unless there really is a strong reason.

Fixes: #1004

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
2020-04-21 14:47:35 +08:00
Penny Zheng
339d2e70e6 config: move mmio-related configs to common dir
mmio devices are required in firecracker, and for now, x86_64 and
aarch64 are all supporting kata containers with firecracker.
So, we need to move mmio-related configs to common dir.

Fixes: #1004

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
2020-04-21 14:47:31 +08:00
Penny Zheng
538528fa9a configs: re-organize security-related configs
There exists a few security-related configs, which are x86-64 specific.
CONFIG_LEGACY_VSYSCALL_NONE=y
CONFIG_RETPOLINE=y

CONFIG_RELOCATABLE and CONFIG_RANDOMIZE_BASE are kinds of tangled on
aarch64, if CONFIG_RANDOMIZE_BASE=y, then CONFIG_RELOCATABLE will be
selected automatically.
CONFIG_RANDOMIZE_BASE will randomize the virtual address at which the
kernel image is loaded, which as a security feature could deter exploit
attempts relying on knowledge of the location of kernel internals.

Fixes: #1004

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
2020-04-21 14:47:27 +08:00
Penny Zheng
beee33b1a4 configs: re-locate configs about linux guest support and optimization
There exists a few configs about linux guest support or optimization
that are not supported on aarch64.
CONFIG_HYPERVISOR_GUEST is only defined under arch/x86/Kconfig and
unfortunately, CONFIG_KVM_GUEST is not supported on aarch64 for now.

Fixes: #1004

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
2020-04-21 14:47:21 +08:00
Penny Zheng
6b2d84b0dc config: re-organize acpi related config
For now, a few configs as follows in common acpi dir are truly x86-spcecific
or disable by default on arm64.
CONFIG_ACPI_CPU_FREQ_PSS=y
CONFIG_ACPI_HOTPLUG_IOAPIC=y
CONFIG_ACPI_LEGACY_TABLES_LOOKUP
CONFIG_ACPI_LPIT=y
CONFIG_ARCH_MIGHT_HAVE_ACPI_PDC=y
CONFIG_ACPI_PROCESSOR_CSTATE=y
CONFIG_ACPI_SYSTEM_POWER_STATES_SUPPORT=y
CONFIG_HAVE_ACPI_APEI_NMI=y
And I also add a few configs which are aarch64-specific.
Like CONFIG_ACPI_REDUCED_HARDWARE_ONLY=y, since ARM64 can run properly
in ACPI hardware reduced mode.

Fixes: #1004

Signed-off-by: Penny Zheng <penny.zheng@arm.com>
2020-04-21 14:47:15 +08:00
Julio Montes
fc8c3a024e
Merge pull request #998 from jongwu/ptp
kernel: Enable ptp_kvm for 5.4 kernel on arm64.
2020-04-20 09:25:34 -05:00
Salvador Fuentes
3b98b259b4
Merge pull request #2626 from chavafg/1.11.0-rc0-branch-bump
# Kata Containers 1.11.0-rc0
2020-04-17 20:14:34 -05:00
Salvador Fuentes
c06d1d65d1
Merge pull request #1019 from chavafg/1.11.0-rc0-branch-bump
# Kata Containers 1.11.0-rc0
2020-04-17 18:21:06 -05:00
Archana Shinde
c95d09a34d
Merge pull request #181 from chavafg/1.11.0-rc0-branch-bump
# Kata Containers 1.11.0-rc0
2020-04-17 14:55:51 -07:00
Salvador Fuentes
eb128f8558
Merge pull request #444 from chavafg/1.11.0-rc0-branch-bump
# Kata Containers 1.11.0-rc0
2020-04-17 16:25:20 -05:00
Salvador Fuentes
7537ad3a7c
Merge pull request #647 from chavafg/1.11.0-rc0-branch-bump
# Kata Containers 1.11.0-rc0
2020-04-17 16:10:21 -05:00
Salvador Fuentes
21711eea08 release: Kata Containers 1.11.0-rc0
- makefile: Make SELinux support configurable
- clh: Boot from persistent memory device
- config: Add scsi_mod.scan=none for virtio-scsi
- katautils: Use config paths set during the build
- version: Update kernel to lts 5.4.32
- clh: virtiofs: Add no_posix_lock option
- versions: Switch to virtio-fs-dev branch for kernel
- v2: Open log fifo with `RDWR` instead of `WRONLY`
- qemu-ppc64le: Switch off large decrementer capability
- versions: Update go to 1.13.9
- qemu_ppc64le: EXpose fs support explicitly
- qemu: Don't crash if virtiofsd path is non existent
- Add SELinux support for running VM Confinement
- clh:  Implment capabilities
- Update go to v1.13.8
- Makefile: Allow change default hypervisor via env var
- clh: Report warning when requested vCPUs exceeds maxVCPU allowed
- clh: Enable memory hotplug
- virtcontainers: check PCI resource format before using it
- Support persistent memory volumes
- versions: Update containerd commit
- virtcontainers: Don't create vfio devices in the guest
- shimv2: move container rootfs mounted flag to container level
- AArch64: officially enable firecracker v0.21.0 on AArch64
- clh: add vfio support

d78ffd65 makefile: Make SELinux support configurable
7aa31685 clh: Boot from persistent memory device
e8fc25a7 version: Update clh to master
bf9758bf katautils: Use config paths set during the build
8c850d9e config: Add scsi_mod.scan=none for virtio-scsi
07d0a4f0 version: Update kernel to lts 5.4.32
ab8050c5 kata_agent: Don't use dax if virtio_fs_cache is 0
6218b2a5 kata_agent: Remove sharedDirVirtioFSOptions
95ccc0f7 agent: Use "virtiofs" instead of "virtio_fs"
4c1cacd3 versions: Switch to virtio-fs-dev branch for kernel
8e0f891e v2: Open log fifo with `RDWR` instead of `WRONLY`
afbd03cf qemu-ppc64le: Switch off large decrementer capability
432f9bea clh: virtiofs: Add no_posix_lock option
0294fcb9 versions: Update go to 1.13.9
fd625b3f qemu: Don't crash if virtiofsd path is non existent
5eec8bdf qemu_ppc64le: EXpose fs support explicitly
e4eb553d virtcontainers: Add SELinux support for running VM Confinement
39e354f6 clh: Implement capabilities
0a1ffc1d types: Make FS sharing disable by default
669b6e32 clh: Report warning when requested vCPUs exceeds maxVCPU allowed
7997218c Makefile: Allow change default hypervisor via env var
aab82f67 clh: Add memory hotplug
e62a8aa9 versions: Update containerd commit
2f948738 clh: Use MemUnit to create VM
b6a7d8d6 utils: Add memory unit abstraction
5e7d2538 clh: add vmInfo method
ebb8fd57 versions: Update clh to latest master
4d2574a7 virtcontainers: Don't create vfio devices in the guest
3b53114a virtcontainers: improve algorithm to check Large bar devices
7aff5466 virtcontainers: check PCI resource format before using it
d0a730c6 shimv2: move container rootfs mounted flag to container level
d60902a9 FC: change minimum supported version of Firecracker to v0.21.1
aadf8c4a AArch64: enable firecracker v0.21.1 on AArch64
44e23493 FC: Fix error of overlong firecracker API unix socket
c3bafd57 FC: Change default API socket path
2945bcd7 FC: Removed redundant `--seccomp-level` jailer parameter
d2cae59e FC: Removed redundant `RescanBlockDevice` action
37b91b33 FC: Remove `logger.options`
2c310fec virtcontainers: handle persistent memory volumes
434b3025 virtcontainers: hotplug block drives that are pmem devices as nvdimm
84e0ee13 virtcontainers: reimplement `createBlockDevices`
abbdf078 virtcontainers: add Pmem attribute to BlockDrive
ee941e5c virtcontainers: Implement function to get the pmem DeviceInfo
9ff44dba virtcontainers: implement function to get the backing file
0a4e2edc virtcontainers: move GetDevicePathAndFsType to utils_linux
2c7f27ec vendor: update govmm
f61eca89 clh: Add comments around clh api
6a4e667f virtiofsd: Check if PID is valid
3251beaa version: Update clh to master
c5184641 clh: Add vfio support
4d034b1e versions: update go to v1.14

Signed-off-by: Salvador Fuentes <salvador.fuentes@intel.com>
2020-04-17 17:51:39 +00:00
Salvador Fuentes
b0c76b0e8f release: Kata Containers 1.11.0-rc0
- release: Tag and fork documentation repo as part of release
- obs: let patch set in order before apply them
- scripts: Disable pie for qemu when static building
- kernel: Enable CONFIG_VIRTIO_PMEM for booting from pmem
- kernel: Fix patch ordering
- tests: Remove performing updates in Fedora dockerfile
- kata-deploy: fix k3s containerd check
- scripts: update configuration script to support QEMU 5.0
- obs: Update SLES version for packaging
- config: enable printk-time for kernel-5.4 for arm64
- actions: change trigger phrase for kata-deploy action
- kernel: enable virtio-fs for arm64.
- add kernel config for gpu
- Optimize the kata qemu binary size
- obs: Remove OpenSUSE Leap 15.0 from obs generation
- pod : optimization Some debian package manager tweaks

d271ee7 obs: let patch set in order before apply them
fbad186 kernel: Enable CONFIG_VIRTIO_PMEM for booting from pmem
652d1fd release: Tag and fork documentation repo as part of release
7e22144 scripts: Disable pie for qemu when static building
93da145 kernel: Fix patch ordering
59f7678 tests: Remove performing updates in Fedora dockerfiles
96f3b99 kata-deploy: fix k3s containerd check
fb42e38 scripts: update configuration script to support QEMU 5.0
9bdc51c obs: Update SLES version for packaging
32986db config: enable printk-time for kernel-5.4 for arm64
9b8f20c kernel: enable virtio-fs for arm64.
12d351d kernel: add usage in readme
1389500 kernel: support force setup
7a17b50 kernel: support bash debug
d248e41 kernel: support build guest kernel for gpu
cbfc7a1 obs: Remove OpenSUSE Leap 15.0 from obs generation
9a6bd12 debian: Install missing ca-certificates package
d527c4f debian: Don't install recommended software
3670074 scripts: Disable a few options to reduce qemu binary size on generic architectures
711eae6 scripts: Set --enable-pie on aarch64 arch
7cdf113 scripts: Relax the version limitation for qemu
0871391 scripts: Remove obsoleted --disable-uuid
878a223 scripts: Disable xen when builing qemu on generic architectures
e92f3db actions: change trigger phrase for kata-deploy action

Signed-off-by: Salvador Fuentes <salvador.fuentes@intel.com>
2020-04-17 17:51:22 +00:00
Salvador Fuentes
b7d1e30c9f release: Kata Containers 1.11.0-rc0
- tests: deleting stale test results when tests failed
- image_builder: Reduce the boundary mb for reducing image size on arm64
- initrd-builder: Don't error if run as non-root
- s390x: Skip rust for s390x
- image_builder: Force mount_dir to be created in /tmp

c29dbae tests: deleting when tests failed
2ac3090 s390x: Skip rust for s390x
9665563 image_builder: Force mount_dir to be created in $TMPDIR
6cae294 initrd-builder: Don't error if run as non-root
005c62a image_builder: Reduce the boundary mb for reducing image size on arm64

Signed-off-by: Salvador Fuentes <salvador.fuentes@intel.com>
2020-04-17 17:51:16 +00:00
Salvador Fuentes
63d9a8696f release: Kata Containers 1.11.0-rc0
- Fix potentianl crash
- sandbox: fix the issue of missing setting hostname
- unify the rustjail's log to contain container id and exec id
- Refactor the way of creating container process

ba3c732 grpc: fix the issue of potential crashes
32431d7 rpc: fix the issue of kill container process
986e666 sandbox: fix the issue of missing setting hostname
7d9bdf7 grpc: Fix the issue passing wrong exec_id to exec process
9220fb8 rustjail: unify the rustjail's log to contain container id and exec id
c1b6838 rustjail: refactoring the way of creating container process
e56b10f rustjail: remove the unused imported crates
ded27f4 oci: add Default and Clone to oci spec objects
7df8ede rustjail: replace protocol spec with oci spec

Signed-off-by: Salvador Fuentes <salvador.fuentes@intel.com>
2020-04-17 17:51:05 +00:00
Salvador Fuentes
36ee5c8b9d release: Kata Containers 1.11.0-rc0
Version bump no changes

Signed-off-by: Salvador Fuentes <salvador.fuentes@intel.com>
2020-04-17 17:50:59 +00:00
Archana Shinde
e72b3d7340
Merge pull request #1008 from amshinde/tag-documentation-repo
release: Tag and fork documentation repo as part of release
2020-04-17 10:34:40 -07:00
Jianyong Wu
3a0d53dba4 kernel: Enable ptp_kvm for 5.4 kernel on arm64.
The enablement of ptp_kvm for arm is under review, see [1].
So we have to apply private patch to enable it in 5.4 kernel.
ptp_kvm can offer the capability of time sync in kata even there
is no network available and higher precision than time sync
service depend on network.

note:
If you want to use this feature on your arm machine, the host kernel
also need apply this patch. we recommend that your host kernel version
is the 5.4, then you can apply this patch smoothly.

[1] https://patchwork.kernel.org/cover/11372743/

Fixes: #997
Signed-off-by: Jianyong Wu <jianyong.wu@arm.com>
2020-04-17 21:49:10 +08:00
Fabiano Fidêncio
33e5ab1bf6
Merge pull request #2624 from andreabolognani/feature-selinux
makefile: Make SELinux support configurable
2020-04-17 15:04:04 +02:00
Julio Montes
6988935b0f
Merge pull request #1016 from jongwu/order
obs: let patch set in order before apply them
2020-04-17 07:10:34 -05:00
Andrea Bolognani
d78ffd653d makefile: Make SELinux support configurable
SELinux support requires libselinux to be available, but that's
not the only factor: for example, Fedora 31 has libselinux but not
a version of selinux-policy that knows about Kata containers, so
enabling SELinux support by default in that case causes usability
issues.

Another issue with the current implementation is that, when
libselinux is absent, SELinux support will be quietly disabled,
which might not be what the user (or packager) intended.

To solve both problems, introduce the new FEATURE_SELINUX user
variable. This variable takes one of three values:

  * check (default): keep the current behavior;

  * yes: enable SELinux support, erroring out if libselinux is
         not present on the system;

  * no: disable SELinux support.

In the future we might want to formalize support for optional
build-time features, but for now this will do.

Fixes: #2623

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
2020-04-17 12:03:22 +02:00