62b5917526
This is a double bump.
Changes 0.0.20171122:
* chacha20poly1305: fast primitives from Andy Polyakov
Samuel Neves and I have spent considerable time and headaches porting,
reworking, and partially rewriting Andy's optimized implementations of
ChaCha20 and Poly1305. We now support the following:
On x86_64:
- Poly1305: integer unit
- ChaCha20: SSSE3
- HChaCha20: SSSE3
- Poly1305: AVX
- ChaCha20: AVX2
- Poly1305: AVX2
- ChaCha20: AVX512
- Poly1305: AVX512
On ARM:
- Poly1305: integer unit
- ChaCha20: NEON
- Poly1305: NEON
On ARM64:
- Poly1305: integer unit
- ChaCha20: NEON
- Poly1305: NEON
On MIPS64:
- Poly1305: integer unit
All others:
- ChaCha20: generic C
- Poly1305: generic C
This is a pretty substantial amount of new handrolled assembly. It will
perhaps MURDER KITTENS, so please tread lightly with this snapshot and adjust
expectations accordingly. I'm looking forward to quickly fixing any issues
folks find while testing.
Performance-wise, this should see increases all around. The biggest speedups
will be on ARM and ARM64, but x86_64 and MIPS64 should also see modest speed
improvements too, especially on Skylake systems supporting AVX512.
* chacha20poly1305: add more test vectors, some of which are weird
Test vectors are pretty important, so we added more to catch odd edge cases
using the following butcher's code:
from cryptography.hazmat.primitives.ciphers.aead import ChaCha20Poly1305
import os
def encode_blob(blob):
a = ""
for i in blob:
a += "\\x" + hex(i)[2:]
return a
enc = [ ]
dec = [ ]
def make_vector(plen, adlen):
key = os.urandom(32)
nonce = os.urandom(8)
p = os.urandom(plen)
ad = os.urandom(adlen)
c = ChaCha20Poly1305(key).encrypt(nonce=bytes(4) + nonce, data=p, associated_data=ad)
out = "{\n"
out += "\t.key\t= \"" + encode_blob(key) + "\",\n"
out += "\t.nonce\t= \"" + encode_blob(nonce) + "\",\n"
out += "\t.assoc\t= \"" + encode_blob(ad) + "\",\n"
out += "\t.alen\t= " + str(len(ad)) + ",\n"
out += "\t.input\t= \"" + encode_blob(p) + "\",\n"
out += "\t.ilen\t= " + str(len(p)) + ",\n"
out += "\t.result\t= \"" + encode_blob(c) + "\"\n"
out += "}"
enc.append(out)
out = "{\n"
out += "\t.key\t= \"" + encode_blob(key) + "\",\n"
out += "\t.nonce\t= \"" + encode_blob(nonce) + "\",\n"
out += "\t.assoc\t= \"" + encode_blob(ad) + "\",\n"
out += "\t.alen\t= " + str(len(ad)) + ",\n"
out += "\t.input\t= \"" + encode_blob(c) + "\",\n"
out += "\t.ilen\t= " + str(len(c)) + ",\n"
out += "\t.result\t= \"" + encode_blob(p) + "\"\n"
out += "}"
dec.append(out)
make_vector(0, 0)
make_vector(0, 8)
make_vector(1, 8)
make_vector(1, 0)
make_vector(129, 7)
make_vector(256, 0)
make_vector(512, 0)
make_vector(513, 9)
make_vector(1024, 16)
make_vector(1933, 7)
make_vector(2011, 63)
print("======== encryption vectors ========")
print(", ".join(enc))
print("\n\n\n======== decryption vectors ========")
print(", ".join(dec))
* wg-quick: document localhost exception and v6 rule
Probably a "kill switch" wants this too:
-m addrtype ! --dst-type LOCAL
so that basic local services can continue to work.
* selftest: allowedips: randomized test mutex update
* allowedips: do not write out of bounds
* device: uninitialize socket first in destruction
* tools: tighten up strtoul parsing
Small fixups.
* qemu: update kernel
* qemu: use unprefixed strip when not cross-compiling
Fedora/Redhat doesn't ship with a prefixed strip, and we don't need
to use it anyway when we're not cross compiling, so don't.
* compat: 3.16.50 got proper rt6_get_cookie
* compat: stable finally backported fix
* compat: new kernels have netlink fixes
* compat: fix compilation with PaX
Usual set of compatibility updates.
* curve25519-neon: compile in thumb mode
In thumb mode, it's not possible to use sp as an operand of and, so
we have to muck around with r3 as a scratch register.
* socket: only free socket after successful creation of new
When an interface is down, the socket port can change freely. A socket
will be allocated when the interface comes up, and if a socket can't be
allocated, the interface doesn't come up.
However, a socket port can change while the interface is up. In this
case, if a new socket with a new port cannot be allocated, it's
important to keep the interface in a consistent state. The choices are
either to bring down the interface or to preserve the old socket. This
patch implements the latter.
* global: switch from timeval to timespec
This gets us nanoseconds instead of microseconds, which is better, and
we can do this pretty much without freaking out existing userspace,
which doesn't actually make use of the nano/microseconds field. The below
test program shows that this won't break existing sizes:
zx2c4@thinkpad ~ $ cat a.c
void main()
{
puts(sizeof(struct timeval) == sizeof(struct timespec) ?
"success" : "failure");
}
zx2c4@thinkpad ~ $ gcc a.c -m64 && ./a.out
success
zx2c4@thinkpad ~ $ gcc a.c -m32 && ./a.out
success
Changes 0.0.20171127:
* compat: support timespec64 on old kernels
* compat: support AVX512BW+VL by lying
* compat: fix typo and ranges
* compat: support 4.15's netlink and barrier changes
* poly1305-avx512: requires AVX512F+VL+BW
Numerous compat fixes which should keep us supporting 3.10-4.15-rc1.
* blake2s: AVX512F+VL implementation
* blake2s: tweak avx512 code
* blake2s: hmac space optimization
Another terrific submission from Samuel Neves: we now have an implementation
of Blake2s using AVX512, which is extremely fast.
* allowedips: optimize
* allowedips: simplify
* chacha20: directly assign constant and initial state
Small performance tweaks.
* tools: fix removing preshared keys
* qemu: use netfilter.org https site
* qemu: take shared lock for untarring
Small bug fixes.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
LinuxKit
LinuxKit, a toolkit for building custom minimal, immutable Linux distributions.
- Secure defaults without compromising usability
- Everything is replaceable and customisable
- Immutable infrastructure applied to building Linux distributions
- Completely stateless, but persistent storage can be attached
- Easy tooling, with easy iteration
- Built with containers, for running containers
- Designed for building and running clustered applications, including but not limited to container orchestration such as Docker or Kubernetes
- Designed from the experience of building Docker Editions, but redesigned as a general-purpose toolkit
- Designed to be managed by external tooling, such as Infrakit or similar tools
- Includes a set of longer-term collaborative projects in various stages of development to innovate on kernel and userspace changes, particularly around security
Subprojects
- LinuxKit kubernetes aims to build minimal and immutable Kubernetes images. (previously
projects/kubernetesin this repository).
Getting Started
Build the linuxkit tool
LinuxKit uses the linuxkit tool for building, pushing and running VM images.
Simple build instructions: use make to build. This will build the tool in bin/. Add this
to your PATH or copy it to somewhere in your PATH eg sudo cp bin/* /usr/local/bin/. Or you can use sudo make install.
If you already have go installed you can use go get -u github.com/linuxkit/linuxkit/src/cmd/linuxkit to install the linuxkit tool.
On MacOS there is a brew tap available. Detailed instructions are at linuxkit/homebrew-linuxkit,
the short summary is
brew tap linuxkit/linuxkit
brew install --HEAD linuxkit
Build requirements from source:
- GNU
make - Docker
- optionally
qemu
Building images
Once you have built the tool, use
linuxkit build linuxkit.yml
to build the example configuration. You can also specify different output formats, eg linuxkit build -format raw-bios linuxkit.yml to
output a raw BIOS bootable disk image, or linuxkit build -format iso-efi linuxkit.yml to output an EFI bootable ISO image. See linuxkit build -help for more information.
Booting and Testing
You can use linuxkit run <name> or linuxkit run <name>.<format> to execute the image you created with linuxkit build <name>.yml.
This will use a suitable backend for your platform or you can choose one, for example VMWare.
See linuxkit run --help.
Currently supported platforms are:
- Local hypervisors
- Cloud based platforms:
- Baremetal:
- x86 and arm64 servers on packet.net
- Raspberry Pi Model 3b
Running the Tests
The test suite uses rtf To
install this you should use make bin/rtf && make install. You will
also need to install expect on your system as some tests use it.
To run the test suite:
cd test
rtf -x run
This will run the tests and put the results in a the _results directory!
Run control is handled using labels and with pattern matching. To run add a label you may use:
rtf -x -l slow run
To run tests that match the pattern linuxkit.examples you would use the following command:
rtf -x run linuxkit.examples
Building your own customised image
To customise, copy or modify the linuxkit.yml to your own file.yml or use one of the examples and then run linuxkit build file.yml to
generate its specified output. You can run the output with linuxkit run file.
The yaml file specifies a kernel and base init system, a set of containers that are built into the generated image and started at boot time. You can specify the type
of artifact to build with the moby tool eg linuxkit build -format vhd linuxkit.yml.
If you want to build your own packages, see this document.
Yaml Specification
The yaml format specifies the image to be built:
kernelspecifies a kernel Docker image, containing a kernel and a filesystem tarball, eg containing modules. The example kernels are built fromkernel/initis the baseinitprocess Docker image, which is unpacked as the base system, containinginit,containerd,runcand a few tools. Built frompkg/init/onbootare the system containers, executed sequentially in order. They should terminate quickly when done.servicesis the system services, which normally run for the whole time the system is upfilesare additional files to add to the image
For a more detailed overview of the options see yaml documentation
Architecture and security
There is an overview of the architecture covering how the system works.
There is an overview of the security considerations and direction covering the security design of the system.
Roadmap
This project was extensively reworked from the code we are shipping in Docker Editions, and the result is not yet production quality. The plan is to return to production quality during Q3 2017, and rebase the Docker Editions on this open source project during this quarter. We plan to start making stable releases on this timescale.
This is an open project without fixed judgements, open to the community to set the direction. The guiding principles are:
- Security informs design
- Infrastructure as code: immutable, manageable with code
- Sensible, secure, and well-tested defaults
- An open, pluggable platform for diverse use cases
- Easy to use and participate in the project
- Built with containers, for portability and reproducibility
- Run with system containers, for isolation and extensibility
- A base for robust products
Development reports
There are weekly development reports summarizing work carried out in the week.
FAQ
See FAQ.
Released under the Apache 2.0 license.