Commit Graph

6765 Commits

Author SHA1 Message Date
David Scott
9bbc0ecbee projects/logging: delete the remaining unused bits
Signed-off-by: David Scott <dave.scott@docker.com>
2018-07-06 21:49:26 +01:00
David Scott
92a9a64a1b examples: add the example from projects/logging
This simplifies the example by adding a service which writes to the
log every 1s and a getty for introspection.

To see the logs:

  /proc/1/root/usr/bin/logread -F

Signed-off-by: David Scott <dave.scott@docker.com>
2018-07-06 21:49:26 +01:00
David Scott
a07ecf60d5 memlogd: use kmsg format for reading the logs
Switch to a more formally-specified `kmsg`-style format for reading
the logs.

- update the spec in docs/logging.md
- check for bad names in pkg/memlogd with unit test

Signed-off-by: David Scott <dave.scott@docker.com>
2018-07-06 21:49:26 +01:00
David Scott
fe64f33430 memlogd: add test cases
- check writing to the log does not block
- check the log doesn't expand -- it should be finite
- check that client connections don't buffer arbitrary amounts of
  data if the client is slow

Signed-off-by: David Scott <dave.scott@docker.com>
2018-07-06 21:49:26 +01:00
David Scott
6aca715ad8 memlogd: replace an unbounded buffer with bounded channel
Previously we had a per-connection

  bytes.Buffer // to be written to the connection
  sync.Cond    // to allow us to Wait for more data

This had the major disadvantage that the buffer was unbounded and so
a slow client could cause memory exhaustion in the server. This patch
replaces these with a single

  chan *logEntry

which is naturally bounded and supports blocking read. We make write
non-blocking using select i.e. we drop messages rather than allocate
more space.

Signed-off-by: David Scott <dave.scott@docker.com>
2018-07-06 21:49:26 +01:00
David Scott
3e742018d6 memlogd: use net.Conn rather than *net.UnixConn
There's no need to be so specific about the connection implementation.

Signed-off-by: David Scott <dave.scott@docker.com>
2018-07-06 21:49:25 +01:00
David Scott
c92af038fb projects/logging: promote pkg/memlogd to a toplevel package
This is an example external logging service which can be enabled by
adding it to the `init` section of the .yml, for example:

    ...
    init:
      - linuxkit/init:35866bb276c264a5f664bfac7456f4b9eeb87a4d
      - linuxkit/runc:v0.4
      - linuxkit/containerd:f2bc1bda1ab18146967fa1a149800aaf14bee81b
      - linuxkit/ca-certificates:v0.4
      - linuxkit/memlogd:cc035e5c9e4011ec1ba97a181a6689fc90965ce9
    onboot:
    ...

Signed-off-by: David Scott <dave.scott@docker.com>
2018-07-06 21:49:25 +01:00
David Scott
638c455dd2 docs: document how to use an external logging service
This is an updated version of projects/logging/README.md

Signed-off-by: David Scott <dave.scott@docker.com>
2018-07-06 21:49:25 +01:00
David Scott
f4bbce7a6c service: use the logging system with runc
If external logging is enabled, this patch sets the stdout and stderr
of the `runc` invocations to one end of a socketpair and the other end is
sent to the logging service. Otherwise we log to files as before.

Signed-off-by: David Scott <dave.scott@docker.com>
2018-07-06 21:49:21 +01:00
David Scott
4dc75bc67b service: use the logging system with containerd
An external logging system exists if the socket

  /var/run/linuxkit-external-logging.sock

exists.

If an external logging system is enabled then create FIFOs for
containerd and send the other end of the FIFOs to the logging service.
Otherwise use /var/log files as before.

Signed-off-by: David Scott <dave.scott@docker.com>
2018-07-06 21:48:57 +01:00
David Scott
1795c15276 projects/logging: start memlogd from init.d
Previously memlogd would always run in the foreground. This patch
adds a `-daemonize` option which binds the /var/run sockets, forks
and execs itself and immediately returns. Therefore the program won't
block (important for an init.d script) but guarantees the sockets will
be available for any program started afterwards.

This also removes the alpine base from the memlogd image as `init`
"containers" are treated as simple file overlays.

Signed-off-by: David Scott <dave.scott@docker.com>
2018-07-06 21:47:50 +01:00
David Scott
6dfd40a639 projects/linuxkit: bind /var/run in the memlogd container
We will place the control sockets in the root /var/run and then share
with all services who need access.

Signed-off-by: David Scott <dave.scott@docker.com>
2018-07-06 21:47:50 +01:00
David Scott
d09fbe04fb projects/logging: use /var/run for sockets by default
Previously we used /tmp, but /var/run is a better place for these.

Signed-off-by: David Scott <dave.scott@docker.com>
2018-07-06 21:47:49 +01:00
David Scott
8727bcb4f3 projects/logging: fix govet composite literal error
This fixes the error

net.UnixAddr composite literal uses unkeyed fields

Signed-off-by: David Scott <dave.scott@docker.com>
2018-07-06 21:47:49 +01:00
David Scott
6eabfdcb96 projects/logging: add modern build.yml and Dockerfile
Signed-off-by: David Scott <dave@recoil.org>
2018-07-06 21:47:49 +01:00
Rolf Neugebauer
2c8438f64b
Merge pull request #3093 from TiejunChina/master-dev
update -rt to 4.14.53-rt34
2018-07-06 11:11:03 +01:00
Tiejun Chen
a1d47ff766 update -rt to 4.14.53-rt34
Signed-off-by: Tiejun Chen <tiejun.china@gmail.com>
2018-07-06 14:02:07 +08:00
Rolf Neugebauer
9088e3f843
Merge pull request #3088 from rn/kernel-up
Update kernel to 4.17.4/4.14.53/4.9.11/4.4.139
2018-07-05 10:15:07 +01:00
Justin Cormack
e1cff3614a
Merge pull request #3090 from rn/june-report
June 2018 LinuxKit report
2018-07-05 10:00:06 +01:00
Rolf Neugebauer
977dec7ae2
Merge pull request #216 from justincormack/gnutar2
Use PAX tar format
2018-07-04 16:08:57 +01:00
Rolf Neugebauer
91bed426f1
Merge pull request #3089 from masterzen/fix/3087-faq-and-motd
Some FAQ and motd update
2018-07-04 15:05:54 +01:00
Brice Figureau
9a1d9fb1eb Add FAQ entries to help troubleshoot running containers
Since I struggled to understand and find information about how to
troubleshoot a running linuxkit instance, I propose to add these two
FAQ entries.
The first one explains why it is possible to not see the `containerd` or
`init` outputs at boot in the console.
The second one gives a few `ctr` example to list containers, running
containers or how to open a shell in a given container.

Signed-off-by: Brice Figureau <brice@daysofwonder.com>
2018-07-04 13:51:19 +02:00
Rolf Neugebauer
b859e7ab4d June 2018 LinuxKit report
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-07-04 11:24:25 +01:00
Rolf Neugebauer
498d4f1a91 Update YAMLs to latest kernels
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-07-04 10:38:55 +01:00
Brice Figureau
f7ff31382b Make sure motd contains the correct ctr command
The listed namespace wasn't correct, and thus the given `ctr` command
wasn't working.

Signed-off-by: Brice Figureau <brice@daysofwonder.com>
2018-07-04 11:32:36 +02:00
Rolf Neugebauer
909a9fac68
Merge pull request #3086 from mat007/fix-windows-build
Fix windows build
2018-07-03 18:19:46 +01:00
Rolf Neugebauer
dcc4a5e799 kernel: Update to 4.17.4/4.14.53/4.9.111/4.4.139
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-07-03 11:49:36 +01:00
Mathieu Champlon
d3e3df3d73 Append .exe to binaries on Windows
Signed-off-by: Mathieu Champlon <mathieu.champlon@docker.com>
2018-07-03 12:03:43 +02:00
Mathieu Champlon
82867aa114 Set proper GOOS for Windows
Signed-off-by: Mathieu Champlon <mathieu.champlon@docker.com>
2018-07-03 12:02:35 +02:00
Rolf Neugebauer
9dac66b831
Merge pull request #3078 from justincormack/dhcp-hook-fix
Fix dhcpcd hooks config
2018-06-29 19:02:24 +01:00
Justin Cormack
f6af74b6bf
Update hashes
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2018-06-29 16:40:52 +01:00
Justin Cormack
157c73cfed
Fix dhcpcd hooks config
These were being added to the incorrect directory.

Also move config file to /etc to be more standard.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2018-06-29 16:31:44 +01:00
Ian Campbell
380e24391d
Merge pull request #3084 from ijc/containerd-v1.1.0-rc.2
Containerd v1.1.0-rc.2
2018-06-29 13:30:04 +01:00
Ian Campbell
ad85828d87 Bump yml to pickup containerd v1.1.0-rc.2
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-06-29 10:48:10 +01:00
Ian Campbell
34666cd87d Bump packages to new alpine w/ containerd v1.1.0-rc.2
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-06-28 17:35:04 +01:00
Ian Campbell
779a611d6b alpine: Bump to containerd v1.1.0-rc.2
Signed-off-by: Ian Campbell <ijc@docker.com>
2018-06-28 12:34:50 -04:00
Rolf Neugebauer
b37527c23d
Merge pull request #3082 from rn/kup
Update kernels to 4.17.3/4.16.18/4.14.52/4.9.110 and then remove 4.16.x (EOLed)
2018-06-27 16:25:57 +01:00
Rolf Neugebauer
0ee4ab7b7f Update YAMLs to latest kernels
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-06-27 14:52:26 +01:00
Rolf Neugebauer
f7cf178ead kernel: Remove 4.16.x as it is EOLed
The previous commit updated to 4.16.18, which is the last
4.16.x kernel. The 4.16.18 kernel was compiled and pushed
but we may as well now remove it as it has been EOLed.

Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-06-27 14:49:08 +01:00
Rolf Neugebauer
e7a7fe9339 kernels: Update to 4.17.3/4.16.18/4.14.52/4.9.110
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-06-27 13:10:44 +01:00
Rolf Neugebauer
e5a3b36d38
Merge pull request #3080 from alanraison/3079-apk-architecture
Use apk --print-arch to discover architecture
2018-06-27 13:03:26 +01:00
Alan Raison
412857be96 Use apk --print-arch to discover architecture
When dealing with apk, `uname -m` doesn't always match the architecture
name that apk uses. Instead `apk --print-arch` is used.

Signed-off-by: Alan Raison <alanraison@users.noreply.github.com>
2018-06-26 23:16:20 +01:00
Rolf Neugebauer
ef47cd9c98
Merge pull request #3077 from zx2c4-forks/wireguard-bump
wireguard: upgrade to 0.0.20180625
2018-06-25 23:28:57 +01:00
Jason A. Donenfeld
bfd0ba0983 wireguard: upgrade to 0.0.20180625
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
2018-06-25 19:58:49 +02:00
Rolf Neugebauer
8f962b9e4a
Merge pull request #3076 from rn/kernel-up
kernel update to 4.16.17/4.14.51/4.4.137/4.4.138
2018-06-22 17:55:07 +01:00
Rolf Neugebauer
e36eae4820 Update YAMLs to latest kernels
Signed-off-by: Rolf Neugebauer <rolf.neugebauer@docker.com>
2018-06-22 16:12:34 +01:00
Justin Cormack
561e9375e8
Merge pull request #3075 from justincormack/binfmt_mount
Binfmt fixes
2018-06-22 16:09:34 +01:00
Justin Cormack
8ac642e4e0
Update yaml files for new package
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2018-06-22 14:18:31 +01:00
Justin Barrick
14ba30f418
Workaround DNS issue with qemu-user-static on alpine by using Debian in the first stage #2710
DNS lookups fail in qemu-user when it is built on Alpine: https://bugs.alpinelinux.org/issues/8131

Until this is resolved, we fetch the binaries from Debian and use those instead. The final stage
of the Dockerfile is still based on scratch.

We can revert this once the Alpine issue is fixed.

Signed-off-by: Justin Barrick <jbarrick@cloudflare.com>
2018-06-22 14:10:50 +01:00
Justin Cormack
a6dbe8ceba
Rather than bind mounting binfmt_misc filesystem, mount a copy.
For some reason, bind mounting does not always seem to work,
sometimes the filesystem is empty. Mounting a fresh copy seems
a better solution, and simplifies things. The container does
need `CAP_SYS_ADMIN` but only on boot.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2018-06-22 14:10:43 +01:00