Terminology
| Term | Definition |
|---|---|
| Dind | Docker in docker, this is the key to our realization of this feature. |
| User docker, UD | The user-docker on RancherOS |
| Other user docker, OUD | The other user-docker daemons we create, these user-docker daemons are Dind mode |
Solution Overview
Introduce some CLIs to allow users to create OUDs.
Rancher1.6 server and rancher-agent can work on user-docker and dind container.
Architecture
UD still runs in the console container, OUD runs on system-docker. Each OUD is a container, and has standclone fs and namespace. Users can access each OUD in the console container.
RancherOS
+-------------------------------------------+
| docker.sock |
| +------UD------+ | +--------------+ |
| | console ----+---> OUD container| |
| | container | | +--------------+ |
| +--------------+ | +--------------+ |
| +---> OUD container| |
| +--------------+ |
| +-------------------------------------+ |
| | +--------------+ | |
| | ROOTFS | system-docker| | |
| | +--------------+ | |
| +-------------------------------------+ |
| |
+-------------------------------------------+
How to run
Download ISO and install
Users can download the ISO here. https://storage.googleapis.com/ros-images/rancheros-mud-3.iso
If users want to use it on AWS, they should boot a VM with RancherOS1.4 and upgrade to a new version:
ros os upgrade -i niusmallnan/os:mud-3
Prepare
Users must switch user-docker to 17.12.1 or earlier version.
ros engine switch docker-17.12.1-ce
Otherwise, it may get these error when creating an user-defined network on system-docker.
Error response from daemon: unable to insert jump to DOCKER-ISOLATION rule in FORWARD chain: (iptables failed: iptables --wait -I FORWARD -j DOCKER-ISOLATION: iptables v1.6.0: Couldn't find target `DOCKER-ISOLATION'
Try `iptables -h' or 'iptables --help' for more information.
(exit status 2))
Create an user-define network, need to use this network when creating an OUD:
system-docker network create --subnet=172.20.0.0/16 dind
Create OUD
Just use ros engine create. For the OUD image, currently only support docker 17.12.1 and 18.03.1.
ros engine create dind1 --network=dind --fixed-ip=172.20.0.2
After the OUD service is created, users can query the OUD service as usual.
ros service list
...
...
disabled volume-efs
disabled volume-nfs
enabled dind1
To make the dind1 service running, can use:
ros service up dind1
After the OUD service is started, users can interact with it as if they were using the docker command.
docker-dind1 ps -a
SSH into OUD container
User can specify an external ssh port with --ssh-port, and ssh keys with --authorized-keys. Both of them are optional.
ros engine create -h
...
...
OPTIONS:
--ssh-port value
--authorized-keys value
For --authorized-keys, user needs to put the key file in one of the following directories:
/var/lib/rancher/
/opt/
/home/
We will generate a random password for each OUD container, which users can see in the container logs. This password is useful when the users do not set the keys.
system-docker logs dind1
======================================
chpasswd: password for 'root' changed
password: xCrw6fEG
======================================
Users can ssh into any OUD container like this:
system-docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
2ca07a25799b rancher/os-dind:17.12.1 "docker-entrypoint..." 5 seconds ago Up 3 seconds 2375/tcp, 0.0.0.0:34791->22/tcp dind1
ssh -p 34791 root@<host-external-ip>
ssh root@<OUD-container-ip>
Remove OUD
Just use ros engine rm:
ros engine rm dind1
Rancher1.6 setup
Run rancher/server with UD
docker run -d --restart=unless-stopped -p 8080:8080 rancher/server:stable
Run rancher/agent in OUD container
# exec into OUD container, also ssh
system-docker exec -it dind1 /bin/sh
# Add an agent to this rancher-server
# "-e CATTLE_CHECK_NAMESERVER=false" need to be added
docker run -e CATTLE_AGENT_IP="172.20.0.2" -e CATTLE_CHECK_NAMESERVER=false --rm --privileged -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/rancher:/var/lib/rancher rancher/agent:v1.2.10 http://<RancherOS-IPADDRESS>:8080/v1/scripts/xxxxxx
Limitations
As Rancher uses the overlay network by default, only one RancherOS host can be used in one Env.
Others
Please submit any bugs, issues for this feature to rancher/os, and add this label area/oud would be very friendly.