mirror of
https://github.com/k3s-io/kubernetes.git
synced 2025-08-05 10:19:50 +00:00
Update the mesos.md getting started guide
This commit is contained in:
Dr. Stefan Schimanski
parent
1c0b765df6
commit
dc7a9c8792
@ -17,26 +17,31 @@ Getting started with Kubernetes on Mesos
|
||||
<!-- TODO: Update, clean up. -->
|
||||
|
||||
Mesos allows dynamic sharing of cluster resources between Kubernetes and other first-class Mesos frameworks such as [Hadoop][1], [Spark][2], and [Chronos][3].
|
||||
Mesos also ensures applications from different frameworks running on your cluster are isolated and that resources are allocated fairly.
|
||||
Mesos also ensures applications from different frameworks running on your cluster are isolated and that resources are allocated fairly among them.
|
||||
|
||||
Running Kubernetes on Mesos allows you to easily move Kubernetes workloads from one cloud provider to another to your own physical datacenter.
|
||||
Mesos clusters can be deployed on nearly every IaaS cloud provider infrastructure or in your own physical datacenter. Kubernetes on Mesos runs on-top of that and therefore allows you to easily move Kubernetes workloads from one of these environments to the other.
|
||||
|
||||
This tutorial will walk you through setting up Kubernetes on a Mesos cluster.
|
||||
It provides a step by step walk through of adding Kubernetes to a Mesos cluster and running the classic GuestBook demo application.
|
||||
The walkthrough presented here is based on the v0.4.x series of the Kubernetes-Mesos project, which itself is based on Kubernetes v0.11.0.
|
||||
It provides a step by step walk through of adding Kubernetes to a Mesos cluster and starting your first pod with an nginx webserver.
|
||||
|
||||
**NOTE:** There are [known issues with the current implementation][11].
|
||||
Please [file an issue against the kubernetes-mesos project][12] if you have problems completing the steps below.
|
||||
**NOTE:** There are [known issues with the current implementation][7] and support for centralized logging and monitoring is not yet available.
|
||||
Please [file an issue against the kubernetes-mesos project][8] if you have problems completing the steps below.
|
||||
|
||||
### Prerequisites
|
||||
|
||||
* Understanding of [Apache Mesos][10]
|
||||
* Mesos cluster on [Google Compute Engine][5]
|
||||
* A VPN connection to the cluster.
|
||||
* Understanding of [Apache Mesos][6]
|
||||
* A running [Mesos cluster on Google Compute Engine][5]
|
||||
* A [VPN connection][10] to the cluster
|
||||
* A machine in the cluster which should become the Kubernetes *master node* with:
|
||||
* GoLang > 1.2
|
||||
* make (i.e. build-essential)
|
||||
* Docker
|
||||
|
||||
**Note**: You *can*, but you *don't have to* deploy Kubernetes-Mesos on the same machine the Mesos master is running on.
|
||||
|
||||
### Deploy Kubernetes-Mesos
|
||||
|
||||
Log into the master node over SSH, replacing the placeholder below with the correct IP address.
|
||||
Log into the future Kubernetes *master node* over SSH, replacing the placeholder below with the correct IP address.
|
||||
|
||||
```bash
|
||||
ssh jclouds@${ip_address_of_master_node}
|
||||
@ -45,74 +50,81 @@ ssh jclouds@${ip_address_of_master_node}
|
||||
Build Kubernetes-Mesos.
|
||||
|
||||
```bash
|
||||
$ git clone https://github.com/mesosphere/kubernetes-mesos k8sm
|
||||
$ mkdir -p bin && sudo docker run --rm -v $(pwd)/bin:/target \
|
||||
-v $(pwd)/k8sm:/snapshot -e GIT_BRANCH=release-0.4 \
|
||||
mesosphere/kubernetes-mesos:build
|
||||
$ git clone https://github.com/GoogleCloudPlatform/kubernetes
|
||||
$ cd kubernetes
|
||||
$ export KUBERNETES_CONTRIB=mesos
|
||||
$ make
|
||||
```
|
||||
|
||||
Set some environment variables.
|
||||
The internal IP address of the master may be obtained via `hostname -i`.
|
||||
|
||||
```bash
|
||||
$ export servicehost=$(hostname -i)
|
||||
$ export mesos_master=${servicehost}:5050
|
||||
$ export KUBERNETES_MASTER=http://${servicehost}:8888
|
||||
$ export KUBERNETES_MASTER_IP=$(hostname -i)
|
||||
$ export KUBERNETES_MASTER=http://${KUBERNETES_MASTER_IP}:8888
|
||||
```
|
||||
|
||||
### Deploy etcd
|
||||
Start etcd and verify that it is running:
|
||||
|
||||
```bash
|
||||
$ sudo docker run -d --hostname $(uname -n) --name etcd -p 4001:4001 -p 7001:7001 coreos/etcd
|
||||
$ sudo docker run -d --hostname $(uname -n) --name etcd -p 4001:4001 -p 7001:7001 quay.io/coreos/etcd:v2.0.12
|
||||
```
|
||||
|
||||
```bash
|
||||
$ sudo docker ps
|
||||
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
|
||||
fd7bac9e2301 coreos/etcd:latest "/etcd" 5s ago Up 3s 2379/tcp, 2380/... etcd
|
||||
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
|
||||
fd7bac9e2301 quay.io/coreos/etcd:v2.0.12 "/etcd" 5s ago Up 3s 2379/tcp, 2380/... etcd
|
||||
```
|
||||
It's also a good idea to ensure your etcd instance is reachable by testing it
|
||||
```bash
|
||||
curl -L http://$servicehost:4001/v2/keys/
|
||||
curl -L http://${KUBERNETES_MASTER_IP}:4001/v2/keys/
|
||||
```
|
||||
If connectivity is OK, you will see an output of the available keys in etcd (if any).
|
||||
|
||||
### Start Kubernetes-Mesos Services
|
||||
Start the kubernetes-mesos API server, controller manager, and scheduler on a Mesos master node:
|
||||
|
||||
Update your PATH to more easily run the Kubernetes-Mesos binaries:
|
||||
```bash
|
||||
$ ./bin/km apiserver \
|
||||
--address=${servicehost} \
|
||||
--mesos_master=${mesos_master} \
|
||||
--etcd_servers=http://${servicehost}:4001 \
|
||||
--service-cluster-ip-range=10.10.10.0/24 \
|
||||
--port=8888 \
|
||||
--cloud_provider=mesos \
|
||||
--v=1 >apiserver.log 2>&1 &
|
||||
|
||||
$ ./bin/km controller-manager \
|
||||
--master=$servicehost:8888 \
|
||||
--mesos_master=${mesos_master} \
|
||||
--v=1 >controller.log 2>&1 &
|
||||
|
||||
$ ./bin/km scheduler \
|
||||
--address=${servicehost} \
|
||||
--mesos_master=${mesos_master} \
|
||||
--etcd_servers=http://${servicehost}:4001 \
|
||||
--mesos_user=root \
|
||||
--api_servers=$servicehost:8888 \
|
||||
--v=2 >scheduler.log 2>&1 &
|
||||
$ export PATH="$(pwd)/_output/local/go/bin:$PATH"
|
||||
```
|
||||
Identify your Mesos master: depending on your Mesos installation this is either a `host:port` like `mesos_master:5050` or a ZooKeeper URL like `zk://zookeeper:2181/mesos`.
|
||||
In order to let Kubernetes survive Mesos master changes, the ZooKeeper URL is recommended for production environments.
|
||||
```bash
|
||||
$ export MESOS_MASTER=<host:port or zk:// url>
|
||||
```
|
||||
Create a cloud config file `mesos-cloud.conf` in the current directory with the following contents:
|
||||
```bash
|
||||
$ cat <<EOF >mesos-cloud.conf
|
||||
[mesos-cloud]
|
||||
mesos-master = ${MESOS_MASTER}
|
||||
EOF
|
||||
```
|
||||
|
||||
Also on the master node, we'll start up a proxy instance to act as a
|
||||
public-facing service router, for testing the web interface a little
|
||||
later on.
|
||||
Now start the kubernetes-mesos API server, controller manager, and scheduler on the master node:
|
||||
|
||||
```bash
|
||||
$ sudo ./bin/km proxy \
|
||||
--bind_address=${servicehost} \
|
||||
--etcd_servers=http://${servicehost}:4001 \
|
||||
--logtostderr=true >proxy.log 2>&1 &
|
||||
$ km apiserver \
|
||||
--address=${KUBERNETES_MASTER_IP} \
|
||||
--etcd-servers=http://${KUBERNETES_MASTER_IP}:4001 \
|
||||
--service-cluster-ip-range=10.10.10.0/24 \
|
||||
--port=8888 \
|
||||
--cloud-provider=mesos \
|
||||
--cloud-config=mesos-cloud.conf \
|
||||
--v=1 >apiserver.log 2>&1 &
|
||||
|
||||
$ km controller-manager \
|
||||
--master=${KUBERNETES_MASTER_IP}:8888 \
|
||||
--cloud-provider=mesos \
|
||||
--cloud-config=./mesos-cloud.conf \
|
||||
--v=1 >controller.log 2>&1 &
|
||||
|
||||
$ km scheduler \
|
||||
--address=${KUBERNETES_MASTER_IP} \
|
||||
--mesos-master=${MESOS_MASTER} \
|
||||
--etcd-servers=http://${KUBERNETES_MASTER_IP}:4001 \
|
||||
--mesos-user=root \
|
||||
--api-servers=${KUBERNETES_MASTER_IP}:8888 \
|
||||
--v=2 >scheduler.log 2>&1 &
|
||||
```
|
||||
|
||||
Disown your background jobs so that they'll stay running if you log out.
|
||||
@ -124,22 +136,20 @@ $ disown -a
|
||||
Interact with the kubernetes-mesos framework via `kubectl`:
|
||||
|
||||
```bash
|
||||
$ bin/kubectl get pods
|
||||
POD IP CONTAINER(S) IMAGE(S) HOST LABELS STATUS
|
||||
$ kubectl get pods
|
||||
NAME READY REASON RESTARTS AGE
|
||||
```
|
||||
|
||||
```bash
|
||||
$ bin/kubectl get services # your service IPs will likely differ
|
||||
NAME LABELS SELECTOR IP PORT
|
||||
kubernetes component=apiserver,provider=kubernetes <none> 10.10.10.2 443
|
||||
# NOTE: your service IPs will likely differ
|
||||
$ kubectl get services
|
||||
NAME LABELS SELECTOR IP(S) PORT(S)
|
||||
k8sm-scheduler component=scheduler,provider=k8sm <none> 10.10.10.113 10251/TCP
|
||||
kubernetes component=apiserver,provider=kubernetes <none> 10.10.10.1 443/TCP
|
||||
```
|
||||
Lastly, use the Mesos CLI tool to validate the Kubernetes scheduler framework has been registered and running:
|
||||
```bash
|
||||
$ mesos state | grep "Kubernetes"
|
||||
"name": "Kubernetes",
|
||||
```
|
||||
Or, look for Kubernetes in the Mesos web GUI by pointing your browser to
|
||||
`http://${mesos_master}`. Make sure you have an active VPN connection.
|
||||
|
||||
Lastly, look for Kubernetes in the Mesos web GUI by pointing your browser to
|
||||
`http://<mesos_master_ip:port>`. Make sure you have an active VPN connection.
|
||||
Go to the Frameworks tab, and look for an active framework named "Kubernetes".
|
||||
|
||||
## Spin up a pod
|
||||
@ -147,175 +157,57 @@ Go to the Frameworks tab, and look for an active framework named "Kubernetes".
|
||||
Write a JSON pod description to a local file:
|
||||
|
||||
```bash
|
||||
$ cat <<EOPOD >nginx.json
|
||||
{ "kind": "Pod",
|
||||
"apiVersion": "v1beta1",
|
||||
"id": "nginx-id-01",
|
||||
"desiredState": {
|
||||
"manifest": {
|
||||
"version": "v1beta1",
|
||||
"containers": [{
|
||||
"name": "nginx-01",
|
||||
"image": "nginx",
|
||||
"ports": [{
|
||||
"containerPort": 80,
|
||||
"hostPort": 31000
|
||||
}],
|
||||
"livenessProbe": {
|
||||
"enabled": true,
|
||||
"type": "http",
|
||||
"initialDelaySeconds": 30,
|
||||
"httpGet": {
|
||||
"path": "/index.html",
|
||||
"port": "8081"
|
||||
}
|
||||
}
|
||||
}]
|
||||
}
|
||||
},
|
||||
"labels": {
|
||||
"name": "foo"
|
||||
} }
|
||||
$ cat <<EOPOD >nginx.yaml
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: nginx
|
||||
spec:
|
||||
containers:
|
||||
- name: nginx
|
||||
image: nginx
|
||||
ports:
|
||||
- containerPort: 80
|
||||
EOPOD
|
||||
```
|
||||
|
||||
Send the pod description to Kubernetes using the `kubectl` CLI:
|
||||
|
||||
```bash
|
||||
$ bin/kubectl create -f nginx.json
|
||||
nginx-id-01
|
||||
$ kubectl create -f nginx.yaml
|
||||
pods/nginx
|
||||
```
|
||||
|
||||
Wait a minute or two while `dockerd` downloads the image layers from the internet.
|
||||
We can use the `kubectl` interface to monitor the status of our pod:
|
||||
|
||||
```bash
|
||||
$ bin/kubectl get pods
|
||||
POD IP CONTAINER(S) IMAGE(S) HOST LABELS STATUS
|
||||
nginx-id-01 172.17.5.27 nginx-01 nginx 10.72.72.178/10.72.72.178 cluster=gce,name=foo Running
|
||||
$ kubectl get pods
|
||||
NAME READY REASON RESTARTS AGE
|
||||
nginx 1/1 Running 0 14s
|
||||
```
|
||||
|
||||
Verify that the pod task is running in the Mesos web GUI. Click on the
|
||||
Kubernetes framework. The next screen should show the running Mesos task that
|
||||
started the Kubernetes pod.
|
||||
|
||||
## Run the Example Guestbook App
|
||||
## What next?
|
||||
|
||||
Following the instructions from the kubernetes-mesos [examples/guestbook][6]:
|
||||
Try out some of the standard [Kubernetes examples][9].
|
||||
|
||||
```bash
|
||||
$ export ex=k8sm/examples/guestbook
|
||||
$ bin/kubectl create -f $ex/redis-master.json
|
||||
$ bin/kubectl create -f $ex/redis-master-service.json
|
||||
$ bin/kubectl create -f $ex/redis-slave-controller.json
|
||||
$ bin/kubectl create -f $ex/redis-slave-service.json
|
||||
$ bin/kubectl create -f $ex/frontend-controller.json
|
||||
**NOTE:** Some examples require Kubernetes DNS to be installed on the cluster.
|
||||
Future work will add instructions to this guide to enable support for Kubernetes DNS.
|
||||
|
||||
$ cat <<EOS >/tmp/frontend-service
|
||||
{
|
||||
"id": "frontend",
|
||||
"kind": "Service",
|
||||
"apiVersion": "v1beta1",
|
||||
"port": 9998,
|
||||
"selector": {
|
||||
"name": "frontend"
|
||||
},
|
||||
"publicIPs": [
|
||||
"${servicehost}"
|
||||
]
|
||||
}
|
||||
EOS
|
||||
$ bin/kubectl create -f /tmp/frontend-service
|
||||
```
|
||||
|
||||
Watch your pods transition from `Pending` to `Running`:
|
||||
|
||||
```bash
|
||||
$ watch 'bin/kubectl get pods'
|
||||
```
|
||||
|
||||
Review your Mesos cluster's tasks:
|
||||
|
||||
```bash
|
||||
$ mesos ps
|
||||
TIME STATE RSS CPU %MEM COMMAND USER ID
|
||||
0:00:05 R 41.25 MB 0.5 64.45 none root 0597e78b-d826-11e4-9162-42010acb46e2
|
||||
0:00:08 R 41.58 MB 0.5 64.97 none root 0595b321-d826-11e4-9162-42010acb46e2
|
||||
0:00:10 R 41.93 MB 0.75 65.51 none root ff8fff87-d825-11e4-9162-42010acb46e2
|
||||
0:00:10 R 41.93 MB 0.75 65.51 none root 0597fa32-d826-11e4-9162-42010acb46e2
|
||||
0:00:05 R 41.25 MB 0.5 64.45 none root ff8e01f9-d825-11e4-9162-42010acb46e2
|
||||
0:00:10 R 41.93 MB 0.75 65.51 none root fa1da063-d825-11e4-9162-42010acb46e2
|
||||
0:00:08 R 41.58 MB 0.5 64.97 none root b9b2e0b2-d825-11e4-9162-42010acb46e2
|
||||
```
|
||||
The number of Kubernetes pods listed earlier (from `bin/kubectl get pods`) should equal to the number active Mesos tasks listed the previous listing (`mesos ps`).
|
||||
|
||||
Next, determine the internal IP address of the front end [service][7]:
|
||||
|
||||
```bash
|
||||
$ bin/kubectl get services
|
||||
NAME LABELS SELECTOR IP PORT
|
||||
kubernetes component=apiserver,provider=kubernetes <none> 10.10.10.2 443
|
||||
redismaster <none> name=redis-master 10.10.10.49 10000
|
||||
redisslave name=redisslave name=redisslave 10.10.10.109 10001
|
||||
frontend <none> name=frontend 10.10.10.149 9998
|
||||
```
|
||||
|
||||
Interact with the frontend application via curl using the front-end service IP address from above:
|
||||
|
||||
```bash
|
||||
$ curl http://${frontend_service_ip_address}:9998/index.php?cmd=get\&key=messages
|
||||
{"data": ""}
|
||||
```
|
||||
|
||||
Or via the Redis CLI:
|
||||
|
||||
```bash
|
||||
$ sudo apt-get install redis-tools
|
||||
$ redis-cli -h ${redis_master_service_ip_address} -p 10000
|
||||
10.233.254.108:10000> dump messages
|
||||
"\x00\x06,world\x06\x00\xc9\x82\x8eHj\xe5\xd1\x12"
|
||||
```
|
||||
#### Test Guestbook App
|
||||
Or interact with the frontend application via your browser, in 2 steps:
|
||||
|
||||
First, open the firewall on the master machine.
|
||||
|
||||
```bash
|
||||
# determine the internal port for the frontend service
|
||||
$ sudo iptables-save|grep -e frontend # -- port 36336 in this case
|
||||
-A KUBE-PORTALS-CONTAINER -d 10.10.10.149/32 -p tcp -m comment --comment frontend -m tcp --dport 9998 -j DNAT --to-destination 10.22.183.23:36336
|
||||
-A KUBE-PORTALS-CONTAINER -d 10.22.183.23/32 -p tcp -m comment --comment frontend -m tcp --dport 9998 -j DNAT --to-destination 10.22.183.23:36336
|
||||
-A KUBE-PORTALS-HOST -d 10.10.10.149/32 -p tcp -m comment --comment frontend -m tcp --dport 9998 -j DNAT --to-destination 10.22.183.23:36336
|
||||
-A KUBE-PORTALS-HOST -d 10.22.183.23/32 -p tcp -m comment --comment frontend -m tcp --dport 9998 -j DNAT --to-destination 10.22.183.23:36336
|
||||
|
||||
# open up access to the internal port for the frontend service
|
||||
$ sudo iptables -A INPUT -i eth0 -p tcp -m state --state NEW,ESTABLISHED -m tcp \
|
||||
--dport ${internal_frontend_service_port} -j ACCEPT
|
||||
```
|
||||
|
||||
Next, add a firewall rule in the Google Cloud Platform Console. Choose Compute >
|
||||
Compute Engine > Networks, click on the name of your mesosphere-* network, then
|
||||
click "New firewall rule" and allow access to TCP port 9998.
|
||||
|
||||
![Google Cloud Platform firewall configuration][8]
|
||||
|
||||
Now, you can visit the guestbook in your browser!
|
||||
|
||||
![Kubernetes Guestbook app running on Mesos][9]
|
||||
**NOTE:** Please be aware that there are [known issues with the current Kubernetes-Mesos implementation][7].
|
||||
|
||||
[1]: http://mesosphere.com/docs/tutorials/run-hadoop-on-mesos-using-installer
|
||||
[2]: http://mesosphere.com/docs/tutorials/run-spark-on-mesos
|
||||
[3]: http://mesosphere.com/docs/tutorials/run-chronos-on-mesos
|
||||
[4]: http://cloud.google.com
|
||||
[5]: https://cloud.google.com/compute/
|
||||
[6]: https://github.com/mesosphere/kubernetes-mesos/tree/v0.4.0/examples/guestbook
|
||||
[7]: https://github.com/GoogleCloudPlatform/kubernetes/blob/v0.11.0/docs/services.md#ips-and-vips
|
||||
[8]: mesos/k8s-firewall.png
|
||||
[9]: mesos/k8s-guestbook.png
|
||||
[10]: http://mesos.apache.org/
|
||||
[11]: https://github.com/mesosphere/kubernetes-mesos/blob/master/docs/issues.md
|
||||
[12]: https://github.com/mesosphere/kubernetes-mesos/issues
|
||||
|
||||
|
||||
[5]: http://open.mesosphere.com/getting-started/cloud/google/mesosphere/
|
||||
[6]: http://mesos.apache.org/
|
||||
[7]: https://github.com/mesosphere/kubernetes-mesos/blob/master/docs/issues.md
|
||||
[8]: https://github.com/mesosphere/kubernetes-mesos/issues
|
||||
[9]: ../../examples/
|
||||
[10]: http://open.mesosphere.com/getting-started/cloud/google/mesosphere/#vpn-setup
|
||||
|
||||
[]()
|
||||
|
||||
Loading…
Reference in New Issue
Block a user