From 93908b9b1444cdd3b530e718024015068d055dc9 Mon Sep 17 00:00:00 2001 From: Elson Rodriguez Date: Sun, 4 Oct 2015 00:38:22 -0700 Subject: [PATCH 1/3] Adds example for running Selenium on Kubernetes. --- examples/selenium/README.md | 196 ++++++++++++++++++ examples/selenium/selenium-hub-rc.yaml | 36 ++++ examples/selenium/selenium-hub-svc.yaml | 15 ++ .../selenium/selenium-node-chrome-rc.yaml | 29 +++ .../selenium/selenium-node-firefox-rc.yaml | 29 +++ examples/selenium/selenium-test.py | 17 ++ 6 files changed, 322 insertions(+) create mode 100644 examples/selenium/README.md create mode 100644 examples/selenium/selenium-hub-rc.yaml create mode 100644 examples/selenium/selenium-hub-svc.yaml create mode 100644 examples/selenium/selenium-node-chrome-rc.yaml create mode 100644 examples/selenium/selenium-node-firefox-rc.yaml create mode 100644 examples/selenium/selenium-test.py diff --git a/examples/selenium/README.md b/examples/selenium/README.md new file mode 100644 index 00000000000..a8b2615ad4c --- /dev/null +++ b/examples/selenium/README.md @@ -0,0 +1,196 @@ + + + + +WARNING +WARNING +WARNING +WARNING +WARNING + +

PLEASE NOTE: This document applies to the HEAD of the source tree

+ +If you are using a released version of Kubernetes, you should +refer to the docs that go with that version. + + +The latest 1.0.x release of this document can be found +[here](http://releases.k8s.io/release-1.0/examples/redis/README.md). +Documentation for other releases can be found at +[releases.k8s.io](http://releases.k8s.io). + +-- + + + + + +## Selenium on Kubernetes + +Selenium is a browser automation tool used primarily for testing web applications. However when Selenium is used in a CI pipeline to test applications, there is often contention around the use of Selenium resources. This example shows you how to deploy Selenium to Kubernetes in a scalable fashion. + +### Prerequisites +This example assumes you have a working Kubernetes cluster and a properly configured kubectl client. See the [Getting Started Guides](../../docs/getting-started-guides/) for details. + +Google Container Engine is also a quick way to get Kubernetes up and running: https://cloud.google.com/container-engine/ + +Your cluster must have 4 CPU and 6 GB of RAM to complete the example up to the scaling portion. + +### Deploy Selenium Grid Hub: +We will be using Selenium Grid Hub to make our Selenium install scalable via a master/worker model. The Selenium Hub is the master, and the Selenium Nodes are the workers(not to be confused with Kubernetes nodes). We only need one hub, but we're using a replication controller to ensure that the hub is always running: +``` +kubectl create --filename=selenium-hub-rc.yaml +``` + +The Selenium Nodes will need to know how to get to the Hub, let's create a service for the nodes to connect to. +``` +kubectl create --filename=selenium-hub-svc.yaml +``` + +### Verify Selenium Hub Deployment +Let's verify our deployment of Selenium hub by connecting to the web console. + +#### Kubernetes Nodes Reachable +If your Kubernetes nodes are reachable from your network, you can verify the hub by hitting it on the nodeport. You can retrieve the nodeport by typing `kubectl describe svc selenium-hub`, however the snippet below automates that: +``` +export NODEPORT=`kubectl get svc --selector='name=selenium-hub' --output=template --template="{{ with index .items 0}}{{with index .spec.ports 0 }}{{.nodePort}}{{end}}{{end}}"` +export NODE=`kubectl get nodes --output=template --template="{{with index .items 0 }}{{.metadata.name}}{{end}}"` + +curl http://$NODE:$NODEPORT +``` + +#### Kubernetes Nodes Unreachable +If you cannot reach your Kubernetes nodes from your network, you can proxy via kubectl. +``` +export PODNAME=`kubectl get pods --selector="name=selenium-hub" --output=template --template="{{with index .items 0}}{{.metadata.name}}{{end}}"` +kubectl port-forward --pod=$PODNAME 4444:4444 +``` + +In a seperate terminal, you can now check the status. +``` +curl http://localhost:4444 +``` + +#### Using Google Container Engine +If you are using Google Container Engine, you can expose your hub via the internet. This is a bad idea for many reasons, but you can do it as follows: +``` +kubectl expose rc selenium-hub --name=selenium-hub-external --labels="name=selenium-hub-external,external=true" --create-external-load-balancer=true +``` + +Then wait a few minutes, eventually your new `selenium-hub-external` service will be assigned an load balancing IP from gcloud. +``` +export INTERNET_IP=`kubectl get svc --selector="name=selenium-hub-external" --output=template --template="{{with index .items 0}}{{with index .status.loadBalancer.ingress 0}}{{.ip}}{{end}}{{end}}"` + +curl http://$INTERNET_IP:4444/ +``` + +### Deploy Firefox and Chrome Nodes: +Now that the Hub is up, we can deploy workers. + +This will deploy 3 Chrome nodes. +``` +kubectl create -f selenium-node-chrome-rc.yaml +``` + +And 3 Firefox nodes to match. +``` +kubectl create -f selenium-node-firefox-rc.yaml +``` + +Once the pods start, you will see them show up in the Selenium Hub interface. + +### Run a Selenium Job +Let's run a quick Selenium job to validate our setup. + +#### Setup Python Environment +First, we need to start a python container that we can attach to. +``` +kubectl run selenium-python --image=google/python-hello +``` + +Next, we need to get inside this container. +``` +export PODNAME=`kubectl get pods --selector="run=selenium-python" --output=template --template="{{with index .items 0}}{{.metadata.name}}{{end}}"` +kubectl exec --stdin=true --tty=true $PODNAME bash +``` + +Once inside, we need to install the Selenium library +``` +pip install selenium +``` + +#### Run Selenium Job with Python +We're all set up, start the python interpreter. +``` +python +``` + +And paste in the contents of selenium-test.py. +```python +from selenium import webdriver +from selenium.webdriver.common.desired_capabilities import DesiredCapabilities + +def check_browser(browser): + driver = webdriver.Remote( + command_executor='http://selenium-hub:4444/wd/hub', + desired_capabilities=getattr(DesiredCapabilities, browser) + ) + driver.get("http://google.com") + assert "google" in driver.page_source + driver.close() + print("Browser %s checks out!" % browser) + + +check_browser("FIREFOX") +check_browser("CHROME") +``` + +You should get +``` +>>> check_browser("FIREFOX") +Browser FIREFOX checks out! +>>> check_browser("CHROME") +Browser CHROME checks out! +``` +Congratulations, your Selenium Hub is up, with Firefox and Chrome nodes! + +### Scale your Firefox and Chrome nodes. + +If you need more Firefox or Chrome nodes, your hardware is the limit: +``` +kubectl scale rc selenium-node-firefox --replicas=10 +kubectl scale rc selenium-node-chrome --replicas=10 +``` + +You now have 10 Firefox and 10 Chrome nodes, happy Seleniuming! + +### Debugging +Sometimes it is neccessary to check on a hung test. Each pod is running VNC. To check on one of the browser nodes via VNC, it's reccomended that you proxy, since we don't want to expose a service for every pod, and the containers have a weak password. Replace POD_NAME with the name of the pod you want to connect to. + +``` +kubectl port-forward --pod=POD_NAME 9000:5900 +``` + +Then connect to localhost:9000 with your VNC client using the password "secret" + +Enjoy your scalable Selenium Grid! + +Adapted from: https://github.com/SeleniumHQ/docker-selenium + +### Teardown + +To remove all created resources, run the following: + +``` +kubectl delete rc selenium-hub +kubectl delete rc selenium-node-chrome +kubectl delete rc selenium-node-firefox +kubectl delete rc selenium-python +kubectl delete svc selenium-hub +kubectl delete svc selenium-hub-external +``` diff --git a/examples/selenium/selenium-hub-rc.yaml b/examples/selenium/selenium-hub-rc.yaml new file mode 100644 index 00000000000..b1d53361cbe --- /dev/null +++ b/examples/selenium/selenium-hub-rc.yaml @@ -0,0 +1,36 @@ +apiVersion: v1 +kind: ReplicationController +metadata: + name: selenium-hub + labels: + name: selenium-hub +spec: + replicas: 1 + selector: + name: selenium-hub + template: + metadata: + labels: + name: selenium-hub + spec: + containers: + - name: selenium-hub + image: selenium/hub:2.47.1 + ports: + - containerPort: 4444 + resources: + limits: + memory: "1000Mi" + cpu: ".5" + livenessProbe: + httpGet: + path: /grid/console + port: 4444 + initialDelaySeconds: 30 + timeoutSeconds: 5 + readinessProbe: + httpGet: + path: /grid/console + port: 4444 + initialDelaySeconds: 30 + timeoutSeconds: 5 diff --git a/examples/selenium/selenium-hub-svc.yaml b/examples/selenium/selenium-hub-svc.yaml new file mode 100644 index 00000000000..4feed0387ae --- /dev/null +++ b/examples/selenium/selenium-hub-svc.yaml @@ -0,0 +1,15 @@ +apiVersion: v1 +kind: Service +metadata: + name: selenium-hub + labels: + name: selenium-hub +spec: + ports: + - port: 4444 + targetPort: 4444 + name: port0 + selector: + name: selenium-hub + type: NodePort + sessionAffinity: None diff --git a/examples/selenium/selenium-node-chrome-rc.yaml b/examples/selenium/selenium-node-chrome-rc.yaml new file mode 100644 index 00000000000..6f78bdbec8f --- /dev/null +++ b/examples/selenium/selenium-node-chrome-rc.yaml @@ -0,0 +1,29 @@ +apiVersion: v1 +kind: ReplicationController +metadata: + name: selenium-node-chrome + labels: + name: selenium-node-chrome +spec: + replicas: 2 + selector: + name: selenium-node-chrome + template: + metadata: + labels: + name: selenium-node-chrome + spec: + containers: + - name: selenium-node-chrome + image: selenium/node-chrome-debug:2.47.1 + ports: + - containerPort: 5900 + env: + - name: HUB_PORT_4444_TCP_ADDR + value: "selenium-hub" + - name: HUB_PORT_4444_TCP_PORT + value: "4444" + resources: + limits: + memory: "1000Mi" + cpu: ".5" diff --git a/examples/selenium/selenium-node-firefox-rc.yaml b/examples/selenium/selenium-node-firefox-rc.yaml new file mode 100644 index 00000000000..c3496182850 --- /dev/null +++ b/examples/selenium/selenium-node-firefox-rc.yaml @@ -0,0 +1,29 @@ +apiVersion: v1 +kind: ReplicationController +metadata: + name: selenium-node-firefox + labels: + name: selenium-node-firefox +spec: + replicas: 2 + selector: + name: selenium-node-firefox + template: + metadata: + labels: + name: selenium-node-firefox + spec: + containers: + - name: selenium-node-firefox + image: selenium/node-firefox-debug:2.47.1 + ports: + - containerPort: 5900 + env: + - name: HUB_PORT_4444_TCP_ADDR + value: "selenium-hub" + - name: HUB_PORT_4444_TCP_PORT + value: "4444" + resources: + limits: + memory: "1000Mi" + cpu: ".5" diff --git a/examples/selenium/selenium-test.py b/examples/selenium/selenium-test.py new file mode 100644 index 00000000000..bfaeeea7b5b --- /dev/null +++ b/examples/selenium/selenium-test.py @@ -0,0 +1,17 @@ +from selenium import webdriver +from selenium.webdriver.common.desired_capabilities import DesiredCapabilities + +def check_browser(browser): + driver = webdriver.Remote( + command_executor='http://selenium-hub:4444/wd/hub', + desired_capabilities=getattr(DesiredCapabilities, browser) + ) + driver.get("http://google.com") + assert "google" in driver.page_source + driver.close() + print("Browser %s checks out!" % browser) + + +check_browser("FIREFOX") +check_browser("CHROME") + From 0bb17fe6df20fd3d330875a4152ea625f84dca09 Mon Sep 17 00:00:00 2001 From: Elson Rodriguez Date: Wed, 7 Oct 2015 19:26:58 -0700 Subject: [PATCH 2/3] Removes "name" labels, improves document flow, adds highlighting. --- examples/selenium/README.md | 55 ++++++++++--------- examples/selenium/selenium-hub-rc.yaml | 6 +- examples/selenium/selenium-hub-svc.yaml | 6 +- .../selenium/selenium-node-chrome-rc.yaml | 6 +- .../selenium/selenium-node-firefox-rc.yaml | 6 +- examples/selenium/selenium-test.py | 16 ++++++ 6 files changed, 56 insertions(+), 39 deletions(-) diff --git a/examples/selenium/README.md b/examples/selenium/README.md index a8b2615ad4c..66a6a3363df 100644 --- a/examples/selenium/README.md +++ b/examples/selenium/README.md @@ -43,12 +43,12 @@ Your cluster must have 4 CPU and 6 GB of RAM to complete the example up to the s ### Deploy Selenium Grid Hub: We will be using Selenium Grid Hub to make our Selenium install scalable via a master/worker model. The Selenium Hub is the master, and the Selenium Nodes are the workers(not to be confused with Kubernetes nodes). We only need one hub, but we're using a replication controller to ensure that the hub is always running: -``` +```console kubectl create --filename=selenium-hub-rc.yaml ``` The Selenium Nodes will need to know how to get to the Hub, let's create a service for the nodes to connect to. -``` +```console kubectl create --filename=selenium-hub-svc.yaml ``` @@ -56,9 +56,9 @@ kubectl create --filename=selenium-hub-svc.yaml Let's verify our deployment of Selenium hub by connecting to the web console. #### Kubernetes Nodes Reachable -If your Kubernetes nodes are reachable from your network, you can verify the hub by hitting it on the nodeport. You can retrieve the nodeport by typing `kubectl describe svc selenium-hub`, however the snippet below automates that: -``` -export NODEPORT=`kubectl get svc --selector='name=selenium-hub' --output=template --template="{{ with index .items 0}}{{with index .spec.ports 0 }}{{.nodePort}}{{end}}{{end}}"` +If your Kubernetes nodes are reachable from your network, you can verify the hub by hitting it on the nodeport. You can retrieve the nodeport by typing `kubectl describe svc selenium-hub`, however the snippet below automates that by using kubectl's template functionality: +```console +export NODEPORT=`kubectl get svc --selector='app=selenium-hub' --output=template --template="{{ with index .items 0}}{{with index .spec.ports 0 }}{{.nodePort}}{{end}}{{end}}"` export NODE=`kubectl get nodes --output=template --template="{{with index .items 0 }}{{.metadata.name}}{{end}}"` curl http://$NODE:$NODEPORT @@ -66,39 +66,40 @@ curl http://$NODE:$NODEPORT #### Kubernetes Nodes Unreachable If you cannot reach your Kubernetes nodes from your network, you can proxy via kubectl. -``` -export PODNAME=`kubectl get pods --selector="name=selenium-hub" --output=template --template="{{with index .items 0}}{{.metadata.name}}{{end}}"` +```console +export PODNAME=`kubectl get pods --selector="app=selenium-hub" --output=template --template="{{with index .items 0}}{{.metadata.name}}{{end}}"` kubectl port-forward --pod=$PODNAME 4444:4444 ``` In a seperate terminal, you can now check the status. -``` +```console curl http://localhost:4444 ``` #### Using Google Container Engine If you are using Google Container Engine, you can expose your hub via the internet. This is a bad idea for many reasons, but you can do it as follows: -``` -kubectl expose rc selenium-hub --name=selenium-hub-external --labels="name=selenium-hub-external,external=true" --create-external-load-balancer=true +```console +kubectl expose rc selenium-hub --name=selenium-hub-external --labels="app=selenium-hub,external=true" --create-external-load-balancer=true ``` -Then wait a few minutes, eventually your new `selenium-hub-external` service will be assigned an load balancing IP from gcloud. -``` -export INTERNET_IP=`kubectl get svc --selector="name=selenium-hub-external" --output=template --template="{{with index .items 0}}{{with index .status.loadBalancer.ingress 0}}{{.ip}}{{end}}{{end}}"` +Then wait a few minutes, eventually your new `selenium-hub-external` service will be assigned a load balanced IP from gcloud. Once `kubectl get svc selenium-hub-external` shows two IPs, run this snippet. +```console +export INTERNET_IP=`kubectl get svc --selector="app=selenium-hub,external=true" --output=template --template="{{with index .items 0}}{{with index .status.loadBalancer.ingress 0}}{{.ip}}{{end}}{{end}}"` curl http://$INTERNET_IP:4444/ ``` +You should now be able to hit `$INTERNET_IP` via your web browser, and so can everyone else on the Internet! ### Deploy Firefox and Chrome Nodes: Now that the Hub is up, we can deploy workers. -This will deploy 3 Chrome nodes. -``` +This will deploy 2 Chrome nodes. +```console kubectl create -f selenium-node-chrome-rc.yaml ``` -And 3 Firefox nodes to match. -``` +And 2 Firefox nodes to match. +```console kubectl create -f selenium-node-firefox-rc.yaml ``` @@ -109,24 +110,24 @@ Let's run a quick Selenium job to validate our setup. #### Setup Python Environment First, we need to start a python container that we can attach to. -``` +```console kubectl run selenium-python --image=google/python-hello ``` Next, we need to get inside this container. -``` +```console export PODNAME=`kubectl get pods --selector="run=selenium-python" --output=template --template="{{with index .items 0}}{{.metadata.name}}{{end}}"` kubectl exec --stdin=true --tty=true $PODNAME bash ``` Once inside, we need to install the Selenium library -``` +```console pip install selenium ``` #### Run Selenium Job with Python We're all set up, start the python interpreter. -``` +```console python ``` @@ -162,7 +163,7 @@ Congratulations, your Selenium Hub is up, with Firefox and Chrome nodes! ### Scale your Firefox and Chrome nodes. If you need more Firefox or Chrome nodes, your hardware is the limit: -``` +```console kubectl scale rc selenium-node-firefox --replicas=10 kubectl scale rc selenium-node-chrome --replicas=10 ``` @@ -170,13 +171,13 @@ kubectl scale rc selenium-node-chrome --replicas=10 You now have 10 Firefox and 10 Chrome nodes, happy Seleniuming! ### Debugging -Sometimes it is neccessary to check on a hung test. Each pod is running VNC. To check on one of the browser nodes via VNC, it's reccomended that you proxy, since we don't want to expose a service for every pod, and the containers have a weak password. Replace POD_NAME with the name of the pod you want to connect to. +Sometimes it is neccessary to check on a hung test. Each pod is running VNC. To check on one of the browser nodes via VNC, it's reccomended that you proxy, since we don't want to expose a service for every pod, and the containers have a weak VNC password. Replace POD_NAME with the name of the pod you want to connect to. -``` -kubectl port-forward --pod=POD_NAME 9000:5900 +```console +kubectl port-forward --pod=POD_NAME 5900:5900 ``` -Then connect to localhost:9000 with your VNC client using the password "secret" +Then connect to localhost:5900 with your VNC client using the password "secret" Enjoy your scalable Selenium Grid! @@ -186,7 +187,7 @@ Adapted from: https://github.com/SeleniumHQ/docker-selenium To remove all created resources, run the following: -``` +```console kubectl delete rc selenium-hub kubectl delete rc selenium-node-chrome kubectl delete rc selenium-node-firefox diff --git a/examples/selenium/selenium-hub-rc.yaml b/examples/selenium/selenium-hub-rc.yaml index b1d53361cbe..6f9b50c1d30 100644 --- a/examples/selenium/selenium-hub-rc.yaml +++ b/examples/selenium/selenium-hub-rc.yaml @@ -3,15 +3,15 @@ kind: ReplicationController metadata: name: selenium-hub labels: - name: selenium-hub + app: selenium-hub spec: replicas: 1 selector: - name: selenium-hub + app: selenium-hub template: metadata: labels: - name: selenium-hub + app: selenium-hub spec: containers: - name: selenium-hub diff --git a/examples/selenium/selenium-hub-svc.yaml b/examples/selenium/selenium-hub-svc.yaml index 4feed0387ae..0b252ede375 100644 --- a/examples/selenium/selenium-hub-svc.yaml +++ b/examples/selenium/selenium-hub-svc.yaml @@ -1,15 +1,15 @@ apiVersion: v1 kind: Service metadata: - name: selenium-hub + name: selenium-hub labels: - name: selenium-hub + app: selenium-hub spec: ports: - port: 4444 targetPort: 4444 name: port0 selector: - name: selenium-hub + app: selenium-hub type: NodePort sessionAffinity: None diff --git a/examples/selenium/selenium-node-chrome-rc.yaml b/examples/selenium/selenium-node-chrome-rc.yaml index 6f78bdbec8f..324c5507380 100644 --- a/examples/selenium/selenium-node-chrome-rc.yaml +++ b/examples/selenium/selenium-node-chrome-rc.yaml @@ -3,15 +3,15 @@ kind: ReplicationController metadata: name: selenium-node-chrome labels: - name: selenium-node-chrome + app: selenium-node-chrome spec: replicas: 2 selector: - name: selenium-node-chrome + app: selenium-node-chrome template: metadata: labels: - name: selenium-node-chrome + app: selenium-node-chrome spec: containers: - name: selenium-node-chrome diff --git a/examples/selenium/selenium-node-firefox-rc.yaml b/examples/selenium/selenium-node-firefox-rc.yaml index c3496182850..399ddd249d2 100644 --- a/examples/selenium/selenium-node-firefox-rc.yaml +++ b/examples/selenium/selenium-node-firefox-rc.yaml @@ -3,15 +3,15 @@ kind: ReplicationController metadata: name: selenium-node-firefox labels: - name: selenium-node-firefox + app: selenium-node-firefox spec: replicas: 2 selector: - name: selenium-node-firefox + app: selenium-node-firefox template: metadata: labels: - name: selenium-node-firefox + app: selenium-node-firefox spec: containers: - name: selenium-node-firefox diff --git a/examples/selenium/selenium-test.py b/examples/selenium/selenium-test.py index bfaeeea7b5b..d7ab8035893 100644 --- a/examples/selenium/selenium-test.py +++ b/examples/selenium/selenium-test.py @@ -1,3 +1,19 @@ +#!/usr/bin/env python + +# Copyright 2015 The Kubernetes Authors All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + from selenium import webdriver from selenium.webdriver.common.desired_capabilities import DesiredCapabilities From e10798664905402e67122bd6d8071e30d914af85 Mon Sep 17 00:00:00 2001 From: Elson Rodriguez Date: Thu, 8 Oct 2015 16:49:37 -0700 Subject: [PATCH 3/3] Ran document cleanup script, changed flags to be consistent. --- examples/selenium/README.md | 49 +++++++++++++++++++++++++++++++------ 1 file changed, 42 insertions(+), 7 deletions(-) diff --git a/examples/selenium/README.md b/examples/selenium/README.md index 66a6a3363df..41168b7cd24 100644 --- a/examples/selenium/README.md +++ b/examples/selenium/README.md @@ -20,7 +20,8 @@ refer to the docs that go with that version. The latest 1.0.x release of this document can be found -[here](http://releases.k8s.io/release-1.0/examples/redis/README.md). +[here](http://releases.k8s.io/release-1.0/examples/selenium/README.md). + Documentation for other releases can be found at [releases.k8s.io](http://releases.k8s.io). @@ -35,28 +36,35 @@ Documentation for other releases can be found at Selenium is a browser automation tool used primarily for testing web applications. However when Selenium is used in a CI pipeline to test applications, there is often contention around the use of Selenium resources. This example shows you how to deploy Selenium to Kubernetes in a scalable fashion. ### Prerequisites -This example assumes you have a working Kubernetes cluster and a properly configured kubectl client. See the [Getting Started Guides](../../docs/getting-started-guides/) for details. + +This example assumes you have a working Kubernetes cluster and a properly configured kubectl client. See the [Getting Started Guides](../../docs/getting-started-guides/) for details. Google Container Engine is also a quick way to get Kubernetes up and running: https://cloud.google.com/container-engine/ Your cluster must have 4 CPU and 6 GB of RAM to complete the example up to the scaling portion. ### Deploy Selenium Grid Hub: + We will be using Selenium Grid Hub to make our Selenium install scalable via a master/worker model. The Selenium Hub is the master, and the Selenium Nodes are the workers(not to be confused with Kubernetes nodes). We only need one hub, but we're using a replication controller to ensure that the hub is always running: + ```console -kubectl create --filename=selenium-hub-rc.yaml +kubectl create --filename=examples/selenium/selenium-hub-rc.yaml ``` The Selenium Nodes will need to know how to get to the Hub, let's create a service for the nodes to connect to. + ```console -kubectl create --filename=selenium-hub-svc.yaml +kubectl create --filename=examples/selenium/selenium-hub-svc.yaml ``` ### Verify Selenium Hub Deployment + Let's verify our deployment of Selenium hub by connecting to the web console. #### Kubernetes Nodes Reachable + If your Kubernetes nodes are reachable from your network, you can verify the hub by hitting it on the nodeport. You can retrieve the nodeport by typing `kubectl describe svc selenium-hub`, however the snippet below automates that by using kubectl's template functionality: + ```console export NODEPORT=`kubectl get svc --selector='app=selenium-hub' --output=template --template="{{ with index .items 0}}{{with index .spec.ports 0 }}{{.nodePort}}{{end}}{{end}}"` export NODE=`kubectl get nodes --output=template --template="{{with index .items 0 }}{{.metadata.name}}{{end}}"` @@ -65,73 +73,91 @@ curl http://$NODE:$NODEPORT ``` #### Kubernetes Nodes Unreachable + If you cannot reach your Kubernetes nodes from your network, you can proxy via kubectl. + ```console export PODNAME=`kubectl get pods --selector="app=selenium-hub" --output=template --template="{{with index .items 0}}{{.metadata.name}}{{end}}"` kubectl port-forward --pod=$PODNAME 4444:4444 ``` In a seperate terminal, you can now check the status. + ```console curl http://localhost:4444 ``` #### Using Google Container Engine + If you are using Google Container Engine, you can expose your hub via the internet. This is a bad idea for many reasons, but you can do it as follows: + ```console kubectl expose rc selenium-hub --name=selenium-hub-external --labels="app=selenium-hub,external=true" --create-external-load-balancer=true ``` Then wait a few minutes, eventually your new `selenium-hub-external` service will be assigned a load balanced IP from gcloud. Once `kubectl get svc selenium-hub-external` shows two IPs, run this snippet. + ```console export INTERNET_IP=`kubectl get svc --selector="app=selenium-hub,external=true" --output=template --template="{{with index .items 0}}{{with index .status.loadBalancer.ingress 0}}{{.ip}}{{end}}{{end}}"` curl http://$INTERNET_IP:4444/ ``` + You should now be able to hit `$INTERNET_IP` via your web browser, and so can everyone else on the Internet! ### Deploy Firefox and Chrome Nodes: + Now that the Hub is up, we can deploy workers. This will deploy 2 Chrome nodes. + ```console -kubectl create -f selenium-node-chrome-rc.yaml +kubectl create --file=examples/selenium/selenium-node-chrome-rc.yaml ``` And 2 Firefox nodes to match. + ```console -kubectl create -f selenium-node-firefox-rc.yaml +kubectl create --file=examples/selenium/selenium-node-firefox-rc.yaml ``` Once the pods start, you will see them show up in the Selenium Hub interface. ### Run a Selenium Job + Let's run a quick Selenium job to validate our setup. #### Setup Python Environment + First, we need to start a python container that we can attach to. + ```console kubectl run selenium-python --image=google/python-hello ``` Next, we need to get inside this container. + ```console export PODNAME=`kubectl get pods --selector="run=selenium-python" --output=template --template="{{with index .items 0}}{{.metadata.name}}{{end}}"` kubectl exec --stdin=true --tty=true $PODNAME bash ``` Once inside, we need to install the Selenium library + ```console pip install selenium ``` #### Run Selenium Job with Python + We're all set up, start the python interpreter. + ```console python ``` And paste in the contents of selenium-test.py. + ```python from selenium import webdriver from selenium.webdriver.common.desired_capabilities import DesiredCapabilities @@ -152,17 +178,20 @@ check_browser("CHROME") ``` You should get + ``` >>> check_browser("FIREFOX") Browser FIREFOX checks out! >>> check_browser("CHROME") Browser CHROME checks out! ``` + Congratulations, your Selenium Hub is up, with Firefox and Chrome nodes! ### Scale your Firefox and Chrome nodes. If you need more Firefox or Chrome nodes, your hardware is the limit: + ```console kubectl scale rc selenium-node-firefox --replicas=10 kubectl scale rc selenium-node-chrome --replicas=10 @@ -171,8 +200,9 @@ kubectl scale rc selenium-node-chrome --replicas=10 You now have 10 Firefox and 10 Chrome nodes, happy Seleniuming! ### Debugging + Sometimes it is neccessary to check on a hung test. Each pod is running VNC. To check on one of the browser nodes via VNC, it's reccomended that you proxy, since we don't want to expose a service for every pod, and the containers have a weak VNC password. Replace POD_NAME with the name of the pod you want to connect to. - + ```console kubectl port-forward --pod=POD_NAME 5900:5900 ``` @@ -195,3 +225,8 @@ kubectl delete rc selenium-python kubectl delete svc selenium-hub kubectl delete svc selenium-hub-external ``` + + + +[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/examples/selenium/README.md?pixel)]() +