From 00c4e9ba01933d4cb84ce711b725af7046bad749 Mon Sep 17 00:00:00 2001 From: Paulo Pires Date: Tue, 2 Jun 2015 22:09:48 +0100 Subject: [PATCH 1/4] Updated to hazelcast-kubernetes-bootstraper 0.3.1. --- examples/hazelcast/hazelcast-controller.yaml | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/examples/hazelcast/hazelcast-controller.yaml b/examples/hazelcast/hazelcast-controller.yaml index cde79303f78..86496ef665f 100644 --- a/examples/hazelcast/hazelcast-controller.yaml +++ b/examples/hazelcast/hazelcast-controller.yaml @@ -17,8 +17,11 @@ spec: - resources: limits: cpu: 1 - image: pires/hazelcast-k8s:0.2 + image: quay.io/pires/hazelcast-kubernetes:0.3.1 name: hazelcast + env: + - name: "DNS_DOMAIN" + value: "cluster.local" ports: - containerPort: 5701 name: hazelcast From 61a4d646a7883ed251b28b1baee46e050a12f798 Mon Sep 17 00:00:00 2001 From: Paulo Pires Date: Tue, 2 Jun 2015 22:10:04 +0100 Subject: [PATCH 2/4] Updated Hazelcast example instructions. --- examples/hazelcast/README.md | 41 +++++++++++++++++++++++++++++------- 1 file changed, 33 insertions(+), 8 deletions(-) diff --git a/examples/hazelcast/README.md b/examples/hazelcast/README.md index bfc5e96a2df..98351aa2573 100644 --- a/examples/hazelcast/README.md +++ b/examples/hazelcast/README.md @@ -1,3 +1,23 @@ +hazelcast-kubernetes +==================== + +Hazelcast clustering for Kubernetes made easy. + +Lean (186MB) JRE 8u45 + Hazelcast 3.4.2 + Kubernetes discovery Docker image, based on `progrium/busybox`. + +## Pre-requisites + +* Docker 1.5+ +* Kubernetes 0.18.0 or newer cluster + +## Kubernetes cluster + +You can test with a local cluster. Check [this other repository from yours truly](https://github.com/pires/kubernetes-vagrant-coreos-cluster). + +## Docker image + +The image is already available at [quay.io/pires](https://quay.io/repository/pires/hazelcast-kubernetes) + ## Cloud Native Deployments of Hazelcast using Kubernetes The following document describes the development of a _cloud native_ [Hazelcast](http://hazelcast.org/) deployment on Kubernetes. When we say _cloud native_ we mean an application which understands that it is running within a cluster manager, and uses this cluster management infrastructure to help implement the application. In particular, in this instance, a custom Hazelcast ```bootstrapper``` is used to enable Hazelcast to dynamically discover Hazelcast nodes that have already joined the cluster. @@ -52,7 +72,7 @@ $ kubectl create -f hazelcast-service.yaml ``` ### Adding replicated nodes -The real power of Kubernetes and Hazelcast lies in easily building a replicated, scalable Hazelcast cluster. +The real power of Kubernetes and Hazelcast lies in easily building a replicated, resizable Hazelcast cluster. In Kubernetes a _Replication Controller_ is responsible for replicating sets of identical pods. Like a _Service_ it has a selector query which identifies the members of it's set. Unlike a _Service_ it also has a desired number of replicas, and it will create or delete _Pods_ to ensure that the number of _Pods_ matches up with it's desired state. @@ -78,18 +98,23 @@ spec: - resources: limits: cpu: 1 - image: pires/hazelcast-k8s:0.2 + image: quay.io/pires/hazelcast-kubernetess:0.3.1 name: hazelcast + env: + - name: "DNS_DOMAIN" + value: "cluster.local" ports: - containerPort: 5701 name: hazelcast ``` -There are a few things to note in this description. First is that we are running the `pires/hazelcast-k8s` image, tag `0.2`. This is a `busybox` installation with JRE 8. However it also adds a custom [`application`](https://github.com/pires/hazelcast-kubernetes-bootstrapper) that finds any Hazelcast nodes in the cluster and bootstraps an Hazelcast instance accordingle. The `HazelcastDiscoveryController` discovers the Kubernetes API Server using the built in Kubernetes discovery service, and then uses the Kubernetes API to find new nodes (more on this later). +There are a few things to note in this description. First is that we are running the `quay.io/pires/hazelcast-kubernetes` image, tag `0.3.1`. This is a `busybox` installation with JRE 8. However it also adds a custom [`application`](https://github.com/pires/hazelcast-kubernetes-bootstrapper) that finds any Hazelcast nodes in the cluster and bootstraps an Hazelcast instance accordingle. The `HazelcastDiscoveryController` discovers the Kubernetes API Server using the built in Kubernetes discovery service, and then uses the Kubernetes API to find new nodes (more on this later). You may also note that we tell Kubernetes that the container exposes the `hazelcast` port. Finally, we tell the cluster manager that we need 1 cpu core. -The bulk of the replication controller config is actually identical to the Hazelcast pod declaration above, it simply gives the controller a recipe to use when creating new pods. The other parts are the ```selector``` which contains the controller's selector query, and the ```replicas``` parameter which specifies the desired number of replicas, in this case 1. +The bulk of the replication controller config is actually identical to the Hazelcast pod declaration above, it simply gives the controller a recipe to use when creating new pods. The other parts are the `selector` which contains the controller's selector query, and the `replicas` parameter which specifies the desired number of replicas, in this case 1. + +Last but not least, we set `DNS_DOMAIN` environment variable according to your Kubernetes clusters DNS configuration. Create this controller: @@ -140,9 +165,9 @@ Now if you list the pods in your cluster, you should see two hazelcast pods: $ kubectl get pods POD IP CONTAINER(S) IMAGE(S) HOST LABELS STATUS CREATED MESSAGE hazelcast-pkyzd 10.244.90.3 e2e-test-minion-vj7k/104.197.8.214 name=hazelcast Running 14 seconds - hazelcast pires/hazelcast-k8s:0.2 Running 2 seconds -hazelcast-ulkws 10.244.66.2 e2e-test-minion-2x1f/146.148.62.37 name=hazelcast Running 7 seconds - hazelcast pires/hazelcast-k8s:0.2 Running 6 seconds + hazelcast quay.io/pires/hazelcast-kubernetes:0.3.1 Running 2 seconds +hazelcast-ulkws 10.244.66.2 e2e-test-minion-2x1f/146.148.62.37 name=hazelcast Running 7 seconds + hazelcast quay.io/pires/hazelcast-kubernetes:0.3.1 Running 6 seconds ``` To prove that this all works, you can use the `log` command to examine the logs of one pod, for example: @@ -152,7 +177,7 @@ $ kubectl log hazelcast-ulkws hazelcast 2015-05-09 22:06:20.016 INFO 5 --- [ main] com.github.pires.hazelcast.Application : Starting Application v0.2-SNAPSHOT on hazelcast-enyli with PID 5 (/bootstrapper.jar started by root in /) 2015-05-09 22:06:20.071 INFO 5 --- [ main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@5424f110: startup date [Sat May 09 22:06:20 GMT 2015]; root of context hierarchy 2015-05-09 22:06:21.511 INFO 5 --- [ main] o.s.j.e.a.AnnotationMBeanExporter : Registering beans for JMX exposure on startup -2015-05-09 22:06:21.549 INFO 5 --- [ main] c.g.p.h.HazelcastDiscoveryController : Asking k8s registry at http://10.100.0.1:80.. +2015-05-09 22:06:21.549 INFO 5 --- [ main] c.g.p.h.HazelcastDiscoveryController : Asking k8s registry at https://kubernetes.default.cluster.local.. 2015-05-09 22:06:22.031 INFO 5 --- [ main] c.g.p.h.HazelcastDiscoveryController : Found 2 pods running Hazelcast. 2015-05-09 22:06:22.176 INFO 5 --- [ main] c.h.instance.DefaultAddressPicker : [LOCAL] [someGroup] [3.4.2] Interfaces is disabled, trying to pick one address from TCP-IP config addresses: [10.244.90.3, 10.244.66.2] 2015-05-09 22:06:22.177 INFO 5 --- [ main] c.h.instance.DefaultAddressPicker : [LOCAL] [someGroup] [3.4.2] Prefer IPv4 stack is true. From db70dd5902542e055fc536467516d608b3cfdbf5 Mon Sep 17 00:00:00 2001 From: Paulo Pires Date: Tue, 2 Jun 2015 22:10:26 +0100 Subject: [PATCH 3/4] Added Hazelcast example Dockerfile. --- examples/hazelcast/Dockerfile | 11 +++++++++++ examples/hazelcast/README.md | 20 -------------------- 2 files changed, 11 insertions(+), 20 deletions(-) create mode 100644 examples/hazelcast/Dockerfile diff --git a/examples/hazelcast/Dockerfile b/examples/hazelcast/Dockerfile new file mode 100644 index 00000000000..55963290c1a --- /dev/null +++ b/examples/hazelcast/Dockerfile @@ -0,0 +1,11 @@ +FROM quay.io/pires/docker-jre:8u45-2 + +MAINTAINER Paulo Pires + +EXPOSE 5701 + +RUN \ + curl -Lskj https://github.com/pires/hazelcast-kubernetes-bootstrapper/releases/download/0.3.1/hazelcast-kubernetes-bootstrapper-0.3.1.jar \ + -o /bootstrapper.jar + +CMD java -jar /bootstrapper.jar diff --git a/examples/hazelcast/README.md b/examples/hazelcast/README.md index 98351aa2573..acabf8fac5a 100644 --- a/examples/hazelcast/README.md +++ b/examples/hazelcast/README.md @@ -1,23 +1,3 @@ -hazelcast-kubernetes -==================== - -Hazelcast clustering for Kubernetes made easy. - -Lean (186MB) JRE 8u45 + Hazelcast 3.4.2 + Kubernetes discovery Docker image, based on `progrium/busybox`. - -## Pre-requisites - -* Docker 1.5+ -* Kubernetes 0.18.0 or newer cluster - -## Kubernetes cluster - -You can test with a local cluster. Check [this other repository from yours truly](https://github.com/pires/kubernetes-vagrant-coreos-cluster). - -## Docker image - -The image is already available at [quay.io/pires](https://quay.io/repository/pires/hazelcast-kubernetes) - ## Cloud Native Deployments of Hazelcast using Kubernetes The following document describes the development of a _cloud native_ [Hazelcast](http://hazelcast.org/) deployment on Kubernetes. When we say _cloud native_ we mean an application which understands that it is running within a cluster manager, and uses this cluster management infrastructure to help implement the application. In particular, in this instance, a custom Hazelcast ```bootstrapper``` is used to enable Hazelcast to dynamically discover Hazelcast nodes that have already joined the cluster. From f059f6ee606f179e979320b37c0865ccf59ad0b7 Mon Sep 17 00:00:00 2001 From: Paulo Pires Date: Thu, 4 Jun 2015 22:02:27 +0100 Subject: [PATCH 4/4] Fixed typo in Hazelcast example README.md. --- examples/hazelcast/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/examples/hazelcast/README.md b/examples/hazelcast/README.md index acabf8fac5a..44c8ab6d70d 100644 --- a/examples/hazelcast/README.md +++ b/examples/hazelcast/README.md @@ -78,7 +78,7 @@ spec: - resources: limits: cpu: 1 - image: quay.io/pires/hazelcast-kubernetess:0.3.1 + image: quay.io/pires/hazelcast-kubernetes:0.3.1 name: hazelcast env: - name: "DNS_DOMAIN"