github/kubernetes
1
0
Fork 0
mirror of https://github.com/k3s-io/kubernetes.git synced 2025-07-23 03:41:45 +00:00
Code Issues Projects Releases Wiki Activity

Merge pull request #39580 from oscerd/hazelcast-example-upgrade

Browse Source 
Automatic merge from submit-queue (batch tested with PRs 40873, 40948, 39580, 41065, 40815)

Upgrade Hazelcast example with the latest hazelcast-kubernetes-bootst…

…rapper 0.8.0, using Deployments instead of Replication Controller.

**What this PR does / why we need it**:
This PR upgrade the Hazelcast example in storage folder. It would be great to be aligned with the latest version of hazelcast-kubernetes-bootstrapper
This commit is contained in:
Kubernetes Submit Queue 2017-02-08 14:14:45 -08:00 committed by GitHub
commit 16f1bd83c0
5 changed files with 125 additions and 172 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
examples/examples_test.go
View File

@ -241,7 +241,7 @@ func TestExampleObjectSchemas(t *testing.T) {
"pod": &api.Pod{}, "pod": &api.Pod{},
}, },
"../examples/storage/hazelcast": { "../examples/storage/hazelcast": {
"hazelcast-controller": &api.ReplicationController{}, "hazelcast-deployment": &extensions.Deployment{},
"hazelcast-service": &api.Service{}, "hazelcast-service": &api.Service{},
}, },
"../examples/meteor": { "../examples/meteor": {

218
examples/storage/hazelcast/README.md
View File

@ -4,7 +4,7 @@ The following document describes the development of a _cloud native_ [Hazelcast]
Any topology changes are communicated and handled by Hazelcast nodes themselves. Any topology changes are communicated and handled by Hazelcast nodes themselves.
This document also attempts to describe the core components of Kubernetes: _Pods_, _Services_, and _Replication Controllers_. This document also attempts to describe the core components of Kubernetes: _Pods_, _Services_, and _Deployments_.
### Prerequisites ### Prerequisites
@ -30,7 +30,7 @@ In this case, we shall not run a single Hazelcast pod, because the discovery mec
### Adding a Hazelcast Service ### Adding a Hazelcast Service
In Kubernetes a _[Service](../../../docs/user-guide/services.md)_ describes a set of Pods that perform the same task. For example, the set of nodes in a Hazelcast cluster. An important use for a Service is to create a load balancer which distributes traffic across members of the set. But a _Service_ can also be used as a standing query which makes a dynamically changing set of Pods available via the Kubernetes API. This is actually how our discovery mechanism works, by relying on the service to discover other Hazelcast pods. In Kubernetes a _[Service](../../../docs/user-guide/services.md)_ describes a set of Pods that perform the same task. For example, the set of nodes in a Hazelcast cluster. An important use for a Service is to create a load balancer which distributes traffic across members of the set. But a _Service_ can also be used as a standing query which makes a dynamically changing set of Pods available via the Kubernetes API. This is actually how our discovery mechanism works, by relying on the service to discover other Hazelcast pods.
Here is the service description: Here is the service description:
@ -65,52 +65,41 @@ $ kubectl create -f examples/storage/hazelcast/hazelcast-service.yaml
The real power of Kubernetes and Hazelcast lies in easily building a replicated, resizable Hazelcast cluster. The real power of Kubernetes and Hazelcast lies in easily building a replicated, resizable Hazelcast cluster.
In Kubernetes a _[Replication Controller](../../../docs/user-guide/replication-controller.md)_ is responsible for replicating sets of identical pods. Like a _Service_ it has a selector query which identifies the members of it's set. Unlike a _Service_ it also has a desired number of replicas, and it will create or delete _Pods_ to ensure that the number of _Pods_ matches up with it's desired state. In Kubernetes a _[_Deployment_](../../../docs/user-guide/deployments.md)_ is responsible for replicating sets of identical pods. Like a _Service_ it has a selector query which identifies the members of its set. Unlike a _Service_ it also has a desired number of replicas, and it will create or delete _Pods_ to ensure that the number of _Pods_ matches up with its desired state.
Replication Controllers will "adopt" existing pods that match their selector query, so let's create a Replication Controller with a single replica to adopt our existing Hazelcast Pod. Deployments will "adopt" existing pods that match their selector query, so let's create a Deployment with a single replica to adopt our existing Hazelcast Pod.
<!-- BEGIN MUNGE: EXAMPLE hazelcast-controller.yaml --> <!-- BEGIN MUNGE: EXAMPLE hazelcast-controller.yaml -->
```yaml ```yaml
apiVersion: v1 apiVersion: extensions/v1beta1
kind: ReplicationController kind: Deployment
metadata: metadata:
name: hazelcast
labels: labels:
name: hazelcast name: hazelcast
name: hazelcast
spec: spec:
replicas: 1
selector:
name: hazelcast
template: template:
metadata: metadata:
labels: labels:
name: hazelcast name: hazelcast
spec: spec:
containers: containers:
- resources: - name: hazelcast
limits: image: quay.io/pires/hazelcast-kubernetes:0.8.0
cpu: 0.1 imagePullPolicy: Always
image: quay.io/pires/hazelcast-kubernetes:0.6.1 env:
name: hazelcast - name: "DNS_DOMAIN"
env: value: "cluster.local"
- name: "DNS_DOMAIN" ports:
value: "cluster.local" - name: hazelcast
- name: POD_NAMESPACE containerPort: 5701
valueFrom:
fieldRef:
fieldPath: metadata.namespace
ports:
- containerPort: 5701
name: hazelcast
``` ```
[Download example](hazelcast-controller.yaml?raw=true) [Download example](hazelcast-deployment.yaml?raw=true)
<!-- END MUNGE: EXAMPLE hazelcast-controller.yaml --> <!-- END MUNGE: EXAMPLE hazelcast-controller.yaml -->
There are a few things to note in this description. First is that we are running the `quay.io/pires/hazelcast-kubernetes` image, tag `0.5`. This is a `busybox` installation with JRE 8 Update 45. However it also adds a custom [`application`](https://github.com/pires/hazelcast-kubernetes-bootstrapper) that finds any Hazelcast nodes in the cluster and bootstraps an Hazelcast instance accordingly. The `HazelcastDiscoveryController` discovers the Kubernetes API Server using the built in Kubernetes discovery service, and then uses the Kubernetes API to find new nodes (more on this later). You may note that we tell Kubernetes that the container exposes the `hazelcast` port.
You may also note that we tell Kubernetes that the container exposes the `hazelcast` port. Finally, we tell the cluster manager that we need 1 cpu core.
The bulk of the replication controller config is actually identical to the Hazelcast pod declaration above, it simply gives the controller a recipe to use when creating new pods. The other parts are the `selector` which contains the controller's selector query, and the `replicas` parameter which specifies the desired number of replicas, in this case 1. The bulk of the replication controller config is actually identical to the Hazelcast pod declaration above, it simply gives the controller a recipe to use when creating new pods. The other parts are the `selector` which contains the controller's selector query, and the `replicas` parameter which specifies the desired number of replicas, in this case 1.
@ -119,127 +108,124 @@ Last but not least, we set `DNS_DOMAIN` environment variable according to your K
Create this controller: Create this controller:
```sh ```sh
$ kubectl create -f examples/storage/hazelcast/hazelcast-controller.yaml $ kubectl create -f examples/storage/hazelcast/hazelcast-deployment.yaml
``` ```
After the controller provisions successfully the pod, you can query the service endpoints: After the controller provisions successfully the pod, you can query the service endpoints:
```sh ```sh
$ kubectl get endpoints hazelcast -o json $ kubectl get endpoints hazelcast -o yaml
{ apiVersion: v1
"kind": "Endpoints", kind: Endpoints
"apiVersion": "v1", metadata:
"metadata": { creationTimestamp: 2016-12-16T08:57:27Z
"name": "hazelcast", labels:
"namespace": "default", name: hazelcast
"selfLink": "/api/v1/namespaces/default/endpoints/hazelcast", name: hazelcast
"uid": "094e507a-2700-11e5-abbc-080027eae546", namespace: default
"resourceVersion": "4094", resourceVersion: "11360"
"creationTimestamp": "2015-07-10T12:34:41Z", selfLink: /api/v1/namespaces/default/endpoints/hazelcast
"labels": { uid: 46447198-70eb-11e6-940c-0800278ab84d
"name": "hazelcast" subsets:
} - addresses:
}, - ip: 10.244.37.2
"subsets": [ targetRef:
{ kind: Pod
"addresses": [ name: hazelcast-1790698550-3heau
{ namespace: default
"ip": "10.244.37.3", resourceVersion: "11359"
"targetRef": { uid: c9c3febd-70eb-11e6-940c-0800278ab84d
"kind": "Pod", ports:
"namespace": "default", - port: 5701
"name": "hazelcast-nsyzn", protocol: TCP
"uid": "f57eb6b0-2706-11e5-abbc-080027eae546",
"resourceVersion": "4093"
}
}
],
"ports": [
{
"port": 5701,
"protocol": "TCP"
}
]
}
]
}
``` ```
You can see that the _Service_ has found the pod created by the replication controller. You can see that the _Service_ has found the pod created by the replication controller.
Now it gets even more interesting. Now it gets even more interesting. Let's scale our cluster to 2 pods:
Let's scale our cluster to 2 pods:
```sh ```sh
$ kubectl scale rc hazelcast --replicas=2 $ kubectl scale deployment hazelcast --replicas 2
``` ```
Now if you list the pods in your cluster, you should see two hazelcast pods: Now if you list the pods in your cluster, you should see two hazelcast pods:
```sh ```sh
$ kubectl get pods $ kubectl get deployment,pods
NAME READY STATUS RESTARTS AGE NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
hazelcast-nanfb 1/1 Running 0 40s deploy/hazelcast 2 2 2 2 1m
hazelcast-nsyzn 1/1 Running 0 2m
kube-dns-xudrp 3/3 Running 0 1h NAME READY STATUS RESTARTS AGE
po/hazelcast-3980717115-k1xsk 1/1 Running 0 1m
po/hazelcast-3980717115-pbhbq 1/1 Running 0 22s
``` ```
To prove that this all works, you can use the `log` command to examine the logs of one pod, for example: To prove that this all works, you can use the `log` command to examine the logs of one pod, for example:
```sh ```sh
$ kubectl log hazelcast-nanfb hazelcast kubectl logs -f hazelcast-39807171
2015-07-10 13:26:34.443 INFO 5 --- [ main] com.github.pires.hazelcast.Application : Starting Application on hazelcast-nanfb with PID 5 (/bootstrapper.jar started by root in /) 15-k1xsk
2015-07-10 13:26:34.535 INFO 5 --- [ main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@42cfcf1: startup date [Fri Jul 10 13:26:34 GMT 2015]; root of context hierarchy 2017-01-30 12:42:50.774 INFO 6 --- [ main] com.github.pires.hazelcast.Application : Starting Application on hazelcast-3980717115-k1xsk with PID 6 (/bootstrapper.jar started by root in /)
2015-07-10 13:26:35.888 INFO 5 --- [ main] o.s.j.e.a.AnnotationMBeanExporter : Registering beans for JMX exposure on startup 2017-01-30 12:42:50.781 INFO 6 --- [ main] com.github.pires.hazelcast.Application : No active profile set, falling back to default profiles: default
2015-07-10 13:26:35.924 INFO 5 --- [ main] c.g.p.h.HazelcastDiscoveryController : Asking k8s registry at https://kubernetes.default.svc.cluster.local.. 2017-01-30 12:42:50.852 INFO 6 --- [ main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@14514713: startup date [Mon Jan 30 12:42:50 GMT 2017]; root of context hierarchy
2015-07-10 13:26:37.259 INFO 5 --- [ main] c.g.p.h.HazelcastDiscoveryController : Found 2 pods running Hazelcast. 2017-01-30 12:42:52.304 INFO 6 --- [ main] o.s.j.e.a.AnnotationMBeanExporter : Registering beans for JMX exposure on startup
2015-07-10 13:26:37.404 INFO 5 --- [ main] c.h.instance.DefaultAddressPicker : [LOCAL] [someGroup] [3.5] Interfaces is disabled, trying to pick one address from TCP-IP config addresses: [10.244.77.3, 10.244.37.3] 2017-01-30 12:42:52.323 INFO 6 --- [ main] c.g.p.h.HazelcastDiscoveryController : Asking k8s registry at https://kubernetes.default.svc.cluster.local..
2015-07-10 13:26:37.405 INFO 5 --- [ main] c.h.instance.DefaultAddressPicker : [LOCAL] [someGroup] [3.5] Prefer IPv4 stack is true. 2017-01-30 12:42:52.857 INFO 6 --- [ main] c.g.p.h.HazelcastDiscoveryController : Found 1 pods running Hazelcast.
2015-07-10 13:26:37.415 INFO 5 --- [ main] c.h.instance.DefaultAddressPicker : [LOCAL] [someGroup] [3.5] Picked Address[10.244.77.3]:5701, using socket ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=5701], bind any local is true 2017-01-30 12:42:52.990 INFO 6 --- [ main] c.h.instance.DefaultAddressPicker : [LOCAL] [someGroup] [3.7.5] Interfaces is disabled, trying to pick one address from TCP-IP config addresses: [10.244.9.2]
2015-07-10 13:26:37.852 INFO 5 --- [ main] com.hazelcast.spi.OperationService : [10.244.77.3]:5701 [someGroup] [3.5] Backpressure is disabled 2017-01-30 12:42:52.990 INFO 6 --- [ main] c.h.instance.DefaultAddressPicker : [LOCAL] [someGroup] [3.7.5] Prefer IPv4 stack is true.
2015-07-10 13:26:37.879 INFO 5 --- [ main] c.h.s.i.o.c.ClassicOperationExecutor : [10.244.77.3]:5701 [someGroup] [3.5] Starting with 2 generic operation threads and 2 partition operation threads. 2017-01-30 12:42:53.002 INFO 6 --- [ main] c.h.instance.DefaultAddressPicker : [LOCAL] [someGroup] [3.7.5] Picked [10.244.9.2]:5701, using socket ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=5701], bind any local is true
2015-07-10 13:26:38.531 INFO 5 --- [ main] com.hazelcast.system : [10.244.77.3]:5701 [someGroup] [3.5] Hazelcast 3.5 (20150617 - 4270dc6) starting at Address[10.244.77.3]:5701 2017-01-30 12:42:53.032 INFO 6 --- [ main] com.hazelcast.system : [10.244.9.2]:5701 [someGroup] [3.7.5] Hazelcast 3.7.5 (20170124 - 111f332) starting at [10.244.9.2]:5701
2015-07-10 13:26:38.532 INFO 5 --- [ main] com.hazelcast.system : [10.244.77.3]:5701 [someGroup] [3.5] Copyright (c) 2008-2015, Hazelcast, Inc. All Rights Reserved. 2017-01-30 12:42:53.032 INFO 6 --- [ main] com.hazelcast.system : [10.244.9.2]:5701 [someGroup] [3.7.5] Copyright (c) 2008-2016, Hazelcast, Inc. All Rights Reserved.
2015-07-10 13:26:38.533 INFO 5 --- [ main] com.hazelcast.instance.Node : [10.244.77.3]:5701 [someGroup] [3.5] Creating TcpIpJoiner 2017-01-30 12:42:53.032 INFO 6 --- [ main] com.hazelcast.system : [10.244.9.2]:5701 [someGroup] [3.7.5] Configured Hazelcast Serialization version : 1
2015-07-10 13:26:38.534 INFO 5 --- [ main] com.hazelcast.core.LifecycleService : [10.244.77.3]:5701 [someGroup] [3.5] Address[10.244.77.3]:5701 is STARTING 2017-01-30 12:42:53.343 INFO 6 --- [ main] c.h.s.i.o.impl.BackpressureRegulator : [10.244.9.2]:5701 [someGroup] [3.7.5] Backpressure is disabled
2015-07-10 13:26:38.672 INFO 5 --- [ cached1] com.hazelcast.nio.tcp.SocketConnector : [10.244.77.3]:5701 [someGroup] [3.5] Connecting to /10.244.37.3:5701, timeout: 0, bind-any: true 2017-01-30 12:42:54.273 INFO 6 --- [ main] com.hazelcast.instance.Node : [10.244.9.2]:5701 [someGroup] [3.7.5] Creating TcpIpJoiner
2015-07-10 13:26:38.683 INFO 5 --- [ cached1] c.h.nio.tcp.TcpIpConnectionManager : [10.244.77.3]:5701 [someGroup] [3.5] Established socket connection between /10.244.77.3:59951 2017-01-30 12:42:54.507 INFO 6 --- [ main] c.h.s.i.o.impl.OperationExecutorImpl : [10.244.9.2]:5701 [someGroup] [3.7.5] Starting 2 partition threads
2015-07-10 13:26:45.699 INFO 5 --- [ration.thread-1] com.hazelcast.cluster.ClusterService : [10.244.77.3]:5701 [someGroup] [3.5] 2017-01-30 12:42:54.508 INFO 6 --- [ main] c.h.s.i.o.impl.OperationExecutorImpl : [10.244.9.2]:5701 [someGroup] [3.7.5] Starting 3 generic threads (1 dedicated for priority tasks)
2017-01-30 12:42:54.525 INFO 6 --- [ main] com.hazelcast.core.LifecycleService : [10.244.9.2]:5701 [someGroup] [3.7.5] [10.244.9.2]:5701 is STARTING
2017-01-30 12:42:54.529 INFO 6 --- [ main] c.h.n.t.n.NonBlockingIOThreadingModel : [10.244.9.2]:5701 [someGroup] [3.7.5] TcpIpConnectionManager configured with Non Blocking IO-threading model: 3 input threads and 3 output threads
2017-01-30 12:42:54.578 INFO 6 --- [ main] com.hazelcast.cluster.impl.TcpIpJoiner : [10.244.9.2]:5701 [someGroup] [3.7.5]
Members [2] {
Member [10.244.37.3]:5701 Members [1] {
Member [10.244.77.3]:5701 this Member [10.244.9.2]:5701 - f9cae801-59da-49d9-b8de-7719abb53844 this
} }
2015-07-10 13:26:47.722 INFO 5 --- [ main] com.hazelcast.core.LifecycleService : [10.244.77.3]:5701 [someGroup] [3.5] Address[10.244.77.3]:5701 is STARTED 2017-01-30 12:42:54.660 INFO 6 --- [ main] com.hazelcast.core.LifecycleService : [10.244.9.2]:5701 [someGroup] [3.7.5] [10.244.9.2]:5701 is STARTED
2015-07-10 13:26:47.723 INFO 5 --- [ main] com.github.pires.hazelcast.Application : Started Application in 13.792 seconds (JVM running for 14.542) 2017-01-30 12:42:54.662 INFO 6 --- [ main] com.github.pires.hazelcast.Application : Started Application in 5.078 seconds (JVM running for 5.771)
2017-01-30 12:44:08.780 INFO 6 --- [thread-Acceptor] c.h.nio.tcp.SocketAcceptorThread : [10.244.9.2]:5701 [someGroup] [3.7.5] Accepting socket connection from /10.244.93.3:45945
2017-01-30 12:44:08.814 INFO 6 --- [cached.thread-1] c.h.nio.tcp.TcpIpConnectionManager : [10.244.9.2]:5701 [someGroup] [3.7.5] Established socket connection between /10.244.9.2:5701 and /10.244.93.3:45945
2017-01-30 12:44:15.785 INFO 6 --- [ration.thread-0] c.h.internal.cluster.ClusterService : [10.244.9.2]:5701 [someGroup] [3.7.5]
Members [2] {
Member [10.244.9.2]:5701 - f9cae801-59da-49d9-b8de-7719abb53844 this
Member [10.244.93.3]:5701 - 4e15667b-ce17-40c2-b045-abe3fb25d48b
}
``` ```
Now let's scale our cluster to 4 nodes: Now let's scale our cluster to 4 nodes:
```sh ```sh
$ kubectl scale rc hazelcast --replicas=4 $ kubectl scale deployment hazelcast --replicas 4
``` ```
Examine the status again by checking the logs and you should see the 4 members connected. Examine the status again by checking a node's logs and you should see the 4 members connected. Something like:
```
(...)
Members [4] {
Member [10.244.9.2]:5701 - f9cae801-59da-49d9-b8de-7719abb53844 this
Member [10.244.93.3]:5701 - 4e15667b-ce17-40c2-b045-abe3fb25d48b
Member [10.244.9.3]:5701 - e0f36fa4-16bf-4009-a034-d4e7a4105003
Member [10.244.93.4]:5701 - 7ac96b48-aa47-4410-885f-1ad0fc3690f0
}
```
### tl; dr; ### tl; dr;
For those of you who are impatient, here is the summary of the commands we ran in this tutorial. For those of you who are impatient, here is the summary of the commands we ran in this tutorial.
```sh ```sh
# create a service to track all hazelcast nodes kubectl create -f service.yaml
kubectl create -f examples/storage/hazelcast/hazelcast-service.yaml kubectl create -f deployment.yaml
kubectl scale deployment hazelcast --replicas 2
# create a replication controller to replicate hazelcast nodes kubectl scale deployment hazelcast --replicas 4
kubectl create -f examples/storage/hazelcast/hazelcast-controller.yaml
# scale up to 2 nodes
kubectl scale rc hazelcast --replicas=2
# scale up to 4 nodes
kubectl scale rc hazelcast --replicas=4
``` ```
### Hazelcast Discovery Source ### Hazelcast Discovery Source

31
examples/storage/hazelcast/hazelcast-controller.yaml
View File

@ -1,31 +0,0 @@
apiVersion: v1
kind: ReplicationController
metadata:
labels:
name: hazelcast
name: hazelcast
spec:
replicas: 1
selector:
name: hazelcast
template:
metadata:
labels:
name: hazelcast
spec:
containers:
- resources:
limits:
cpu: 0.1
image: quay.io/pires/hazelcast-kubernetes:0.6.1
name: hazelcast
env:
- name: "DNS_DOMAIN"
value: "cluster.local"
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
ports:
- containerPort: 5701
name: hazelcast

22
examples/storage/hazelcast/hazelcast-deployment.yaml Normal file
View File

@ -0,0 +1,22 @@
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: hazelcast
labels:
name: hazelcast
spec:
template:
metadata:
labels:
name: hazelcast
spec:
containers:
- name: hazelcast
image: quay.io/pires/hazelcast-kubernetes:0.8.0
imagePullPolicy: Always
env:
- name: "DNS_DOMAIN"
value: "cluster.local"
ports:
- name: hazelcast
containerPort: 5701

24
examples/storage/hazelcast/image/Dockerfile
View File

@ -1,24 +0,0 @@
# Copyright 2016 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
FROM quay.io/pires/docker-jre:8u45-2
EXPOSE 5701
RUN \
curl -Lskj https://github.com/pires/hazelcast-kubernetes-bootstrapper/releases/download/0.5/hazelcast-kubernetes-bootstrapper-0.5.jar \
-o /bootstrapper.jar
CMD java -jar /bootstrapper.jar