mirror of
https://github.com/k3s-io/kubernetes.git
synced 2025-08-14 06:15:45 +00:00
Add distributed Minio example
This commit is contained in:
Nitish
parent
ee59ecb9ff
commit
6dae1a7580
195
examples/storage/minio-distributed/README.md
Normal file
195
examples/storage/minio-distributed/README.md
Normal file
@ -0,0 +1,195 @@
|
||||
# Cloud Native Deployment of Distributed Minio using Kubernetes
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [Introduction](#introduction)
|
||||
- [Prerequisites](#prerequisites)
|
||||
- [Quickstart](#quickstart)
|
||||
- [Step 1: Create Minio Headless Service](#step-1-create-minio-headless-service)
|
||||
- [Step 2: Create Minio Statefulset](#step-2-create-minio-statefulset)
|
||||
- [Step 3: Create LoadBalancer Service](#step-3-create-minio-service)
|
||||
- [Step 4: Resource cleanup](#step-4-resource-cleanup)
|
||||
|
||||
## Introduction
|
||||
Minio is an AWS S3 compatible, object storage server built for cloud applications and devops. Minio is _cloud native_, meaning Minio understands that it
|
||||
is running within a cluster manager, and uses the cluster management infrastructure for allocation of compute and storage resources.
|
||||
|
||||
The following document describes the process to deploy [distributed Minio](https://docs.minio.io/docs/distributed-minio-quickstart-guide) server on Kubernetes.
|
||||
This example uses the [official Minio Docker image](https://hub.docker.com/r/minio/minio/~/dockerfile/) from Docker Hub.
|
||||
|
||||
This example uses some of the core components of Kubernetes:
|
||||
|
||||
- [_Pods_](https://kubernetes.io/docs/user-guide/pods/)
|
||||
- [_Services_](https://kubernetes.io/docs/user-guide/services/)
|
||||
- [_Statefulsets_](https://kubernetes.io/docs/tutorials/stateful-application/basic-stateful-set/)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
This example assumes that you have a Kubernetes version >=1.5 cluster installed and running, and that you have installed the [`kubectl`](../../../docs/user-guide/kubectl/kubectl.md)
|
||||
command line tool somewhere in your path. Please see the
|
||||
[getting started guides](../../../docs/getting-started-guides/)
|
||||
for installation instructions for your platform.
|
||||
|
||||
## Quickstart
|
||||
|
||||
Run the below commands to get started quickly
|
||||
|
||||
```sh
|
||||
kubectl create -f examples/storage/minio/minio-distributed-headless-service.yaml
|
||||
kubectl create -f examples/storage/minio/minio-distributed-statefulset.yaml
|
||||
kubectl create -f examples/storage/minio/minio-distributed-service.yaml
|
||||
```
|
||||
|
||||
## Step 1: Create Minio Headless Service
|
||||
|
||||
Headless Service controls the domain within which we create StatefulSets. The domain managed by this Service takes the form: `$(service name).$(namespace).svc.cluster.local` (where “cluster.local” is the cluster domain), and the pods in this domain take the form: `$(pod-name-{i}).$(service name).$(namespace).svc.cluster.local`. This is required to get a DNS resolvable URL for each of the pods created within the Statefulset.
|
||||
|
||||
This is the Headless service description.
|
||||
|
||||
```sh
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: minio
|
||||
labels:
|
||||
app: minio
|
||||
spec:
|
||||
clusterIP: None
|
||||
ports:
|
||||
- port: 9000
|
||||
name: minio
|
||||
selector:
|
||||
app: minio
|
||||
```
|
||||
[Download example] (minio-distributed-headless-service.yaml?raw=true)
|
||||
|
||||
Create the Headless Service
|
||||
|
||||
```sh
|
||||
kubectl create -f minio-distributed-headless-service.yaml
|
||||
```
|
||||
The response should be like this:
|
||||
|
||||
```sh
|
||||
service "minio" created
|
||||
```
|
||||
|
||||
# Step 2: Create Minio Statefulset
|
||||
|
||||
A StatefulSet provides a deterministic name and a unique identity to each pod, making it easy to deploy stateful distributed applications. To launch distributed Minio you need to pass drive locations as parameters to the minio server command. Then, you’ll need to run the same command on all the participating pods. StatefulSets offer a perfect way to handle this requirement.
|
||||
|
||||
This is the Statefulset description.
|
||||
|
||||
```sh
|
||||
apiVersion: apps/v1beta1
|
||||
kind: StatefulSet
|
||||
metadata:
|
||||
name: minio
|
||||
spec:
|
||||
serviceName: "minio"
|
||||
replicas: 4
|
||||
template:
|
||||
metadata:
|
||||
annotations:
|
||||
pod.alpha.kubernetes.io/initialized: "true"
|
||||
labels:
|
||||
app: minio
|
||||
spec:
|
||||
containers:
|
||||
- name: minio
|
||||
env:
|
||||
- name: MINIO_ACCESS_KEY
|
||||
value: "minio"
|
||||
- name: MINIO_SECRET_KEY
|
||||
value: "minio123"
|
||||
image: minio/minio
|
||||
command: ["minio"]
|
||||
args: ["server", "http://minio-0.minio.default.svc.cluster.local/data", "http://minio-1.minio.default.svc.cluster.local/data", "http://minio-2.minio.default.svc.cluster.local/data", "http://minio-3.minio.default.svc.cluster.local/data"]
|
||||
ports:
|
||||
- containerPort: 9000
|
||||
hostPort: 9000
|
||||
# These volume mounts are persistent. Each pod in the PetSet
|
||||
# gets a volume mounted based on this field.
|
||||
volumeMounts:
|
||||
- name: data
|
||||
mountPath: /data
|
||||
# These are converted to volume claims by the controller
|
||||
# and mounted at the paths mentioned above.
|
||||
volumeClaimTemplates:
|
||||
- metadata:
|
||||
name: data
|
||||
annotations:
|
||||
volume.alpha.kubernetes.io/storage-class: anything
|
||||
spec:
|
||||
accessModes: [ "ReadWriteOnce" ]
|
||||
resources:
|
||||
requests:
|
||||
storage: 10Gi
|
||||
```
|
||||
|
||||
[Download example] (minio-distributed-statefulset.yaml?raw=true)
|
||||
|
||||
Create the Statefulset
|
||||
|
||||
```sh
|
||||
kubectl create -f minio-distributed-statefulset.yaml
|
||||
```
|
||||
|
||||
The response should be like this
|
||||
|
||||
```sh
|
||||
statefulset "minio" created
|
||||
```
|
||||
|
||||
# Step 3: Create Minio Service
|
||||
|
||||
Now that you have a Minio statefulset running, you may either want to access it internally (within the cluster) or expose it as a Service onto an external (outside of your cluster, maybe public internet) IP address, depending on your use case. You can achieve this using Services. There are 3 major service types — default type is ClusterIP, which exposes a service to connection from inside the cluster. NodePort and LoadBalancer are two types that expose services to external traffic.
|
||||
|
||||
In this example, we expose the Minio Deployment by creating a LoadBalancer service. This is the service description.
|
||||
|
||||
```sh
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: minio-service
|
||||
spec:
|
||||
type: LoadBalancer
|
||||
ports:
|
||||
- port: 9000
|
||||
targetPort: 9000
|
||||
protocol: TCP
|
||||
selector:
|
||||
app: minio-server
|
||||
```
|
||||
|
||||
[Download example] (minio-distributed-service.yaml?raw=true)
|
||||
|
||||
```sh
|
||||
kubectl create -f minio-distributed-service.yaml
|
||||
```
|
||||
|
||||
The response should be like this
|
||||
|
||||
```sh
|
||||
service "minio-service" created
|
||||
```
|
||||
To check if the service was created successfully, run the command
|
||||
|
||||
```sh
|
||||
kubectl get svc minio-service
|
||||
```
|
||||
|
||||
You should get a response like this
|
||||
```sh
|
||||
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
|
||||
minio-service 10.55.248.23 104.199.249.165 9000:31852/TCP 1m
|
||||
```
|
||||
|
||||
# Step 4: Resource cleanup
|
||||
|
||||
Once you are done, cleanup the cluster using
|
||||
```sh
|
||||
kubectl delete statefulset minio \
|
||||
&& kubectl delete svc minio \
|
||||
&& kubectl delete svc minio-service
|
||||
```
|
||||
@ -0,0 +1,13 @@
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: minio
|
||||
labels:
|
||||
app: minio
|
||||
spec:
|
||||
clusterIP: None
|
||||
ports:
|
||||
- port: 9000
|
||||
name: minio
|
||||
selector:
|
||||
app: minio
|
||||
@ -0,0 +1,14 @@
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: minio-service
|
||||
labels:
|
||||
app: minio
|
||||
spec:
|
||||
ports:
|
||||
- port: 9000
|
||||
targetPort: 9000
|
||||
protocol: "TCP"
|
||||
name: minio
|
||||
selector:
|
||||
app: minio
|
||||
@ -0,0 +1,44 @@
|
||||
apiVersion: apps/v1beta1
|
||||
kind: StatefulSet
|
||||
metadata:
|
||||
name: minio
|
||||
spec:
|
||||
serviceName: "minio"
|
||||
replicas: 4
|
||||
template:
|
||||
metadata:
|
||||
annotations:
|
||||
pod.alpha.kubernetes.io/initialized: "true"
|
||||
labels:
|
||||
app: minio
|
||||
spec:
|
||||
containers:
|
||||
- name: minio
|
||||
env:
|
||||
- name: MINIO_ACCESS_KEY
|
||||
value: "minio"
|
||||
- name: MINIO_SECRET_KEY
|
||||
value: "minio123"
|
||||
image: minio/minio
|
||||
command: ["minio"]
|
||||
args: ["server", "http://minio-0.minio.default.svc.cluster.local/data", "http://minio-1.minio.default.svc.cluster.local/data", "http://minio-2.minio.default.svc.cluster.local/data", "http://minio-3.minio.default.svc.cluster.local/data"]
|
||||
ports:
|
||||
- containerPort: 9000
|
||||
hostPort: 9000
|
||||
# These volume mounts are persistent. Each pod in the PetSet
|
||||
# gets a volume mounted based on this field.
|
||||
volumeMounts:
|
||||
- name: data
|
||||
mountPath: /data
|
||||
# These are converted to volume claims by the controller
|
||||
# and mounted at the paths mentioned above.
|
||||
volumeClaimTemplates:
|
||||
- metadata:
|
||||
name: data
|
||||
annotations:
|
||||
volume.alpha.kubernetes.io/storage-class: anything
|
||||
spec:
|
||||
accessModes: [ "ReadWriteOnce" ]
|
||||
resources:
|
||||
requests:
|
||||
storage: 10Gi
|
||||
Loading…
Reference in New Issue
Block a user