第五十八节 k8s1.30.x 安装Redis集群

cat /etc/hosts
---
192.168.80.31 lyc-80-31
192.168.80.32 lyc-80-32
192.168.80.33 lyc-80-33

系统版本Rocky linux 8.10
docker版本  26.1.3
k8s版本     v1.31.3

集群已免密互信，初始化配置一设置，k8s集群已安装

存储规划：
   使用 OpenEBS 来作为存储引擎，OpenEBS 是一个开源的、可扩展的存储平台，
   它提供了一种简单的方式来创建和管理持久化存储卷。它支持各种存储后端，包括但不限于
   ZFS、Btrfs、XFS 等。同时，OpenEBS 具有高度的可扩展性和可配置性，可以满足不同的存储需求。
   使用 redis-trib 管理 Redis 集群。 
   
   redis-trib 是官方提供的 Redis Cluster 的管理工具，可以检查集群、创建删除节点、删除key、
   批量下发实例命令等等。
   
   如果没有安装 OpenEBS，需要先安装 ：
   wget https://openebs.github.io/charts/openebs-operator.yaml

cat openebs-operator.yaml | grep image
---
image: openebs/node-disk-manager:2.1.0
image: openebs/node-disk-operator:2.1.0
image: openebs/node-disk-exporter:2.1.0
image: openebs/provisioner-localpv:3.4.0

准备镜像
docker pull openebs/node-disk-manager:2.1.0
docker pull openebs/node-disk-operator:2.1.0
docker pull openebs/node-disk-exporter:2.1.0
docker pull openebs/provisioner-localpv:3.4.0

执行安装
kubectl apply -f openebs-operator.yaml

查看pod调度状态
kubectl get pod -A

导入docker镜像
docker load -i  openebs_node-disk-manager.tar
docker load -i  openebs_node-disk-exporter.tar
docker load -i  openebs_node-disk-operator.tar
docker load -i  openebs_provisioner-localpv.tar

查看pod状态
kubectl get pod -n openebs
kubectl get pod -n openebs -o wide

所有的 OpenEBS pods 都处于Running状态表示正常

创建 StorageClass ，这里使用 OpenEBS 的 hostpath 模式：
vim redis-local-storage.yml
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: redis-local-storage
  annotations:
    openebs.io/cas-type: local
    cas.openebs.io/config: |
      - name: StorageType
        value: hostpath
      - name: BasePath
        value: /data/openebs/redis
provisioner: openebs.io/local
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer
---

kubectl apply -f redis-local-storage.yml

创建一个redis的命名空间

vim redis-ns.yaml
---
apiVersion: v1
kind: Namespace
metadata:
  name: redis
  labels:
    name: redis
---

kubectl apply -f redis-ns.yaml 
kubectl get ns

vim redis.yaml
---
apiVersion: v1
kind: Service
metadata:
  name: redis-hl
  namespace: redis
  labels:
    app: redis-hl
spec:
  ports:
  - port: 6379
    name: client
  clusterIP: None
  selector:
    app: redis
 
---
apiVersion: v1
kind: Service
metadata:
  name: redis-np
  namespace: redis
  labels:
    app: redis-np
spec:
  type: NodePort
  ports:
  - port: 6379
    name: client
    nodePort: 31639
    targetPort: 6379
  selector:
    app: redis
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: redis-config
  namespace: redis
  labels:
    app: redis-config
data:
  redis.conf: |+ 
    cluster-enabled yes
    cluster-config-file nodes.conf
    cluster-node-timeout 5000
    cluster-require-full-coverage no
    cluster-migration-barrier 1
    appendonly yes
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: redis
  namespace: redis
spec:
  serviceName: "redis-hl"
  replicas: 6
  selector: 
    matchLabels: 
      app: redis
  template:
    metadata:
      labels:
        app: redis
    spec:
      containers:
      - name: redis
        image: redis:6.2.3
        ports:
        - containerPort: 6379
          name: server
        command: ["redis-server", "/etc/redis/redis.conf"]
        volumeMounts: 
          - name: redis-config
            mountPath: /etc/redis/redis.conf
            subPath: redis.conf
          - name: redis-data
            mountPath: /data 
      volumes:
      - name: redis-config
        configMap: 
          name: redis-config
  volumeClaimTemplates:
    - metadata:
        name: redis-data
      spec:
        accessModes: ["ReadWriteOnce"]
        storageClassName: redis-local-storage
        resources:
          requests:
            storage: 5Gi
---

kubectl apply -f redis.yaml

kubectl get pod -n redis
---
redis-0   1/1     Running   0          93s
redis-1   1/1     Running   0          77s
redis-2   1/1     Running   0          39s
redis-3   1/1     Running   0          34s
redis-4   1/1     Running   0          29s
redis-5   1/1     Running   0          24s

vim redis-trib.yml
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis-trib
  namespace: redis
  labels:
    app: redis-trib
spec:
  replicas: 1
  selector:
    matchLabels:
      app: redis-trib
  template:
    metadata:
      name: redis-trib
      labels:
        app: redis-trib
    spec:
     containers:
     - name: redis-trib
       image: sunnywang/redis-tools-ubuntu:v0.5.1
       imagePullPolicy: IfNotPresent
       args:
       - /bin/bash
       - -c
       - sleep 3600
---

kubectl apply -f redis-trib.yml
提前准备镜像
docker pull sunnywang/redis-tools-ubuntu:v0.5.1

kubectl get pod -n redis -o wide

【格式化redis】
kubectl get pod -n redis

进入到redis-trib pod中
kubectl exec -it redis-trib-6996bf58dd-xh8fz  -n redis  -- /bin/bash

【初始化主节点】
规划 redis-0、redis-1、redis-2 为主节点
redis-trib.py create `dig +short redis-0.redis-hl.redis.svc.cluster.local`:6379 `dig +short redis-1.redis-hl.redis.svc.cluster.local`:6379 `dig +short redis-2.redis-hl.redis.svc.cluster.local`:6379

为 redis-0 添加 redis-3 副本节点：
redis-trib.py replicate   --master-addr `dig +short redis-0.redis-hl.redis.svc.cluster.local`:6379   --slave-addr `dig +short redis-3.redis-hl.redis.svc.cluster.local`:6379

为 redis-1 添加 redis-4 副本节点：
redis-trib.py replicate   --master-addr `dig +short redis-1.redis-hl.redis.svc.cluster.local`:6379   --slave-addr `dig +short redis-4.redis-hl.redis.svc.cluster.local`:6379

为 redis-2 添加 redis-5 副本节点：
redis-trib.py replicate   --master-addr `dig +short redis-2.redis-hl.redis.svc.cluster.local`:6379   --slave-addr `dig +short redis-5.redis-hl.redis.svc.cluster.local`:6379

连接任意 redis 节点，查看集群状态：
redis-cli -h redis-0.redis-hl -c
cluster info

查看集群状态
cluster nodes

测试读写
 set a b 
 get a