camellia redis proxy v1.3.3对redis主从进行读写分离(非写死,自动识别故障转移)
1 概述
camellia-redis-proxy是一款高性能的redis代理(https://github.com/netease-im/camellia),使用netty4开发,主要特性如下:
- 支持代理到redis-standalone、redis-sentinel、redis-cluster。
- 支持其他proxy作为后端(如双写迁移场景),如 twemproxy 、codis 等。
- 支持 kvrocks 、 pika 、 tendis 等作为后端。
- 支持自定义分片。
- 支持读写分离。
- 支持双(多)写,可以proxy直连双写,也可以基于mq(如kafka)双写,也可以基于插件体系自定义双写规则。
- 支持多租户,即租户A路由到redis1,租户B路由到redis2(可以通过不同的clientname区分,也可以通过不同的password区分)。
支持多租户动态路由,支持自定义的动态路由数据源(内置:本地配置文件、nacos、etcd等,也可以自定义)。 - 支持自定义插件,并且内置了很多插件,可以按需使用(包括:大key监控、热key监控、热key缓存、key命名空间、ip黑白名单、速率控制等等)。
- 支持丰富的监控,可以监控客户端连接数、调用量、方法耗时、大key、热key、后端redis连接数和耗时等,并且支持以http接口形式获取监控数据。
- 等等其他优点。
本文介绍camellia-redis-proxy为哨兵模式部署的redis主从实例进行能读写分离,是能识别redis故障转移的读写分离。
2 部署redis一主两从
cd /tmp
helm pull --untar stable/redis-ha
helm install myredis -n default --set hardAntiAffinity=false ./redis-ha
3 部署camellia redis proxy
将如下三个文件apply到k8s集群即可。
apiVersion: apps/v1
kind: Deployment
metadata:
name: deploy-db-camellia-test
labels:
app: pod-db-camellia-test
spec:
replicas: 1
selector:
matchLabels:
app: pod-db-camellia-test
template:
metadata:
labels:
app: pod-db-camellia-test
spec:
enableServiceLinks: false
containers:
- name: container-camellia
image: swr.cn-south-1.myhuaweicloud.com/migrator/camellia-redis-proxy:1.3.3
command:
- java
- "-XX:+UseG1GC"
- "-XX:+UseContainerSupport"
- "-Dio.netty.tryReflectionSetAccessible=true"
- "--add-opens"
- "java.base/java.lang=ALL-UNNAMED"
- "--add-opens"
- "java.base/java.io=ALL-UNNAMED"
- "--add-opens"
- "java.base/java.math=ALL-UNNAMED"
- "--add-opens"
- "java.base/java.net=ALL-UNNAMED"
- "--add-opens"
- "java.base/java.nio=ALL-UNNAMED"
- "--add-opens"
- "java.base/java.security=ALL-UNNAMED"
- "--add-opens"
- "java.base/java.text=ALL-UNNAMED"
- "--add-opens"
- "java.base/java.time=ALL-UNNAMED"
- "--add-opens"
- "java.base/java.util=ALL-UNNAMED"
- "--add-opens"
- "java.base/jdk.internal.access=ALL-UNNAMED"
- "--add-opens"
- "java.base/jdk.internal.misc=ALL-UNNAMED"
- "--add-opens"
- "java.base/sun.net.util=ALL-UNNAMED"
- "-Xms100m"
- "-Xmx4096m"
- "-server"
- "org.springframework.boot.loader.JarLauncher"
resources:
requests:
memory: "100Mi"
cpu: "1"
limits:
memory: "4Gi"
cpu: "1"
ports:
- containerPort: 6380
name: db
protocol: TCP
volumeMounts:
- name: cm-db-camellia-test
mountPath: /opt/camellia-redis-proxy/BOOT-INF/classes/application.yml
subPath: application-sentinel-singlewrite-multiread.yml
- name: cm-db-camellia-test
mountPath: /opt/camellia-redis-proxy/BOOT-INF/classes/resource-sentinel-singlewrite-multiread.json
subPath: resource-sentinel-singlewrite-multiread.json
volumes:
- name: cm-db-camellia-test
configMap:
name: cm-db-camellia-test
apiVersion: v1
kind: ConfigMap
metadata:
name: cm-db-camellia-test
data:
application-sentinel-singlewrite-multiread.yml: |
server:
port: 6380
spring:
application:
name: camellia-redis-proxy-server
camellia-redis-proxy:
console-port: 16379
password: pass123
monitor-enable: false
monitor-interval-seconds: 60
plugins: #plugin list
- monitorPlugin
- bigKeyPlugin
- hotKeyPlugin
config:
"check.redis.resource.valid.not.active.threshold.sceonds": 300
transpond:
type: local
local:
type: complex
json-file: resource-sentinel-singlewrite-multiread.json
redis-conf:
preheat: false
close-idle-connection: true
check-idle-connection-threshold-seconds: 600
close-idle-connection-delay-seconds: 60
resource-sentinel-singlewrite-multiread.json: |
{
"type": "simple",
"operation": {
"read": "redis-sentinel-slaves://@myredis-redis-ha-announce-0:26379/mymaster?withMaster=false",
"type": "rw_separate",
"write": "redis-sentinel://@myredis-redis-ha-announce-1:26379/mymaster"
}
}
apiVersion: v1
kind: Service
metadata:
annotations:
prometheus.io/path: /metrics
prometheus.io/port: "16379"
prometheus.io/scrape: "true"
prometheus.io/scrape_slow: "true"
prometheus.io/services: "false"
name: svc-db-camellia-test
namespace: default
spec:
ports:
- name: port-6380
port: 6380
protocol: TCP
targetPort: 6380
- name: port-16379
port: 16379
protocol: TCP
targetPort: 16379
selector:
app: pod-db-camellia-test
sessionAffinity: None
type: ClusterIP
4 测试
redis-0是主。
在camellia查看upstream信息,看见10.247.192.224:6379,记住这个IP地址。
将redis-0删除后,重启的redis-0变成从,但是从camellia代理去写入key依然是成功,说明camellia是自动识别了redis的故障转移。
对camellia执行info后,看见upstream信息如下,10.247.217.97:6379和删除redis-0之前看到10.247.192.224:6379,是两个地址,说明自动识别了主从的故障转移。
5 小结
本文介绍camellia-redis-proxy为哨兵模式部署的redis主从实例进行能读写分离,redis主从的故障转移是被代理识别的。