Linux——redis主从复制、哨兵模式
一、redis 的安全加固:
- 对redis数据库访问的角度
- auth // 验证登录redis 数据库的用户
- acl // 设置redis用户的权限
- 将配置完成的ACL策略写入配置文件
- config rewrite //目前redis生效的配置全部写入到默认配置文件的尾部
- 写入到acl文件中,在加载配置文件时,找到acl文件
- aclfile 配置选项在配置文件中启用
- acl save 将目前acl list 列出的权限规则写入acl文件
- redis 默认采用明文数据传输,数据传输的加密
- TLS
- 在配置文件中,指定加密证书以及秘钥即可
- 一般只需要配置服务端的证书和秘钥,也可以同时配置客户端的秘钥
- 加密通信配置参数
- tls-port 在指定端口启用tls加密
- tls-cert-file redis.crt // 服务端证书
- tls-key-file redis.key // 服务端秘钥
- tls-key-file-pass secret // 秘钥访问密码
- tls-client-cert-file client.crt // 客户端证书
- tls-client-key-file client.key // 客户端秘钥
- tls-client-key-file-pass secret // 客户端秘钥访问密码
二、redis的水平扩展
在进行水平扩展时,容易遇到以下问题:
- 数据不一致 // 主从复制
- 自动的故障转移 // sential哨兵模式 redis 自带集群功能
- 持续监控redis实例集群
- 异常信息的通知 // sential 哨兵模式
- 水平扩展的操作复杂度
- 通过redis本身特性配置完成,因此操作相对而言比较简单
- 在redis集群模式下,增加新的节点也是非常容易
三、主从复制
redis从节点完全复制主节点的数据,从而成为主节点的数据备份,因此可以在主节点出现故障的情况下,接替主节点功能。
redis的主从复制在数据同步方面默认采用异步策略,采用异步数据来保证数据同步的低延迟高性能。
redis主从复制过程:
- 主从复制建立之后,首先从节点将请求一次全量复制,获取主节点中保存的所有数据;
- 在主从通信良好的情况下,主节点的redis 实例将持续向redis从节点,发送任何导致主节点数据集发生改变的语句,键删除与新建、redis过期的键、
- 在主从复制过程中,如果因为意外或者网络问题,导致主从之间的复制中断,那么从节点恢复与主节点通信后,会再次请求一次全量数据同步,由主节点发送全量数据集后,继续给从节点发送数据改动语句。
redis 主从复制注意事项:
- 主从复制采用异步同步
- 一个主节点可以有多个从节点
- 从节点也可以成为别的从节点的主节点,从而形成一个树状结构
- 主从同步不会阻塞主节点的任何正常操作,因为主节点性能表现几乎不受主从复制的影响
- 使用从节点来实现redis集群的水平扩展,保证数据安全和高可用
- 因为使用从节点作为主节点数据备份,那么主节点可以不用考虑数据持久化配置的 ,可以在从节点上配置rdb或者AOF持久化,降低主节点和磁盘的交互,进一步提升主节点的运行效率
- 主从的故障转移,需要结合哨兵机制
主从复制建立之后,从节点一般默认处于只读状态,只读状态修改配置的方式修改。
replica-read-only yes // 从节点只读 no 从节点可写
四、配置过程:
注意实验环境下防火墙和selinux 均关闭
打开一个新的虚拟机
[root@redis-replica ~]# dnf -y install redis
[root@redis-replica ~]# vim /etc/redis/redis.conf
[root@redis-replica ~]# systemctl restart redis
[root@redis-replica ~]# ss -anput | grep 6379
tcp LISTEN 0 511 192.168.110.137:6379 0.0.0.0:* users:(("redis-server",pid=33967,fd=6))
tcp ESTAB 0 0 192.168.110.137:43791 192.168.110.133:6379 users:(("redis-server",pid=33967,fd=7))
[root@redis-replica ~]# redis-cli -h 192.168.110.137
192.168.110.137:6379> keys *
1) "sset1"
2) "bike:1"
3) "newke"
4) "bike2"
5) "site1"
6) "alice"
7) "set2"
8) "eee"
9) "a_kay"
10) "bike:3"
11) "bike1"
12) "bike:2"
13) "c_key"
14) "queue2"
15) "game_event"
16) "counter"
17) "b_ksy"
18) "bike"
19) "set1"
20) "newqueue"
主节点:
127.0.0.1:6379> LPOP newqueue
"user1"
127.0.0.1:6379> LRANGE newqueue 0 -1
(empty array)
从节点验证:
192.168.110.137:6379> LRANGE newqueue 0 -1
(empty array)
主节点:
127.0.0.1:6379> lpush newqueue newitem item2 item3
(integer) 3
127.0.0.1:6379> LRANGE newqueue 0 -1
1) "item3"
2) "item2"
3) "newitem"
从节点验证:
192.168.110.137:6379> LRANGE newqueue 0 -1
1) "item3"
2) "item2"
3) "newitem"
192.168.110.137:6379> exit
从节点只读:
192.168.110.137:6379> set a b
(error) READONLY You can't write against a read only replica.
192.168.110.137:6379> config get replica-read-only
1) "replica-read-only"
2) "yes"
192.168.110.137:6379> CONFIG SET replica-read-only no
OK
192.168.110.137:6379> set a b
OK
192.168.110.137:6379> get a
"b"
192.168.110.137:6379> CONFIG SET replica-read-only yes
OK
集群模式:
redis的集群提供了一种将数据分散到不同节点进行保存的数据,在保存数据的节点中,又可以设置主从复制,所以保存数据的节点down之后,从节点可以自动接替主节点,无需手工干预。redis集群具有很高的抗风险能力,扩展节点的配置也非常的简单。
redis集群模式之下,不同的redis实例不考虑数据同步,每一个集群内的工作节点上都保存不同的数据,通过hash一致性算法确定每一个键的保存位置。redis集群模式具备以下特性:
- 数据将分散到不同的节点中
- redis集群内部部分节点down 不会导致集群down,只有集群中大部分节点down的情况下,集群才会down。
- redis集群可以实现故障的自动转移
主从模式下主从节点数据完全一致,结合上图架构可以理解为同一个数据同时保存在主从节点上。
redis集群模式如何分散数据?
集群中进行数据存储时,结合hash slot完成数据存储。简单理解:一个redis集群中一共有 16384 个hash slot,当需要进行数据存储时,给新建的键分配一个新的 hash slot , 一个hash slot 对应一组键值对。这些hash slot皆可以分配给不同的节点,每个节点保存对应hash slot 的键值对。
redis 集群模式之下,做好下面两点就可以保证redis集群的可用性和完整性。
- 确保集群中每一个主节点有自己的从节点,同时从节点也需要在集群内
- 保证每一个主节点都分配了合理数量的hash slot
redis 的集群端口一般是 数据端口 + 10000
数据端口:6379 集群端口 16379
[root@redis-master ~]# systemctl stop redis.service
[root@redis-master ~]#
[root@redis-master ~]# mkdir /redis-cluster/700{0..5} -p
[root@redis-master ~]# ls /redis-cluster/
7000 7001 7002 7003 7004 7005
[root@redis-master ~]# cd /redis-cluster/700
-bash: cd: /redis-cluster/700: No such file or directory
[root@redis-master ~]# cd /redis-cluster/7000
[root@redis-master 7000]# vim redis.conf
[root@redis-master 7000]# vim /etc/redis/redis.conf
[root@redis-master 7000]# vim redis.conf
[root@redis-master 7000]# cp redis.conf ../7001/
[root@redis-master 7000]# cp redis.conf ../7002/
[root@redis-master 7000]# cp redis.conf ../7003/
[root@redis-master 7000]# cp redis.conf ../7004/
[root@redis-master 7000]# cp redis.conf ../7005/
[root@redis-master 7000]# sed -i "s/7000/7001/g" /redis-cluster/7001/redis.conf
[root@redis-master 7000]# cat /redis-cluster/7001/redis.conf
port 7001
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes
logfile /redis-cluster/7001/redis-7001.log
[root@redis-master 7000]# sed -i "s/7000/7002/g" /redis-cluster/7002/redis.conf
[root@redis-master 7000]# sed -i "s/7000/7003/g" /redis-cluster/7003/redis.conf
[root@redis-master 7000]# sed -i "s/7000/7004/g" /redis-cluster/7004/redis.conf
[root@redis-master 7000]# sed -i "s/7000/7005/g" /redis-cluster/7005/redis.conf
[root@redis-master 7000]# chown -R redis /redis-cluster/
[root@redis-master 7000]# vim /etc/redis/redis.conf
[root@redis-master 7000]# vim redis.conf
[root@redis-master 7000]# echo "daemonize yes" >> redis.conf
[root@redis-master 7000]# echo "pidfile /redis-cluster/7000/7000.pid" >> redis.conf
[root@redis-master 7000]# echo "daemonize yes" >> ../7001/redis.conf
[root@redis-master 7000]# echo "pidfile /redis-cluster/7001/7001.pid" >> ../7001/redis.conf
[root@redis-master 7000]# echo "daemonize yes" >> ../7002/redis.conf
[root@redis-master 7000]# echo "pidfile /redis-cluster/7002/7002.pid" >> ../7002/redis.conf
[root@redis-master 7000]# echo "pidfile /redis-cluster/7003/7003.pid" >> ../7003/redis.conf
[root@redis-master 7000]# echo "daemonize yes" >> ../7003/redis.conf
[root@redis-master 7000]# echo "daemonize yes" >> ../7004/redis.conf
[root@redis-master 7000]# echo "daemonize yes" >> ../7005/redis.conf
[root@redis-master 7000]# echo "pidfile /redis-cluster/7004/7004.pid" >> ../7004/redis.conf
[root@redis-master 7000]# echo "pidfile /redis-cluster/7005/7005.pid" >> ../7005/redis.conf
[root@redis-master 7000]# redis-server ./redis.conf
[root@redis-master 7000]# ss -anput | grep 7000
tcp LISTEN 0 511 0.0.0.0:17000 0.0.0.0:* users:(("redis-server",pid=3617,fd=10))
tcp LISTEN 0 511 0.0.0.0:7000 0.0.0.0:* users:(("redis-server",pid=3617,fd=6))
tcp LISTEN 0 511 [::]:17000 [::]:* users:(("redis-server",pid=3617,fd=11))
tcp LISTEN 0 511 [::]:7000 [::]:* users:(("redis-server",pid=3617,fd=7))
[root@redis-master 7000]# cd ../7001/
[root@redis-master 7001]# redis-server ./redis.conf
[root@redis-master 7001]# ss -anput | grep 7001
tcp LISTEN 0 511 0.0.0.0:17001 0.0.0.0:* users:(("redis-server",pid=3642,fd=10))
tcp LISTEN 0 511 0.0.0.0:7001 0.0.0.0:* users:(("redis-server",pid=3642,fd=6))
tcp LISTEN 0 511 [::]:17001 [::]:* users:(("redis-server",pid=3642,fd=11))
tcp LISTEN 0 511 [::]:7001 [::]:* users:(("redis-server",pid=3642,fd=7))
[root@redis-master 7001]# cd ../7002
[root@redis-master 7002]# redis-server ./redis.conf
[root@redis-master 7002]# ss -anput | grep 7002
tcp LISTEN 0 511 0.0.0.0:17002 0.0.0.0:* users:(("redis-server",pid=3651,fd=10))
tcp LISTEN 0 511 0.0.0.0:7002 0.0.0.0:* users:(("redis-server",pid=3651,fd=6))
tcp LISTEN 0 511 [::]:17002 [::]:* users:(("redis-server",pid=3651,fd=11))
tcp LISTEN 0 511 [::]:7002 [::]:* users:(("redis-server",pid=3651,fd=7))
[root@redis-master 7002]# cd ../7003
[root@redis-master 7003]# redis-server ./redis.conf
[root@redis-master 7003]# ss -anput | grep 7003
tcp LISTEN 0 511 0.0.0.0:17003 0.0.0.0:* users:(("redis-server",pid=3660,fd=10))
tcp LISTEN 0 511 0.0.0.0:7003 0.0.0.0:* users:(("redis-server",pid=3660,fd=6))
tcp LISTEN 0 511 [::]:17003 [::]:* users:(("redis-server",pid=3660,fd=11))
tcp LISTEN 0 511 [::]:7003 [::]:* users:(("redis-server",pid=3660,fd=7))
[root@redis-master 7003]# cd ../7004
[root@redis-master 7004]# redis-server ./redis.conf
[root@redis-master 7004]# ss -anput | grep 7004
tcp LISTEN 0 511 0.0.0.0:17004 0.0.0.0:* users:(("redis-server",pid=3670,fd=10))
tcp LISTEN 0 511 0.0.0.0:7004 0.0.0.0:* users:(("redis-server",pid=3670,fd=6))
tcp LISTEN 0 511 [::]:17004 [::]:* users:(("redis-server",pid=3670,fd=11))
tcp LISTEN 0 511 [::]:7004 [::]:* users:(("redis-server",pid=3670,fd=7))
[root@redis-master 7004]# cd ../7005
[root@redis-master 7005]# redis-server ./redis.conf
[root@redis-master 7005]# ss -anput | grep 7005
tcp LISTEN 0 511 0.0.0.0:17005 0.0.0.0:* users:(("redis-server",pid=3678,fd=10))
tcp LISTEN 0 511 0.0.0.0:7005 0.0.0.0:* users:(("redis-server",pid=3678,fd=6))
tcp LISTEN 0 511 [::]:17005 [::]:* users:(("redis-server",pid=3678,fd=11))
tcp LISTEN 0 511 [::]:7005 [::]:* users:(("redis-server",pid=3678,fd=7))
[root@redis-master 7005]# ls
7005.pid appendonly.aof nodes.conf redis-7005.log redis.conf
[root@redis-master 7005]# cat nodes.conf
015b10e7a627b8f0326d00f8225370734ce3d07c :0@0 myself,master - 0 0 0 connected
vars currentEpoch 0 lastVoteEpoch 0
[root@redis-master 7005]# redis-c
redis-check-aof redis-check-rdb redis-cli
[root@redis-master 7005]# redis-cli --cluster create 127.0.0.1:7000 \
> 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 \
> --cluster-replicas 1
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 127.0.0.1:7004 to 127.0.0.1:7000
Adding replica 127.0.0.1:7005 to 127.0.0.1:7001
Adding replica 127.0.0.1:7003 to 127.0.0.1:7002
>>> Trying to optimize slaves allocation for anti-affinity
[WARNING] Some slaves are in the same host as their master
M: f145e0a3b35c40faafad2b1c8884091394903d7d 127.0.0.1:7000
slots:[0-5460] (5461 slots) master
M: 963bb6b558d56ac90cecf7ae979eebe9d1bc0058 127.0.0.1:7001
slots:[5461-10922] (5462 slots) master
M: caa5055128fb506f175c26bc2467eda55c4c9665 127.0.0.1:7002
slots:[10923-16383] (5461 slots) master
S: c211b4e3c423331ab4f9d9237c1c4522fd977608 127.0.0.1:7003
replicates f145e0a3b35c40faafad2b1c8884091394903d7d
S: f59c330f1edc2aeea17ba37abecc813e3e6b3554 127.0.0.1:7004
replicates 963bb6b558d56ac90cecf7ae979eebe9d1bc0058
S: 015b10e7a627b8f0326d00f8225370734ce3d07c 127.0.0.1:7005
replicates caa5055128fb506f175c26bc2467eda55c4c9665
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
.
>>> Performing Cluster Check (using node 127.0.0.1:7000)
M: f145e0a3b35c40faafad2b1c8884091394903d7d 127.0.0.1:7000
slots:[0-5460] (5461 slots) master
1 additional replica(s)
M: 963bb6b558d56ac90cecf7ae979eebe9d1bc0058 127.0.0.1:7001
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
S: 015b10e7a627b8f0326d00f8225370734ce3d07c 127.0.0.1:7005
slots: (0 slots) slave
replicates caa5055128fb506f175c26bc2467eda55c4c9665
S: c211b4e3c423331ab4f9d9237c1c4522fd977608 127.0.0.1:7003
slots: (0 slots) slave
replicates f145e0a3b35c40faafad2b1c8884091394903d7d
M: caa5055128fb506f175c26bc2467eda55c4c9665 127.0.0.1:7002
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
S: f59c330f1edc2aeea17ba37abecc813e3e6b3554 127.0.0.1:7004
slots: (0 slots) slave
replicates 963bb6b558d56ac90cecf7ae979eebe9d1bc0058
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
[root@redis-master 7005]# cat nodes.conf
c211b4e3c423331ab4f9d9237c1c4522fd977608 127.0.0.1:7003@17003 slave f145e0a3b35c40faafad2b1c8884091394903d7d 0 1725507593185 1 connected
f145e0a3b35c40faafad2b1c8884091394903d7d 127.0.0.1:7000@17000 master - 1725507595507 1725507592989 1 connected 0-5460
f59c330f1edc2aeea17ba37abecc813e3e6b3554 127.0.0.1:7004@17004 slave 963bb6b558d56ac90cecf7ae979eebe9d1bc0058 0 1725507595004 2 connected
963bb6b558d56ac90cecf7ae979eebe9d1bc0058 127.0.0.1:7001@17001 master - 0 1725507593992 2 connected 5461-10922
015b10e7a627b8f0326d00f8225370734ce3d07c 127.0.0.1:7005@17005 myself,slave caa5055128fb506f175c26bc2467eda55c4c9665 0 1725507593000 3 connected
caa5055128fb506f175c26bc2467eda55c4c9665 127.0.0.1:7002@17002 master - 0 1725507594000 3 connected 10923-16383
vars currentEpoch 6 lastVoteEpoch 0
[root@redis-master 7005]# redis-cli -p 7000 -c
127.0.0.1:7000> set testa testtest
OK
127.0.0.1:7000> get testa
"testtest"
127.0.0.1:7000> exit
[root@redis-master 7005]# redis-cli -p 7001 -c
127.0.0.1:7001> get testa
-> Redirected to slot [2133] located at 127.0.0.1:7000
"testtest"
127.0.0.1:7000> exit
[root@redis-master 7005]# cat nodes.conf
c211b4e3c423331ab4f9d9237c1c4522fd977608 127.0.0.1:7003@17003 slave f145e0a3b35c40faafad2b1c8884091394903d7d 0 1725507593185 1 connected
f145e0a3b35c40faafad2b1c8884091394903d7d 127.0.0.1:7000@17000 master - 1725507595507 1725507592989 1 connected 0-5460
f59c330f1edc2aeea17ba37abecc813e3e6b3554 127.0.0.1:7004@17004 slave 963bb6b558d56ac90cecf7ae979eebe9d1bc0058 0 1725507595004 2 connected
963bb6b558d56ac90cecf7ae979eebe9d1bc0058 127.0.0.1:7001@17001 master - 0 1725507593992 2 connected 5461-10922
015b10e7a627b8f0326d00f8225370734ce3d07c 127.0.0.1:7005@17005 myself,slave caa5055128fb506f175c26bc2467eda55c4c9665 0 1725507593000 3 connected
caa5055128fb506f175c26bc2467eda55c4c9665 127.0.0.1:7002@17002 master - 0 1725507594000 3 connected 10923-16383
vars currentEpoch 6 lastVoteEpoch 0
## 模拟7000端口redis 实例宕机
[root@redis-master 7005]# cd ../7000
[root@redis-master 7000]# cat 7000.pid
3617
[root@redis-master 7000]# kill -9 3617
[root@redis-master 7000]# cd -
/redis-cluster/7005
[root@redis-master 7005]# cat nodes.conf
c211b4e3c423331ab4f9d9237c1c4522fd977608 127.0.0.1:7003@17003 master - 0 1725507903053 7 connected 0-5460
f145e0a3b35c40faafad2b1c8884091394903d7d 127.0.0.1:7000@17000 master,fail - 1725507898600 1725507896000 1 disconnected
f59c330f1edc2aeea17ba37abecc813e3e6b3554 127.0.0.1:7004@17004 slave 963bb6b558d56ac90cecf7ae979eebe9d1bc0058 0 1725507903766 2 connected
963bb6b558d56ac90cecf7ae979eebe9d1bc0058 127.0.0.1:7001@17001 master - 0 1725507902751 2 connected 5461-10922
015b10e7a627b8f0326d00f8225370734ce3d07c 127.0.0.1:7005@17005 myself,slave caa5055128fb506f175c26bc2467eda55c4c9665 0 1725507904000 3 connected
caa5055128fb506f175c26bc2467eda55c4c9665 127.0.0.1:7002@17002 master - 0 1725507903000 3 connected 10923-16383
vars currentEpoch 7 lastVoteEpoch 0
[root@redis-master 7005]# redis-cli -p 7002 -c
127.0.0.1:7002> keys *
(empty array)
127.0.0.1:7002> get testa
-> Redirected to slot [2133] located at 127.0.0.1:7003
"testtest"
127.0.0.1:7003> exit
[root@redis-master 7005]# cd -
/redis-cluster/7000
[root@redis-master 7000]# pwd
/redis-cluster/7000
[root@redis-master 7000]# redis-server ./redis.conf
[root@redis-master 7000]# cat nodes.conf
963bb6b558d56ac90cecf7ae979eebe9d1bc0058 127.0.0.1:7001@17001 master - 0 1725508058257 2 connected 5461-10922
c211b4e3c423331ab4f9d9237c1c4522fd977608 127.0.0.1:7003@17003 master - 0 1725508058257 7 connected 0-5460
f145e0a3b35c40faafad2b1c8884091394903d7d 127.0.0.1:7000@17000 myself,slave c211b4e3c423331ab4f9d9237c1c4522fd977608 0 1725508058253 7 connected
f59c330f1edc2aeea17ba37abecc813e3e6b3554 127.0.0.1:7004@17004 slave 963bb6b558d56ac90cecf7ae979eebe9d1bc0058 0 1725508058257 2 connected
015b10e7a627b8f0326d00f8225370734ce3d07c 127.0.0.1:7005@17005 slave caa5055128fb506f175c26bc2467eda55c4c9665 0 1725508058257 3 connected
caa5055128fb506f175c26bc2467eda55c4c9665 127.0.0.1:7002@17002 master - 0 1725508058257 3 connected 10923-16383
vars currentEpoch 7 lastVoteEpoch 0
[root@redis-master 7000]# cat nodes.conf
963bb6b558d56ac90cecf7ae979eebe9d1bc0058 127.0.0.1:7001@17001 master - 0 1725508058257 2 connected 5461-10922
c211b4e3c423331ab4f9d9237c1c4522fd977608 127.0.0.1:7003@17003 master - 0 1725508058257 7 connected 0-5460
f145e0a3b35c40faafad2b1c8884091394903d7d 127.0.0.1:7000@17000 myself,slave c211b4e3c423331ab4f9d9237c1c4522fd977608 0 1725508058253 7 connected
f59c330f1edc2aeea17ba37abecc813e3e6b3554 127.0.0.1:7004@17004 slave 963bb6b558d56ac90cecf7ae979eebe9d1bc0058 0 1725508058257 2 connected
015b10e7a627b8f0326d00f8225370734ce3d07c 127.0.0.1:7005@17005 slave caa5055128fb506f175c26bc2467eda55c4c9665 0 1725508058257 3 connected
caa5055128fb506f175c26bc2467eda55c4c9665 127.0.0.1:7002@17002 master - 0 1725508058257 3 connected 10923-16383
vars currentEpoch 7 lastVoteEpoch 0
[root@redis-master 7000]# 000 m