CLUSTERDOWN Hash slot not served问题复现
1、一主两从，
172.31.4.18:6381> cluster nodes
f5e774bd5cd8f5bcba53f1297015f3245dd3f18c 172.31.4.20:6383@16383 master - 0 1726278342891 10 connected 5461-10922
9a79febbfb1d7a8f7a6ba3fb11e86b0f122f71e7 172.31.4.21:6386@16386 slave 401730ba6e1c2cd8b60243990f0965606edfc083 0 1726278344894 2 connected
23bb415a72cf125c8205be6b1de4f8f5a0a636c8 172.31.4.18:6381@16381 myself,master - 0 1726278336000 1 connected 0-5460
602fbfe65cf83178c0b897441013ec0bae590c96 172.31.4.20:6385@16385 slave 401730ba6e1c2cd8b60243990f0965606edfc083 0 1726278341890 2 connected
3dec7d5dd8139854f766417611026b07785c0a76 172.31.4.19:6382@16382 slave 23bb415a72cf125c8205be6b1de4f8f5a0a636c8 0 1726278342000 1 connected
83d609a16f5fc2605ff8fd698197533c6ad29334 172.31.4.18:6383@16383 master - 0 1726278343000 9 connected
401730ba6e1c2cd8b60243990f0965606edfc083 172.31.4.18:6385@16385 master - 0 1726278343000 2 connected 10923-16383
4e25e8f87029a22bbba7aa5f6fb2edba4a342da5 172.31.4.21:6384@16384 slave f5e774bd5cd8f5bcba53f1297015f3245dd3f18c 0 1726278343892 10 connected
172.31.4.18:6381> exit
2、从172.31.4.22上发4个并发
redis-benchmark -h 172.31.4.18 -p 6383 -a Redis_6.2.1_Sc -t set,get -e -c 4 -d 1 -n 100000000000000
[cpcn@redis01 redis6383]$ netstat -anltp |grep 4.22
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
tcp 0 31 172.31.4.18:6383 172.31.4.22:54100 ESTABLISHED 30548/redis-server
tcp 0 31 172.31.4.18:6383 172.31.4.22:54098 ESTABLISHED 30548/redis-server
tcp 0 7 172.31.4.18:6385 172.27.16.58:4422 ESTABLISHED 30559/redis-server
tcp 0 31 172.31.4.18:6383 172.31.4.22:54102 ESTABLISHED 30548/redis-server
tcp 0 31 172.31.4.18:6383 172.31.4.22:54096 ESTABLISHED 30548/redis-server
tcp 0 0 172.31.4.18:16383 172.31.4.21:46224 ESTABLISHED 30548/redis-server
3、172.31.4.18 6383 切为从，老链接还在：
172.31.4.18:6383> cluster failover takeover
OK
172.31.4.18:6383> cluster nodes
83d609a16f5fc2605ff8fd698197533c6ad29334 172.31.4.18:6383@16383 myself,slave f5e774bd5cd8f5bcba53f1297015f3245dd3f18c 0 1726278372000 10 connected
401730ba6e1c2cd8b60243990f0965606edfc083 172.31.4.18:6385@16385 master - 0 1726278374588 2 connected 10923-16383
4e25e8f87029a22bbba7aa5f6fb2edba4a342da5 172.31.4.21:6384@16384 slave f5e774bd5cd8f5bcba53f1297015f3245dd3f18c 0 1726278373587 10 connected
f5e774bd5cd8f5bcba53f1297015f3245dd3f18c 172.31.4.20:6383@16383 master - 0 1726278372840 10 connected 5461-10922
23bb415a72cf125c8205be6b1de4f8f5a0a636c8 172.31.4.18:6381@16381 master - 0 1726278371000 1 connected 0-5460
9a79febbfb1d7a8f7a6ba3fb11e86b0f122f71e7 172.31.4.21:6386@16386 slave 401730ba6e1c2cd8b60243990f0965606edfc083 0 1726278369000 2 connected
602fbfe65cf83178c0b897441013ec0bae590c96 172.31.4.20:6385@16385 slave 401730ba6e1c2cd8b60243990f0965606edfc083 0 1726278371000 2 connected
3dec7d5dd8139854f766417611026b07785c0a76 172.31.4.19:6382@16382 slave 23bb415a72cf125c8205be6b1de4f8f5a0a636c8 0 1726278370582 1 connected
172.31.4.18:6383>
4、将 172.31.4.18 6383 下线，该进程还在，老链接还在。
[cpcn@redis01 redis6381]$ ./bin/redis-cli -a Redis_6.2.1_Sc --cluster del-node 172.31.4.18:6383 83d609a16f5fc2605ff8fd698197533c6ad29334
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Removing node 83d609a16f5fc2605ff8fd698197533c6ad29334 from cluster 172.31.4.18:6383
>>> Sending CLUSTER FORGET messages to the cluster...
>>> Sending CLUSTER RESET SOFT to the deleted node.
[cpcn@redis01 redis6383]$ ps -ef |grep redis
avahi 1161 1 0 Sep12 ? 00:00:00 avahi-daemon: running [redis01.local]
cpcn 30529 1 7 09:07 ? 00:03:50 /usr/local/redis6381/bin/redis-server 172.31.4.18:6381 [cluster]
cpcn 30559 1 8 09:07 ? 00:03:57 /usr/local/redis6385/bin/redis-server 172.31.4.18:6385 [cluster]
cpcn 31308 1 0 09:57 ? 00:00:00 /usr/local/redis6383/bin/redis-server 172.31.4.18:6383 [cluster]
cpcn 31314 29933 0 09:57 pts/1 00:00:00 grep --color=auto redis
[cpcn@redis01 redis6383]$ netstat -anltp |grep 4.22
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
tcp 0 31 172.31.4.18:6383 172.31.4.22:54100 ESTABLISHED 30548/redis-server
tcp 0 31 172.31.4.18:6383 172.31.4.22:54098 ESTABLISHED 30548/redis-server
tcp 0 7 172.31.4.18:6385 172.27.16.58:4422 ESTABLISHED 30559/redis-server
tcp 0 31 172.31.4.18:6383 172.31.4.22:54102 ESTABLISHED 30548/redis-server
tcp 0 31 172.31.4.18:6383 172.31.4.22:54096 ESTABLISHED 30548/redis-server
tcp 0 0 172.31.4.18:16383 172.31.4.21:46224 ESTABLISHED 30548/redis-server
172.31.4.18:6383> cluster nodes
83d609a16f5fc2605ff8fd698197533c6ad29334 172.31.4.18:6383@16383 myself,master - 0 1726278605000 11 connected
172.31.4.18:6383>
5、下线 172.31.4.18 6383 集群信息：
172.31.4.20:6383> cluster nodes
401730ba6e1c2cd8b60243990f0965606edfc083 172.31.4.18:6385@16385 master - 0 1726277099000 2 connected 10923-16383
4e25e8f87029a22bbba7aa5f6fb2edba4a342da5 172.31.4.21:6384@16384 slave f5e774bd5cd8f5bcba53f1297015f3245dd3f18c 0 1726277098872 8 connected
23bb415a72cf125c8205be6b1de4f8f5a0a636c8 172.31.4.18:6381@16381 master - 0 1726277099874 1 connected 0-5460
602fbfe65cf83178c0b897441013ec0bae590c96 172.31.4.20:6385@16385 slave 401730ba6e1c2cd8b60243990f0965606edfc083 0 1726277098000 2 connected
9a79febbfb1d7a8f7a6ba3fb11e86b0f122f71e7 172.31.4.21:6386@16386 slave 401730ba6e1c2cd8b60243990f0965606edfc083 0 1726277097871 2 connected
f5e774bd5cd8f5bcba53f1297015f3245dd3f18c 172.31.4.20:6383@16383 myself,master - 0 1726277096000 8 connected 5461-10922
3dec7d5dd8139854f766417611026b07785c0a76 172.31.4.19:6382@16382 slave 23bb415a72cf125c8205be6b1de4f8f5a0a636c8 0 1726277097000 1 connected
172.31.4.20:6383>
此时的状态，下线172.31.4.18 6383，该进程还在，老链接还在。172.31.4.22上并发过的链接 报槽位错误：
Error from server: CLUSTERDOWN Hash slot not served(overall: 0.244)
Error from server: CLUSTERDOWN Hash slot not served(overall: 0.244)
Error from server: CLUSTERDOWN Hash slot not served(overall: 0.244)
6、解决的办法，是尽快关闭下线的 redis 进程：
172.31.4.18:6383> cluster nodes
83d609a16f5fc2605ff8fd698197533c6ad29334 172.31.4.18:6383@16383 myself,master - 0 1726278605000 11 connected
172.31.4.18:6383>
172.31.4.18:6383>
172.31.4.18:6383> shutdown
not connected> exit
[cpcn@redis01 redis6383]$
[cpcn@redis01 redis6383]$
[cpcn@redis01 redis6383]$ ps -ef |grep redis
avahi 1161 1 0 Sep12 ? 00:00:00 avahi-daemon: running [redis01.local]
cpcn 30529 1 7 09:07 ? 00:03:49 /usr/local/redis6381/bin/redis-server 172.31.4.18:6381 [cluster]
cpcn 30559 1 8 09:07 ? 00:03:56 /usr/local/redis6385/bin/redis-server 172.31.4.18:6385 [cluster]
cpcn 31302 29933 0 09:57 pts/1 00:00:00 grep --color=auto redis
此时172.31.4.22上并发过的链接中断，4.22上的redis-benchmark进程中断，报：
报：
Error from server: CLUSTERDOWN Hash slot not served(overall: 0.237)
Error: Server closed the connection avg_msec=0.240 (overall: 0.238)
[cpcn@redis003 bin]$
[cpcn@redis003 bin]$
[cpcn@redis003 bin]$ ps -ef |grep redis
cpcn 67809 67655 0 10:08 pts/0 00:00:00 grep --color=auto redis
[cpcn@redis003 bin]$
7附：连接日志：
[cpcn@redis003 bin]$ redis-benchmark -h 172.31.4.18 -p 6383 -a Redis_6.2.1_Sc -t set,get -e -c 4 -d 1 -n 100000000000000
Error from server: MOVED 13782 172.31.4.18:6385rall: -nan)
Error from server: MOVED 13782 172.31.4.18:6385
Error from server: MOVED 13782 172.31.4.18:6385242 (overall: 0.226)