Linux-lvs

LVS调试算法
ipvs scheduler：根据其调度时是否考虑各RS当前的负载状态

分为两种：静态方法和动态方法

静态方法
仅根据算法本身进行调度

1、RR：roundrobin，轮询

2、WRR：Weighted RR，加权轮询

3、SH：Source Hashing，实现session sticky，源IP地址hash；将来自于同一个IP地址的请求始终发往第一次挑中的RS，从而实现会话绑定

4、DH：Destination Hashing；目标地址哈希，第一次轮询调度至RS，后续将发往同一个目标地址的请求始终转发至第一次挑中的RS，典型使用场景是正向代理缓存场景中的负载均衡，如：宽带运营商

动态方法
主要根据每RS当前的负载状态及调度算法进行调度Overhead=value 较小的RS将被调度

1、LC：least connections 适用于长连接应用

​ Overhead=activeconns*256+inactiveconns

2、WLC：Weighted LC，默认调度方法

​ Overhead=(activeconns*256+inactiveconns)/weight

3、SED：Shortest Expection Delay，初始连接高权重优先

​ Overhead=(activeconns+1)*256/weight

4、NQ：Never Queue，第一轮均匀分配，后续SED

5、LBLC：Locality-Based LC，动态的DH算法，使用场景：根据负载状态实现正向代理

6、LBLCR：LBLC with Replication，带复制功能的LBLC，解决LBLC负载不均衡问题，从负载重的复制到负载轻的RS

内核版本 4.15 版本后新增调度算法：FO和OVF
FO（Weighted Fail Over）调度算法： 
        在此FO算法中，遍历虚拟服务所关联的真实服务器链表，
        找到还未过载（未设置IP_VS_DEST_F_OVERLOAD标志）的且权重最高的真实服务器，进行调度

OVF（Overflow-connection）调度算法：
        基于真实服务器的活动连接数量和权重值实现。
        将新连接调度到权重值最高的真实服务器，
        直到其活动连接数量超过权重值，之后调度到下一个权重值最高的真实服务器,
        在此OVF算法中，遍历虚拟服务相关联的真实服务器链表，
        找到权重值最高的可用真实服务器。一个可用的真实服务器需要同时满足以下条件：

        -未过载（未设置IP_VS_DEST_F_OVERLOAD标志）
        -真实服务器当前的活动连接数量小于其权重值
        -其权重值不为零
        
        
LVS NAT模型实验

环境：

共四台主机
一台： internet client ：10.0.0.100/24   GW:无

一台：lvs  
eth1 NAT 10.0.0.11/24
eth0 桥接 172.25.0.11/24

两台RS：
RS1: 172.25.0.101/24    GW： 172.25.0.11
RS2: 172.25.0.102/24    GW： 172.25.0.11


配置过程

#LVS启用IP_FORWORD功能
[root@lvs ~]#vim /etc/sysctl.conf
net.ipv4.ip_forward = 1
[root@LVS ~]#sysctl  -p

[root@lvs ~]#ipvsadm -A -t 10.0.0.11:80 -s rr
[root@lvs ~]#ipvsadm -a -t 10.0.0.11:80 -r 172.25.0.101 -m
[root@lvs ~]#ipvsadm -a -t 10.0.0.11:80 -r 172.25.0.102 -m

[root@lvs ~]# ipvsadm -Ln                                             
IP Virtual Server version 1.2.1 (size=4096)                           
Prot LocalAddress:Port Scheduler Flags                                
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn 
TCP  10.0.0.11:80 rr                                                  
  -> 172.25.0.101:80              Masq    1      0          0         
  -> 172.25.0.102:80              Masq    1      0          0         

#测试
[root@ex-host-01 ~]# while true; do curl 10.0.0.11 ; sleep 0.5;done
LAN-RS1 172.25.0.101
LAN-RS2 172.25.0.102
LAN-RS1 172.25.0.101
LAN-RS2 172.25.0.102
LAN-RS1 172.25.0.101
LAN-RS2 172.25.0.102
LAN-RS1 172.25.0.101
LAN-RS2 172.25.0.102
LAN-RS1 172.25.0.101
LAN-RS2 172.25.0.102


[root@lvs ~]# cat /proc/net/ip_vs_conn
Pro FromIP   FPrt ToIP     TPrt DestIP   DPrt State       Expires PEName PEData
TCP 0A000064 ED6E 0A00000B 0050 AC190066 0050 TIME_WAIT        63
TCP 0A000064 ED82 0A00000B 0050 AC190066 0050 TIME_WAIT        69
TCP 0A000064 ED7C 0A00000B 0050 AC190065 0050 TIME_WAIT        67
TCP 0A000064 ED66 0A00000B 0050 AC190066 0050 TIME_WAIT        61
TCP 0A000064 ED86 0A00000B 0050 AC190066 0050 TIME_WAIT        70
TCP 0A000064 ED62 0A00000B 0050 AC190066 0050 TIME_WAIT        60
TCP 0A000064 ED70 0A00000B 0050 AC190065 0050 TIME_WAIT        64
TCP 0A000064 ED64 0A00000B 0050 AC190065 0050 TIME_WAIT        61
TCP 0A000064 ED7A 0A00000B 0050 AC190066 0050 TIME_WAIT        67
TCP 0A000064 ED68 0A00000B 0050 AC190065 0050 TIME_WAIT        62


#保存规则
[root@LVS ~]#ipvsadm -Sn > /etc/sysconfig/ipvsadm
[root@lvs ~]# cat /etc/sysconfig/ipvsadm
-A -t 10.0.0.11:80 -s rr
-a -t 10.0.0.11:80 -r 172.25.0.101:80 -m -w 1
-a -t 10.0.0.11:80 -r 172.25.0.102:80 -m -w 1


#清除规则
[root@lvs ~]# ipvsadm -C
[root@lvs ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn


#重新加载规则
[root@lvs ~]# ipvsadm -R < /etc/sysconfig/ipvsadm
[root@lvs ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  10.0.0.11:80 rr
  -> 172.25.0.101:80              Masq    1      0          0         
  -> 172.25.0.102:80              Masq    1      0          0   

#开机加载ipvs规则
[root@LVS ~]#ipvsadm -C
[root@LVS ~]#ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn


[root@rs1 ~]# tail /var/log/httpd/access_log 
10.0.0.100 - - [25/Apr/2024:21:34:30 +0800] "GET / HTTP/1.1" 200 21 "-" "curl/7.29.0"
10.0.0.100 - - [25/Apr/2024:21:34:31 +0800] "GET / HTTP/1.1" 200 21 "-" "curl/7.29.0"
10.0.0.100 - - [25/Apr/2024:21:34:31 +0800] "GET / HTTP/1.1" 200 21 "-" "curl/7.29.0"
10.0.0.100 - - [25/Apr/2024:21:34:32 +0800] "GET / HTTP/1.1" 200 21 "-" "curl/7.29.0"


#修改调度算法为 WRR 和后端服务器的端口
[root@lvs ~]# ipvsadm -E -t 10.0.0.11:80 -s wrr
[root@lvs ~]# ipvsadm -e -t 10.0.0.11:80 -r 172.25.0.101:80 -m -w 6
[root@lvs ~]# ipvsadm -e -t 10.0.0.11:80 -r 172.25.0.102:80 -m -w 1

[root@lvs ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  10.0.0.11:80 wrr
  -> 172.25.0.101:80              Masq    6      0          160       
  -> 172.25.0.102:80              Masq    1      0          73        

实验：LVS-DR单一网段

DR模型中各主机上均需要配置VIP，解决地址冲突的方式有三种：

(1) 在前端网关做静态绑定

(2) 在各RS使用arptables

(3) 在各RS修改内核参数，来限制arp响应和通告的级别

限制响应级别：arp_ignore

-0：默认值，表示可使用本地任意接口上配置的任意地址进行响应

-1：仅在请求的目标IP配置在本地主机的接收到请求报文的接口上时，才给予响应

限制通告级别：arp_announce

-0：默认值，把本机所有接口的所有信息向每个接口的网络进行通告

-1：尽量避免将接口信息向非直接连接网络进行通告

-2：必须避免将接口信息向非本网络进行通告

配置要点

Director 服务器采用双IP桥接网络，一个是VIP，一个DIP
Web服务器采用和DIP相同的网段和Director连接
每个Web服务器配置VIP
每个web服务器可以出外网

范例：

环境：五台主机
一台：客户端 10.0.0.100/16 GW:10.0.0.11
一台：ROUTER
eth0 :NAT  172.25.0.11/24 VIP
eth1: 桥接 10.0.0.11/16
启用 IP_FORWARD
一台：LVS
eth0: 172.25.0.100/24 GW:172.25.0.11
两台RS：
RS1：172.25.0.101/24  GW：172.25.0.11
RS2：172.25.0.102/24  GW：172.25.0.11
配置过程

#在LVS服务器上实现
[root@lvs ~]# ifconfig lo:1 172.25.0.200/32
[root@lvs ~]# ipvsadm -A -t 172.25.0.200:80 -s rr 
[root@lvs ~]# ipvsadm -a -t 172.25.0.200:80 -r 172.25.0.101
[root@lvs ~]# ipvsadm -a -t 172.25.0.200:80 -r 172.25.0.102
[root@lvs ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  172.25.0.200:80 rr
  -> 172.25.0.101:80              Route   1      0          0         
  -> 172.25.0.102:80              Route   1      0          0       

#在后端RS服务器上实现
[root@RS1 ~]#ifconfig lo:1 172.25.0.200/32
[root@RS1 ~]#echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
[root@RS1 ~]#echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
[root@RS1 ~]#echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
[root@RS1 ~]#echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce

范例：
CLIENT：
[root@ex-host-01 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:bd:bf:41 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.100/16 brd 10.0.255.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::21c:4aef:e249:489d/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
       
[root@ex-host-01 ~]#  ip route 
default via 10.0.0.11 dev ens33 
10.0.0.0/16 dev ens33 proto kernel scope link src 10.0.0.100 metric 100 


[root@router ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:c3:fc:3a brd ff:ff:ff:ff:ff:ff
    inet 172.25.0.11/24 brd 172.25.0.255 scope global noprefixroute eno16777736
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:fec3:fc3a/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
3: ens36: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:c3:fc:44 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.11/16 brd 10.0.255.255 scope global noprefixroute ens36
       valid_lft forever preferred_lft forever
    inet6 fe80::5e6b:117d:9a98:a79e/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
[root@router ~]# ip route 
default via 10.0.0.1 dev ens36 proto static metric 101 
default via 172.25.0.2 dev eno16777736 proto static metric 102 
10.0.0.0/16 dev ens36 proto kernel scope link src 10.0.0.11 metric 101 
172.25.0.0/24 dev eno16777736 proto kernel scope link src 172.25.0.11 metric 102 


[root@lvs ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet 172.25.0.200/0 scope global lo:1
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:8a:6c:13 brd ff:ff:ff:ff:ff:ff
    inet 172.25.0.100/24 brd 172.25.0.255 scope global eno16777736
       valid_lft forever preferred_lft forever
    inet 172.25.0.199/32 scope global eno16777736
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:fe8a:6c13/64 scope link 
       valid_lft forever preferred_lft forever
[root@lvs ~]# ip route 
default via 172.25.0.11 dev eno16777736  proto static  metric 100 
172.25.0.0/24 dev eno16777736  proto kernel  scope link  src 172.25.0.100  metric 100 

VIP映射RIP
[root@lvs ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  172.25.0.200:80 rr
  -> 172.25.0.101:80              Route   1      0          0         
  -> 172.25.0.102:80              Route   1      0          0         

[root@rs1 ~]#echo 1 >   /proc/sys/net/ipv4/conf/all/arp_ignore
[root@rs1 ~]#echo 2 >   /proc/sys/net/ipv4/conf/all/arp_announce 
[root@rs1 ~]#echo 1 >   /proc/sys/net/ipv4/conf/lo/arp_ignore
[root@rs1 ~]#echo 2 >   /proc/sys/net/ipv4/conf/lo/arp_announce 
[root@rs1 ~]#ifconfig lo:1 172.25.0.200/32
[root@rs1 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet 172.25.0.200/0 scope global lo:1
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:4e:b0:81 brd ff:ff:ff:ff:ff:ff
    inet 172.25.0.101/24 brd 172.25.0.255 scope global eno16777736
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:fe4e:b081/64 scope link 
       valid_lft forever preferred_lft forever

[root@rs2 ~]#echo 1 >   /proc/sys/net/ipv4/conf/all/arp_ignore
[root@rs2 ~]#echo 2 >   /proc/sys/net/ipv4/conf/all/arp_announce 
[root@rs2 ~]#echo 1 >   /proc/sys/net/ipv4/conf/lo/arp_ignore
[root@rs2 ~]#echo 2 >   /proc/sys/net/ipv4/conf/lo/arp_announce
[root@rs2 ~]#ifconfig lo:1 172.25.0.200/32
[root@rs2 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet 172.25.0.200/0 scope global lo:1
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
3: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:58:26:2b brd ff:ff:ff:ff:ff:ff
    inet 172.25.0.102/24 brd 172.25.0.255 scope global eno16777736
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:fe58:262b/64 scope link 
       valid_lft forever preferred_lft forever


测试：
[root@ex-host-01 ~]# while true; do curl 172.25.0.200 ; sleep 0.3;done
LAN-RS2 172.25.0.102
LAN-RS1 172.25.0.101
LAN-RS2 172.25.0.102
LAN-RS1 172.25.0.101
LAN-RS2 172.25.0.102
LAN-RS1 172.25.0.101
LAN-RS2 172.25.0.102
LAN-RS1 172.25.0.101
LAN-RS2 172.25.0.102
LAN-RS1 172.25.0.101
LAN-RS2 172.25.0.102
LAN-RS1 172.25.0.101
LAN-RS2 172.25.0.102
LAN-RS1 172.25.0.101

[root@rs1 ~]# tail /etc/httpd/logs/access_log
10.0.0.100 - - [30/Jul/2024:19:54:04 +0800] "GET / HTTP/1.1" 200 21 "-" "curl/7.29.0"
10.0.0.100 - - [30/Jul/2024:19:54:04 +0800] "GET / HTTP/1.1" 200 21 "-" "curl/7.29.0"
10.0.0.100 - - [30/Jul/2024:19:54:05 +0800] "GET / HTTP/1.1" 200 21 "-" "curl/7.29.0"
10.0.0.100 - - [30/Jul/2024:21:17:55 +0800] "GET / HTTP/1.1" 200 21 "-" "curl/7.29.0"
10.0.0.100 - - [30/Jul/2024:21:17:56 +0800] "GET / HTTP/1.1" 200 21 "-" "curl/7.29.0"
10.0.0.100 - - [30/Jul/2024:21:17:57 +0800] "GET / HTTP/1.1" 200 21 "-" "curl/7.29.0"
10.0.0.100 - - [30/Jul/2024:21:17:57 +0800] "GET / HTTP/1.1" 200 21 "-" "curl/7.29.0"
10.0.0.100 - - [30/Jul/2024:21:17:58 +0800] "GET / HTTP/1.1" 200 21 "-" "curl/7.29.0"
10.0.0.100 - - [30/Jul/2024:21:17:59 +0800] "GET / HTTP/1.1" 200 21 "-" "curl/7.29.0"
10.0.0.100 - - [30/Jul/2024:21:17:59 +0800] "GET / HTTP/1.1" 200 21 "-" "curl/7.29.0"


[root@rs2 ~]# tail /etc/httpd/logs/access_log
10.0.0.100 - - [28/Apr/2024:10:29:45 +0800] "GET / HTTP/1.1" 200 21 "-" "curl/7.29.0"
10.0.0.100 - - [28/Apr/2024:10:29:45 +0800] "GET / HTTP/1.1" 200 21 "-" "curl/7.29.0"
10.0.0.100 - - [28/Apr/2024:10:29:46 +0800] "GET / HTTP/1.1" 200 21 "-" "curl/7.29.0"
10.0.0.100 - - [28/Apr/2024:11:53:36 +0800] "GET / HTTP/1.1" 200 21 "-" "curl/7.29.0"
10.0.0.100 - - [28/Apr/2024:11:53:37 +0800] "GET / HTTP/1.1" 200 21 "-" "curl/7.29.0"
10.0.0.100 - - [28/Apr/2024:11:53:38 +0800] "GET / HTTP/1.1" 200 21 "-" "curl/7.29.0"
10.0.0.100 - - [28/Apr/2024:11:53:38 +0800] "GET / HTTP/1.1" 200 21 "-" "curl/7.29.0"
10.0.0.100 - - [28/Apr/2024:11:53:39 +0800] "GET / HTTP/1.1" 200 21 "-" "curl/7.29.0"
10.0.0.100 - - [28/Apr/2024:11:53:40 +0800] "GET / HTTP/1.1" 200 21 "-" "curl/7.29.0"
10.0.0.100 - - [28/Apr/2024:11:53:40 +0800] "GET / HTTP/1.1" 200 21 "-" "curl/7.29.0"

LVS-DR模式多网段案例

配置

[root@rs1 ~]#cat lvs_dr_rs.sh 
#!/bin/bash

vip=192.168.0.200
mask='255.255.255.255'
dev=lo:1
#rpm -q httpd &> /dev/null || yum -y install httpd &>/dev/null
#service httpd start &> /dev/null && echo "The httpd Server is Ready!"
echo "<h1>
hostname
</h1>" > /var/www/html/index.html

case $1 in
start)
    echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
    echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
    echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
    echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
    ifconfig $dev $vip netmask $mask #broadcast $vip up
    #route add -host $vip dev $dev
    echo "The RS Server is Ready!"
    ;;
stop)
    ifconfig $dev down
    echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore
    echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore
    echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce
    echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce
    echo "The RS Server is Canceled!"
    ;;
*) 
    echo "Usage: $(basename $0) start|stop"
    exit 1
    ;;
esac

[root@rs1 ~]#bash lvs_dr_rs.sh  start
[root@rs2 ~]#bash lvs_dr_rs.sh  start

[root@LVS ~]#cat lvs_dr_vs.sh 
#!/bin/bash
vip='192.168.0.200'
iface='lo:1'
mask='255.255.255.255'
port='80'
rs1='172.25.0.101'
rs2='172.25.0.102'
scheduler='wrr'
type='-g'
#rpm -q ipvsadm &> /dev/null || yum -y install ipvsadm &> /dev/null

case $1 in
start)
    ifconfig $iface $vip netmask $mask #broadcast $vip up
    iptables -F

    ipvsadm -A -t ${vip}:${port} -s $scheduler
    ipvsadm -a -t ${vip}:${port} -r ${rs1} $type -w 1
    ipvsadm -a -t ${vip}:${port} -r ${rs2} $type -w 1
    echo "The VS Server is Ready!"
    ;;
stop)
    ipvsadm -C
    ifconfig $iface down
    echo "The VS Server is Canceled!"
    ;;
*)
    echo "Usage: $(basename $0) start|stop"
    exit 1
    ;;
esac

[root@LVS ~]#bash lvs_dr_vs.sh start

[root@Router ~]#nmcli connection modify eno16777736 +ipv4.addresses  192.168.0.11/24 
[root@Router ~]#nmcli connection reload
[root@Router ~]#nmcli connection up eno16777736
[root@Router ~]#ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:c3:fc:3a brd ff:ff:ff:ff:ff:ff
    inet 172.25.0.11/24 brd 172.25.0.255 scope global noprefixroute eno16777736
       valid_lft forever preferred_lft forever
    inet 192.168.0.11/24 brd 192.168.0.255 scope global noprefixroute eno16777736
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:fec3:fc3a/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
3: ens36: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:c3:fc:44 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.11/16 brd 10.0.255.255 scope global noprefixroute ens36
       valid_lft forever preferred_lft forever
    inet6 fe80::5e6b:117d:9a98:a79e/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever

实战案例：实现双主的LVS-DR模式
 

[root@keepalive01 ~]#vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
    global_defs {
        notification_email {
            root@localhost
        }
        notification_email_from keepalived@localhost
        smtp_server 127.0.0.1
        smtp_connect_timeout 30
        router_id ka1.apep.org                #另一个节点为ka2.apep.org
        vrrp_mcast_group4 224.0.100.10
    }

vrrp_instance VI_1 {
    state MASTER                                #在另一个结点上为BACKUP
    interface eno16777736
    virtual_router_id 66
    priority 100                                #在另一个结点上为80
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {
        172.25.0.200/24 dev eno16777736 label eno16777736:1      #指定VIP
    }
}

vrrp_instance VI_2 {
    state BACKUP                                #在另一个结点上为MASTER
    interface eno16777736
    virtual_router_id  88
    priority 80                                 #在另一个结点上为100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 654321
    }
    virtual_ipaddress {
        172.25.0.201/24 dev eno16777736 label eno16777736:2      #指定VIP2
    }
}
virtual_server 172.25.0.200 80 {  
    delay_loop 6
    lb_algo rr
    lb_kind DR
    protocol TCP
    sorry_server 127.0.0.1 80
    real_server 172.25.0.101 80 {  #指定RS1地址
        weight 1
        HTTP_GET {
            url {
                path /
                status_code 200
            }
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }

    }
    real_server 172.25.0.102 80 {                  #指定RS2地址
        weight 1
        HTTP_GET {
            url {
                path /
                status_code 200
            }
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }

    }   

}

virtual_server 172.25.0.201 80 {                       #指定VIP2
    delay_loop 6
    lb_algo rr
    lb_kind DR
    protocol TCP
    sorry_server 127.0.0.1 80
    real_server 172.25.0.101 80 {                      #指定RS3地址
        weight 1
        HTTP_GET {
            url {
                path /
                status_code 200
            }
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }

    }
    real_server 172.25.0.102 80 {                      #指定RS4地址
        weight 1
        HTTP_GET {
            url {
                path /
                status_code 200
            }
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }   
}

RS运行的配置脚本 
vim lvs_dr_rs.sh

#!/bin/bash

vip=172.25.0.200
mask='255.255.255.255'
dev=lo:1
#rpm -q httpd &> /dev/null || yum -y install httpd &>/dev/null
#service httpd start &> /dev/null && echo "The httpd Server is Ready!"
echo "<h1>
`hostname`
</h1>" > /var/www/html/index.html

case $1 in
start)
    echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
    echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
    echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
    echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
    ifconfig $dev $vip netmask $mask #broadcast $vip up
    #route add -host $vip dev $dev
    echo "The RS Server is Ready!"
    ;;
stop)
    ifconfig $dev down
    echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore
    echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore
    echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce
    echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce
    echo "The RS Server is Canceled!"
    ;;
*)
    echo "Usage: $(basename $0) start|stop"
    exit 1
    ;;
esac

[root@rs1 ~]# bash lvs_dr_rs.sh start
[root@rs2 ~]# bash lvs_dr_rs.sh start

访问测试结果
[root@ex-host-01 ~]# curl 172.25.0.200
<h1>
rs2
</h1>
[root@ex-host-01 ~]# curl 172.25.0.200
<h1>
rs1
</h1>

[root@keepalive01 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  172.25.0.200:80 rr
  -> 172.25.0.101:80              Route   1      1          124       
  -> 172.25.0.102:80              Route   1      0          123   
  
模拟故障
#第一台RS1故障，自动切换至RS2
[root@rs1 ~]#chmod 0 /var/www/html/index.html 

[root@keepalive01 ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  172.25.0.200:80 rr
  -> 172.25.0.102:80              Route   1      0          259       
TCP  172.25.0.201:80 rr
  -> 172.25.0.102:80              Route   1      0          0           

#后端RS服务器都故障，启动Sorry Server
#需要keepalived服务器开启httpd服务


#ka1故障，VIP自动切换至ka2
[root@keepalive02 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:04:1d:9c brd ff:ff:ff:ff:ff:ff
    inet 172.25.0.99/24 brd 172.25.0.255 scope global eno16777736
       valid_lft forever preferred_lft forever
    inet 172.25.0.201/24 scope global secondary eno16777736:2
       valid_lft forever preferred_lft forever
    inet 172.25.0.200/24 scope global secondary eno16777736:1
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:fe04:1d9c/64 scope link 
       valid_lft forever preferred_lft forever

#ka1恢复后，VIP恢复至ka1
[root@keepalive01 ~]# killall keepalived
[root@keepalive01 ~]# systemctl restart keepalived.service 
[root@keepalive01 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eno16777736: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:8a:6c:13 brd ff:ff:ff:ff:ff:ff
    inet 172.25.0.100/24 brd 172.25.0.255 scope global eno16777736
       valid_lft forever preferred_lft forever
    inet 172.25.0.200/24 scope global secondary eno16777736:1
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:fe8a:6c13/64 scope link 
       valid_lft forever preferred_lft forever