LVS keepalived搭建负载均衡学习手札（LVS keepalived builds load balancing learning hand）

1、LVS+keepalived搭建负载均衡学习手札（LVS + keepalived builds load balancing learning hand）LVS+keepalived build a load balancing letters learning (1) (2009-10-23 10:04:12) reprint we label: it.Installation environment: environment centos4.4LB:58 (VIP:88)Real-server1:87Real-server

2、2:89Important point: the HA keepalvied scheme for LVS is completely taken up by a keepalived.conf file, and keepalived uses the VRRP protocol, which is explained below:VRRP (Virtual, Router, Redundancy, Protocol, virtual routing Redundancy Protocol) is a fault tolerant protocol that dynam

3、ically allocates the responsibility of a virtual router to one of the VRRP routers on the lan. The VRRP router, which controls the IP address of the virtual router, is called the primary router, which is responsible for forwarding packets to these virtual IP addresses. Once the primary router is not

4、 available, this selection process provides a dynamic failover mechanism, which allows the IP address of the virtual router to act as the default master first hop router for the terminal host. The advantage of using VRRP is the availability of a higher default path without the need to configure dyna

5、mic routing or routing discovery protocols on each terminal host. The VRRP package is sent in the IP package.Install now:1. Install ipvsadm on the VIP machineWGet /software/kernel-2.6/ipvsadm-1.24.tar.gzSet up a connection file before installation, or else it will go

6、wrongLn -s, /usr/src/kernels/2.6.9-42.EL-i686/, /usr/src/linux must be consistent with the current running kernelTar -zxvf ipvsadm-1.24.tar.gzCD ipvsadm-1.24Make &makeinstallSo far, ipvsadm even installed successfullyDo some validation of the installation below:1. first execute the ipvsadm command2.

7、lsmod |grep ip_vsIp_vs_rr 59531Ip_vs 831373, ip_vs_rrVerify that the ipvsadm of the VIP machine has no problem.Two. Next is the important installation of the keepalived:WGet /software/keepalived-1.1.17.tar.gzTar -zxvf keepalived-1.1.17.tar.gzCD keepalived-1.1.17./configure -p

8、refix=/ -mandir=/usr/local/share/man/ -with-kernel-dir=/usr/src/kernels/2.6.9-42.EL-smp-i686/Configure, if correct, will display:Keepalived configuration-Keepalived version: 1.1.15Compiler: GCCCompiler flags: -g -O2Extra Lib: -lpopt, -lssl, -lcryptoUse IPVS Framework: Yes # support LVSIPVS, sync, da

9、emon, support: YesUse VRRP Framework: YesUse LinkWatch: NoUse Debug flags: NoMake &make installSo far, the lvs+keepalived installation is complete.Three. Next, configure keepalived.conf:VI /etc/keepalived/keepalived.confHeres my configuration:Configuration, File, for, keepalived!# global configurati

10、on:Global_defs Notification_email Admin # email address, the machine need to open SMTP serviceNotification_email_from rootlocalhostSmtp_server _ _ smtp connect timeout 30router id lvs _ _ devel # 负载均衡器标示: 同一局域网内, 应该是唯一的.# vrrp配置.i vrrp _ mcv (_ groupgroup (vi _ 1# vrrp实例配置vrrp _ forum vi _

11、1 # 定义一个实例# 设置为master state.eth0 interfacevirtual _ router _ id 51 # 主、备机的virtual _ id一定要相同 _ routerabout 100 # 主机值较大, 备份机值较小(5 # _ vrrp multicast广播周期秒数, 检查间隔where auth _ # vrrp认证方式 type.auth _ 1111 # vrrp口令字.virtual _ ipaddress (88. # (如果有多个vip, 继续换行填写).# lvs配置.virtual _ server 192.168.2

12、.188 80 it _ loop 6 # (每隔6秒查询realserver状态)lb _ algorithm # (负载均衡调度 算法, 常用wlc rr, rr)._, dr. # lb (负载均衡转发规则, 一般包括dr, nat, tun)persistence _ timeout (50 # 会话保持, 同一ip的连接50秒内被分配到同一台realserver)protocol (tcp # 用tcp协议检查realserver状态)im sorry _ server 80 # realserver全部失败, vip指向本机80端口.real _ server

13、87 80 a 3 # (权重)tcp (# 通过tcpcheck判断realserver的健康状态 _.(3 # 重连次数 _ get _ retryit _ before _ retry 3 # 重练间隔时间connect _ # 健康检查端口 port 80_ 3 # 连接超时时间 connect timeoutreal _ server 89 80 weight 1tcp (_.note _ get _ retry 3it _ before _ retry 3connect _ port 80_ 3 connect timeout至此: ke

14、epalived配置完成.执行 / etc / init.d / keepalived 启动.两台上执行相同的脚本即可, 脚本如下 四.接下来配置real - server.root test1 # in / usr / local / bin / lvs _ real#. / bin / shvip = 88 (直接路由模式的vip必须跟服务器对外提供服务的ip地址在同一个网段)/ etc / rc.d / init.d / functions$1.start)echo start tunl portifconfig lo: $0 $vip vip netmask 25

15、55 broadcast.(如果有多vip, 依次往下写)echo 2 / proc / sys / net / ipv4 / conf / all / arp _ announceecho 1 / proc / sys / net / ipv4 / conf / all / arp _ ignoreecho 2 / proc / sys / net / ipv4 / conf / lo / arp _ announceecho 1 / proc / sys / net / ipv4 / conf / lo / arp _ ignore;stop)echo stop tu

16、nl portifconfig lo: 0.echo 0 / proc / sys / net / ipv4 / conf / all / arp _ announceecho 0 / proc / sys / net / ipv4 / conf / all / arp _ ignoreecho 0 / proc / sys / net / ipv4 / conf / lo / arp _ announceecho 0 / proc / sys / net / ipv4 / conf / lo / arp _ ignore;*)echo usage: $0 start | stopexit 1

17、esac下面是田老师对以上脚本的说明:1、 vip (virtual ip).直接路由模式的vip必须跟服务器对外提供服务的ip地址在同一个网段, 并且lvs 负载均衡器和其他所有提供相同功能的服务器都使用这个vip.2、 vip被绑定在环回接口lo0: 0上, 其广播地址是其本身, 子网掩码是55.这与标准的网络地址设置有很大的不同.采用这种可变长掩码方式把网段划分成只含一个主机地址的目的是避免ip地址冲突.3、 echo 1, 2 这段的作用是抑制arp广播.如果不做arp抑制 echo, 将会有众多的机器向其他宣称: 嗨! 我是奥巴马, 我在这里呢!, 这样就乱

18、套了.解释:1 - 允许多个网络介质位于同一子网段内, 每个网络界面依据是否内核指派路由该数据包经过此界面来确认是否回答arp查询 (这个实现是由来源地址确定路由的时候决定的), 换句话说, 允许控制使用某一块网卡 (通常是第一块) 回应arp询问. (做负载均衡的时候, 可以考虑用echo 1 / proc / sys / net / ipv4 / conf / all / arp _ filter这样的方式就可以解决, 当然利用:echo 2 / proc / sys / net / ipv4 / conf / all / arp _ announcearpecho 1 / proc /

19、sys / net / ipv4 / conf / all / arp _ ignore两条命令配合使用更好, 因为arp _ announcearp和arp _ ignore 似乎是对arp _ filter的更细节控制的实现.)使用 / usr / local / bin / lvs _ real start | stop 来启动和关闭启动后:root test1 # ip add1: lo: mtu, up 16436 qdisc noqueuelink / network 00:00:00: 00: 00: 00: 00: 00: 00 (00:00:00inet

20、/ 8 (55 scope host loinet 88 / 32 (88 scope global 10: 0inet6: 1 / 128 scope hostvalid _ ift forever preferred _ ift forever2: eth0: mtu 1500 qdisc pfifo _ almost qlen 1000link / ether 00: 0c: 29: 72, 73: b5 (ff: ff: ff: ff: ff: ffinet 87 / 24 (192.168.2

21、.255 scope global eth0fe80 inet6: 20c: 29ff: fe72: 73b5 / 64 scope linkvalid _ ift forever preferred _ ift forever3: sit0: mtu 1480 qdisc nooplink / sit , germany关闭后:root test1 # ip add1: lo: mtu, up 16436 qdisc noqueuelink / 00: 00: 00: loopback 00:00 a.m. brd 00:00: 00: 00: 00: 00fo

22、r / 8) 55 hosts the broomsinet6: 1 / 128 games hostvalid _ lft forever preferred _ lft.2: eth0: broadcast, multicast, up mtu 1500 qdisc pfifo _ fast qlen 1000link / ether at 0c: 29: 72, 73: b5) ff: ff: ff: ff: ff: fffor 87 / 24) 55 brooms global eth0inet6

23、 fe80: 20c: 29ff: fe72: 73b5 / 64 brooms linkvalid _ lft forever preferred _ lft.3: sit0: mtu noarp 1480 qdisc nooplink / sit brd 五: 接下来验证lvs + keepalivedvip上 / etc / init.d / keepalived start两台realserver上执行上述的lvs _ real start在vip机器上查看:root yuhao linux ipvsadm-1.24 # ipvsadmip virtual

24、 server version 1.2.0 (size = 4096)prot localaddress: port scheduler flags- remoteaddress: port forward weight activeconn inactconntcp 88 http rr persistent 50- 87 http route 3 2 0- 89 http route 1 0 0这时访问web也是正常的. 其中 activeconn表示活跃连接数: established状态 inactconn 是不活跃连接

25、数: 除了established的状态 (syn _ recv, time _ wait, since _ wait1等)然后我关掉一台realserver的apache, 再次查看root yuhao linux ipvsadm-1.24 # ipvsadmip virtual server version 1.2.0 (size = 4096)prot localaddress: port scheduler flags- remoteaddress: port forward weight activeconn inactconntcp 88 http rr per

26、sistent 50- 89 http route 1 0 0可见keepalived已经发现一台web挂掉, 并将其踢出负载均衡.六.以下附上一些日志:keepalived启动日志:oct 22 17: 07: 39 yuhao linux keepalived: starting keepalived v1.1.17 (10 / 222009)oct 22 17: 07: 39 yuhao linux keepalived: remove a zombie pid file / var / run / vrrp.pidoct 22 17: 07: 39 yuhao l

27、inux keepalived: remove a zombie pid file / var / run / checkers.pidoct 22 17: 07: 39 yuhao linux keepalived _ healthcheckers: using i - bmsr nic polling threads.oct 22 17: 07: 39 yuhao linux keepalived _ healthcheckers: netlink reflector reports ip 58 addedoct 22 17: 07: 39 yuhao linux k

28、eepalived _ healthcheckers: netlink reflector reports ip 88 addedoct 22 17: 07: 39 yuhao linux keepalived _ healthcheckers: proposed kernel netlink reflector10月22日17:07:39豪Linux keepalived_healthcheckers：注册内核netlink命令通道10月22日17:07:39豪Linux简单：开始检查孩子的过程中，PID = 597210月22日17:07:39豪Linux keepa

29、lived_vrrp：使用mii-bmsr NIC轮询线程10月22日17:07:39豪keepalived_vrrp：Linux Netlink反射器IP 58补充报道10月22日17:07:39豪keepalived_vrrp：Linux Netlink反射器IP 88补充报道10月22日17:07:39豪Linux简单：启动VRRP的子进程，PID = 597310月22日17:07:39豪Linux简单：简单的启动成功10月22日17:07:39豪Linux keepalived_healthcheckers：打开文件“/ etc /简单/简

30、单配置的。10月22日17:07:39豪Linux keepalived_vrrp：注册内核netlink反射器10月22日17:07:40豪Linux keepalived_healthcheckers：配置使用：7482字节10月22日17:07:40豪Linux keepalived_vrrp：注册内核netlink命令通道10月22日17:07:40豪Linux keepalived_vrrp：注册gratutious ARP共享信道10月22日17:07:40豪Linux keepalived_vrrp：打开文件“/ etc /简单/简单配置的。10月22日17:07:40豪Linux keepalived_vrrp：配置使用：37230字节10月22日17:07