lvs+keepalived linux负载均衡实现

背景:
        随着你的网站业务量的增长你网站的服务器压力越来越大？需要负载均衡方案！商业的硬件如F5又太贵，你们又是创业型互联公司如何有效节约成本，节省不必要的浪费？同时实现商业硬件一样的高性能高可用的功能？有什么好的负载均衡可伸张可扩展的方案吗？答案是肯定的！有！我们利用LVS+Keepalived基于完整开源软件的架构可以为你提供一个负载均衡及高可用的服务器.

LVS+Keepalived 介绍
LVS
LVS是Linux Virtual Server的简写，意即Linux虚拟服务器，是一个虚拟的服务器集群系统。本项目在1998年5月由章文嵩博士成立，是中国国内最早出现的自由软件项目之一。目前有三种IP负载均衡技术（VS/NAT、VS/TUN和VS/DR）；
十种调度算法（rrr|wrr|lc|wlc|lblc|lblcr|dh|sh|sed|nq）。

操作系统:

Centos 6.0 64位版本
Keepalvied
               Keepalived在这里主要用作RealServer的健康状态检查

IP信息列表：   
名称                           IP   

LVS-DR-Master     61.164.122.6  //负责请求分发，需安装lvs+keepalived        


WEB1-Realserver   61.164.122.9  //负责接受lvs分发的http请求，不需安装lvs+keepalived           


WEB2-Realserver   61.164.122.10 //负责接受lvs分发的http请求，不需安装lvs+keepalived        


GateWay           61.164.122.1          

IP信息列表：
名称                           IP
LVS-DR-Master     61.164.122.6  //负责请求分发，需安装lvs+keepalived     
WEB1-Realserver   61.164.122.9  //负责接受lvs分发的http请求，不需安装lvs+keepalived        
WEB2-Realserver   61.164.122.10 //负责接受lvs分发的http请求，不需安装lvs+keepalived     
GateWay           61.164.122.1

 三. 安装LVS和Keepalvied软件包(LVS-DR-Master 服务器安装)

   1. 下载相关软件包

Shell代码  

1. #mkdir /usr/local/src/lvs   
2. #cd /usr/local/src/lvs   
3. #wget http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.24.tar.gz   
4. #wget http://www.keepalived.org/software/keepalived-1.1.15.tar.gz  

#mkdir /usr/local/src/lvs
#cd /usr/local/src/lvs
#wget http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.24.tar.gz
#wget http://www.keepalived.org/software/keepalived-1.1.15.tar.gz

   2. 安装LVS和Keepalived

Shell代码  

1. #uname -r   
2. 2.6.32-71.el6.x86_64   
3. #ln -s /usr/src/kernels/2.6.32-71.el6.x86_64/  /usr/src/linux   
4.   
5. #tar zxvf ipvsadm-1.24.tar.gz   
6. #cd ipvsadm-1.24  
7. #make && make install   
8. #find / -name ipvsadm  # 查看ipvsadm的位置   
9.   
10. #tar zxvf keepalived-1.1.15.tar.gz   
11. #cd keepalived-1.1.15  
12. #./configure                      #查看keepalived配置. 要和FAQ2中的配置一致   
13. #make && make install   
14. #find / -name keepalived  # 查看keepalived位置 ,默认的路径是 /usr/local/etc和/usr/local/sbin                  
15.   
16. #cp /usr/local/etc/rc.d/init.d/keepalived /etc/rc.d/init.d/   
17. #cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/   
18. #mkdir /etc/keepalived   
19. #cp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/   
20. #cp /usr/local/sbin/keepalived /usr/sbin/   
21. #service keepalived start|stop     #做成系统启动服务方便管理.  

#uname -r
	2.6.32-71.el6.x86_64
	#ln -s /usr/src/kernels/2.6.32-71.el6.x86_64/  /usr/src/linux

	#tar zxvf ipvsadm-1.24.tar.gz
	#cd ipvsadm-1.24
	#make && make install
	#find / -name ipvsadm  # 查看ipvsadm的位置

	#tar zxvf keepalived-1.1.15.tar.gz
	#cd keepalived-1.1.15
	#./configure                      #查看keepalived配置. 要和FAQ2中的配置一致
 #make && make install
	#find / -name keepalived  # 查看keepalived位置 ,默认的路径是 /usr/local/etc和/usr/local/sbin               

	#cp /usr/local/etc/rc.d/init.d/keepalived /etc/rc.d/init.d/
	#cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/
	#mkdir /etc/keepalived
	#cp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/
	#cp /usr/local/sbin/keepalived /usr/sbin/
	#service keepalived start|stop     #做成系统启动服务方便管理.

    FAQ:1.在安装keepalive过程中可能会出现的问题:

缺少依赖包, 搜索下载对应包安装即可, 我安装过程中报Popt libraries is required

Shell代码  

1. wget http://rpm5.org/files/popt/popt-1.16.tar.gz   
2. tar zxvf popt-1.16.tar.gz   
3. cd popt-1.16  
4. ./configure   
5. make   
6. makeinstall  

wget http://rpm5.org/files/popt/popt-1.16.tar.gz
tar zxvf popt-1.16.tar.gz
cd popt-1.16
./configure
make
makeinstall

Shell代码  

1. Keepalived configuration   
2. ------------------------   
3. Keepalived version       : 1.2.2  
4. Compiler                 : gcc   
5. Compiler flags           : -g -O2   
6. Extra Lib                : -lpopt -lssl -lcrypto    
7. Use IPVS Framework       : No   
8. IPVS sync daemon support : No   
9. Use VRRP Framework       : Yes   
10. Use Debug flags          : No  

Keepalived configuration
------------------------
Keepalived version       : 1.2.2
Compiler                 : gcc
Compiler flags           : -g -O2
Extra Lib                : -lpopt -lssl -lcrypto 
Use IPVS Framework       : No
IPVS sync daemon support : No
Use VRRP Framework       : Yes
Use Debug flags          : No

    如果显示 IPVS sync daemon support: NO,   安装是不起作用的. 需执行  ln -s /usr/src/kernels/2.6.18-164.el5-i686/ /usr/src/linux. 然后configure 如下, 配置正常, 进行 make && make install 编译安装

Shell代码  

1. Keepalived configuration   
2. ------------------------   
3. Keepalived version       : 1.2.2  
4. Compiler                 : gcc   
5. Compiler flags           : -g -O2   
6. Extra Lib                : -lpopt -lssl -lcrypto    
7. Use IPVS Framework       : Yes   
8. IPVS sync daemon support : Yes   
9. IPVS use libnl           : No   
10. Use VRRP Framework       : Yes   
11. Use Debug flags          : No  

Keepalived configuration
------------------------
Keepalived version       : 1.2.2
Compiler                 : gcc
Compiler flags           : -g -O2
Extra Lib                : -lpopt -lssl -lcrypto 
Use IPVS Framework       : Yes
IPVS sync daemon support : Yes
IPVS use libnl           : No
Use VRRP Framework       : Yes
Use Debug flags          : No

 四. 配置LVS实现负载均衡

 1.配置Realserver(client端)脚本.

Shell代码  

1. #vi /usr/local/sbin/realserver.sh   
2. #!/bin/bash   
3. # description: Config realserver lo and apply noarp    
4. #Written by :NetSeek http://www.linuxtone.org   
5.   
6. SNS_VIP=61.164.122.8  
7.     
8. . /etc/rc.d/init.d/functions   
9.     
10. case "$1" in   
11. start)   
12.        ifconfig lo:0 $SNS_VIP netmask 255.255.255.255 broadcast $SNS_VIP   
13.        /sbin/route add -host $SNS_VIP dev lo:0  
14.        echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore   
15.        echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce   
16.        echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore   
17.        echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce   
18.        sysctl -p >/dev/null 2>&1  
19.        echo "RealServer Start OK"  
20.     
21.        ;;   
22. stop)   
23.        ifconfig lo:0 down   
24.        route del $SNS_VIP >/dev/null 2>&1  
25.        echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore   
26.        echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce   
27.        echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore   
28.        echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce   
29.        echo "RealServer Stoped"  
30.        ;;   
31. *)   
32.        echo "Usage: $0 {start|stop}"  
33.        exit 1  
34. esac   
35.     
36. exit 0  

#vi /usr/local/sbin/realserver.sh
        #!/bin/bash
        # description: Config realserver lo and apply noarp 
        #Written by :NetSeek http://www.linuxtone.org
        
        SNS_VIP=61.164.122.8
         
        . /etc/rc.d/init.d/functions
         
        case "$1" in
        start)
               ifconfig lo:0 $SNS_VIP netmask 255.255.255.255 broadcast $SNS_VIP
               /sbin/route add -host $SNS_VIP dev lo:0
               echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
               echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
               echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
               echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
               sysctl -p >/dev/null 2>&1
               echo "RealServer Start OK"
         
               ;;
        stop)
               ifconfig lo:0 down
               route del $SNS_VIP >/dev/null 2>&1
               echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
               echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
               echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
               echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
               echo "RealServer Stoped"
               ;;
        *)
               echo "Usage: $0 {start|stop}"
               exit 1
        esac
         
        exit 0

 五．利用Keepalvied实现负载均衡和和高可用性

  1.配置在主负载均衡服务器上配置keepalived.conf
  #vi /etc/keepalived/keepalived.conf

Shell代码  

1.   ! Configuration File for keepalived   
2.   
3. global_defs {   
4.    notification_email {   
5.          [email protected]   
6.    }   
7.    notification_email_from [email protected]   
8.    smtp_server 127.0.0.1  
9.   # smtp_connect_timeout 30  
10.    router_id LVS_DEVEL   
11. }   
12.   
13. # 20081013 written by :netseek   
14. # VIP1   
15. vrrp_instance VI_1 {   
16.     state MASTER             #备份服务器上将MASTER改为BACKUP      
17.     interface eth0   
18.     virtual_router_id 51  
19.     priority 100    # 备份服务上将100改为99  
20.     advert_int 1  
21.     authentication {   
22.         auth_type PASS   
23.         auth_pass 1111  
24.     }   
25.     virtual_ipaddress {   
26.         61.164.122.8      
27.         #(如果有多个VIP，继续换行填写.)   
28.     }   
29. }   
30.   
31. virtual_server 61.164.122.8 80 {   
32.     delay_loop 6                  #(每隔10秒查询realserver状态)   
33.     lb_algo wrr                  #(lvs 算法)   
34.     lb_kind DR                  #(Direct Route)   
35.     persistence_timeout 60        #(同一IP的连接60秒内被分配到同一台realserver)   
36.     protocol TCP                #(用TCP协议检查realserver状态)   
37.   
38.     real_server 61.164.122.9 80 {   
39.         weight 3               #(权重)   
40.         TCP_CHECK {   
41.         connect_timeout 10       #(10秒无响应超时)   
42.         nb_get_retry 3  
43.         delay_before_retry 3  
44.         connect_port 80  
45.         }   
46.     }   
47.     real_server 61.164.122.10 80 {   
48.         weight 3  
49.         TCP_CHECK {   
50.         connect_timeout 10  
51.         nb_get_retry 3  
52.         delay_before_retry 3  
53.         connect_port 80  
54.         }   
55.      }   
56. }   
57.          

! Configuration File for keepalived

global_defs {
   notification_email {
         [email protected]
   }
   notification_email_from [email protected]
   smtp_server 127.0.0.1
  # smtp_connect_timeout 30
   router_id LVS_DEVEL
}

# 20081013 written by :netseek
# VIP1
vrrp_instance VI_1 {
    state MASTER             #备份服务器上将MASTER改为BACKUP   
    interface eth0
    virtual_router_id 51
    priority 100    # 备份服务上将100改为99
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        61.164.122.8   
        #(如果有多个VIP，继续换行填写.)
    }
}

virtual_server 61.164.122.8 80 {
    delay_loop 6                  #(每隔10秒查询realserver状态)
    lb_algo wrr                  #(lvs 算法)
    lb_kind DR                  #(Direct Route)
    persistence_timeout 60        #(同一IP的连接60秒内被分配到同一台realserver)
    protocol TCP                #(用TCP协议检查realserver状态)

    real_server 61.164.122.9 80 {
        weight 3               #(权重)
        TCP_CHECK {
        connect_timeout 10       #(10秒无响应超时)
        nb_get_retry 3
        delay_before_retry 3
        connect_port 80
        }
    }
    real_server 61.164.122.10 80 {
        weight 3
        TCP_CHECK {
        connect_timeout 10
        nb_get_retry 3
        delay_before_retry 3
        connect_port 80
        }
     }
}

   #/etc/init.d/keepalived start  启动keepalived 服务，keepalived就能利用keepalived.conf 配置文件，实现负载均衡和高可用.
     整个LVS负均衡HA方案，由keepalived.conf一个文件即可搞定！

参考http://bbs.linuxtone.org/thread-1077-1-1.html