LVS+heartbeat+ldirectord高可用负载均衡集群解决方案
LVS是Linux Virtual Server的简写，意即Linux虚拟服务器，是一个虚拟的服务器集群系统。本项目由章文嵩博士在1998年5月创立，是中国国内最早出现的自由软件项目之一。其作用是实现LB（Load Balance）将大量的并发访问或数据流量分担到多台节点设备上分别处理，减少用户等待响应的时间；将单个重负载的运算分担到多台节点设备上做并行处理，每个节点设备处理结束后，将结果汇总，返回给用户，系统处理能力得到大幅度提高。
HA（High Avalibaility）高可用性，即双机热备的方式保证服务的高可用性。
本文高可用负载均衡集群解决方案，是在LVS实现负载均衡的基础之上，采用双机热备director来保证lvs个高可用性，并通过heartbeat实时监听主director的运行状态，另外使用ldirectord实时监测director后端Real Server的健康状态，将不能正常提供服务的主机离线。
整套方案拓扑架构如图：
下面开始详述实现步骤：
一、两台Real Server：RS1、RS2的配置
初始化步骤：配置ip地址、主机名、yum源等不在赘述
<ol class="dp-rb"> <li class="alt"><span><span>echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore </span></span></li> <li><span>echo 2> /proc/sys/net/ipv4/conf/lo/arp_announce </span></li> <li class="alt"><span>echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore </span></li> <li><span>echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce </span></li> <li class="alt"><span><span class="comment">#定义内核参数，禁止real server响应ARP广播请求</span><span> </span></span></li> </ol>
Ps：由于使用DR模型故要屏蔽arp对后端realserver的解析
<ol class="dp-rb"> <li class="alt"><span><span>ifconfig lo:0 192.168.0.123 broadcast 192.168.0.123 netmask 255.255.255.255 up </span></span></li> <li><span><span class="comment">#在回环端口配置对外通信的VIP，并锁定其广播域</span><span> </span></span></li> <li class="alt"><span>route add -host 192.168.0.123 dev lo:0 </span></li> <li><span><span class="comment">#配置主机路由，使之对外发送数据包均使用VIP</span><span> </span></span></li> </ol>
<ol class="dp-rb"> <li class="alt"><span><span>yum install -y httpd </span></span></li> <li><span><span class="comment">#安装安装web服务</span><span> </span></span></li> <li class="alt"><span>service httpd start </span></li> <li><span>echo <span class="string">'<h>Real Server 1</h>'</span><span> > /var/www/html/index.html </span></span></li> <li class="alt"><span><span class="comment">#添加real server测试页面</span><span> </span></span></li> <li><span>mkdir /var/www/html/test/ </span></li> <li class="alt"><span>echo <span class="string">'ok'</span><span> > /var/www/html/test/index.html </span></span></li> <li><span><span class="comment">#为ldirectord提供检测页面</span><span> </span></span></li> <li>service httpd start</li> </ol>
Ps：这里仅以RS1为实例进行说明，RS2同此法操作。
二、高可用HA director节点的配置
初始化步骤：配置ip、yum源等。
Ps：两台director节点都使用双网卡，eth0与real server同网段，eth1使用网卡直连用于传输心跳信号。
<ol class="dp-rb"> <li class="alt"><span><span>vim /etc/hosts </span></span></li> <li><span>192.168.0.71 node1.a.org node1 </span></li> <li class="alt"><span>192.168.0.72 node2.a.org node1 </span></li> <li><span><span class="comment">#用于实现两台director节点间域名解析，此操作node1、node2相同</span><span> </span></span></li> </ol>
<ol class="dp-rb"> <li class="alt"><span><span>vim /etc/sysconfig/network </span></span></li> <li><span><span class="comment">#设置主机名</span><span> </span></span></li> <li class="alt"><span>hostname node1.a.org </span></li> <li><span><span class="comment">#修改主机名使之立即生效</span><span> </span></span></li> </ol>
使heartbeat信息加密传输
<ol class="dp-rb"> <li class="alt"><span><span>ssh-keygen -t rsa </span></span></li> <li><span><span class="comment">#生成密钥</span><span> </span></span></li> <li class="alt"><span>ssh-copy-id -i .ssh/id_rsa.pub root<span class="variable">@node2</span><span>.a.org </span></span></li> <li><span><span class="comment">#将公钥复制给node2</span><span> </span></span></li> <li class="alt"><span>ssh node2 -- ‘ifconfig’ </span></li> <li><span><span class="comment">#执行命令测试，此时应该显示node2的ip信息</span><span> </span></span></li> </ol>
安装heartbeat
<ol class="dp-rb"> <li class="alt"><span><span>cd heartbeat </span></span></li> <li><span>yum localinstall -y --nogpgcheck ./* </span></li> <li class="alt"><span><span class="comment">#安装此目录中的所有rpm包</span><span> </span></span></li> </ol>
<ol class="dp-xml"> <li class="alt"><span><span>cd /usr/share/doc/heartbeat-2.1.4 </span></span></li> <li><span>cp authkeys /etc/ha.d/ </span></li> <li class="alt"><span>cp haresources /etc/ha.d/ </span></li> <li><span>cp ha.cf /etc/ha.d/ </span></li> <li class="alt"><span>#拷贝heartbeat所需配置文件到指定目录下 </span></li> </ol>
<ol class="dp-rb"> <li class="alt"><span><span>vim /etc/ha.d/ha.cf </span></span></li> <li><span>bcast eth1 </span></li> <li class="alt"><span><span class="comment">#定义心跳信息从那一块网卡传输</span><span> </span></span></li> <li><span>node node1.a.org </span></li> <li class="alt"><span>node node2.a.org </span></li> <li><span><span class="comment">#添加这两行，用于指明心跳信号传输范围</span><span> </span></span></li> <li class="alt"><span>vim /etc/ha.d/authkeys </span></li> <li><span>auth 2 </span></li> <li class="alt"><span>2 sha1 [键入随机数] </span></li> <li><span>chmod 400 authkeys </span></li> <li class="alt"><span><span class="comment">#保存退出并修改权限400</span><span> </span></span></li> <li><span>vim /etc/ha.d/haresource </span></li> <li class="alt"> <span>node1.a.org 192.168.0.123/24/eth0/192.168.0.255 </span><span style="COLOR: rgb(255,0,0)">ldirectord::</span><span style="COLOR: rgb(51,102,255)"><span style="COLOR: rgb(255,0,0)"><span class="Apple-style-span">ldirectord.cf httpd</span></span></span> </li> <li><span><span class="comment">#末行添加主节点域名，vip资源，广播地址，ldirectord资源，以及用户提供显示错误页面的httpd资源</span></span></li> </ol>
同步配置文件到node2
<ol class="dp-rb"> <li class="alt"><span><span>/usr/lib/heartbeat/ha_propagate </span></span></li> <li><span><span class="comment">#脚本用来同步ha.cf和authkeys文件到node2</span><span> </span></span></li> <li class="alt"><span>scp haresources node2:/etc/ha.d/ </span></li> <li><span><span class="comment">#复制haresource到nod2</span><span> </span></span></li> </ol>
配置ldirectord，同步配置文件
<ol class="dp-xml"> <li class="alt"><span><span>cp /usr/share/doc/heartbeat-ldirectord-2.1.4/ldirectord.cf /etc/ha.d/ldirectord.cf </span></span></li> <li><span>#复制ldirector的配置文件 </span></li> </ol>
内容如下配置
<ol class="dp-xml"> <li class="alt"><span><span class="attribute">checktimeout</span><span>=</span><span class="attribute-value">3</span><span> </span></span></li> <li><span>#当DR收不到realserver的回应，设定几秒后判定realserver当机或挂掉了，预设5秒。 </span></li> <li class="alt"><span><span class="attribute">checkinterval</span><span>=</span><span class="attribute-value">1</span><span> </span></span></li> <li><span>#查询间隔，每个几秒侦测一次realserver </span></li> <li class="alt"><span><span class="attribute">autoreload</span><span>=</span><span class="attribute-value">yes</span><span> </span></span></li> <li><span>#配置文件发生改变是否自动重读 </span></li> <li class="alt"><span><span class="attribute">quiescent</span><span>=</span><span class="attribute-value">yes</span><span> </span></span></li> <li><span>#静态链接，yes：表示侦测realserver宕机，将其权值至零（���果开启了persistent参数不要用yes）；no：表示侦测到宕机realserver，随即将其对应条目从ipvsadm中删除。 </span></li> <li class="alt"><span><span class="attribute">virtual</span><span>=</span><span class="attribute-value">192</span><span>.168.0.123:80 </span></span></li> <li><span> <span class="attribute">real</span><span>=</span><span class="attribute-value">192</span><span>.168.0.51:80 gate 4 </span></span></li> <li class="alt"><span> <span class="attribute">real</span><span>=</span><span class="attribute-value">192</span><span>.168.0.52:80 gate 2 </span></span></li> <li><span> <span class="attribute">fallback</span><span>=</span><span class="attribute-value">127</span><span>.0.0.1:80 gate #realserver全部失败，vip指向本机80端口。 </span></span></li> <li class="alt"><span> <span class="attribute">service</span><span>=</span><span class="attribute-value">http</span><span> </span></span></li> <li><span> <span class="attribute">request</span><span>=</span><span class="attribute-value">"test/index.html"</span><span> #用于健康检测的url </span></span></li> <li class="alt"><span> <span class="attribute">receive</span><span>=</span><span class="attribute-value">"ok"</span><span> #用于健康检测的url包含的关键字 </span></span></li> <li><span> <span class="attribute">scheduler</span><span>=</span><span class="attribute-value">wlc</span><span> </span></span></li> <li class="alt"><span> #<span class="attribute">persistent</span><span>=</span><span class="attribute-value">600</span><span> </span></span></li> <li><span> #持久链接：表示600s之内同意ip将访问同一台realserver </span></li> <li class="alt"><span> <span class="attribute">protocol</span><span>=</span><span class="attribute-value">tcp</span><span> </span></span></li> <li><span> <span class="attribute">checktype</span><span>=</span><span class="attribute-value">negotiate</span><span> </span></span></li> <li class="alt"><span> #检查类型：negotiate，表示DR发送请求，realserver恢复特定字符串才表示服务正常；connect，表示DR能够连线realserver即正常。 </span></li> <li><span> <span class="attribute">checkport</span><span>=</span><span class="attribute-value">80</span><span> </span></span></li> </ol>
启动两个节点上的heartbeat
<ol class="dp-xml"> <li class="alt"><span><span>service heartbeat start </span></span></li> <li><span>ssh node2 -- 'service heartbeat start' </span></li> <li class="alt"><span>#启动两节点heartbeat服务 </span></li> </ol>
Ps：ldirectord同vip一样是DR节点争夺的资源，由heartbeart启动
tail -f /var/log/messages
应出现：heartbeat: info: Running /etc/ha.d/resource.d/ldirectord ldirectord.cf start
ip、ldirectord和httpd资源被主节点成功启动
或查看ipvsadm -ln
在主节点上安装httpd服务，并创建测试页
<ol class="dp-rb"> <li class="alt"><span><span>yum install -y httpd </span></span></li> <li> <span><span class="comment">#安装安装web服务 </span><span> </span></span><font class="Apple-style-span" color="#000000"><br></font> </li> <li><span>echo <span class="string">'<h2>sorry, web service is down, please try again later...</h2>'</span><span> > /var/www/html/index.html </span></span></li> <li class="alt"><span><span class="comment">#node1上添加报错提示 </span><span> </span></span></li> <li><span>echo <span class="string">'sorry, web service is down, please try again later...'</span><span> > /var/www/html/index.html </span></span></li> <li class="alt"><span><span class="comment">#node2上添加报错提示，</span><span style="COLOR: rgb(0,0,255)"><span class="comment">此处字体不同以示区别</span> </span></span></li> </ol>
三、测试
使用浏览器访问web页面，多次刷新RS1和RS2都能访问到
ab测试wlc算法的负载均衡
关闭node1上的heartbeat模拟主节点宕机，并进行ab测试，node2上查看ipvsadm
依旧能够从备份DR节点node2访问web服务并实现负载均衡
重启node1上heartbeat，关闭RS1的httpd服务，查看ipvsadm
idirectord立即检测到，并将RS1的权值至0
继续关闭RS2的httpd服务，查看ipvsadm
RS2权值至0，本地权值至1
刷新浏览器页面
主节点node1 web服务反馈realserver宕机提示信息
此时关闭node1上heartbeat，刷新页面
备份节点node2 web服务反馈realserver宕机提示信息
至此LVS+heartbeat+ldirectord高可用负载均衡集群解决方案的实现流程已叙述完毕，以上内容在您实现过程中出现的任何问题，欢迎在此交流；并请各位高手前辈大神达人予以批评指正，谢谢！
相关推荐
liupengqwert 2020-10-28
快乐de馒头 2020-06-16
CurrentJ 2020-05-12
azhuye 2020-11-12
YzhilongY 2020-08-31
crazyjingling 2020-08-16
swtIrene 2020-08-14
slovyz 2020-08-14
tinydu 2020-08-09
tinydu 2020-08-03
Studynutlee 2020-08-03
快乐de馒头 2020-07-29
yungame 2020-07-27
buaichidoufu 2020-07-28
wanjichun 2020-07-26
极地雪狼 2020-07-26
yungame 2020-07-04
畅聊架构 2020-06-28