redis集群部署

fsl

2020-05-04

搭建Redis集群

准备6台redis服务器,具体要求如下:
ip地址 I端口 etho/ 日志文件名
192.168.4.51 6051 都可以接收连接请求 redis51.log
192.168.4.52 6052 都可以接收连接请求 redis52.log
192.168.4.53 6053 都可以接收连接请求 redis53.log
192.168.4.54 6054 都可以接收连接请求 redis54.log
192.168.4.55 6055 都可以接收连接请求 redis55.log
192.168.4.56 6056 都可以接收连接请求 redis56.log

环境准备

1、在6台redis服务器上运行服务，按照如下要求修改配置文件，重启redis服务

redis版本3以上

vim  /etc/redis/6379.conf
..........
bind 192.168.4.56
port 6056

//开启集群

################################ REDIS CLUSTER
815 cluster-enabled yes                             //开启集群
159 pidfile /var/run/redis_56.pid                  //pid文件
823 cluster-config-file nodes-6056.conf         //当前集群主机信息

1）部署ruby脚本运行环境（在51上面执行）

[ redis-4.0.8]# cd /root/redis-cluster/

[ redis-cluster]# ls

redis-3.2.1.gem  ruby-devel-2.0.0.648-30.el7.x86_64.rpm

[ redis-cluster]# yum -y install  ruby rubygems

[ redis-cluster]# rpm -ivh –nodeps \ 

 ruby-devel-2.0.0.648-30.el7.x86_64.rpm
warning: ruby-devel-2.0.0.648-30.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID f4a80eb5: NOKEY
Preparing...                          ################################# [100%]
Updating / installing...
   1:ruby-devel-2.0.0.648-30.el7      ################################# [100%]
[ redis-cluster]# which gem
/usr/bin/gem
[ redis-cluster]# gem install redis        
Successfully installed redis-3.2.1
Parsing documentation for redis-3.2.1
Installing ri documentation for redis-3.2.1
1 gem installed

源码编译安装ruby

------------------------------------------------------------
[ ~]# yum -y install gcc make gcc-c++

[ ~]# wget http://cache.ruby-lang.org/pub/ruby/2.1/ruby-2.1.4.tar.gz

[ ~]#tar -xzvf ruby-2.1.4.tar.gz 

[ ~]# cd ruby-2.1.4

[ ruby-2.1.4]# ./configure --enable-pthread --prefix=/usr/local/ruby

[ ruby-2.1.4]# make && make install

[ ruby-2.1.4]# cat << EOF >> /etc/profile

export PATH=/usr/local/ruby/bin:$PATH

EOF

[ ruby-2.1.4]# source /etc/profile

[ ruby-2.1.4]# ruby -v

ruby 2.1.4p265 (2014-10-27 revision 48166) [x86_64-linux]

[ redis-cluster]# gem install redis        

Successfully installed redis-3.2.1
Parsing documentation for redis-3.2.1
Installing ri documentation for redis-3.2.1
1 gem installed

------------------------------------------------------------

2）生成创建集群的脚本

[ redis-cluster]# cd /root/redis/redis-4.0.8/src/
[ src]# cp redis-trib.rb /usr/local/bin/
[ src]# ll /usr/local/bin/redis-trib.rb
-rwxr-xr-x. 1 root root 65991 Sep 27 16:12 /usr/local/bin/redis-trib.rb

3）创建集群

[ src]# redis-trib.rb  create --replicas 1 \ 
192.168.4.51:6051  192.168.4.52:6052 \ 
192.168.4.53:6053 192.168.4.54:6054  \ 
192.168.4.55:6055 192.168.4.56:6056

//--replicas 1 给每一个主配置一个从库
[ log]# redis-trib.rb create --replicas 1 192.168.4.51:6351 192.168.4.52:6352    192.168.4.53:6353 192.168.4.54:6354    192.168.4.55:6355 192.168.4.56:6356
>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
192.168.4.51:6351
192.168.4.52:6352
192.168.4.53:6353
Adding replica 192.168.4.55:6355 to 192.168.4.51:6351
Adding replica 192.168.4.56:6356 to 192.168.4.52:6352
Adding replica 192.168.4.54:6354 to 192.168.4.53:6353
...
...
...
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

槽位信息说明：
slots //槽位总16384个范围（0-16383）
key/values x=99
crc16 x =23 % 16384 =5609

4）查看集群信息，任意一台主机访问本机的redis服务查看即可

cluster info 查看集群信息
cluster nodes 查看集群节点信息
[ log]# redis-cli  -h  192.168.4.52  -p  6352 
-----------------------------------------------------------
[ ~]# redis-cli -c -h 192.168.4.52 -p 6052 //-c 连接集群
192.168.4.52:6352>
192.168.4.52:6352> CLUSTER INFO
cluster_state:ok            //状态
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:2
cluster_stats_messages_ping_sent:367
cluster_stats_messages_pong_sent:327
cluster_stats_messages_meet_sent:5
cluster_stats_messages_sent:699
cluster_stats_messages_ping_received:327
cluster_stats_messages_pong_received:372
cluster_stats_messages_received:699

192.168.4.52:6352> CLUSTER NODES        //查看集群节点信息
63afbb5e7d63b1f142358634578a3488e3c9e634 192.168.4.54: 

slave bc5c4e082a5a3391b634cf433a6486c867cfc44b 0 1538039278871 4 connected

bc5c4e082a5a3391b634cf433a6486c867cfc44b 192.168.4.53: 

master - 0 1538039278571 3 connected 10923-16383

28e06c5f24a2b6c6412f81369e09bc9653cc51ff 192.168.4.56: 

slave 8568fbd73cb296cad6915d524e34761b2114af47 0 1538039278069 6 connected

7e8d9121f44d8331ff55b45c218b87df9bda1b70 192.168.4.55: 

slave a3083123bc5c87a76aab2655171634d4ee84f418 0 1538039278000 5 connected

8568fbd73cb296cad6915d524e34761b2114af47 192.168.4.52: 

myself,master - 0 1538039277000 2 connected 5461-10922

a3083123bc5c87a76aab2655171634d4ee84f418 192.168.4.51: 

master - 0 1538039277869 1 connected 0-5460
192.168.4.52:6352>

5）测试集群

命令：
redis-cli -c -h ip地址 -p 端口

[ log]# redis-cli  -c -h 192.168.4.51 -p 6351
192.168.4.51:6351> set name jim
-> Redirected to slot [5798] located at 192.168.4.52:6352
OK
192.168.4.52:6352> get name
"jim"
192.168.4.52:6352> set class linux
OK
192.168.4.52:6352> get class
"linux"
192.168.4.52:6352> set pay 26800
-> Redirected to slot [4013] located at 192.168.4.51:6351
OK
192.168.4.51:6351> get pay
"26800"

集群不能用的情况：
有半数或者半数以上的主库机器挂掉，集群就不能用

了
把一个从库升级成主，没有从库，集群不能用（前提是：有半数或者半数以上

的主库机器挂掉）
一个主库挂掉，它的从库自动顶替为主库，正常使用

（前提是：有半数或者半数以上的主库机器能用），挂掉的主库修复好后，

会成为从库，不会抢占为主

6）集群节点选举策略（三主，三从）

停止某个主库的redis服务，

对应的从库会自动升级为主库
先查看节点信息的主从情况

[ log]# redis-cli  -c -h 192.168.4.51 -p 6351
192.168.4.51:6351> CLUSTER  nodes        
...
8568fbd73cb296cad6915d524e34761b2114af47 192.168.4.52: 

master - 0 1538040400840 2 connected 5461-10922

28e06c5f24a2b6c6412f81369e09bc9653cc51ff 192.168.4.56: 

slave 8568fbd73cb296cad6915d524e34761b2114af47 0 1538040400000 6 connected
...
192.168.4.51:6351>

看谁是谁的从库，如：
看节点前后的编号id是否有相同的
如：8568fbd73cb296cad6915d524e

34761b2114af47
发现52的从库为56
停止主库52

[ log]#  redis-cli    -h 192.168.4.52 -p 6352  shutdown

[ log]# redis-cli  -c -h 192.168.4.51 -p 6351
192.168.4.51:6351> CLUSTER NODES
...
8568fbd73cb296cad6915d524e34761b2114af47 192.168.4.52: 

master,fail - 1538041052349 1538041051000 2 disconnected        //52的主库坏掉

28e06c5f24a2b6c6412f81369e09bc9653cc51ff 192.168.4.56: 

master - 0 1538041066000 7 connected 5461-10922        //56成为主库

...

开启52，发现52成为从库

[ redis-4.0.8]# /etc/init.d/redis_6352 start 
Starting Redis server...

[ log]# redis-cli  -c -h 192.168.4.51 -p 6351
192.168.4.51:6351> CLUSTER NODES
8568fbd73cb296cad6915d524e34761b2114af47 192.168.4.52: 

slave 28e06c5f24a2b6c6412f81369e09bc9653cc51ff 0 1538041254000 7 connected

2 案例2：管理redis集群

2.1 问题
具体要求如下：
练习添加主机
练习删除主机

2.2 步骤
实现此案例需要按照如下步骤进行。

步骤一：添加主机

1）部署一台新redis服务器，ip为192.168.4.58，装包，初始化，启用集群配置，重启服务

2）添加集群4.58（添加master节点）

格式：redis-trib.rb 选项参数
选项： add-node

添加主机（不指定角色为主）
由于之前是在51上面创建ruby脚本，所以只有51上面有

redis-trib.rb命令，

在51上面执行

[ ~]# redis-trib.rb add-node 192.168.4.58:6358 192.168.4.51:6351
>>> Adding node 192.168.4.58:6358 to cluster 192.168.4.51:6351
>>> Performing Cluster Check (using node 192.168.4.51:6351)
S: a3083123bc5c87a76aab2655171634d4ee84f418 192.168.4.51:6351
   slots: (0 slots) slave
   replicates 7e8d9121f44d8331ff55b45c218b87df9bda1b70
M: 7e8d9121f44d8331ff55b45c218b87df9bda1b70 192.168.4.55:6355
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
S: 8568fbd73cb296cad6915d524e34761b2114af47 192.168.4.52:6352
   slots: (0 slots) slave
   replicates 28e06c5f24a2b6c6412f81369e09bc9653cc51ff
M: bc5c4e082a5a3391b634cf433a6486c867cfc44b 192.168.4.53:6353
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
S: 63afbb5e7d63b1f142358634578a3488e3c9e634 192.168.4.54:6354
   slots: (0 slots) slave
   replicates bc5c4e082a5a3391b634cf433a6486c867cfc44b
M: 28e06c5f24a2b6c6412f81369e09bc9653cc51ff 192.168.4.56:6356
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 192.168.4.58:6358 to make it join the cluster.
[OK] New node added correctly.

3）检查集群主机的状态信息

选项：check 检查权限

[ ~]# redis-trib.rb  check 192.168.4.58:6358        //查看状态
>>> Performing Cluster Check (using node 192.168.4.58:6358)
M: c5e0da48f335c46a2ec199faa99b830f537dd8a0 192.168.4.58:6358
   slots: (0 slots) master        //发现没有hash槽
   0 additional replica(s)
M: 7e8d9121f44d8331ff55b45c218b87df9bda1b70 192.168.4.55:6355
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
...
S: a3083123bc5c87a76aab2655171634d4ee84f418 192.168.4.51:6351
   slots: (0 slots) slave
   replicates 7e8d9121f44d8331ff55b45c218b87df9bda1b70
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

4）手动对集群进行分片迁移

选项：reshard 重新分配hash槽

[ ~]# redis-trib.rb  reshard   192.168.4.58:6358  

How many slots do you want to move (from 1 to 16384)?4096   
//拿出多少个hash 槽给主机192.168.4.58

What is the receiving node ID?  c5e0da48f335c46a2ec199faa99b830f537dd8a0   
//主机192.168.4.58的id值

Source node #1:all      //从当前所有的主里面获取hash槽
Do you want to proceed with the proposed reshard plan (yes/no)?yes
...
Moving slot 12283 from 192.168.4.53:6353 to 192.168.4.58:6358: 
Moving slot 12284 from 192.168.4.53:6353 to 192.168.4.58:6358: 
Moving slot 12285 from 192.168.4.53:6353 to 192.168.4.58:6358: 
Moving slot 12286 from 192.168.4.53:6353 to 192.168.4.58:6358: 
Moving slot 12287 from 192.168.4.53:6353 to 192.168.4.58:6358:

再次查看发现4.58有4096个hash slot

[ ~]# redis-trib.rb  check 192.168.4.58:6358
>>> Performing Cluster Check (using node 192.168.4.58:6358)
M: c5e0da48f335c46a2ec199faa99b830f537dd8a0 192.168.4.58:6358
   slots:0-1364,5461-6826,10923-12287 (4096 slots) master
   0 additional replica(s)

5）删除master角色的主机

先删除主机占用的hash槽

[ ~]# redis-trib.rb  reshard 192.168.4.58:6358
How many slots do you want to move (from 1 to 16384)?4096           
//移除hash 槽的个数
What is the receiving node ID?  bc5c4e082a5a3391b634cf433a6486c867cfc44b 
    //要移动给谁的id即目标主机（这里可以随机写一个master的ID）  
Source node #1: c5e0da48f335c46a2ec199faa99b830f537dd8a0
//从谁那移动即源主机（这里写4.58的ID）  
Source node #2:done           //设置完毕
...
    Moving slot 12282 from c5e0da48f335c46a2ec199faa99b830f537dd8a0
    Moving slot 12283 from c5e0da48f335c46a2ec199faa99b830f537dd8a0
    Moving slot 12284 from c5e0da48f335c46a2ec199faa99b830f537dd8a0
    Moving slot 12285 from c5e0da48f335c46a2ec199faa99b830f537dd8a0
    Moving slot 12286 from c5e0da48f335c46a2ec199faa99b830f537dd8a0
    Moving slot 12287 from c5e0da48f335c46a2ec199faa99b830f537dd8a0
Do you want to proceed with the proposed reshard plan (yes/no)?yes      //提交
... 
Moving slot 12282 from 192.168.4.58:6358 to 192.168.4.53:6353: 
Moving slot 12283 from 192.168.4.58:6358 to 192.168.4.53:6353: 
Moving slot 12284 from 192.168.4.58:6358 to 192.168.4.53:6353: 
Moving slot 12285 from 192.168.4.58:6358 to 192.168.4.53:6353: 
Moving slot 12286 from 192.168.4.58:6358 to 192.168.4.53:6353: 
Moving slot 12287 from 192.168.4.58:6358 to 192.168.4.53:6353:

删除集群主机4.58(删除之后redis服务自动关闭)

[ ~]# redis-trib.rb del-node 192.168.4.58:6358 \ 
 c5e0da48f335c46a2ec199faa99b830f537dd8a0    //删除谁+删除的id

>>> Removing node c5e0da48f335c46a2ec199faa99b830f537dd8a0 from cluster 192.168.4.58:6358
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.

6）添加从节点主机，随机添加

[ ~]# redis-trib.rb  add-node  --slave \ 
192.168.4.57:6357  192.168.4.51:6351
>>> Adding node 192.168.4.57:6357 to cluster 192.168.4.51:6351
>>> Performing Cluster Check (using node 192.168.4.51:6351)
……
……
[OK] All 16384 slots covered.
Automatically selected master 192.168.4.51:6351
>>> Send CLUSTER MEET to node 192.168.4.57:6357 to make it join the cluster.
Waiting for the cluster to join.
>>> Configure node as replica of 192.168.4.51:6351.
[OK] New node added correctly.

7）移除从节点，从节点主机没有槽位范围，直接移除即可

命令格式：
redis-trib.rb del-node 192.168.4.57:6357 主机id值

[ ~]# redis-trib.rb  del-node 192.168.4.57:6357  \ 
f6649ea99b2f01faca26217691222c17a3854381

>>> Removing node f6649ea99b2f01faca26217691222c17a3854381 
       from cluster 192.168.4.57:6351
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.

附加操作：

删除主机
redis-trib.rb reshard 192.168.4.51:6051
4096 移除hash槽的个数
主机id 目标主机id
主机id 源主机id
done 设置
yes 提交
all 所有主机

redis-trib.rb del-node 192.168.4.57:6357 f6649ea99b2f01faca26217691222c17a3854381

添加slave角色主机
1、主机运行redis服务，并设置集群配置
2、添加slave角色主机

. /redis-trib.rb add-node --slave --master-id id值
192.168.2.92: 6392 192.168.2.94:6394

如果不指定主节点的id的话，会把新节点随机添加为从节点最少的主的从。

redis-trib.rb add-node --slave 要添加的主机、端口目标主机、端口

[ ~]# redis-trib.rb add-node --slave 192.168.2.92:6392 192.168.2.94:6393

cluster reset

rm -rf /var/lib/redis/6379
mkdir /var/lib/redis/6379
chmod a+w /var/lib/redis/6379
rm -rf /var/run/redis_6379.pid
/etc/init.d/redis_6379 start
/etc/init.d/redis_6379 status

redis-cli -c -h 192.168.4.54 -p 6054
cluster info

redis-trib.rb add-node --slave 192.168.2.92:6392 192.168.2.94:6393

redis redis集群搭建 ruby 集群服务器编程语言