Docker+Redis3集群环境搭建
拓扑：
实验目标:
client通过192.168.100.67访问到下列redis集群
容器ID为9cb25bcd52d1的ip地址：172.17.0.5 port: 7005 7006
容器ID为91dac3ea23c9的ip地址：172.17.0.4 port: 7003 7004
容器ID为e2189fc1d4d9的ip地址：172.17.0.2 port: 7001 7002
创建一个基础的rides镜像,包含基础包、ruby、redis接口
我们创建一个基础的rides镜像，这其中要安装必须的一些基础包，ruby等，Dockerfile如下，借用网上的稍作修改,本来打算把gem install redis一起在RUN命令中，但是连不到官方服务器，只能通过国外IP下载放到宿主机目录/root/redissoft作为共享卷让容器读取
[root@localhost CentOS7]# pwd
/root/centos7
[root@localhost centos7]# more Dockerfile
#
# MAINTAINER [email protected]
# Dockerizing CentOS7: Dockerfile for building CentOS images
#
FROM centos:centos7.1.1503
MAINTAINER [email protected]
ENV TZ "Asia/Shanghai"
ENV TERM xterm
ADD aliyun-mirror.repo /etc/yum.repos.d/CentOS-Base.repo
ADD aliyun-epel.repo /etc/yum.repos.d/epel.repo
VOLUME ["/root/redissoft"]
RUN yum install -y curl wget tar bzip2 unzip vim-enhanced passwd sudo yum-utils hostname net-tools rsync man && \
yum install -y gcc gcc-c++ git make automake cmake patch logrotate Python-devel libpng-devel libjpeg-devel && \
yum install -y --enablerepo=epel pwgen python-pip && \
yum install -y ruby rubygems && \
yum clean all
通过Dockfiler创建redis/centos:v7.1 镜像，并且安装redis接口，
[root@localhost centos7]# docker build -t redis/centos:v7.1 .
[root@localhost centos7]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
redis/centos v7.1 5812d7509345 About an hour ago 601.4 MB
现在有了基础镜像但是还没有安装gem install redis,所以我们进入容器通过gem install redis安装redis接口
[root@localhost ~]#docker run -it -v /root/redissoft:/root/redissoft redis/centos:v1
[root@9cb25bcd7895 ~]#cd /root/redissoft
[root@9cb25bcd7895 ~]#gem install redis-3.2.1.gem
[root@9cb25bcd7895 ~]#exit
创建redis/nodemodule：v1镜像
再创建镜像redis/nodemodule镜像用于redis节点容器的新建
[root@localhost ~]#docker commit -t 9cb25bcd7895 redis/nodemodule:v1
注：由于是实验并没有深究gem install redis-3.2.1.gem整合到Dockfile中，实际应用尽量将操作都要整合到Dockfile，其实以下所有操作都可以作为一个Dockfile来创建，后续再探索；
启动容器
[root@localhost centos7]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
redis/nodemodule v1 e5ea8f7834df 3 minutes ago 602.1 MB
#此镜像作为创建3个节点容器的模板，此镜像已经安装了ruby环境和基础环境包， 没有安装redis3.0.7；
[root@localhost ~]#docker run -itd -p 7001:7001 -p 7002:7002 -v /root/redissoft:/root/redissoft --name redisNodeA7000-70001 redis/nodemodule:v1 /bin/bash
[root@localhost ~]#docker run -itd -p 7003:7003 -p 7004:7004 -v /root/redissoft:/root/redissoft --name redisNodeA7003-70004 redis/nodemodule:v1 /bin/bash
[root@localhost ~]#docker run -itd -p 7005:7005 -p 7006:7006 -v /root/redissoft:/root/redissoft --name redisNodeA7005-70006 redis/nodemodule:v1 /bin/bash
[root@localhost ~]# docker ps #之所以用-v /root/redissoft:/root/redissoft参数是让容器能取到redis安装包
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
e2189fc1d4d9 redis/nodemodule:v1 "/bin/bash" 55 minutes ago Up 55 minutes 0.0.0.0:7001-7002->7001-7002/tcp redisNodeA7000-70001
9cb25bcd52d1 redis/nodemodule:v1 "/bin/bash" About an hour ago Up About an hour 0.0.0.0:7005-7006->7005-7006/tcp redisNodeA7005-70006
91dac3ea23c9 redis/nodemodule:v1 "/bin/bash" About an hour ago Up About an hour 0.0.0.0:7003-7004->7003-7004/tcp redisNodeA7003-70004
容器ID为9cb25bcd52d1的ip地址：172.17.0.5 port: 7005 7006
容器ID为91dac3ea23c9的ip地址：172.17.0.4 port: 7003 7004
容器ID为e2189fc1d4d9的ip地址：172.17.0.2 port: 7001 7002
安装redis3.0.7
下面的操作实例只记录在9cb25bcd52d1：172.17.0.5 上面的操作， 一定要在每个节点都进行操作，对应的端口目录，修改对面的端口
[root@localhost redissoft]# docker-enter 9cb25bcd52d1
Last login: Tue Mar 15 08:13:51 UTC 2016
[root@9cb25bcd52d1 ~]# cd /root/redissoft/
[root@9cb25bcd52d1 redissoft]# ls
redis-3.0.7.tar.gz redis-3.2.1.gem
[root@9cb25bcd52d1 redissoft]#
[root@9cb25bcd52d1 local]# cp redis-3.0.7.tar.gz /usr/local/
[root@9cb25bcd52d1 local]# tar -xvf redis-3.0.7.tar.gz
[root@9cb25bcd52d1 local]# mv redis-3.0.7 redis3.0
[root@9cb25bcd52d1 local]# cd redis3.0
[root@9cb25bcd52d1 local]# make && make install
创建节点配置文件
保证3个节点都有对应的端口目录
[root@9cb25bcd52d1 redis3.0]# mkdir -p /usr/local/cluster
[root@9cb25bcd52d1 redis3.0]# mkdir -p /usr/local/cluster/7005
[root@9cb25bcd52d1 redis3.0]# mkdir -p /usr/local/cluster/7006
[root@9cb25bcd52d1 redis3.0]# cp -rf /usr/local/redis3.0/* /usr/local/cluster/7005/
[root@9cb25bcd52d1 redis3.0]# cp -rf /usr/local/redis3.0/* /usr/local/cluster/7006/
[root@9cb25bcd52d1 redis3.0]#
修个节点配置文件
保证3个节点都要在相应的配置文件内配置相应的端口号，只修改列出的项目；
[root@9cb25bcd52d1 redis3.0]# vi /usr/local/cluster/7005/redis.conf
port 7005
daemonize yes
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes
[root@9cb25bcd52d1 redis3.0]# vi /usr/local/cluster/7006/redis.conf
port 7006
daemonize yes
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes
启动服务
保证3个节点都要起来
[root@9cb25bcd52d1 cluster]# cd /usr/local/cluster/7005/src/
[root@9cb25bcd52d1 src]# ./redis-server ../redis.conf
[root@9cb25bcd52d1 src]# cd /usr/local/cluster/7006/src/
[root@9cb25bcd52d1 src]# ./redis-server ../redis.conf
[root@9cb25bcd52d1 src]# ps -ef |grep redis
root 3000 1 0 09:12 ? 00:00:00 ./redis-server *:7005 [cluster]
root 3004 1 0 09:13 ? 00:00:00 ./redis-server *:7006 [cluster]
root 3008 58 0 09:13 ? 00:00:00 grep --color=auto redis
[root@9cb25bcd52d1 src]#
创建集群
cd /usr/local/redis3.0/src
./redis-trib.rb create --replicas 1 172.17.0.2:7001 172.17.0.2:7002 172.17.0.4:7003 172.17.0.4:7004 172.17.0.5:7005 172.17.0.5:7006
>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
172.17.0.2:7001
172.17.0.4:7003
172.17.0.5:7005
Adding replica 172.17.0.4:7004 to 172.17.0.2:7001
Adding replica 172.17.0.2:7002 to 172.17.0.4:7003
Adding replica 172.17.0.5:7006 to 172.17.0.5:7005
M: b9597621be3c89721e980bf21211349d816719f8 172.17.0.2:7001
slots:0-5460 (5461 slots) master
S: 198746c9c20225b2f35312e3f787590ae8fbc450 172.17.0.2:7002
replicates ae4adef75beba6f20129f7a7263ec301b85862c5
M: ae4adef75beba6f20129f7a7263ec301b85862c5 172.17.0.4:7003
slots:5461-10922 (5462 slots) master
S: 3f89d374e0e6461b566527d5a9dffe2d91bf43dd 172.17.0.4:7004
replicates b9597621be3c89721e980bf21211349d816719f8
M: 1b26d10b8a6ee3aa731123716e68bd3e911d805e 172.17.0.5:7005
slots:10923-16383 (5461 slots) master
S: 75e63758ce188ab1ccaa4e9bc195d560b9a7c417 172.17.0.5:7006
replicates 1b26d10b8a6ee3aa731123716e68bd3e911d805e
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join...
>>> Performing Cluster Check (using node 172.17.0.2:7001)
M: b9597621be3c89721e980bf21211349d816719f8 172.17.0.2:7001
slots:0-5460 (5461 slots) master
M: 198746c9c20225b2f35312e3f787590ae8fbc450 172.17.0.2:7002
slots: (0 slots) master
replicates ae4adef75beba6f20129f7a7263ec301b85862c5
M: ae4adef75beba6f20129f7a7263ec301b85862c5 172.17.0.4:7003
slots:5461-10922 (5462 slots) master
M: 3f89d374e0e6461b566527d5a9dffe2d91bf43dd 172.17.0.4:7004
slots: (0 slots) master
replicates b9597621be3c89721e980bf21211349d816719f8
M: 1b26d10b8a6ee3aa731123716e68bd3e911d805e 172.17.0.5:7005
slots:10923-16383 (5461 slots) master
M: 75e63758ce188ab1ccaa4e9bc195d560b9a7c417 172.17.0.5:7006
slots: (0 slots) master
replicates 1b26d10b8a6ee3aa731123716e68bd3e911d805e
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
创建结束后我们会发现有：
3个主节点：172.17.0.2:7001 172.17.0.2:7003 172.17.0.2:7005 这正是我们想要的结果
集群测试
发现3个set语句都把数据存在不同节点的slot里面了，读取的时候也从相应节点进行读取
[root@e2189fc1d4d9 ~]# redis-cli -c -p 7001
127.0.0.1:7001> set foo bar
-> Redirected to slot [12182] located at 172.17.0.5:7005
OK
172.17.0.5:7005> set hello newjoyful
-> Redirected to slot [866] located at 172.17.0.2:7001
OK
172.17.0.2:7001> set test newjoy
-> Redirected to slot [6918] located at 172.17.0.4:7003
OK
172.17.0.4:7003> get foo
-> Redirected to slot [12182] located at 172.17.0.5:7005
"bar"
172.17.0.5:7005> get hello
-> Redirected to slot [866] located at 172.17.0.2:7001
"newjoyful"
172.17.0.2:7001> get test
-> Redirected to slot [6918] located at 172.17.0.4:7003
"newjoy"
172.17.0.4:7003>
在宿主机192.168.100.67上安装redis3.0.7之后同样可以连接redis集群环境，因为容器与主机有端口映射关系
[root@localhost ~]# cp redis-3.0.7.tar.gz /usr/local/
[root@localhost ~]# cd /usr/local/
[root@localhost ~]# tar -xvf redis-3.0.7.tar.gz
[root@localhost ~]# mv redis-3.0.7 redis3.0
[root@localhost ~]# cd redis3.0/
[root@localhost ~]# make
[root@localhost ~]# make install
[root@localhost ~]# netstat -an |grep :::700
tcp6 0 0 :::7001 :::* LISTEN
tcp6 0 0 :::7002 :::* LISTEN
tcp6 0 0 :::7003 :::* LISTEN
tcp6 0 0 :::7004 :::* LISTEN
tcp6 0 0 :::7005 :::* LISTEN
tcp6 0 0 :::7006 :::* LISTEN
[root@localhost ~]# redis-cli -c -p 7003
127.0.0.1:7003> get hello
-> Redirected to slot [866] located at 172.17.0.2:7001
"newjoyful"
172.17.0.2:7001> get test
-> Redirected to slot [6918] located at 172.17.0.4:7003
"newjoy"
172.17.0.4:7003>
作为结尾，我们应该把现在运行的容器生成Image，但是觉得没有意义，如果容器重启之后IP地址可能会发生变化配置也会随着变化，所以这次试验只合适进行学习、测试
Error parsing reference: "redis/redisNodeA7001_7002:v1" is not a valid repository/tag
[root@localhost ~]# docker commit redisNodeA7000-70001 redis/redisnodea7001_7002:v1
sha256:4bef9995c1fd4214719ae810c8dda347c8f10d0431d27f84cc1c81cf94231cb1
[root@localhost ~]# docker commit redisNodeA7003-70004 redis/redisnodeb7003_7004:v1
sha256:37a5ea1d992b3406fb98751f2798f1f339f6f5bc6ed001fc0103da35ca79f4f4
[root@localhost ~]# docker commit redisNodeA7005-70006 redis/redisnodec7005_7006:v1
sha256:263ccb2779787d917e24a5e0ed7ba29c1a24fb4adc544fdb8948365526f4fb56
[root@localhost ~]#
[root@localhost ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
redis/redisnodec7005_7006 v1 263ccb277978 About a minute ago 751.5 MB
redis/redisnodeb7003_7004 v1 37a5ea1d992b About a minute ago 752.9 MB
redis/redisnodea7001_7002 v1 4bef9995c1fd 2 minutes ago 752.9 MB
redis/nodemodule v1 e5ea8f7834df About an hour ago 602.1 MB
redis/centos v7.1 5812d7509345 About an hour ago 601.4 MB
Ubuntu latest 07c86167cdc4 11 days ago 187.9 MB
redis latest 4f5f397d4b7c 12 days ago 177.5 MB
centos centos7.1.1503 30e55721fefe 5 months ago 212.1 MB
本文要用到的相关附件到安科网资源站下载：
------------------------------------------分割线------------------------------------------
具体下载目录在 /2016年资料/3月/17日/Docker+Redis3集群环境搭建/
------------------------------------------分割线------------------------------------------