Kubernetes集群之部署ETCD

kubernetes集群部署

官方提供的三种部署方式
1.minikube
minikube是一个工具,可以在本地快速运行一个单点的kubernetes,仅用于尝试K8S或日常开发的测试环境使用
部署地址:https://kubernetes.io/docs/setup/minkube/
2.kubeadm
kubeadm也是一个工具,提供kubeadm init和kubeadm join,用于快速部署kubernetes集群
部署地址:https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm/
3.二进制包
从官方下载发行版的二进制包,手动部署每个组件,组成kubernetes集群
地址:https://github.com/kubernetes/kubernetes/releases
https://github.com/kubernetes/kubernetes/releases?after=v1.13.1
这里我们使用二进制分组安装的方式搭建 kubernetes
kubernetes部署思路

  1. 自签ssl证书
  2. etcd数据库集群部署
  3. node安装docker
  4. flannel容器集群网络部署
  5. 部署master组件
  6. 部署node组件
  7. 部署一个测试示例
  8. 部署web ui(dashboard)
  9. 部署集群内部dns解析服务(coredns)
    kubernetes单节点拓扑图
    Kubernetes集群之部署ETCD
    一、master、node1、node2环境部署
    master:关闭网络管理器,清空iptabels,关闭核心防护,编辑主机名
    [ ~]# hostnamectl set-hostname master
    [ ~]# su
    [ ~]# systemctl stop NetworkManager
    [ ~]# systemctl disable NetworkManager
    Removed symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service.
    Removed symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service.
    Removed symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service.
    [ ~]# setenforce 0
    [ ~]# sed -i ‘s/^SELINUX=.*/SELINUX=disabled/‘ /etc/selinux/config
    [ ~]# iptables -F

    node1:

    [ ~]# hostnamectl set-hostname master1
    [ ~]# su
    [ ~]# systemctl stop NetworkManager
    [ ~]# systemctl disable NetworkManager
    Removed symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service.
    Removed symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service.
    Removed symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service.
    [ ~]# setenforce 0
    [ ~]# sed -i ‘s/^SELINUX=.*/SELINUX=disabled/‘ /etc/selinux/config
    [ ~]# iptables -F

    node2:

    [ ~]# hostnamectl set-hostname node02
    [ ~]# su
    [ ~]# systemctl stop NetworkManager
    [ ~]# systemctl disable NetworkManager
    Removed symlink/etc/systemd/system/multi-user.target.wants/NetworkManager.service.
    Removed symlink/etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service.
    Removed symlink/etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service.
    [ ~]# setenforce 0
    [ ~]# sed -i ‘s/^SELINUX=.*/SELINUX=disabled/‘ /etc/selinux/config
    [ ~]# iptables -F

    自签SSL证书
    各个组件及使用的证书
    etcd:ca.pem、server.pem、server-key.pem
    flannel:ca.pem、server.pem、server-key.pem
    kube-apiserver:ca.pem、server.pem、server-key.pem
    kubelet:ca.pem、server.pem
    kube-proxy:ca.pem、kube-proxy.pem、kube-proxy-key.pem
    kubectl:ca.pem、admin.pem、admin-key.pem
    1.首先要生成CA证书,各组件之间的通讯必须有ca证书

    [ ~]# mkdir k8s
    [ ~]# cd k8s
    [ k8s]# pwd
    /root/k8s
    [ k8s]# mkdir /abc
    [ k8s]# mount.cifs //192.168.0.88/linuxs /abc
    Password for //192.168.0.88/linuxs:  
    [ k8s]# cp /abc/k8s/etcd* .
    [ k8s]# ll
    total 8
    -rwxr-xr-x. 1 root root 1088 Apr 29 00:13 etcd-cert.sh
    -rwxr-xr-x. 1 root root 1764 Apr 29 00:13 etcd.sh

    2.etcd-cert.sh用来创建关于etcd的CA证书

    [ k8s]# cat etcd-cert.sh 
    #ca-config.json是ca证书的配置文件
    cat > ca-config.json <<EOF
    {
    "signing": {
    "default": {
      "expiry": "87600h"
    },
    "profiles": {
      "www": {
         "expiry": "87600h", #expiry    有效期10年
         "usages": [
            "signing",
            "key encipherment", #使用密钥验证
            "server auth",
            "client auth"
        ]
      }
    }
    }
    }
    EOF
    #ca-csr.json是ca证书的签名文件
    cat > ca-csr.json <<EOF
    {
    "CN": "etcd CA",
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "Beijing",
            "ST": "Beijing"
        }
    ]
    }
    EOF
    cfssl gencert -initca ca-csr.json | cfssljson -bare ca -
    #-----------------------
    #server-csr.json是指定etcd三个节点之间的通信验证
    cat > server-csr.json <<EOF
    {
    "CN": "etcd",
    "hosts": [
    "192.168.191.130",
    "192.168.191.131",
    "192.168.191.132"
    ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "BeiJing",
            "ST": "BeiJing"
        }
    ]
    }
    EOF
    cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=www server-csr.json | cfssljson -bare server

    3.etcd.sh用来创建启动脚本和配置文件

    [ k8s]# cat etcd.sh 
    #!/bin/bash
    # example: ./etcd.sh etcd01 192.168.191.130 etcd02=https://192.168.191.131:2380,etcd03=https://192.168.191.132:2380
    ETCD_NAME=$1
    ETCD_IP=$2
    ETCD_CLUSTER=$3
    WORK_DIR=/k8s/etcd
    cat <<EOF >$WORK_DIR/cfg/etcd
    #[Member]
    ETCD_NAME="${ETCD_NAME}"
    ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
    ETCD_LISTEN_PEER_URLS="https://${ETCD_IP}:2380"
    ETCD_LISTEN_CLIENT_URLS="https://${ETCD_IP}:2379"
    #[Clustering]
    ETCD_INITIAL_ADVERTISE_PEER_URLS="https://${ETCD_IP}:2380"
    ETCD_ADVERTISE_CLIENT_URLS="https://${ETCD_IP}:2379"
    ETCD_INITIAL_CLUSTER="etcd01=https://${ETCD_IP}:2380,${ETCD_CLUSTER}"
    ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
    ETCD_INITIAL_CLUSTER_STATE="new"
    EOF
    cat <<EOF >/usr/lib/systemd/system/etcd.service
    [Unit]
    Description=Etcd Server
    After=network.target
    After=network-online.target
    Wants=network-online.target
    [Service]
    Type=notify
    EnvironmentFile=${WORK_DIR}/cfg/etcd
    ExecStart=${WORK_DIR}/bin/etcd --name=\${ETCD_NAME} --data-dir=\${ETCD_DATA_DIR} --listen-peer-urls=\${ETCD_LISTEN_PEER_URLS} --listen-client-urls=\${ETCD_LISTEN_CLIENT_URLS},http://127.0.0.1:2379 --advertise-client-urls=\${ETCD_ADVERTISE_CLIENT_URLS} --initial-advertise-peer-urls=\${ETCD_INITIAL_ADVERTISE_PEER_URLS} --initial-cluster=\${ETCD_INITIAL_CLUSTER} --initial-cluster-token=\${ETCD_INITIAL_CLUSTER_TOKEN} --initial-cluster-state=new --cert-file=${WORK_DIR}/ssl/server.pem --key-file=${WORK_DIR}/ssl/server-key.pem --peer-cert-file=${WORK_DIR}/ssl/server.pem --peer-key-file=${WORK_DIR}/ssl/server-key.pem --trusted-ca-file=${WORK_DIR}/ssl/ca.pem --peer-trusted-ca-file=${WORK_DIR}/ssl/ca.pem
    Restart=on-failure
    LimitNOFILE=65536
    [Install]
    WantedBy=multi-user.target
    EOF
    systemctl daemon-reload
    systemctl enable etcd
    systemctl restart etcd

    4.下载cfssl官方脚本包
    注:
    1.cfssl 生成证书工具
    2.cfssljson 通过传入json文件生成证书
    3.cfssl-certinfo 查看证书信息

    [ k8s]# vim cfssl.sh
    curl -L https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 -o /usr/local/bin/cfssl
    curl -L https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 -o /usr/local/bin/cfssljson
    curl -L https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64 -o /usr/local/bin/cfssl-certinfo
    chmod +x /usr/local/bin/cfssl*
    [ k8s]# bash cfssl.sh 
    % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
    100  9.8M  100  9.8M    0     0   106k      0  0:01:35  0:01:35 --:--:-- 98678
    % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
    100 2224k  100 2224k    0     0   316k      0  0:00:07  0:00:07 --:--:--  455k
    % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
    100 6440k  100 6440k    0     0   531k      0  0:00:12  0:00:12 --:--:--  736k

    5.验证

    [ k8s]# ll /usr/local/bin/*
    -rwxr-xr-x. 1 root root 10376657 Apr 29 00:20 /usr/local/bin/cfssl
    -rwxr-xr-x. 1 root root  6595195 Apr 29 00:21 /usr/local/bin/cfssl-certinfo
    -rwxr-xr-x. 1 root root  2277873 Apr 29 00:20 /usr/local/bin/cfssljson
    [ k8s]# rm -rf cfssl.sh

    6.创建etcd组件证书临时目录,定义ca证书配置

    [ k8s]# mkdir etcd-cert
    [ k8s]# mv etcd-cert.sh etcd-cert
    [ k8s]# cd etcd-cert/
    [ etcd-cert]# ls
    etcd-cert.sh
    [ etcd-cert]# cat > ca-config.json <<EOF
    {
    "signing": {
    "default": {
      "expiry": "87600h"
    },
    "profiles": {
      "www": {
         "expiry": "87600h",
         "usages": [
            "signing",
            "key encipherment",
            "server auth",
            "client auth"
        ]
      }
    }
    }
    }
    EOF
    [ etcd-cert]# ls
    ca-config.json  etcd-cert.sh

    7.ca证书签名

    [ etcd-cert]# cat > ca-csr.json <<EOF
    {
    "CN": "etcd CA",
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "Beijing",
            "ST": "Beijing"
        }
    ]
    }
    EOF
    [ etcd-cert]# ls
    ca-config.json  ca-csr.json  etcd-cert.sh

    8.生成证书,生成ca-key.pem ca.pem这两个证书文件

    [ etcd-cert]# cfssl gencert -initca ca-csr.json | cfssljson -bare ca -
    2020/04/29 00:44:16 [INFO] generating a new CA key and certificate from CSR
    2020/04/29 00:44:16 [INFO] generate received request
    2020/04/29 00:44:16 [INFO] received CSR
    2020/04/29 00:44:16 [INFO] generating key: rsa-2048
    2020/04/29 00:44:16 [INFO] encoded CSR
    2020/04/29 00:44:16 [INFO] signed certificate with serial number 527285287467326079906972398205016440554485642971
    [ etcd-cert]# ls
    ca-config.json  ca.csr  ca-csr.json  ca-key.pem  ca.pem  etcd-cert.sh

    9.指定etcd三个节点之间的通信验证

    [ etcd-cert]#  cat > server-csr.json <<EOF
    {
    "CN": "etcd",
    "hosts": [
    "192.168.191.130",
    "192.168.191.131",
    "192.168.191.132"
    ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "BeiJing",
            "ST": "BeiJing"
        }
    ]
    }
    EOF
    [ etcd-cert]# ls
    ca-config.json  ca.csr  ca-csr.json  ca-key.pem  ca.pem  etcd-cert.sh  server-csr.json

    10.生成etcd的server证书和密钥

    [ etcd-cert]# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=www server-csr.json | cfssljson -bare server
    2020/04/29 00:48:07 [INFO] generate received request
    2020/04/29 00:48:07 [INFO] received CSR
    2020/04/29 00:48:07 [INFO] generating key: rsa-2048
    2020/04/29 00:48:07 [INFO] encoded CSR
    2020/04/29 00:48:07 [INFO] signed certificate with serial number 79028110669307243971733075611743333137367463128
    2020/04/29 00:48:07 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
    websites. For more information see the Baseline Requirements for the Issuance and Management
    of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
    specifically, section 10.2.3 ("Information Requirements").
    [ etcd-cert]# ls
    ca-config.json  ca-csr.json  ca.pem        server.csr       server-key.pem
    ca.csr          ca-key.pem   etcd-cert.sh  server-csr.json  server.pem

    三个材料:
    -config.json 证书配置参数
    -csr.json 证书签名文件参数
    .csr 证书签名文件
    生成结果:
    -key.pem 密钥
    .pem 证书
    二、Etcd数据库集群部署
    二进制包下载地址:https://github.com/etcd-io/etcd/releases
    查看集群状态:

    /k8s/etcd/bin/etcdctl 
    –ca-file=/k8s/etcd/ssl/ca.pem 
    –cert-file=/k8s/etcd/ssl/server.pem --key-file=/k8s/etcd/ssl/server-key.pem 
    –endpoints=“https://192.168.191.130:2379,https://192.168.191.131:2379,https://192.168.191.132:2379”

    1.将安装包拉倒master节点

    [ etcd-cert]# cp /abc/k8s/etcd-v3.3.10-linux-amd64.tar.gz /root/k8s/
    [ etcd-cert]# cd ..
    [ k8s]# pwd
    /root/k8s
    [ k8s]# ls
    etcd-cert  etcd.sh  etcd-v3.3.10-linux-amd64.tar.gz
    [ k8s]# tar xf etcd-v3.3.10-linux-amd64.tar.gz 
    [ k8s]# ls
    etcd-cert  etcd.sh  etcd-v3.3.10-linux-amd64  etcd-v3.3.10-linux-amd64.tar.gz
    [ k8s]# cd etcd-v3.3.10-linux-amd64/
    [ etcd-v3.3.10-linux-amd64]# ls
    Documentation  etcd  etcdctl  README-etcdctl.md  README.md  READMEv2-etcdctl.md

    2.创建etcd的工作目录,下面还有配置文件cfg,命令bin,证书ssl的目录

    [ etcd-v3.3.10-linux-amd64]# mkdir /k8s/etcd/{cfg,bin,ssl} -p
    [ etcd-v3.3.10-linux-amd64]# cd /k8s
    [ k8s]# tree .
    .
    └── etcd
    ├── bin
    ├── cfg
    └── ssl

    3.将证书文件和命令文件复制过来

    [ k8s]# mv /root/k8s/etcd-v3.3.10-linux-amd64/etcd* /k8s/etcd/bin/
    [ k8s]# cp /root/k8s/etcd-cert/*.pem /k8s/etcd/ssl/
    [ k8s]# cd/etcd/bin/etcd/etcdctl/cfg/ssl
    [ ssl]#ls
    ca-key.pem     ca.pem
    server-key.pem    server.pem

    4.编辑etcd的配置文件和启动脚本

    #!/bin/bash
    # example: ./etcd.sh etcd01 192.168.191.130 etcd02=https://192.168.191.131:2380,etcd03=https://192.168.191.132:2380
    ETCD_NAME=$1
    ETCD_IP=$2
    ETCD_CLUSTER=$3
    WORK_DIR=/k8s/etcd
    cat <<EOF >$WORK_DIR/cfg/etcd
    #[Member]
    ETCD_NAME="${ETCD_NAME}"
    ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
    ETCD_LISTEN_PEER_URLS="https://${ETCD_IP}:2380"
    ETCD_LISTEN_CLIENT_URLS="https://${ETCD_IP}:2379"
    #[Clustering]
    ETCD_INITIAL_ADVERTISE_PEER_URLS="https://${ETCD_IP}:2380"
    ETCD_ADVERTISE_CLIENT_URLS="https://${ETCD_IP}:2379"
    ETCD_INITIAL_CLUSTER="etcd01=https://${ETCD_IP}:2380,${ETCD_CLUSTER}"
    ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
    ETCD_INITIAL_CLUSTER_STATE="new"
    EOF
    cat <<EOF >/usr/lib/systemd/system/etcd.service
    [Unit]
    Description=Etcd Server
    After=network.target
    After=network-online.target
    Wants=network-online.target
    [Service]
    Type=notify
    EnvironmentFile=${WORK_DIR}/cfg/etcd
    ExecStart=${WORK_DIR}/bin/etcd --name=\${ETCD_NAME} --data-dir=\${ETCD_DATA_DIR} --listen-peer-urls=\${ETCD_LISTEN_PEER_URLS} --listen-client-urls=\${ETCD_LISTEN_CLIENT_URLS},http://127.0.0.1:2379 --advertise-client-urls=\${ETCD_ADVERTISE_CLIENT_URLS} --initial-advertise-peer-urls=\${ETCD_INITIAL_ADVERTISE_PEER_URLS} --initial-cluster=\${ETCD_INITIAL_CLUSTER} --initial-cluster-token=\${ETCD_INITIAL_CLUSTER_TOKEN} --initial-cluster-state=new --cert-file=${WORK_DIR}/ssl/server.pem --key-file=${WORK_DIR}/ssl/server-key.pem --peer-cert-file=${WORK_DIR}/ssl/server.pem --peer-key-file=${WORK_DIR}/ssl/server-key.pem --trusted-ca-file=${WORK_DIR}/ssl/ca.pem --peer-trusted-ca-file=${WORK_DIR}/ssl/ca.pem
    Restart=on-failure
    LimitNOFILE=65536
    [Install]
    WantedBy=multi-user.target
    EOF
    systemctl daemon-reload
    systemctl enable etcd
    systemctl restart etcd
    [ k8s]# sh etcd.sh  
    Created symlink from /etc/systemd/system/multi-user.target.wants/etcd.service to /usr/lib/systemd/system/etcd.service.
    Job for etcd.service failed because the control process exited with error code. See "systemctl status etcd.service" and "journalctl -xe" for details.
    [ etcd]# cd /k8s
    [ k8s]# ll /usr/lib/systemd/system/ | grep etcd
    -rw-r--r--. 1 root root  923 Apr 29 08:11 etcd.service

    5.第一步产生配置文件,启动脚本生成到systemd下
    端口,2379是提供给外部端口,2380是内部集群通讯端口,最多65536端口

    [ k8s]# cd /root/k8s/
    [ k8s]# ls
    etcd-cert  etcd.sh  etcd-v3.3.10-linux-amd64  etcd-v3.3.10-linux-amd64.tar.gz
    [ k8s]# pwd
    /root/k8s
    [ k8s]# bash etcd.sh etcd01 192.168.191.130 etcd02=https://192.168.191.131:2380,etcd03=https://192.168.191.132:2380

    此时进入一个等待状态,查找别的etcd集群节点,查找不到过5分钟默认退出
    这里再去两个node节点部署etcd
    6.查看etcd单节点是否开启

    [ k8s]# netstat -natp | grep etcd
    tcp        0      0 192.168.247.149:2379    0.0.0.0:*               LISTEN      26604/etcd          
    tcp        0      0 127.0.0.1:2379          0.0.0.0:*               LISTEN      26604/etcd          
    tcp        0      0 192.168.247.149:2380    0.0.0.0:*               LISTEN      26604/etcd

    7.复制证书去另外两个节点

    [ k8s]# scp -r /k8s :/k8s
    ‘s password: 
    etcd                                                                                  100%  523   252.4KB/s   00:00    
    etcd                                                                                  100%   18MB  95.0MB/s   00:00    
    etcdctl                                                                               100%   15MB 124.7MB/s   00:00    
    ca-key.pem                                                                            100% 1679   725.6KB/s   00:00    
    ca.pem                                                                                100% 1265   287.0KB/s   00:00    
    server-key.pem                                                                        100% 1675   813.0KB/s   00:00    
    server.pem

    8.复制启动脚本去另外两个节点

    [ k8s]# scp /usr/lib/systemd/system/etcd.service :/usr/lib/systemd/system/
    ‘s password: 
    etcd.service                                                                          100%  923   814.8KB/s   00:00    
    [ k8s]# scp /usr/lib/systemd/system/etcd.service :/usr/lib/systemd/system/
    ‘s password: 
    etcd.service

    9.相关文件复制过去了,需要修改/k8s/etcd/cfg/etcd配置文件中有些参数
    node1:

    [ k8s]# vim /k8s/etcd/cfg/etcd
    #[Member]
    ETCD_NAME="etcd02"
    ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
    ETCD_LISTEN_PEER_URLS="https://192.168.191.131:2380"
    ETCD_LISTEN_CLIENT_URLS="https://192.168.191.131:2379"
    #[Clustering]
    ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.191.131:2380"
    ETCD_ADVERTISE_CLIENT_URLS="https://192.168.191.131:2379"
    ETCD_INITIAL_CLUSTER="etcd01=https://192.168.191.130:2380,etcd02=https://192.168.191.131:2380,etcd03=https://192.168.191.132:2380"
    ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
    ETCD_INITIAL_CLUSTER_STATE="new"

    node2:

    [ k8s]# vim /k8s/etcd/cfg/etcd
    #[Member]
    ETCD_NAME="etcd03"
    ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
    ETCD_LISTEN_PEER_URLS="https://192.168.191.132:2380"
    ETCD_LISTEN_CLIENT_URLS="https://192.168.191.132:2379"
    #[Clustering]
    ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.191.132:2380"
    ETCD_ADVERTISE_CLIENT_URLS="https://192.168.191.132:2379"
    ETCD_INITIAL_CLUSTER="etcd01=https://192.168.191.130:2380,etcd02=https://192.168.191.131:2380,etcd03=https://192.168.191.132:2380"
    ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
    ETCD_INITIAL_CLUSTER_STATE="new"

    10.节点开启etcd服务

    [ ~]# systemctl start etcd
    [ ~]# systemctl status etcd
    ● etcd.service - Etcd Server
    Loaded: loaded (/usr/lib/systemd/system/etcd.service; disabled; vendor preset: disabled)
    Active: active (running) since Wed 2020-04-29 08:42:15 CST; 9s ago
    [ ~]# systemctl start etcd
    [ ~]# systemctl status etcd
    ● etcd.service - Etcd Server
    Loaded: loaded (/usr/lib/systemd/system/etcd.service; disabled; vendor preset: disabled)
    Active: active (running) since Wed 2020-04-29 08:42:15 CST; 9s ago

    11.此时在主节点master1上重新执行脚本命令

    [ k8s]# cd /root/k8s/
    [ k8s]# ls
    etcd-cert  etcd.sh  etcd-v3.3.10-linux-amd64  etcd-v3.3.10-linux-amd64.tar.gz
    [ k8s]# bash etcd.sh etcd01 192.168.191.130 etcd02=https://192.168.191.131:2380,etcd03=https://192.168.191.132:2380
    [ k8s]#

    检查集群状态

    /k8s/etcd/bin/etcdctl --ca-file=/k8s/etcd/ssl/ca.pem --cert-file=/k8s/etcd/ssl/server.pem --key-file=/k8s/etcd/ssl/server-key.pem --endpoints="https://192.168.191.130:2379,https://192.168.191.131:2379,https://192.168.191.132:2379" cluster-health
    [ k8s]# cd /k8s/etcd/ssl/
    [ ssl]# pwd
    /k8s/etcd/ssl
    [ ssl]# ls
    ca-key.pem  ca.pem  server-key.pem  server.pem
    [ ssl]# /k8s/etcd/bin/etcdctl > --ca-file=ca.pem > --cert-file=server.pem --key-file=server-key.pem > --endpoints="https://192.168.191.130:2379,https://192.168.191.131:2379,https://192.168.191.132:2379" > cluster-health
    member 8f4e6ce663f0d49a is healthy: got healthy result from https://192.168.191.131:2379
    member b6230d9c6f20feeb is healthy: got healthy result from https://192.168.191.132:2379
    member d618618928dffeba is healthy: got healthy result from https://192.168.191.130:2379
    cluster is healthy

    到这里我们的master和两个node节点的etcd服务就部署完成了
    两个node节点都是要部署docker环境的,在部署etcd之前两个node节点就已经部署好了docker环境。