使用docker快速部署Elasticsearch集群的方法

æ¬æå°ä½¿ç¨Docker容å¨ï¼ä½¿ç¨docker-composeç¼æï¼å¿«éé¨ç½²Elasticsearch é群ï¼å¯ç¨äºå¼åç¯å¢ï¼åæºå¤å®ä¾ï¼æç产ç¯å¢é¨ç½²ã

注æï¼6.xçæ¬å·²ç»ä¸è½éè¿ -Epath.config åæ°å»æå®éç½®æ件çå è½½ä½ç½®ï¼æ档说æï¼

For the archive distributions, the config directory location defaults to $ES_HOME/config. The location of the >config directory can be changed via the ES_PATH_CONF environment variable as follows:
ES_PATH_CONF=/path/to/my/config ./bin/elasticsearch
Alternatively, you can export the ES_PATH_CONF environment variable via the command line or via your shell profile.

å³äº¤ç»ç¯å¢åé ES_PATH_CONF æ¥è®¾å®äºï¼å®æ¹ææ¡£ï¼ï¼åæºé¨ç½²å¤ä¸ªå®ä¾ä¸ä¸ä½¿ç¨å®¹å¨çåå­¦å¤å¤æ³¨æã

åå¤å·¥ä½

å®è£ Docker & docker-compose

è¿éæ¨è¿ä½¿ç¨ daocloud å个å éå®è£ï¼

#docker
curl -sSL https://get.daocloud.io/docker | sh

#docker-compose
curl -L \
https://get.daocloud.io/docker/compose/releases/download/1.23.2/docker-compose-`uname -s`-`uname -m` \
> /usr/local/bin/docker-compose

chmod +x /usr/local/bin/docker-compose

#æ¥çå®è£ç»æ
docker-compose -v

æ°æ®ç®å½

#å建æ°æ®/æ¥å¿ç®å½ è¿éæ们é¨ç½²3个èç¹
mkdir /opt/elasticsearch/data/{node0,nod1,node2} -p
mkdir /opt/elasticsearch/logs/{node0,nod1,node2} -p
cd /opt/elasticsearch
#æéæä¹å¾æµé¼å¦ ç»äº privileged ä¹ä¸è¡ ç´¢æ§0777好äº
chmod 0777 data/* -R && chmod 0777 logs/* -R

#é²æ­¢JVMæ¥é
echo vm.max_map_count=262144 >> /etc/sysctl.conf
sysctl -p

docker-compse ç¼ææå¡

å建ç¼ææ件

vim docker-compose.yml

åæ°è¯´æ

- cluster.name=elasticsearch-cluster

é群å称

- node.name=node0
- node.master=true
- node.data=true

èç¹å称ãæ¯å¦å¯ä½ä¸ºä¸»èç¹ãæ¯å¦å­å¨æ°æ®

- bootstrap.memory_lock=true

éå®è¿ç¨çç©çåå­å°åé¿å交æ¢ï¼swappedï¼æ¥æé«æ§è½

- http.cors.enabled=true
- http.cors.allow-origin=*

å¼å¯cors以便使ç¨Headæ件

- "ES_JAVA_OPTS=-Xms512m -Xmx512m"

JVMåå­å¤§å°éç½®

- "discovery.zen.ping.unicast.hosts=elasticsearch_n0,elasticsearch_n1,elasticsearch_n2"
- "discovery.zen.minimum_master_nodes=2"

ç±äº5.2.1åççæ¬æ¯ä¸æ¯æå¤æ­çï¼æ以éè¦æå¨æå®é群åèç¹çtcpæ°æ®äº¤äºå°åï¼ç¨äºé群çèç¹åç°åfailoverï¼é»è®¤ç¼ºç9300端å£ï¼å¦è®¾å®äºå¶å®ç«¯å£éå¦è¡æå®ï¼è¿éæ们ç´æ¥åå©å®¹å¨éä¿¡ï¼ä¹å¯ä»¥å°åèç¹ç9300æ å°è³å®¿ä¸»æºï¼éè¿ç½ç»ç«¯å£éä¿¡ã

设å®failoveréåçquorum = nodes/2 + 1

å½ç¶ï¼ä¹å¯ä»¥æè½½èªå·±çéç½®æ件ï¼ESéåçéç½®æ件æ¯/usr/share/elasticsearch/config/elasticsearch.ymlï¼æè½½å¦ä¸:

volumes:
 - path/to/local/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml:ro

docker-compose.yml

version: '3'
services:
 elasticsearch_n0:
  image: elasticsearch:6.6.2
  container_name: elasticsearch_n0
  privileged: true
  environment:
   - cluster.name=elasticsearch-cluster
   - node.name=node0
   - node.master=true
   - node.data=true
   - bootstrap.memory_lock=true
   - http.cors.enabled=true
   - http.cors.allow-origin=*
   - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
   - "discovery.zen.ping.unicast.hosts=elasticsearch_n0,elasticsearch_n1,elasticsearch_n2"
   - "discovery.zen.minimum_master_nodes=2"
  ulimits:
   memlock:
    soft: -1
    hard: -1
  volumes:
   - ./data/node0:/usr/share/elasticsearch/data
   - ./logs/node0:/usr/share/elasticsearch/logs
  ports:
   - 9200:9200
 elasticsearch_n1:
  image: elasticsearch:6.6.2
  container_name: elasticsearch_n1
  privileged: true
  environment:
   - cluster.name=elasticsearch-cluster
   - node.name=node1
   - node.master=true
   - node.data=true
   - bootstrap.memory_lock=true
   - http.cors.enabled=true
   - http.cors.allow-origin=*
   - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
   - "discovery.zen.ping.unicast.hosts=elasticsearch_n0,elasticsearch_n1,elasticsearch_n2"
   - "discovery.zen.minimum_master_nodes=2"
  ulimits:
   memlock:
    soft: -1
    hard: -1
  volumes:
   - ./data/node1:/usr/share/elasticsearch/data
   - ./logs/node1:/usr/share/elasticsearch/logs
  ports:
   - 9201:9200
 elasticsearch_n2:
  image: elasticsearch:6.6.2
  container_name: elasticsearch_n2
  privileged: true
  environment:
   - cluster.name=elasticsearch-cluster
   - node.name=node2
   - node.master=true
   - node.data=true
   - bootstrap.memory_lock=true
   - http.cors.enabled=true
   - http.cors.allow-origin=*
   - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
   - "discovery.zen.ping.unicast.hosts=elasticsearch_n0,elasticsearch_n1,elasticsearch_n2"
   - "discovery.zen.minimum_master_nodes=2"
  ulimits:
   memlock:
    soft: -1
    hard: -1
  volumes:
   - ./data/node2:/usr/share/elasticsearch/data
   - ./logs/node2:/usr/share/elasticsearch/logs
  ports:
   - 9202:9200

è¿éæ们åå«ä¸ºnode0/node1/node2å¼æ¾å®¿ä¸»æºç9200/9201/9202ä½ä¸ºhttpæå¡ç«¯å£ï¼åå®ä¾çtcpæ°æ®ä¼ è¾ç¨é»è®¤ç9300éè¿å®¹å¨ç®¡çéä¿¡ã

å¦æéè¦å¤æºé¨ç½²ï¼åå°ESçtransport.tcp.port: 9300端å£æ å°è³å®¿ä¸»æºxxxx端å£ï¼discovery.zen.ping.unicast.hostså¡«åå主æºä»£ççå°åå³å¯ï¼

#æ¯å¦å¶ä¸­ä¸å°å®¿ä¸»æºä¸º192.168.1.100
  ...
  - "discovery.zen.ping.unicast.hosts=192.168.1.100:9300,192.168.1.101:9300,192.168.1.102:9300"
  ...
ports:
 ...
 - 9300:9300

å建并å¯å¨æå¡

[root@localhost elasticsearch]# docker-compose up -d
[root@localhost elasticsearch]# docker-compose ps
   Name          Command        State        Ports       
--------------------------------------------------------------------------------------------
elasticsearch_n0  /usr/local/bin/docker-entr ...  Up   0.0.0.0:9200->9200/tcp, 9300/tcp
elasticsearch_n1  /usr/local/bin/docker-entr ...  Up   0.0.0.0:9201->9200/tcp, 9300/tcp
elasticsearch_n2  /usr/local/bin/docker-entr ...  Up   0.0.0.0:9202->9200/tcp, 9300/tcp

#å¯å¨å¤±è´¥æ¥çé误
[root@localhost elasticsearch]# docker-compose logs
#æå¤æ¯ä¸äºè®¿é®æé/JVM vm.max_map_count ç设置é®é¢

æ¥çé群ç¶æ

192.168.20.6 æ¯æçæå¡å¨å°å

访é®http://192.168.20.6:9200/_cat/nodes?vå³å¯æ¥çé群ç¶æï¼

ip     heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
172.25.0.3      36     98 79  3.43  0.88   0.54 mdi    *   node0
172.25.0.2      48     98 79  3.43  0.88   0.54 mdi    -   node2
172.25.0.4      42     98 51  3.43  0.88   0.54 mdi    -   node1

éªè¯ Failover

éè¿é群æ¥å£æ¥çç¶æ

模æ主èç¹ä¸çº¿ï¼é群å¼å§é举æ°ç主èç¹ï¼å¹¶å¯¹æ°æ®è¿è¡è¿ç§»ï¼éæ°åçã

[root@localhost elasticsearch]# docker-compose stop elasticsearch_n0
Stopping elasticsearch_n0 ... done

é群ç¶æ(注ææ¢ä¸ªhttpç«¯å£ å主èç¹ä¸çº¿äº)ï¼downæçèç¹è¿å¨é群中ï¼ç­å¾ä¸æ®µæ¶é´ä»æªæ¢å¤åå°±ä¼è¢«ååº

ip     heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
172.25.0.2      57     84  5  0.46  0.65   0.50 mdi    -   node2
172.25.0.4      49     84  5  0.46  0.65   0.50 mdi    *   node1
172.25.0.3                            mdi    -   node0

ç­å¾ä¸æ®µæ¶é´

ip     heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
172.25.0.2      44     84  1  0.10  0.33   0.40 mdi    -   node2
172.25.0.4      34     84  1  0.10  0.33   0.40 mdi    *   node1

æ¢å¤èç¹ node0

[root@localhost elasticsearch]# docker-compose start elasticsearch_n0
Starting elasticsearch_n0 ... done

ç­å¾ä¸æ®µæ¶é´

ip     heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
172.25.0.2      52     98 25  0.67  0.43   0.43 mdi    -   node2
172.25.0.4      43     98 25  0.67  0.43   0.43 mdi    *   node1
172.25.0.3      40     98 46  0.67  0.43   0.43 mdi    -   node0

éå Head æ件è§å¯

git clone git://github.com/mobz/elasticsearch-head.git
cd elasticsearch-head
npm install
npm run start

é群ç¶æå¾ç¤ºæ´å®¹æçåºæ°æ®èªå¨è¿ç§»çè¿ç¨

1ãé群正常 æ°æ®å®å¨åå¸å¨3个èç¹ä¸

2ãä¸çº¿ node1 主èç¹ é群å¼å§è¿ç§»æ°æ®

è¿ç§»ä¸­

è¿ç§»å®æ

3ãæ¢å¤ node1 èç¹

é®é¢å°è®°

elasticsearch watermark

é¨ç½²å®åå建索å¼åç°æäºåçå¤äº Unsigned ç¶æï¼æ¯ç±äº elasticsearch watermarkï¼low,high,flood_stageçéå®é æçï¼é»è®¤ç¡¬ç使ç¨çé«äº85%å°±ä¼åè­¦ï¼å¼ååï¼æå¨å³æ好äºï¼æ°æ®ä¼åçå°åèç¹ï¼ç产èªè¡å³æ­ã

curl -X PUT http://192.168.20.6:9201/_cluster/settings \
-H 'Content-type':'application/json' \
-d '{"transient":{"cluster.routing.allocation.disk.threshold_enabled": false}}'

以ä¸å°±æ¯æ¬æçå¨é¨å容ï¼å¸æ对大家ç学习ææ帮å©ï¼ä¹å¸æ大家å¤å¤æ¯æèæ¬ä¹å®¶ã