Hadoop HA cluster 搭建步骤
环境准备和计划:
5台 Centos 7 虚拟机, 规划如下:
HostName | Softwares | Process | comments |
kencentos1 | JDK, hadoop | namenode, zkfc (active), resourceManager | namenode 1, resourceManger |
kencentos2 | JDK, hadoop | namenode, zkfc (active), resourceManager | namenode 2, resourceManger |
kencentosClient1 | JDK, hadoop,zookeeper | QuorumPeefMain(zookeeper),journnode, datanode,nodeManger | datanode 3 |
kencentosClient2 | JDK, hadoop,zookeeper | QuorumPeefMain(zookeeper),journnode, datanode,nodeManger | datanode 1 |
kencentosClient3 | JDK, hadoop,zookeeper | QuorumPeefMain(zookeeper),journnode, datanode,nodeManger | datanode 2 |
Hadoop: 2.8.3,
JDK: 1.8.60
Zookeeper:3.4.9
step1 :centos 7 准备,这个百度上搜搜很容易装好。
step2: JDK 安装和环境变量设置。百度一下都可以解决。
Step3: 下载Hadoop 和 Zookeeper。去Apache 官方找到下载的地方。
Step4:hadoop HA集群配置
这一步需要在 dfs.journalnode.edits.dir 指定的地方建相对应的空文件夹。
core-site.xml:
<configuration>
<property>
<name>hadoop.tmp.dir</name>
<value>/opt/hadoop/data</value>
<description>Abase for other temporary directories.</description>
</property>
<property>
<name>fs.defaultFS</name>
<value>hdfs://kenns</value>
</property>
<property>
<name>ha.zookeeper.quorum</name>
<value>kencentosClient1:2181,kencentosClient3:2181,kencentosClient2:2181</value>
</property>
</configuration>
hdfs-site.xml:
这一步需要在hadoop.tmp.dir指定的地方建相对应的空文件夹。
<configuration>
<property>
<name>dfs.replication</name>
<value>2</value>
</property>
<property>
<name>dfs.block.size</name>
<value>64M</value>
</property>
<property>
<name>dfs.nameservices</name>
<value>kenns</value>
</property>
<property>
<name>dfs.ha.namenodes.kenns</name>
<value>nn1,nn2</value>
</property>
<property>
<name>dfs.namenode.rpc-address.kenns.nn1</name>
<value>kencentos1:9000</value>
</property>
<property>
<name>dfs.namenode.http-address.kenns.nn1</name>
<value>kencentos1:50070</value>
</property>
<property>
<name>dfs.namenode.rpc-address.kenns.nn2</name>
<value>kencentos2:9000</value>
</property>
<property>
<name>dfs.namenode.http-address.kenns.nn2</name>
<value>kencentos2:50070</value>
</property>
<property>
<name>dfs.namenode.shared.edits.dir</name>
<value>qjournal://kencentosClient1:8485;kencentosClient2:8485;kencentosClient3:8485/kenns</value>
</property>
<!-- JournalNode data location -->
<property>
<name>dfs.journalnode.edits.dir</name>
<value>/opt/hadoop/journal</value>
</property>
<!-- name node fail over -->
<property>
<name>dfs.ha.automatic-failover.enabled</name>
<value>true</value>
</property>
<!-- fail over way -->
<property>
<name>dfs.client.failover.proxy.provider.kenns</name>
<value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
</property>
<property>
<name>dfs.ha.fencing.methods</name>
<value>
sshfence
shell(/bin/true)
</value>
</property>
<property>
<name>dfs.ha.fencing.ssh.private-key-files</name>
<value>/root/.ssh/id_rsa</value>
</property>
<property>
<name>dfs.ha.fencing.ssh.connect-timeout</name>
<value>30000</value>
</property>
</configuration>
mapred-site.xml:
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>
yarn-site.xml:
<configuration>
<property>
<name>yarn.resourcemanager.ha.enabled</name>
<value>true</value>
</property>
<!-- cluster id -->
<property>
<name>yarn.resourcemanager.cluster-id</name>
<value>yrc</value>
</property>
<!-- RM name -->
<property>
<name>yarn.resourcemanager.ha.rm-ids</name>
<value>rm1,rm2</value>
</property>
<!-- RM address -->
<property>
<name>yarn.resourcemanager.hostname.rm1</name>
<value>kencentos1</value>
</property>
<property>
<name>yarn.resourcemanager.hostname.rm2</name>
<value>kencentos2</value>
</property>
<!-- ZK cluster -->
<property>
<name>yarn.resourcemanager.zk-address</name>
<value>kencentosClient1:2181,kencentosClient2:2181,kencentosClient3:2181</value>
</property>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
</configuration>
slaves:
kencentosClient3
kencentosClient2
kencentosClient1
Step 5: 配置SSH 免密登陆,各个机器之间。这个可以百度一下,要保证各个节点之间都要能连上。
Step6:将配置好的hadoop copy到其它节点。
scp -r
step7:启动zookeeper集群
分别在kencentosClient1,kencentosClient2, kencentosClient3 机器上执行启动指令:./zkServer.sh start 查看zookeeper状态:./zkServer.sh status, 正确的状态是一个leader,两个follower。
这里如果没有这只zookeeper 的home,请在zookeeper 的bin 下面执行。
step 8: 启动journalnode
分别在kencentosClient1,kencentosClient2,kencentosClient3机器上执行启动指令:sbin/hadoop-daemon.sh start journalnode。 启动成功后会多出一个JournalNode进程。
step9:格式化namenode
选取一个namenode 节点,执行:hdfs namenode -format
Step10:格式化zkfc
在格式化的namenode上执行格式化指令: hdfs zkfc -formatZK
格式化成功后会在zookeeper集群建立新的文件路径(该路径下存放zkfc监控namenode节点的信息)
step11:启动格式化过的namenode:
hadoop-daemon.sh start namenode
Step 12: 在另外一个namenode上执行下面的命令,此步骤是copy格式化好的配置到另外一个节点上。
hdfs namenode -bootstrapStandby
Step13: 启动集群。
完成上面的步骤,就可以stop namenode或者不停也可以。执行下面的命令在namenode 上:
./start-dfs.sh
然后:
./start-yarn.sh.
可以用各自的namenode:50070 去查看是否都已经成功启动了。