大数据学习5(基于yarn的高可用集群搭建)

 NN-1NN-2DNZKZKFCJNNRSNM(和DN在一起)
node0001*   **  
node0002 ***** *
node0003  ** ***
node0004  **  **

1,修改配置文件

cd /opt/sxt/hadoop-2.6.5/etc/hadoop

mv mapred-site.xml.template mapred-site.xml

vi mapred-site.xml    (插入)

<property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
</property>

vi yarn-site.xml

<property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>
<property>
   <name>yarn.resourcemanager.ha.enabled</name>
   <value>true</value>
 </property>
 <property>
   <name>yarn.resourcemanager.cluster-id</name>
   <value>cluster1</value>
 </property>
 <property>
   <name>yarn.resourcemanager.ha.rm-ids</name>
   <value>rm1,rm2</value>
 </property>
 <property>
   <name>yarn.resourcemanager.hostname.rm1</name>
   <value>node0003</value>
 </property>
 <property>
   <name>yarn.resourcemanager.hostname.rm2</name>
   <value>node0004</value>
 </property>
 <property>
   <name>yarn.resourcemanager.zk-address</name>
   <value>node0002:2181,node0003:2181,node0004:2181</value>
 </property>

2,分发

scp mapred-site.xml  yarn-site.xml node0002:`pwd`

scp mapred-site.xml  yarn-site.xml node0003:`pwd`

scp mapred-site.xml  yarn-site.xml node0004:`pwd`

3,两个ResourceManager相互免密钥

node0003:cd .ssh

ssh-keygen -t dsa -P ‘‘ -f ./id_dsa

cat id_dsa.pub >> authorized_keys

scp id_dsa.pub node0004:`pwd`/node0003.pub

node0004:cd .ssh

cat node0003.pub >> authorized_keys

ssh-keygen -t dsa -P ‘‘ -f ./id_dsa

cat id_dsa.pub >> authorized_keys

scp id_dsa.pub node0003:`pwd`/node0004.pub

node0003:

cat node0004.pub >> authorized_keys

相互访问没问题ok

4,启动

zkServer.sh start(启动zookeeper,002,003,004)

start-dfs.sh  (启动hdfs,mapreduce基于hdfs,0001)

start-yarn.sh   (启动yarn,0001)

jps 查看 0002,0003,0004 都有NodeManager进程

yarn-daemon.sh start resourcemanager    (启动resourcemanager ,0003,0004)

jps 查看 0003,0004 都有ResourceManager 进程

5,浏览器访问

node0003:8088

6,停止

node0003,node0004: yarn-daemon.sh stop resourcemanager (停止resourcemanager)

node0001: stop-yarn.sh (停止nodemanager)

node0001: stop-dfs.sh

node0002,node0003,node0004: zkServer.sh stop