hadoop伪分布式环境搭建
下载安装包
Apache Hadoop 官网下载hadoop安装包 hadoop-2.9.1.tar.gz,并解压。
[root@d1 local]# pwd /usr/local [root@d1 local]# tar -zxvf hadoop-2.9.1.tar.gz [root@d1 local]# ln -s hadoop-2.9.1 hadoop
修改配置
修改解压后的目录中的文件夹etc/hadoop下的xml配置文件(如果文件不存在,则自己创建)
hadoop-env.sh修改以下配置:
[root@d1 hadoop]# pwd /usr/local/hadoop [root@d1 hadoop]# vi etc/hadoop/hadoop-env.sh export JAVA_HOME=/usr/java/jdk1.8.0_181-amd64
slaves文件修改为以下配置:
[root@d1 hadoop]# vi etc/hadoop/slaves d1
注:以下四个XML配置文件,需在标签<configuration>和</configuration>之间增加配置项。
[root@d1 hadoop]# vi etc/hadoop/mapred-site.xml <configuration> <property> <name>mapreduce.framework.name</name> <value>yarn</value> </property> </configuration>
core-site.xml(其中“d1”是在/etc/hosts中设置的host,如果未设置,则换为localhost):
[root@d1 hadoop]# vi etc/hadoop/core-site.xml <?xml version="1.0" encoding="UTF-8"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <configuration> <property> <name>fs.default.name</name> <value>hdfs://d1:9000</value> </property> <property> <name>dfs.replication</name> <value>1</value> </property> <property> <name>hadoop.tmp.dir</name> <value>/hadoop/hadoop_tmp_dir</value> </property> <property> <name>dfs.namenode.name.dir</name> <value>/hadoop/dfs/name</value> </property> <property> <name>dfs.datanode.data.dir</name> <value>/hadoop/dfs/data</value> </property> </configuration>
yarn-site.xml:
[root@d1 hadoop]# vi etc/hadoop/yarn-site.xml <?xml version="1.0"?> <configuration> <property> <name>yarn.nodemanager.aux-services</name> <value>mapreduce_shuffle</value> </property> </configuration>
启动服务
格式化HDFS:
bin/hadoop namenode -format
启动HDFS:
sbin/start-dfs.sh
启动YARN:
sbin/start-yarn.sh
验证是否启动成功
[root@d1 hadoop]# jps 21697 Jps 20612 SecondaryNameNode 20308 NameNode 20933 NodeManager 20827 ResourceManager 20414 DataNode [root@d1 hadoop]#
参照:http://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/SingleCluster.html
相关推荐
changjiang 2020-11-16
minerd 2020-10-28
WeiHHH 2020-09-23
Aleks 2020-08-19
WeiHHH 2020-08-17
飞鸿踏雪0 2020-07-26
tomli 2020-07-26
deyu 2020-07-21
strongyoung 2020-07-19
eternityzzy 2020-07-19
Elmo 2020-07-19
飞鸿踏雪0 2020-07-09
飞鸿踏雪0 2020-07-04
xieting 2020-07-04
WeiHHH 2020-06-28
genshengxiao 2020-06-26
Hhanwen 2020-06-25