hadoop 提交 mapreduce假死的问题
hadoop版本是CDH5.3.3的,安装了伪分布模式,一直以来程序运行好好的,突然间运行mapreduce程序卡在running job上 ,mapreduce已经提交到yarn上去了 一直卡着没动 ,纠结了N天,总算搞定了,
之前还以为是我的内存不够 ,我把内存设置成6G了 ,没跑别的应用 ,查看内存使用情况
[ehp@hadoop-ehp hadoop-2.5.0-cdh5.3.3]$ free -m total used free shared buffers cached Mem: 5852 3585 2267 0 143 2567 -/+ buffers/cache: 873 4978 Swap: 4095 0 4095
显然不是内存的问题。
使用hadoop dfsadmin -report 查看hdfs 的状况
[ehp@hadoop-ehp hadoop-2.5.0-cdh5.3.3]$ hadoop dfsadmin -report DEPRECATED: Use of this script to execute hdfs command is deprecated. Instead use the hdfs command for it. 15/08/19 10:09:30 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Configured Capacity: 16702676992 (15.56 GB) Present Capacity: 574803968 (548.18 MB) DFS Remaining: 574779392 (548.15 MB) DFS Used: 24576 (24 KB) DFS Used%: 0.00% Under replicated blocks: 0 Blocks with corrupt replicas: 0 Missing blocks: 0 Missing blocks (with replication factor 1): 0 ------------------------------------------------- Live datanodes (1): Name: 192.168.1.120:50010 (hadoop-ehp.hyman.com) Hostname: hadoop-ehp.hyman.com Decommission Status : Normal Configured Capacity: 16702676992 (15.56 GB) DFS Used: 24576 (24 KB) Non DFS Used: 16127873024 (15.02 GB) DFS Remaining: 574779392 (548.15 MB) DFS Used%: 0.00% DFS Remaining%: 3.44% Configured Cache Capacity: 0 (0 B) Cache Used: 0 (0 B) Cache Remaining: 0 (0 B) Cache Used%: 100.00% Cache Remaining%: 0.00% Xceivers: 1 Last contact: Wed Aug 19 10:09:28 EDT 2015
[ehp@hadoop-ehp hadoop-2.5.0-cdh5.3.3]$ df -hl Filesystem Size Used Avail Use% Mounted on /dev/sda2 16G 15G 549M 97% / tmpfs 2.9G 72K 2.9G 1% /dev/shm /dev/sda1 194M 29M 156M 16% /boot
很明显是硬盘空间不足导致。
解决方法:
1.增加硬盘的容量。
相关推荐
tomli 2020-07-26
changjiang 2020-11-16
minerd 2020-10-28
WeiHHH 2020-09-23
Aleks 2020-08-19
WeiHHH 2020-08-17
飞鸿踏雪0 2020-07-26
deyu 2020-07-21
strongyoung 2020-07-19
eternityzzy 2020-07-19
Elmo 2020-07-19
飞鸿踏雪0 2020-07-09
飞鸿踏雪0 2020-07-04
xieting 2020-07-04
WeiHHH 2020-06-28
genshengxiao 2020-06-26
Hhanwen 2020-06-25