Cygwin中运行hadoop mapreduce任务必须设置中间暂存目录
cygwin上运行hadoop mr任务,一直报错:
2012-04-20 00:47:28,937 WARN org.apache.hadoop.mapred.TaskTracker: Error running child java.io.FileNotFoundException: File C:/home/Administrator/hadoop-0.20.2/hadooptmp/mapred/local/taskTracker/jobcache/job_201204200043_0002/attempt_201204200043_0002_m_000001_0/work/tmp does not exist. at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:361) at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:245) at org.apache.hadoop.mapred.TaskRunner.setupWorkDir(TaskRunner.java:519) at org.apache.hadoop.mapred.Child.main(Child.java:155) 2012-04-20 00:47:28,937 INFO org.apache.hadoop.mapred.TaskRunner: Runnning cleanup for the task 2012-04-20 00:47:28,937 INFO org.apache.hadoop.mapred.TaskTracker: Error cleaning upjava.lang.NullPointerException
最终解决:
在mapred-site.xml中添加:
<property> <name>mapred.child.tmp</name> <value>/home/Administrator/hadoop-0.20.2/child/temp</value> <description> To set the value of tmp directory for map and reduce tasks. If the value is an absolute path, it is directly assigned. Otherwise, it is prepended with task's working directory. The java tasks are executed with option -Djava.io.tmpdir='the absolute path of the tmp dir'. Pipes and streaming are set with environment variable, TMPDIR='the absolute path of the tmp dir' </description> </property>
相关推荐
changjiang 2020-11-16
minerd 2020-10-28
WeiHHH 2020-09-23
Aleks 2020-08-19
WeiHHH 2020-08-17
飞鸿踏雪0 2020-07-26
tomli 2020-07-26
deyu 2020-07-21
strongyoung 2020-07-19
eternityzzy 2020-07-19
Elmo 2020-07-19
飞鸿踏雪0 2020-07-09
飞鸿踏雪0 2020-07-04
xieting 2020-07-04
WeiHHH 2020-06-28
genshengxiao 2020-06-26
Hhanwen 2020-06-25