hadoop eclipse 3（转）
2012年5月3日
《转载》hadoopcdh3u3eclipse插件编译
1.编译环境
操作系统：debian6amd64，安装ant和maven2这两个java打包工具。
hadoop：hadoop-0.20.2-cdh3u3.tar.gz
eclipse：eclipse-java-indigo-SR2-win32.zip
2.编译hadoop
解压源码hadoop-0.20.2-cdh3u3.tar.gz并进入，执行ant，自动下载依赖并编译。
3.编译eclipseplugin
解压eclipse。
进入hadoop源码的src/contrib/eclipse-plugin目录下，执行：
ant-Declipse.home=/eclipse解压目录/-Dversion=0.20.2-cdh3u3jar。
4.测试
在hadoop源码的build/contrib/eclipse-plugin中有hadoop-eclipse-plugin-0.20.2-cdh3u3.jar。拷贝至eclipse的plugins目录下，启动eclipse。启动后报错：
Aninternalerroroccurredduring:"ConnectingtoDFSlocalhost".
查看eclipse错误日志，显示：
java.lang.NoClassDefFoundError:org/apache/hadoop/thirdparty/guava/common/collect/LinkedListMultimap
还有一个错误：
java.lang.NoClassDefFoundError:org/codehaus/jackson/map/JsonMappingException
说明eclipse找不到guava和jackson包。
5.修复bug
首先，在hadoop源码的lib目录下拷贝出guava-r09-jarjar.jar,jackson-mapper-asl-1.5.2.jar包。
5.1方法一
把guava-r09-jarjar.jar,jackson-mapper-asl-1.5.2.jar中的字节码（org目录）解压至hadoop-eclipse-plugin-0.20.2-cdh3u3.jar的classes下。
5.2方法二
把guava-r09-jarjar.jar,jackson-mapper-asl-1.5.2.jar放到hadoop-eclipse-plugin-0.20.2-cdh3u3.jar的lib目录下。
然后，修改该包META-INF目录下的MANIFEST.MF，将classpath修改为一下内容：
Bundle-ClassPath:classes/,lib/hadoop-core.jar,lib/guava-r09-jarjar.jar,lib/jackson-mapper-asl-1.5.2.jar
方法二理论上可以，但我测试未成功。
posted@2012-05-0318:39riverphoenix阅读(505)评论(0)编辑
wenti
Aninternalerroroccurredduring:"Map/Reducelocationstatusupdater".
org/codehaus/jackson/map/JsonMappingException
Aninternalerroroccurredduring:"ConnectingtoDFShadoop".
org/apache/commons/configuration/Configuration
posted@2012-05-0317:06riverphoenix阅读(213)评论(1)编辑
<zhuan>Hadoop格式化HDFS报错java.net.UnknownHostException:localhost.localdomain:localhost.localdomain
异常描述
在对HDFS格式化，执行hadoopnamenode-format命令时，出现未知的主机名的问题，异常信息如下所示：
[plain]viewplaincopyprint?
[shirdrn@localhostbin]$hadoopnamenode-format
11/06/2207:33:31INFOnamenode.NameNode:STARTUP_MSG:
/************************************************************
STARTUP_MSG:StartingNameNode
STARTUP_MSG:host=java.net.UnknownHostException:localhost.localdomain:localhost.localdomain
STARTUP_MSG:args=[-format]
STARTUP_MSG:version=0.20.0
STARTUP_MSG:build=https://svn.apache.org/repos/asf/hadoop/core/branches/branch-0.20-r763504;compiledby'ndaley'onThuApr905:18:40UTC2009
************************************************************/
Re-formatfilesystemin/tmp/hadoop/hadoop-shirdrn/dfs/name?(YorN)Y
11/06/2207:33:36INFOnamenode.FSNamesystem:fsOwner=shirdrn,shirdrn
11/06/2207:33:36INFOnamenode.FSNamesystem:supergroup=supergroup
11/06/2207:33:36INFOnamenode.FSNamesystem:isPermissionEnabled=true
11/06/2207:33:36INFOmetrics.MetricsUtil:UnabletoobtainhostName
java.net.UnknownHostException:localhost.localdomain:localhost.localdomain
atjava.net.InetAddress.getLocalHost(InetAddress.java:1353)
atorg.apache.hadoop.metrics.MetricsUtil.getHostName(MetricsUtil.java:91)
atorg.apache.hadoop.metrics.MetricsUtil.createRecord(MetricsUtil.java:80)
atorg.apache.hadoop.hdfs.server.namenode.FSDirectory.initialize(FSDirectory.java:73)
atorg.apache.hadoop.hdfs.server.namenode.FSDirectory.<init>(FSDirectory.java:68)
atorg.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:370)
atorg.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:853)
atorg.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:947)
atorg.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:964)
11/06/2207:33:36INFOcommon.Storage:Imagefileofsize97savedin0seconds.
11/06/2207:33:36INFOcommon.Storage:Storagedirectory/tmp/hadoop/hadoop-shirdrn/dfs/namehasbeensuccessfullyformatted.
11/06/2207:33:36INFOnamenode.NameNode:SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG:ShuttingdownNameNodeatjava.net.UnknownHostException:localhost.localdomain:localhost.localdomain
************************************************************/
我们通过执行hostname命令可以看到：
[plain]viewplaincopyprint?
[shirdrn@localhostbin]#hostname
localhost.localdomain
也就是说，Hadoop在格式化HDFS的时候，通过hostname命令获取到的主机名是localhost.localdomain，然后在/etc/hosts文件中进行映射的时候，没有找到，看下我的/etc/hosts内容：
[plain]viewplaincopyprint?
[root@localhostbin]#cat/etc/hosts
#Donotremovethefollowingline,orvariousprograms
#thatrequirenetworkfunctionalitywillfail.
127.0.0.1localhostlocalhost
192.168.1.103localhostlocalhost
也就说，通过localhost.localdomain根本无法映射到一个IP地址，所以报错了。
此时，我们查看一下/etc/sysconfig/network文件：
[plain]viewplaincopyprint?
NETWORKING=yes
NETWORKING_IPV6=yes
HOSTNAME=localhost.localdomain
可见，执行hostname获取到这里配置的HOSTNAME的值。
解决方法
修改/etc/sysconfig/network中HOSTNAME的值为localhost，或者自己指定的主机名，保证localhost在/etc/hosts文件中映射为正确的IP地址，然后重新启动网络服务：
[plain]viewplaincopyprint?
[root@localhostbin]#/etc/rc.d/init.d/networkrestart
Shuttingdowninterfaceeth0:[OK]
Shuttingdownloopbackinterface:[OK]
Bringinguploopbackinterface:[OK]
Bringingupinterfaceeth0:
DeterminingIPinformationforeth0...done.
[OK]
这时，再执行格式化HDFS命令，以及启动HDFS集群就正常了。
格式化：
[plain]viewplaincopyprint?
[shirdrn@localhostbin]$hadoopnamenode-format
11/06/2208:02:37INFOnamenode.NameNode:STARTUP_MSG:
/************************************************************
STARTUP_MSG:StartingNameNode
STARTUP_MSG:host=localhost/127.0.0.1
STARTUP_MSG:args=[-format]
STARTUP_MSG:version=0.20.0
STARTUP_MSG:build=https://svn.apache.org/repos/asf/hadoop/core/branches/branch-0.20-r763504;compiledby'ndaley'onThuApr905:18:40UTC2009
************************************************************/
11/06/2208:02:37INFOnamenode.FSNamesystem:fsOwner=shirdrn,shirdrn
11/06/2208:02:37INFOnamenode.FSNamesystem:supergroup=supergroup
11/06/2208:02:37INFOnamenode.FSNamesystem:isPermissionEnabled=true
11/06/2208:02:37INFOcommon.Storage:Imagefileofsize97savedin0seconds.
11/06/2208:02:37INFOcommon.Storage:Storagedirectory/tmp/hadoop/hadoop-shirdrn/dfs/namehasbeensuccessfullyformatted.
11/06/2208:02:37INFOnamenode.NameNode:SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG:ShuttingdownNameNodeatlocalhost/127.0.0.1
************************************************************/
启动：
[plain]viewplaincopyprint?
[shirdrn@localhostbin]$start-all.sh
startingnamenode,loggingto/home/shirdrn/eclipse/eclipse-3.5.2/hadoop/hadoop-0.20.0/logs/hadoop-shirdrn-namenode-localhost.out
localhost:startingdatanode,loggingto/home/shirdrn/eclipse/eclipse-3.5.2/hadoop/hadoop-0.20.0/logs/hadoop-shirdrn-datanode-localhost.out
localhost:startingsecondarynamenode,loggingto/home/shirdrn/eclipse/eclipse-3.5.2/hadoop/hadoop-0.20.0/logs/hadoop-shirdrn-secondarynamenode-localhost.out
startingjobtracker,loggingto/home/shirdrn/eclipse/eclipse-3.5.2/hadoop/hadoop-0.20.0/logs/hadoop-shirdrn-jobtracker-localhost.out
localhost:startingtasktracker,loggingto/home/shirdrn/eclipse/eclipse-3.5.2/hadoop/hadoop-0.20.0/logs/hadoop-shirdrn-tasktracker-localhost.out
查看：
[plain]viewplaincopyprint?
[shirdrn@localhostbin]$jps
8192TaskTracker
7905DataNode
7806NameNode
8065JobTracker
8002SecondaryNameNode
8234Jps
http://blog.csdn.net/shirdrn/article/details/6562292