Maven编译hadoop-2.2.0-src
Maven编译Hadoop-2.2.0-src
1.环境
Apache Maven 3.1.1 (0728685237757ffbf44136acec0402957f723d9a; 2013-09-17 23:22:22+0800)
Maven home: /opt/maven
Java version: 1.7.0_45, vendor: Oracle Corporation
Java home: /usr/java/jdk1.7.0_45/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "2.6.18-371.3.1.el5", arch: "amd64", family: "unix"
2.执行
mvn package -DskipTests -Pdist,native,docs -Dtar
3.插曲1
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:testCompile (default-testCompile) on project hadoop-auth: Compilation failure: Compilation failure:
[ERROR] /home/chuan/trunk/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/AuthenticatorTestCase.java:[84,13] cannot access org.mortbay.component.AbstractLifeCycle
[ERROR] class file for org.mortbay.component.AbstractLifeCycle not found
打补丁
Patch :https://issues.apache.org/jira/browse/HADOOP-10110
方法：修改hadoop源码目录下的hadoop-common-project/hadoop-auth/pom.xml
添加或修改如下部分内容：
<dependency>
<groupId>org.mortbay.jetty</groupId>
<artifactId>jetty-util</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mortbay.jetty</groupId>
<artifactId>jetty</artifactId>
<scope>test</scope>
</dependency>
4.插曲2
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.6:run (make) on project hadoop-common: An Ant BuildException has occured: Execute failed: java.io.IOException: Cannot run program "cmake" (in directory "/opt/hadoop-2.2.0-src/hadoop-common-project/hadoop-common/target/native"): error=2, No such file or directory -> [Help 1]
没装cmake
人家就是想要这个地方 这可不行 果断
yum install cmake
5.插曲3
[ERROR] Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:2.2.0:protoc (compile-protoc) on project hadoop-common: org.apache.maven.plugin.MojoExecutionException: 'protoc --version' did not return a version -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:2.2.0:protoc (compile-protoc) on project hadoop-common: org.apache.maven.plugin.MojoExecutionException: 'protoc --version' did not return a version
没装protobuf-2.5.0。
下载protobuf-2.5.0.tar.gz 解压 3部曲编译安装。
6.继续(这次学能了，带着debug)
mvn package -Pdist,native -DskipTests -Dtar -e -X
等了好久好久
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Hadoop Main ................................ SUCCESS [2:18.165s]
[INFO] Apache Hadoop Project POM ......................... SUCCESS [50.526s]
[INFO] Apache Hadoop Annotations ......................... SUCCESS [1:14.522s]
[INFO] Apache Hadoop Assemblies .......................... SUCCESS [0.760s]
[INFO] Apache Hadoop Project Dist POM .................... SUCCESS [5.983s]
[INFO] Apache Hadoop Maven Plugins ....................... SUCCESS [13.161s]
[INFO] Apache Hadoop Auth ................................ SUCCESS [10.647s]
[INFO] Apache Hadoop Auth Examples ....................... SUCCESS [5.010s]
[INFO] Apache Hadoop Common .............................. SUCCESS [4:32.249s]
[INFO] Apache Hadoop NFS ................................. SUCCESS [18.927s]
[INFO] Apache Hadoop Common Project ...................... SUCCESS [0.134s]
[INFO] Apache Hadoop HDFS ................................ SUCCESS [5:29.380s]
[INFO] Apache Hadoop HttpFS .............................. SUCCESS [1:43.219s]
[INFO] Apache Hadoop HDFS BookKeeper Journal ............. SUCCESS [44.693s]
[INFO] Apache Hadoop HDFS-NFS ............................ SUCCESS [13.217s]
[INFO] Apache Hadoop HDFS Project ........................ SUCCESS [0.219s]
[INFO] hadoop-yarn ....................................... SUCCESS [1.957s]
[INFO] hadoop-yarn-api ................................... SUCCESS [1:36.233s]
[INFO] hadoop-yarn-common ................................ SUCCESS [1:31.987s]
[INFO] hadoop-yarn-server ................................ SUCCESS [0.334s]
[INFO] hadoop-yarn-server-common ......................... SUCCESS [26.932s]
[INFO] hadoop-yarn-server-nodemanager .................... SUCCESS [49.536s]
[INFO] hadoop-yarn-server-web-proxy ...................... SUCCESS [7.882s]
[INFO] hadoop-yarn-server-resourcemanager ................ SUCCESS [1:01.936s]
[INFO] hadoop-yarn-server-tests .......................... SUCCESS [6.642s]
[INFO] hadoop-yarn-client ................................ SUCCESS [11.930s]
[INFO] hadoop-yarn-applications .......................... SUCCESS [0.437s]
[INFO] hadoop-yarn-applications-distributedshell ......... SUCCESS [12.064s]
[INFO] hadoop-mapreduce-client ........................... SUCCESS [1.363s]
[INFO] hadoop-mapreduce-client-core ...................... SUCCESS [58.193s]
[INFO] hadoop-yarn-applications-unmanaged-am-launcher .... SUCCESS [11.262s]
[INFO] hadoop-yarn-site .................................. SUCCESS [0.328s]
[INFO] hadoop-yarn-project ............................... SUCCESS [17.453s]
[INFO] hadoop-mapreduce-client-common .................... SUCCESS [43.479s]
[INFO] hadoop-mapreduce-client-shuffle ................... SUCCESS [6.048s]
[INFO] hadoop-mapreduce-client-app ....................... SUCCESS [46.449s]
[INFO] hadoop-mapreduce-client-hs ........................ SUCCESS [9.770s]
[INFO] hadoop-mapreduce-client-jobclient ................. SUCCESS [17.249s]
[INFO] hadoop-mapreduce-client-hs-plugins ................ SUCCESS [9.929s]
[INFO] Apache Hadoop MapReduce Examples .................. SUCCESS [12.193s]
[INFO] hadoop-mapreduce .................................. SUCCESS [6.817s]
[INFO] Apache Hadoop MapReduce Streaming ................. SUCCESS [10.957s]
[INFO] Apache Hadoop Distributed Copy .................... SUCCESS [32.407s]
[INFO] Apache Hadoop Archives ............................ SUCCESS [4.950s]
[INFO] Apache Hadoop Rumen ............................... SUCCESS [18.817s]
[INFO] Apache Hadoop Gridmix ............................. SUCCESS [10.909s]
[INFO] Apache Hadoop Data Join ........................... SUCCESS [6.662s]
[INFO] Apache Hadoop Extras .............................. SUCCESS [11.594s]
[INFO] Apache Hadoop Pipes ............................... SUCCESS [3.737s]
[INFO] Apache Hadoop Tools Dist .......................... SUCCESS [3.623s]
[INFO] Apache Hadoop Tools ............................... SUCCESS [0.127s]
[INFO] Apache Hadoop Distribution ........................ SUCCESS [2:55.243s]
[INFO] Apache Hadoop Client .............................. SUCCESS [25.929s]
[INFO] Apache Hadoop Mini-Cluster ........................ SUCCESS [2.516s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 33:38.762s
[INFO] Finished at: Sat Dec 28 21:59:12 CST 2013
[INFO] Final Memory: 68M/200M
[INFO] ------------------------------------------------------------------------
注：第一次运行，一般要下很多jar包
有时候会出现下载的时候下面这样的东西引起的一大片错误
Connection to http://repo.maven.apache.org refused
只要重新执行
mvn package -Pdist,native -DskipTests -Dtar -e -X
就能接着下，一般是网络问题。