oozie CDH4和hadoop CDH4整合出现的问题(一)
最近在使用oozie和hadoop进行开发
hadoop版本：hadoop-2.0.0-cdh4.3.0
oozie版本：oozie-3.3.2-cdh4.3.0
在使用oozie执行里面的例子时，
oozie在向hadoop提交任务时报出了以下错误：
Caused by: com.google.protobuf.ServiceException: java.net.ConnectException: Call From slave4/10.95.3.65 to 0.0.0.0:8030 failed on connection exception: java.net.ConnectException: 拒绝连接; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:212) at com.sun.proxy.$Proxy28.registerApplicationMaster(Unknown Source) at org.apache.hadoop.yarn.api.impl.pb.client.AMRMProtocolPBClientImpl.registerApplicationMaster(AMRMProtocolPBClientImpl.java:100) ... 12 more Caused by: java.net.ConnectException: Call From slave4/10.95.3.65 to 0.0.0.0:8030 failed on connection exception: java.net.ConnectException: 拒绝连接; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:525) at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:782) at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:729) at org.apache.hadoop.ipc.Client.call(Client.java:1229) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:202) ... 14 more Caused by: java.net.ConnectException: 拒绝连接 at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:692) at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:207) at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:528) at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:492) at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:499) at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:593) at org.apache.hadoop.ipc.Client$Connection.access$2000(Client.java:241) at org.apache.hadoop.ipc.Client.getConnection(Client.java:1278) at org.apache.hadoop.ipc.Client.call(Client.java:1196) ... 15 more 2013-08-22 11:03:06,534 ERROR [main] org.apache.hadoop.yarn.service.CompositeService: Error starting services org.apache.hadoop.mapreduce.v2.app.MRAppMaster org.apache.hadoop.yarn.YarnException: java.lang.reflect.UndeclaredThrowableException at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.register(RMCommunicator.java:160) at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.start(RMCommunicator.java:111) at org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.start(RMContainerAllocator.java:213) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter.start(MRAppMaster.java:689) at org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.start(MRAppMaster.java:904) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.run(MRAppMaster.java:1146) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1142) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1097) Caused by: java.lang.reflect.UndeclaredThrowableException at org.apache.hadoop.yarn.exceptions.impl.pb.YarnRemoteExceptionPBImpl.unwrapAndThrowException(YarnRemoteExceptionPBImpl.java:135) at org.apache.hadoop.yarn.api.impl.pb.client.AMRMProtocolPBClientImpl.registerApplicationMaster(AMRMProtocolPBClientImpl.java:103) at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.register(RMCommunicator.java:147) ... 11 more Caused by: com.google.protobuf.ServiceException: java.net.ConnectException: Call From slave4/10.95.3.65 to 0.0.0.0:8030 failed on connection exception: java.net.ConnectException: 拒绝连接; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:212) at com.sun.proxy.$Proxy28.registerApplicationMaster(Unknown Source) at org.apache.hadoop.yarn.api.impl.pb.client.AMRMProtocolPBClientImpl.registerApplicationMaster(AMRMProtocolPBClientImpl.java:100) ... 12 more Caused by: java.net.ConnectException: Call From slave4/10.95.3.65 to 0.0.0.0:8030 failed on connection exception: java.net.ConnectException: 拒绝连接; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:525) at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:782) at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:729) at org.apache.hadoop.ipc.Client.call(Client.java:1229) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:202) ... 14 more Caused by: java.net.ConnectException: 拒绝连接 at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:692) at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:207) at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:528) at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:492) at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:499) at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:593) at org.apache.hadoop.ipc.Client$Connection.access$2000(Client.java:241) at org.apache.hadoop.ipc.Client.getConnection(Client.java:1278) at org.apache.hadoop.ipc.Client.call(Client.java:1196) ... 15 more
通过异常信息，可以猜测应该是在执行任务时，配置没有读到，所以使用了默认的0.0.0.0:8030地址。为了验证是否是这个原因造成的，我们可以修改oozie/conf目录下的log4j的日志级别为调试。
然后通过查看和跟踪hadoop的源代码，验证了猜想的正确性，由于oozie在提交job的时候，会读取oozie/conf/hadop-conf/core-site.xml中的配置，异常我们只要在这个配置文件中添加配置：
<property>
<name>yarn.resourcemanager.scheduler.address</name>
<value>master:8030</value>
</property>再重启oozie服务，上述异常消失。
相关推荐
WeiHHH 2020-06-28
eternityzzy 2015-05-26
changjiang 2014-08-28
BigPig 2013-09-04
QAnyang 2014-01-01
springqingchun 2014-01-01
xuplus 2014-01-01
zwjcyz 2014-01-01
whspringer 2013-09-26
fujing 2013-06-27
workflow 2012-05-16
wendll 2018-09-28
liubang000 2018-04-23
liushi 2017-10-15
心中永远的正能量 2017-10-15
xuyoucheng00 2018-03-05
小李子栏 2016-11-19
SunWuKongHadoop 2015-11-15
wangzy0 2014-12-13