Java API访问HA方式的HDFS
1、介绍
对于namenode是HA高可用集群时,客户端远程访问hdfs有两种实现方法:
(1)将所有关于namenode的参数写入Configuration对象中
(2)将配置文件core-site.xml和hdfs-site.xml文件复制到项目的src目录下
2、方式一
(1)样例代码
import java.net.URI; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.FileSystem; import org.apache.hadoop.fs.Path; public class HADemo { public static void main(String[] args) { // TODO Auto-generated method stub Configuration conf = new Configuration(); conf.set("fs.defaultFS", "hdfs://my"); conf.set("dfs.nameservices", "my"); conf.set("dfs.ha.namenodes.my", "nn1,nn2"); conf.set("dfs.namenode.rpc-address.my.nn1", "192.168.12.156:8020"); conf.set("dfs.namenode.rpc-address.my.nn2", "192.168.157:8020"); conf.set("dfs.client.failover.proxy.provider.my", "org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider"); try { FileSystem fs = FileSystem.get(new URI("hdfs://my"), conf,"root"); boolean b=fs.mkdirs(new Path("/a")); System.out.println(b); fs.copyFromLocalFile(new Path("D:\\a.txt"), new Path("/a/a.txt")); fs.close(); } catch (Exception e) { System.out.println(e); } } }
(2)运行结果
log4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.lib.MutableMetricsFactory).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
true
(3)HDFS命令验证
[root@node1 ~]# hdfs dfs -ls /a
Found 2 items
-rw-r--r-- 3 root supergroup 13 2018-08-08 09:11 /a/a.txt
[root@node1 ~]# hdfs dfs -cat /a/a.txt
Hello,Hadoop!
[root@node1 ~]#