HDFS FileSystem常用方法

工程所依赖的jar包， hadoop-1.1.2/*.jar、hadoop-1.1.2/lib/*.jar

 编写Java程序通过URL访问HDFS

public class HDFSTest {
	public static final String HDFSPATH ="hdfs://centos:9000/hello.txt";
	@Test
	public void testAcceHDFSForURL() throws IOException{
		URL url = new URL(HDFSPATH);
		InputStream inputStream = url.openStream();
		IOUtils.copyBytes(inputStream, System.out, 1024, true);
	}
}
java.net.MalformedURLException: unknown protocol: hdfs

 只是这样访问，程序会有异常，畸形的URL，就是不能解析HDFS这种协议，使用setURLStreamHandlerFactory()让URL能够识别hdfs协议

public class HDFSTest {
	//URL如果不是用IP地址，使用主机名，需要在hosts文件中配置主机名到IPd地址的映射
	public static final String HDFSPATH ="hdfs://192.168.56.101:9000/hello.txt";
	@Test
	public void testAcceHDFSForURL() throws IOException{
		URL.setURLStreamHandlerFactory(new FsUrlStreamHandlerFactory());
		URL url = new URL(HDFSPATH);
		InputStream inputStream = url.openStream();
		/**
			InputStream：输入流			OutputStream：输出流			int：缓冲区大小			boolean：传输结束后是否关闭流
		**/
		IOUtils.copyBytes(inputStream, System.out, 1024, true);
	}
}

使用HDFS提供的接口访问HDFS，功能更加强大，操作类FileSystem

删除HDFS中的所有文件、目录

[root@centos Downloads]# hadoop fs -rmr hdfs://centos:9000/*
Warning: $HADOOP_HOME is deprecated.
Deleted hdfs://centos:9000/file1
Deleted hdfs://centos:9000/hello.txt
Deleted hdfs://centos:9000/usr

[root@centos Downloads]# hadoop fs -lsr /
Warning: $HADOOP_HOME is deprecated.
[root@centos Downloads]# hadoop fs -lsr /
Warning: $HADOOP_HOME is deprecated.