删除hdfs文件报 permission denied的处理

FireFox

2012-08-02

关注关注

删除hdfs文件报permissiondenied的处理,

参考:

http://blog.sina.com.cn/s/blog_5f3895a00100w2pu.html

分M/R和客户端程序两种情况.

caseA:M/R，只要在执行的机器上加入java参数

-Dhadoop.job.ugi=shentingting,supergroup

caseB:客户端连接去操作，此时加入参数就不管用了,应该修改配置xml文件

(hdfs-site.xml)

<property>
  <name>dfs.permissions</name>
  <value>false</value>
  <description>
    If "true", enable permission checking in HDFS.
    If "false", permission checking is turned off,
    but all other behavior is unchanged.
    Switching from one parameter value to the other does not change the mode,
    owner or group of files or directories.默认为true，如果true，那么要删除别人创建的文件(夹)就会报 Permission denied: user=xxx, access=WRITE, inode...
  </description>
</property>

此处一改，第一种情况也解决了.但是要重启集群环境.

客户端直接访问集群报权限错误:

[FS] INFO [main] HdfsFile.<init>(44) | --分布式文件路径:hdfs://192.168.0.220:9000/usr/local/hadoop/hadoop-root/tmp/oak/local/stt
[FS] ERROR [main] HdfsFile.delete(233) | [分布式文件系统]删除遇到错误
org.apache.hadoop.security.AccessControlException: org.apache.hadoop.security.AccessControlException: Permission denied: user=chenjd, access=WRITE, inode=".svn":shentingting:supergroup:rwxr-xr-x
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
	at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:95)
	at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:57)
	at org.apache.hadoop.hdfs.DFSClient.delete(DFSClient.java:830)
	at org.apache.hadoop.hdfs.DistributedFileSystem.delete(DistributedFileSystem.java:234)
	at com.mw.fs.impl.dfs.HdfsFileSystem.delete(HdfsFileSystem.java:211)
	at com.mw.fs.impl.dfs.HdfsFile.delete(HdfsFile.java:231)
	at com.mw.fs.utils.MLFileUtils.delete(MLFileUtils.java:143)
	at com.mw.fs.utils.MLFileUtils.delete(MLFileUtils.java:182)
	at com.mw.fs.utils.MLFileUtils.delete(MLFileUtils.java:178)
	at com.mw.fs.utils.MLFileUtils.delete(MLFileUtils.java:178)
	at com.mw.fs.utils.MLFileUtils.delete(MLFileUtils.java:198)
	at test.TestDFS.delete(TestDFS.java:117)
	at test.TestDFS.main(TestDFS.java:27)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)

hdfs