hadoop的超时设置
from http://blog.163.com/zhengjiu_520/blog/static/3559830620130510443956/
最近在测试mysql往HDFS备份的性能问题，发现对mysql执行xtrabackup进行限速之后经常出现HDFS写入不正常的情况。
1. 错误如下：
11/10/12 10:50:44 WARN hdfs.DFSClient: DFSOutputStream ResponseProcessor exception for block blk_8540857362443890085_4343699470java.net.SocketTimeoutException: 66000 millis timeout while waiting for channel to be ready for read. ch : java.nio.channels.SocketChannel[connected local=/172.*.*.*:14707 remote=/*.*.*.24:80010]
2. 修改超时设置
后来发现HDFS写入过程中有两个超时设置： dfs.socket.timeout和 dfs.datanode.socket.write.timeout；有些地方以为只是需要修改后面 的dfs.datanode.socket.write.timeout项就可以，其实看报错是READ_TIMEOUT。对应在hbase中的默认值如下：
// Timeouts for communicating with DataNode for streaming writes/reads
public static int READ_TIMEOUT = 60 * 1000; //其实是超过了这个值
public static int READ_TIMEOUT_EXTENSION = 3 * 1000;
public static int WRITE_TIMEOUT = 8 * 60 * 1000;
public static int WRITE_TIMEOUT_EXTENSION = 5 * 1000; //for write pipeline
3. 修改配置文件
所以找出来是超时导致的，所以在hadoop-site.xml[hdfs-site.xml]配置文件中添加如下配置：
<property>
<name>dfs.datanode.socket.write.timeout</name>
<value>3000000</value>
</property>
<property>
<name>dfs.socket.timeout</name>
<value>3000000</value>
</property>
</configuration>
附：备份命令
innobackupex-1.5.1 –user=* –tmpdir=$BAKDEST –stream=tar $BAKDEST –slave-info | $COMPRESS | java hdfs write