AccessControlException: Can't continue with getBlockLocalPathInfo()解决方案

Hadoop的Job执行MR的时候抛AccessControlException异常，详情如下：

org.apache.hadoop.hdfs.DFSClient: Short circuit access failed 
org.apache.hadoop.security.AccessControlException: 
org.apache.hadoop.security.AccessControlException: Can't continue with getBlockLocalPathInfo() authorization.
The user XXX is not allowed to call getBlockLocalPathInfo
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:513)

 

 产生问题的原因定位：

org.apache.hadoop.hdfs.server.datanode.DataNode;
public BlockLocalPathInfo getBlockLocalPathInfo(Block block,
      Token<BlockTokenIdentifier> token) throws IOException ;

 

 具体的执行如下：

private boolean shouldTryShortCircuitRead(InetSocketAddress targetAddr)
	throws IOException {
	if (shortCircuitLocalReads && isLocalAddress(targetAddr)) {
		return true;
	}
	return false;
}

 

1.读取DFSConfigKeys.DFS_CLIENT_READ_SHORTCIRCUIT_KEY对应的值如果是true那么走getLocalBlockReader(...)如果是false那么走getBlockReader(...)

2.如果走getLocalBlockReader(...)那么需要通过BlockReaderLocal.newBlockReader(...)创建BlockReader

3.执行getBlockPathInfo(...)的时候会执行DataNode的getBlockLocalPathInfo方法从而在checkBlockLocalPathAccess的时候告知权限不足.

解决方式：

1.直接禁用

conf.setBoolean(DFSConfigKeys.DFS_CLIENT_READ_SHORTCIRCUIT_KEY, false);