php访问HDFS

假定您的主机上已经安装了LAMP环境

1. 安装所需的依赖包

  #yum install automake libtool flex bison pkgconfig gcc-c++ boost-devel libevent-devel zlib-devel python-devel ruby-devel php php-devel

2. 从http://thrift.apache.org/download/下载thrift源码包

3. 安装thrift

    ./configure –prefix=/usr/local/thrift

    make && make install

4. 安装php扩展

    cd /path/to/thrift-0.8.0/lib/php/src/ext/thrift_protocol

    phpize

    ./configure

    make && make install

    修改php.ini，添加extension=thrift_protocol.so

5. 从http://hadoop.apache.org/releases.html#Download下载hadoop源码包

6. 解压并编译hadoop(注意必须先安装好java编译器，ant编译环境，以及libtool, autoconf等工具)

    tar zxvf hadoop-1.0.4.tar.gz

    cd hadoop-1.0.4

    ant compile

7. 启动hdfs以及thrift代理(hdfs集群配置请参考官方文档，不再累述)

    cd /path/to/hadoop-1.0.4

    bin/start-dfs.sh

     cd /path/to/hadoop-1.0.4/src/contrib/thriftfs/scripts/

     ./start_thrift_server.sh [port] (如果port为空，则随机一个port)

8. 准备php依赖库

    cp -r /path/to/thrift-0.8.0/lib/php  /usr/local/thrift/lib

    mkdir -p /usr/local/thrift/lib/php/src/packages

    cp -r /path/to/hadoop-1.0.4/src/contrib/thriftfs/gen-php/ /usr/local/thrift/lib/php/src/packages/hadoopfs/

9. php测试代码

   

<?php

error_reporting(E_ALL);
ini_set('display_errors', 'on');

$GLOBALS['THRIFT_ROOT'] = '/usr/local/thrift/lib/php/src';
define('THRIFT_ROOT', $GLOBALS['THRIFT_ROOT']);

/**
 * Init Autloader
 */
require_once THRIFT_ROOT . '/Thrift.php';
require_once THRIFT_ROOT . '/transport/TSocket.php';
require_once THRIFT_ROOT . '/transport/TBufferedTransport.php';
require_once THRIFT_ROOT . '/protocol/TBinaryProtocol.php';

$socket = new TSocket('192.168.1.12', 11511);
$socket->setSendTimeout(10000);
$socket->setRecvTimeout(20000);
$transport = new TBufferedTransport($socket);
$protocol = new TBinaryProtocol($transport);

require_once $THRIFT_ROOT . '/packages/hadoopfs/ThriftHadoopFileSystem.php';
$client = new ThriftHadoopFileSystemClient($protocol);

$transport->open();
try{
	#$pathname = new Pathname(array('pathname' => '/user/hadoop/dir4test'));
	#$fp = $client->open($pathname);
	#var_dump($client->stat($pathname));
	#var_dump($client->read($fp, 0, 1024));
	
	$pathname = new Pathname(array('pathname' => '/user/hadoop/thrift'));
	if(!$client->exists($pathname)){
		$client->mkdirs($pathname);
		print("Created dir ". $pathname->pathname);
		var_dump($client->stat($pathname));
	}
}
catch(Exception $ex){
	print_r($ex);
}
$transport->close();
?>