码迷，mamicode.com
首页 > Web开发 > 详细

PHP和Golang使用Thrift1和Thrift2访问Hbase0.96.2(ubuntu12.04)

时间：2014-08-22 02:46:06      阅读：433      评论：0      收藏：0      [点我收藏+]

标签：des   style   blog   http   color   java   使用   os   

目录：
　　一、Thrift1和Thrift2的简要介绍
　　　　1) 写在前面
　　　　2) Thrift1和Thrift2的区别 
　　二、Thrift0.9.2的安装
　　　　1) 安装依赖插件
　　　　2) Thrift0.9.2的编译 
　　　　3) Thrift0.9.2编译后，配置成可执行文件
　　三、Hbase0.96.2的安装
　　　　1) Hbase的介绍
　　　　2) Hbase的安装
　　　　3) Hbase基于源码的Thrift生成接口文件
　　四、PHP和Golang使用Thrift1和Thrift2访问Hbase
　　　　1) PHP基于Thrift1访问Hbase
　　　　2) Golang基于Thrift2访问Hbase
　　五、常见问题
　　六、扩展阅读
 
目录：
　　一、Thrift1和Thrift2的简要介绍
　　　　1) 写在前面
　　从Hbase0.94.11开始有两套thrift接口（可以叫Thrift1和Thrift2），根据官方文档，Thrift1很可能被抛弃，但网上的文章基本是介绍Thrift1的，本文则兼容介绍Thrift1和Thrift2，目前在网上是比较详细的介绍Thrift2文章了。
　　Thrift2相比较Thrift1做了简化和合成，但不提供查询所有Table、创建Table、删除Table的功能了。日常在工作中应该也很少会在Hbase中使用Thrift来创建Table。
　　从Thrift0.9.1开始支持golang语言，相关使用和demo可以参考这里《Golang、Php、Python、Java基于Thrift0.9.1实现跨语言调用》，不过由于写本文时Thrift0.9.1中对Golang访问Hbase支持的并不友好，在0.9.2中的Thrift2已经进行了改进可以很好的支持Golang访问Hbase。所以后面在使用Golang访问Hbase时，使用的是Thrift0.9.2。
　　网上关于Thrift1的文章一搜索有很多，也就不在这里多描述，在后面的演示中Thrift2中90%的方法都会有，不过在实际使用中可能还需要根据你的系统要求进行二次封装 。
 
　　　　2) Thrift1和Thrift2的区别
　　现在官网也没有太详细的介绍，下面的对比也是从IDL描述中找到的。这是一位网友整理的，后面的扩展阅读中链接了他的文章，更多的介绍可以去那里看。

 

Thrift

Thrift2

结构

struct TCell

struct ColumnDescriptor

struct TRegionInfo

struct Mutation

struct BatchMutation

struct TIncrement

struct TColumn

struct TRowResult

struct TScan

struct TTimeRange

struct TColumn

struct TColumnValue

struct TColumnIncrement

struct TResult

struct TGet

struct TPut

struct TDelete

struct TIncrement

struct TScan

struct TRowMutations

异常

exception IOError

exception IllegalArgument

exception AlreadyExists

exception TIOError

exception TIllegalArgument

其他

 

union TMutation

enum TDeleteType

enum TDurability

服务

名称为：Hbase

void enableTable()

void disableTable()

bool isTableEnabled()

void compact()

void majorCompact()

list<Text> getTableNames()

map<Text,ColumnDescriptor> getColumnDescriptors()

list<TRegionInfo> getTableRegions()

void createTable()

void deleteTable()

list<TCell> get()

list<TCell> getVer()

list<TCell> getVerTs()

list<TRowResult> getRow()

list<TRowResult> getRowWithColumns()

list<TRowResult> getRowTs()

list<TRowResult> getRowWithColumnsTs()

list<TRowResult> getRows()

list<TRowResult> getRowsWithColumns()

list<TRowResult> getRowsTs()

list<TRowResult> getRowsWithColumnsTs()

void mutateRow()

void mutateRowTs()

void mutateRows()

void mutateRowsTs()

i64 atomicIncrement()

void deleteAll()

void deleteAllTs()

void deleteAllRow()

void increment()

void incrementRows()

void deleteAllRowTs()

ScannerID scannerOpenWithScan()

ScannerID scannerOpen()

ScannerID scannerOpenWithStop()

ScannerID scannerOpenWithPrefix()

ScannerID scannerOpenTs()

ScannerID scannerOpenWithStopTs()

list<TRowResult> scannerGet()

list<TRowResult> scannerGetList()

void scannerClose()

list<TCell> getRowOrBefore()

TRegionInfo getRegionInfo()

名称为：THBaseService

bool exists(...)

TResult get(...)

list<TResult> getMultiple(...)

void put(...)

bool checkAndPut(...)

void putMultiple(...)

void deleteSingle(...)

list<TDelete> deleteMultiple(...)

bool checkAndDelete(...)

TResult increment(...)

i32 openScanner(...)

list<TResult> getScannerRows(...)

void closeScanner(...)

void mutateRow(...)

list<TResult> getScannerResults(...)
 
　　二、Thrift0.9.2的安装
　　　　1) 安装依赖插件
?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
root@m1:/home/hadoop#sudo apt-get install libboost-dev libboost-test-dev libboost-program-options-dev libevent-dev automake libtool flex bison pkg-config g++ libssl-dev
#下载thirft0.9.2版本
root@m1:/home/hadoop# git clone https://github.com/apache/thrift.git thrift-git0.9.2
Cloning into ‘thrift-git0.9.2‘...
remote: Counting objects: 37221, done.
remote: Compressing objects: 100% (195/195), done.
remote: Total 37221 (delta 312), reused 407 (delta 272)
Receiving objects: 100% (37221/37221), 9.61 MiB | 74 KiB/s, done.
Resolving deltas: 100% (25830/25830), done.
root@m1:/home/hadoop#cd thrift-git0.9.2
# 切换成相应所要用的分支，如果你编译通过，就不用像我现在这样切换了
root@m2:/home/hadoop/thrift-git0.9.2# git checkout --track origin/0.9.2
Branch 0.9.2 set up to track remote branch 0.9.2 from origin.
Switched to a new branch ‘0.9.2’
　　　　2) Thrift0.9.2的编译 
　　#编译Thrift0.9.2版本，注意这个版本和0.9.1编译方法不一样，详情可以参考/home/hadoop/thrift-git0.9.2/compiler/cpp/README.md中的说明
?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
root@m1:/home/hadoop/thrift-git0.9.2# mkdir -p compiler/cpp/build
root@m1:/home/hadoop/thrift-git0.9.2# cd compiler/cpp/build
root@m1:/home/hadoop/thrift-git0.9.2/compiler/cpp/build# cmake ..
-- The C compiler identification is GNU
-- The CXX compiler identification is GNU
-- Check for working C compiler: /usr/bin/gcc
-- Check for working C compiler: /usr/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Thrift package version: 0.9.2
-- Thrift version: 0.9.2 (0.9.2)
-- Found FLEX: /usr/bin/flex (found version "2.5.35")
-- Found BISON: /usr/bin/bison (found version "2.5")
-- Configuring done
-- Generating done
-- Build files have been written to: /home/hadoop/thrift-git0.9.2/compiler/cpp/build
root@m1:/home/hadoop/thrift-git0.9.2/compiler/cpp/build# make
[  3%] [BISON][thrifty] Building parser with bison 2.5
[  6%] Generating thrifty.h
[  9%] [FLEX][thriftl] Building scanner with flex 2.5.35
Scanning dependencies of target libparse
[ 12%] Building CXX object CMakeFiles/libparse.dir/thrifty.cc.o
[ 15%] Building CXX object CMakeFiles/libparse.dir/thriftl.cc.o
Linking CXX static library liblibparse.a
[ 15%] Built target libparse
Scanning dependencies of target thrift
[ 18%] Building CXX object CMakeFiles/thrift.dir/src/main.cc.o
[ 21%] Building C object CMakeFiles/thrift.dir/src/md5.c.o
[ 25%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_generator.cc.o
[ 28%] Building CXX object CMakeFiles/thrift.dir/src/parse/t_typedef.cc.o
[ 31%] Building CXX object CMakeFiles/thrift.dir/src/parse/parse.cc.o
[ 34%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_c_glib_generator.cc.o
[ 37%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_cpp_generator.cc.o
[ 40%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_java_generator.cc.o
[ 43%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_as3_generator.cc.o
[ 46%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_csharp_generator.cc.o
[ 50%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_py_generator.cc.o
[ 53%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_rb_generator.cc.o
[ 56%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_perl_generator.cc.o
[ 59%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_php_generator.cc.o
[ 62%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_erl_generator.cc.o
[ 65%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_cocoa_generator.cc.o
[ 68%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_st_generator.cc.o
[ 71%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_ocaml_generator.cc.o
[ 75%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_hs_generator.cc.o
[ 78%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_xsd_generator.cc.o
[ 81%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_html_generator.cc.o
[ 84%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_js_generator.cc.o
[ 87%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_javame_generator.cc.o
[ 90%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_delphi_generator.cc.o
[ 93%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_go_generator.cc.o
[ 96%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_d_generator.cc.o
[100%] Building CXX object CMakeFiles/thrift.dir/src/generate/t_lua_generator.cc.o
Linking CXX executable thrift
[100%] Built target thrift
root@m1:/home/hadoop/thrift-git0.9.2/compiler/cpp/build#
root@m1:/home/hadoop/thrift-git0.9.2/compiler/cpp/build# ./thrift -version
Thrift version 0.9.2
　　　　3) Thrift0.9.2编译后，配置成可执行文件
　　#将编译后的Thrift0.9.2放到可执行程序目录，并命名为thrift092
?
1
2
3
4
5
6
root@m1:/home/hadoop/thrift-git0.9.2/compiler/cpp/build# whereis thrift
thrift: /usr/local/bin/thrift
root@m1:/home/hadoop/thrift-git0.9.2/compiler/cpp/build# cp thrift /usr/local/bin/thrift092
root@m1:/home/hadoop/thrift-git0.9.2/compiler/cpp/build# thrift092 -version
Thrift version 0.9.2
root@m1:/home/hadoop/thrift-git0.9.2/compiler/cpp/build#
　　三、Hbase0.96.2的安装
　　　　1) Hbase的介绍
　　hbase的介绍，参考这里：http://baike.baidu.com/view/1993870.htm
 
　　　　2) Hbase的安装
　　后面的测试，也是基于上面这篇文章的环境测试的。
 
　　　　3) Hbase基于源码的Thrift生成接口文件
　　#下载Hbase的源码包（后面会使用里面的Thrift1和Thrift2）
?
1
2
root@m1:/home/hadoop# wget http://mirrors.hust.edu.cn/apache/hbase/hbase-0.96.2/hbase-0.96.2-src.tar.gz
root@m1:/home/hadoop# tar xzvf hbase-0.96.2-src.tar.gz
　　#创建测试目录
?
1
2
root@m1:/home/hadoop# mkdir thrift_hbase
root@m1:/home/hadoop# cd thrift_hbase/
　　#生成Php基于Hbase的Thrift1的接口文件，由于Thrift0.9.1版本对Golang访问Hbase支持不够后，后面Golang访问Hbase用Thrift0.9.2来测试
?
1
2
3
4
5
6
7
root@m1:/home/hadoop/thrift_hbase# thrift --gen php /home/hadoop/hbase-0.96.2/hbase-thrift/src/main/resources/org/apache/hadoop/hbase/thrift/Hbase.thrift
root@m1:/home/hadoop/thrift_hbase# ll
总用量 16
drwxr-xr-x  4 root   root   4096 Aug 19 21:19 ./
drwxr-xr-x 47 hadoop hadoop 4096 Aug 19 21:15 ../
drwxr-xr-x  3 root   root   4096 Aug 19 21:19 gen-php/
root@m1:/home/hadoop/thrift_hbase#
　　四、PHP和Golang使用Thrift1和Thrift2访问Hbase
　　　　1) PHP基于Thrift1访问Hbase
　　#将Php用到的Thrift1包复制到thrift_hbase里面
?
1
2
3
4
5
root@m1:/home/hadoop/thrift_hbase# mkdir libphp                            
root@m1:/home/hadoop/thrift_hbase# cp -r /home/hadoop/thrift-git/lib/php/src/* libphp/
root@m1:/home/hadoop/thrift_hbase# cp -r /home/hadoop/thrift-git/lib/php/lib/Thrift libphp/
root@m1:/home/hadoop/thrift_hbase# cp -r gen-php/Hbase libphp/
root@m1:/home/hadoop/thrift_hbase# rm -rf gen-php/
　　#编写Hbase.php文件
?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
<?php
/**
 * PHP基于Hbase0.96.2的Thrift1访问Hbase
 *
 * @author     迦壹
 * @copyright  Copyright (c) 2014 (http://idoall.org)
 * @license    GNU General Public License 2.0
 * @version    1
 */
 $startTime = getMillisecond();
$GLOBALS[‘THRIFT_ROOT‘] = ‘./libphp‘;   # 指定库目录，可以是绝对路径或是相对路径
require_once $GLOBALS[‘THRIFT_ROOT‘].‘/Thrift/ClassLoader/ThriftClassLoader.php‘;
  
use Thrift\ClassLoader\ThriftClassLoader;
use Thrift\Type\TMessageType;
use Thrift\Type\TType;
use Thrift\Exception\TException;
use Thrift\Factory\TStringFuncFactory;
use Thrift\StringFunc\TStringFunc;
use Thrift\StringFunc\Core;
use Thrift\Protocol\TBinaryProtocol;
use Thrift\Transport\TSocket;
use Thrift\Transport\TSocketPool;
use Thrift\Transport\TBufferedTransport;
use Hbase\HbaseClient;
use Hbase\ColumnDescriptor;
use Hbase\Mutation;
 
$loader = new ThriftClassLoader();
$loader->registerNamespace(‘Thrift‘, $GLOBALS[‘THRIFT_ROOT‘]); # 加载thrift
$loader->register();
 
require_once( $GLOBALS[‘THRIFT_ROOT‘].‘/Hbase/Hbase.php‘ );
require_once( $GLOBALS[‘THRIFT_ROOT‘].‘/Hbase/Types.php‘ );
  
$socket = new TSocket( ‘m1‘, 9090 );
$socket->setSendTimeout( 10000 ); // Ten seconds (too long for production, but this is just a demo ;)
$socket->setRecvTimeout( 20000 ); // Twenty seconds
$transport = new TBufferedTransport( $socket );
$protocol = new TBinaryProtocol( $transport );
$client = new HbaseClient( $protocol );
 
$transport->open();
$tmpendTime = getMillisecond();
echo "----建立连接用时:".$tmpendTime."-".$startTime."=".($tmpendTime-$startTime)."毫秒\n";
 
//测试表的名称
$tempTableName = "idoallorg_student";
$tmpstartTime = getMillisecond();
$tables = $client->getTableNames();
sort($tables);
echo "----列出所有的Table：----\n";
foreach ( $tables as $name ) {
  echo( "\tfound: {$name}\n" );
  if ( $name == $tempTableName ) {
    if ($client->isTableEnabled( $tempTableName )) {
      echo( "\t表已经存在，即将关闭: {$tempTableName}\n");
      $client->disableTable( $tempTableName );
    }
    echo( "\t删除表: {$tempTableName}\n" );
    $client->deleteTable( $tempTableName );
  }
}
$tmpendTime = getMillisecond();
echo "----列出所有Table用时:".$tmpendTime."-".$tmpstartTime."=".($tmpendTime-$tmpstartTime)."毫秒\n";
echo "\r\n";
 
 
 
echo( "----创建表: {$tempTableName}----\n" );
$tmpstartTime = getMillisecond();
$columns = array(
  new ColumnDescriptor( array(
    ‘name‘ => ‘id‘,
    ‘maxVersions‘ => 10
  ) ),
  new ColumnDescriptor( array(
    ‘name‘ => ‘name:‘
  ) ),
  new ColumnDescriptor( array(
    ‘name‘ => ‘score:‘
  ) )
);
try {
  $client->createTable( $tempTableName, $columns );
} catch ( AlreadyExists $ae ) {
  echo( "WARN: {$ae->message}\n" );
}
$tmpendTime = getMillisecond();
echo "----创建表: {$tempTableName}用时:".$tmpendTime."-".$tmpstartTime."=".($tmpendTime-$tmpstartTime)."毫秒\n";
 
//读取创建后的表结构
//$tmpstartTime = getMillisecond();
echo( "----查看表结构： {$tempTableName}:----\n" );
$tmpstartTime = getMillisecond();
$descriptors = $client->getColumnDescriptors( $tempTableName );
asort( $descriptors );
foreach ( $descriptors as $col ) {
  echo( "\tcolumn: {$col->name}, maxVer: {$col->maxVersions}\n" );
}
 
$tmpendTime = getMillisecond();
echo "----查看表结构： {$tempTableName}用时:".$tmpendTime."-".$tmpstartTime."=".($tmpendTime-$tmpstartTime)."毫秒\n";
echo "\r\n";
 
//
echo "----向表{$tempTableName} 中写入1条数据----\n";
$tmpstartTime = getMillisecond();
$row = ‘1‘;
$valid = "foobar-000000";
$valid1 = "foobar-11111111";
 
$mutations = array(
    new Mutation(array(
            ‘column‘ => ‘id:‘,
            ‘value‘ => 1,
        )),
    new Mutation(array(
            ‘column‘ => ‘score:a‘,
            ‘value‘ => $valid,
        )),
    new Mutation(array(
            ‘column‘ => ‘name:a‘,
            ‘value‘ => "idoall.org",
        )),
);
$client->mutateRow($tempTableName, $row, $mutations, null);
$tmpendTime = getMillisecond();
echo "----向表{$tempTableName} 中写入1条数据用时:".$tmpendTime."-".$tmpstartTime."=".($tmpendTime-$tmpstartTime)."毫秒\n";
echo "\r\n";
 
//
echo "----获取刚写入的score列的数据->get----\n";
$tmpstartTime = getMillisecond();
$arr = $client->get($tempTableName, $row, "score:a", null);
foreach ($arr as $k => $v) {
    // $k = TCell
    echo "\t ------ get one : value = {$v->value}\n";
    echo "\t ------ get one : timestamp = {$v->timestamp}\n";
}
$tmpendTime = getMillisecond();
echo "----获取刚写入的 score列的数据->get用时:".$tmpendTime."-".$tmpstartTime."=".($tmpendTime-$tmpstartTime)."毫秒\n";
echo "\r\n";
 
//
echo "----获取刚写入的数据->getRow----\n";
$tmpstartTime = getMillisecond();
$result = $client->getRow($tempTableName, $row, null);
foreach ($result as $key => $TRowResult) {
    echo "\trowkey:$TRowResult->row\tcolumns(array):\r\n";
    foreach ($TRowResult->columns as $key => $value) {
        echo "\t\t\tkey:$key\tvalue:$value->value\ttimestamp:$value->timestamp\n";
    }
}
$tmpendTime = getMillisecond();
echo "----获取刚写入的数据->getRow用时:".$tmpendTime."-".$tmpstartTime."=".($tmpendTime-$tmpstartTime)."毫秒\n";
echo "\r\n";
 
 
//
echo "----测试更新数据----\n";
$tmpstartTime = getMillisecond();
try {
  $mutations = array(
    new Mutation( array(
      ‘column‘ => ‘score:a‘,
      ‘value‘ => $valid1
    ) ),
  );
  $client->mutateRow( $tempTableName, $row, $mutations, null );
} catch ( IOError $e ) {
  echo( "expected error: {$e->message}\n" );
}
$tmpendTime = getMillisecond();
echo "----测试更新数据用时:".$tmpendTime."-".$tmpstartTime."=".($tmpendTime-$tmpstartTime)."毫秒\n";
echo "\r\n";
 
 
echo "----批量更新10条数据----\n";
$tmpstartTime = getMillisecond();
$mutationsbatch = array();
for($i=0;$i<10;$i++)
{
    $rowkey=$i+10;
    $mutations = array(
        new Mutation(array(
                ‘column‘ => ‘id:int‘,
                ‘value‘ => $i+10,
            )),
        new Mutation(array(
                ‘column‘ => ‘score:txt‘,
                ‘value‘ => "score".$i,
            )),
        new Mutation(array(
                ‘column‘ => ‘name:txt‘,
                ‘value‘ => "idoall.org".$i,
            )),
    );
    array_push($mutationsbatch, new \Hbase\BatchMutation(array(‘row‘=>$rowkey, ‘mutations‘=>$mutations)));
}
$client->mutateRows($tempTableName, $mutationsbatch,null);
$tmpendTime = getMillisecond();
echo "----批量更新10条数据用时:".$tmpendTime."-".$tmpstartTime."=".($tmpendTime-$tmpstartTime)."毫秒\n";
echo "\r\n";
 
 
echo "----scanner get scannerOpen-----\r\n";
$tmpstartTime = getMillisecond();
$startRow = 0;
$scan = $client->scannerOpen($tempTableName, $startRow, array (‘column‘ => ‘score‘, ‘column‘=>‘name‘), null);
$nbRows = 20;
$arr = $client->scannerGetList($scan, $nbRows);
echo ‘count of result :‘.count($arr)."\n";
foreach ($arr as $k => $TRowResult) {
    echo "\trow:$TRowResult->row\tcolumns(array):";
    foreach ($TRowResult->columns as $key => $value) {
        echo "key:$key\tvalue:$value->value\ttimestamp:$value->timestamp\n";
    }
}
$client->scannerClose($scan);
$tmpendTime = getMillisecond();
echo "----scanner get scannerOpen用时:".$tmpendTime."-".$tmpstartTime."=".($tmpendTime-$tmpstartTime)."毫秒\n";
echo "\r\n";
 
 
echo "----通过RowKey的范围获取数据 scannerOpenWithStop-----\r\n";
$tmpstartTime = getMillisecond();
$result = $client->scannerOpenWithStop($tempTableName , $startRow, $nbRows, array (‘column‘ => ‘score‘, ),null);
$recordArray = array();
while (true) {
    $record = $client->scannerGet($result);
    if ($record == NULL) {
        break;
    }
    foreach($record as $TRowResult) {
        echo "\trow:$TRowResult->row\tcolumns(array):";
        foreach ($TRowResult->columns as $key => $value) {
            echo "key:$key\tvalue:$value->value\ttimestamp:$value->timestamp\n";
        }
    }
}
 
$tmpendTime = getMillisecond();
echo "----通过RowKey的范围获取数据 scannerOpenWithStop用时:".$tmpendTime."-".$tmpstartTime."=".($tmpendTime-$tmpstartTime)."毫秒\n";
 
echo "\r\n";
 
 
//删除表
echo "----删除表 {$tempTableName}----\n";
$tmpstartTime = getMillisecond();
try {
    $tmpstartTime_disableTable = getMillisecond();
    $client->disableTable( $tempTableName );
    $tmpendTime_disableTable = getMillisecond();
    echo "\t其中关闭表 {$tempTableName}用时:".$tmpendTime_disableTable."-".$tmpstartTime_disableTable."=".($tmpendTime_disableTable-$tmpstartTime_disableTable)."毫秒\n";
    $tmpstartTime_disableTable = getMillisecond();
    $client->deleteTable($tempTableName);
    $tmpendTime_disableTable = getMillisecond();
    echo "\t其中删除表 {$tempTableName}用时:".$tmpendTime_disableTable."-".$tmpstartTime_disableTable."=".($tmpendTime_disableTable-$tmpstartTime_disableTable)."毫秒\n";
} catch ( AlreadyExists $ae ) {
  echo( "WARN: {$ae->message}\n" );
}
$tmpendTime = getMillisecond();
echo "----删除表 {$tempTableName}用时:".$tmpendTime."-".$tmpstartTime."=".($tmpendTime-$tmpstartTime)."毫秒\n";
 
$endTime = getMillisecond();
  
 echo "\r\n";
echo "PHP调用 总计 用时:".$endTime."-".$startTime."=".($endTime-$startTime)."毫秒\n";
  
function getMillisecond() {
list($t1, $t2) = explode(‘ ‘, microtime());    
return (float)sprintf(‘%.0f‘, (floatval($t1) + floatval($t2)) * 1000); 
}
 
$transport->close();
?>
　　#启动Hbase的Thrift，如果不了解，看下这篇文章吧，本文的环境都是基于这篇文章的《ubuntu12.04+hadoop2.2.0+zookeeper3.4.5+hbase0.96.2+hive0.13.1分布式环境部署》
?
1
root@m1:/home/hadoop/thrift_hbase# /home/hadoop/hbase-0.96.2-hadoop2/bin/hbase-daemon.sh start thrift
　　#运行Hbase.php，如果你也可以看到下面和我一样的提示，那么恭喜你也成功地使用PHP通过Thrift1访问到Hbase
?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
root@m1:/home/hadoop/thrift_hbase# php hbase.php
----建立连接用时:1408545184136-1408545184102=34毫秒
----列出所有的Table：----
        found: demo_table
        found: hbase2hive_idoall
        found: hive2hbase_idoall
        found: test_idoall_org
----列出所有Table用时:1408545184229-1408545184136=93毫秒
 
----创建表: idoallorg_student----
----创建表: idoallorg_student用时:1408545185026-1408545184229=797毫秒
----查看表结构： idoallorg_student:----
        column: id:, maxVer: 10
        column: name:, maxVer: 3
        column: score:, maxVer: 3
----查看表结构： idoallorg_student用时:1408545185047-1408545185026=21毫秒
 
----向表idoallorg_student 中写入1条数据----
----向表idoallorg_student 中写入1条数据用时:1408545185288-1408545185047=241毫秒
 
----获取刚写入的score列的数据->get----
         ------ get one : value = foobar-000000
         ------ get one : timestamp = 1408545184901
----获取刚写入的 score列的数据->get用时:1408545185373-1408545185289=84毫秒
 
----获取刚写入的数据->getRow----
        rowkey:1        columns(array):
                        key:id: value:1 timestamp:1408545184901
                        key:name:a      value:idoall.org        timestamp:1408545184901
                        key:score:a     value:foobar-000000     timestamp:1408545184901
----获取刚写入的数据->getRow用时:1408545185383-1408545185373=10毫秒
 
----测试更新数据----
----测试更新数据用时:1408545185389-1408545185384=5毫秒
 
----批量更新10条数据----
----批量更新10条数据用时:1408545185442-1408545185389=53毫秒
 
----scanner get scannerOpen-----
count of result :11
        row:1   columns(array):key:name:a       value:idoall.org        timestamp:1408545184901
        row:10  columns(array):key:name:txt     value:idoall.org0       timestamp:1408545185053
        row:11  columns(array):key:name:txt     value:idoall.org1       timestamp:1408545185053
        row:12  columns(array):key:name:txt     value:idoall.org2       timestamp:1408545185053
        row:13  columns(array):key:name:txt     value:idoall.org3       timestamp:1408545185053
        row:14  columns(array):key:name:txt     value:idoall.org4       timestamp:1408545185053
        row:15  columns(array):key:name:txt     value:idoall.org5       timestamp:1408545185053
        row:16  columns(array):key:name:txt     value:idoall.org6       timestamp:1408545185053
        row:17  columns(array):key:name:txt     value:idoall.org7       timestamp:1408545185053
        row:18  columns(array):key:name:txt     value:idoall.org8       timestamp:1408545185053
        row:19  columns(array):key:name:txt     value:idoall.org9       timestamp:1408545185053
----scanner get scannerOpen用时:1408545185484-1408545185443=41毫秒
 
----通过RowKey的范围获取数据 scannerOpenWithStop-----
        row:1   columns(array):key:score:a      value:foobar-11111111   timestamp:1408545185003
        row:10  columns(array):key:score:txt    value:score0    timestamp:1408545185053
        row:11  columns(array):key:score:txt    value:score1    timestamp:1408545185053
        row:12  columns(array):key:score:txt    value:score2    timestamp:1408545185053
        row:13  columns(array):key:score:txt    value:score3    timestamp:1408545185053
        row:14  columns(array):key:score:txt    value:score4    timestamp:1408545185053
        row:15  columns(array):key:score:txt    value:score5    timestamp:1408545185053
        row:16  columns(array):key:score:txt    value:score6    timestamp:1408545185053
        row:17  columns(array):key:score:txt    value:score7    timestamp:1408545185053
        row:18  columns(array):key:score:txt    value:score8    timestamp:1408545185053
        row:19  columns(array):key:score:txt    value:score9    timestamp:1408545185053
----通过RowKey的范围获取数据 scannerOpenWithStop用时:1408545185536-1408545185484=52毫秒
 
----删除表 idoallorg_student----
        其中关闭表 idoallorg_student用时:1408545186819-1408545185536=1283毫秒
        其中删除表 idoallorg_student用时:1408545186974-1408545186819=155毫秒
----删除表 idoallorg_student用时:1408545186974-1408545185536=1438毫秒
 
本次调用 总计 用时:1408545186974-1408545184102=2872毫秒
　　　　2) Golang基于Thrift2访问Hbase
　　#生成Golang基于Hbase的Thrift2的接口文件
?
1
root@m1:/home/hadoop/thrift_hbase# thrift092 -r --gen go /home/hadoop/hbase-0.96.2/hbase-thrift/src/main/resources/org/apache/hadoop/hbase/thrift2/hbase.thrift
　　#将Thrift2生成的开发库复制到GOPATH的src中
　　#删除掉0.9.1的文件，可能作者比较匆忙，在我使用0.9.2生成Thrift2的时候，还存在Thrift1的接口，这会对我们的测试带来干扰（后来我下载thrift1.0.0 dev时发现已经删除了这个文件，如果你生成IDL后的接口文件中，不包含这个文件，可以不用删除）
?
1
2
root@m1:/home/hadoop/thrift_hbase# rm gen-go/hbase/t_h_base_service.go
root@m1:/home/hadoop/thrift_hbase# cp -r gen-go/hbase $GOPATH/src
　　#编写hbase.go文件
?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
//
//Golang基于Hbase0.96.2的Thrift2访问Hbase
//
//@author     迦壹
//@copyright  Copyright (c) 2014 (http://idoall.org)
//@license    GNU General Public License 2.0
//@version    1
 
package main
 
import (
    "encoding/binary"
    "fmt"
    "git.apache.org/thrift.git/lib/go/thrift"
    "hbase"
    "net"
    "os"
    "reflect"
    "strconv"
    "time"
)
 
const (
    HOST       = "m1"
    PORT       = "9090"
    TESTRECORD = 10
)
 
func main() {
    startTime := currentTimeMillis()
    logformatstr_ := "----%s\n"
    logformatstr := "----%s 用时:%d-%d=%d毫秒\n\n"
    logformattitle := "建立连接"
    rowkey := "1"
    temptable := "test_idoall_org"
 
    protocolFactory := thrift.NewTBinaryProtocolFactoryDefault()
 
    transport, err := thrift.NewTSocket(net.JoinHostPort(HOST, PORT))
    if err != nil {
        fmt.Fprintln(os.Stderr, "error resolving address:", err)
        os.Exit(1)
    }
 
    client := hbase.NewTHBaseServiceClientFactory(transport, protocolFactory)
    if err := transport.Open(); err != nil {
        fmt.Fprintln(os.Stderr, "Error opening socket to "+HOST+":"+PORT, " ", err)
        os.Exit(1)
    }
    tmpendTime := currentTimeMillis()
    fmt.Printf(logformatstr, logformattitle, tmpendTime, startTime, (tmpendTime - startTime))
    defer transport.Close()
 
    //--------------Exists
    logformattitle = "调用Exists方法"
    fmt.Printf(logformatstr_, logformattitle)
    tmpstartTime := currentTimeMillis()
    //
    isexists, err := (client.Exists([]byte(temptable), &hbase.TGet{Row: []byte(rowkey)}))
    fmt.Printf("rowkey{%s} in table{%s} Exists:%t\t", rowkey, temptable, isexists)
    if err != nil {
        fmt.Printf("Exists err:%s\n", err)
    }
    fmt.Println("")
    tmpendTime = currentTimeMillis()
    fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime - tmpstartTime))
 
    //--------------put
    logformattitle = "调用Put方法写数据"
    fmt.Printf(logformatstr_, logformattitle)
    tmpstartTime = currentTimeMillis()
    cvarr := []*hbase.TColumnValue{
        &hbase.TColumnValue{
            Family:    []byte("name"),
            Qualifier: []byte("idoall.org"),
            Value:     []byte("welcome idoall.org")}}
    temptput := hbase.TPut{Row: []byte(rowkey), ColumnValues: cvarr}
    err = client.Put([]byte(temptable), &temptput)
    if err != nil {
        fmt.Printf("Put err:%s\n", err)
    } else {
        fmt.Println("Put done")
    }
    tmpendTime = currentTimeMillis()
    fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime - tmpstartTime))
 
    //------------Get---------------
    logformattitle = "调用Get方法获取新增加的数据"
    fmt.Printf(logformatstr_, logformattitle)
    tmpstartTime = currentTimeMillis()
    //
    result, err := (client.Get([]byte(temptable), &hbase.TGet{Row: []byte(rowkey)}))
    if err != nil {
        fmt.Printf("Get err:%s\n", err)
    } else {
        fmt.Println("Rowkey:" + string(result.Row))
        for _, cv := range result.ColumnValues {
            printscruct(cv)
        }
    }
    tmpendTime = currentTimeMillis()
    fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime - tmpstartTime))
 
    //--------------put update
    logformattitle = "调用Put update方法‘修改‘数据"
    fmt.Printf(logformatstr_, logformattitle)
    tmpstartTime = currentTimeMillis()
    cvarr = []*hbase.TColumnValue{
        &hbase.TColumnValue{
            Family:    []byte("name"),
            Qualifier: []byte("idoall.org"),
            Value:     []byte("welcome idoall.org---update")}}
    temptput = hbase.TPut{Row: []byte(rowkey), ColumnValues: cvarr}
    err = client.Put([]byte(temptable), &temptput)
    if err != nil {
        fmt.Printf("Put update err:%s\n", err)
    } else {
        fmt.Println("Put update done")
    }
    tmpendTime = currentTimeMillis()
    fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime - tmpstartTime))
 
    //------------Get  update---------------
    logformattitle = "调用Get方法获取‘修改‘后的数据"
    fmt.Printf(logformatstr_, logformattitle)
    tmpstartTime = currentTimeMillis()
    //
    result, err = (client.Get([]byte(temptable), &hbase.TGet{Row: []byte(rowkey)}))
    if err != nil {
        fmt.Printf("Get update err:%s\n", err)
    } else {
        fmt.Println("update Rowkey:" + string(result.Row))
        for _, cv := range result.ColumnValues {
            printscruct(cv)
        }
    }
    tmpendTime = currentTimeMillis()
    fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime - tmpstartTime))
 
    //------------DeleteSingle------------
    logformattitle = "调用DeleteSingle方法删除一条数据"
    fmt.Printf(logformatstr_, logformattitle)
    tmpstartTime = currentTimeMillis()
    tdelete := hbase.TDelete{Row: []byte(rowkey)}
    err = client.DeleteSingle([]byte(temptable), &tdelete)
    if err != nil {
        fmt.Printf("DeleteSingle err:%s\n", err)
    } else {
        fmt.Printf("DeleteSingel done\n")
    }
 
    tmpendTime = currentTimeMillis()
    fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime - tmpstartTime))
 
    //-------------PutMultiple----------------
    logformattitle = "调用PutMultiple方法添加" + strconv.Itoa(TESTRECORD) + "条数据"
    fmt.Printf(logformatstr_, logformattitle)
    tmpstartTime = currentTimeMillis()
 
    var tputArr []*hbase.TPut
    for i := 0; i < TESTRECORD; i++ {
        putrowkey := strconv.Itoa(i)
 
        tputArr = append(tputArr, &hbase.TPut{
            Row: []byte(putrowkey),
            ColumnValues: []*hbase.TColumnValue{
                &hbase.TColumnValue{
                    Family:    []byte("name"),
                    Qualifier: []byte("idoall.org"),
                    Value:     []byte(time.Now().String())}}})
    }
 
    err = client.PutMultiple([]byte(temptable), tputArr)
    if err != nil {
        fmt.Printf("PutMultiple err:%s\n", err)
    } else {
        fmt.Printf("PutMultiple done\n")
    }
 
    tmpendTime = currentTimeMillis()
    fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime - tmpstartTime))
 
    //------------------GetMultiple-----------------------------
    logformattitle = "调用GetMultiple方法获取" + strconv.Itoa(TESTRECORD) + "数据"
    fmt.Printf(logformatstr_, logformattitle)
    tmpstartTime = currentTimeMillis()
    //
    var tgets []*hbase.TGet
    for i := 0; i < TESTRECORD; i++ {
        putrowkey := strconv.Itoa(i)
        tgets = append(tgets, &hbase.TGet{
            Row: []byte(putrowkey)})
    }
    results, err := client.GetMultiple([]byte(temptable), tgets)
    if err != nil {
        fmt.Printf("GetMultiple err:%s", err)
    } else {
        fmt.Printf("GetMultiple Count:%d\n", len(results))
        for _, k := range results {
            fmt.Println("Rowkey:" + string(k.Row))
            for _, cv := range k.ColumnValues {
                printscruct(cv)
            }
        }
    }
    tmpendTime = currentTimeMillis()
    fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime - tmpstartTime))
    //-------------------TMutation
    //TMutation包含一个TGet一个TPut，就不做测试了
    //可以和MutateRow结合使用
    //
 
    //-------------------OpenScanner
    logformattitle = "调用OpenScanner方法"
    fmt.Printf(logformatstr_, logformattitle)
    tmpstartTime = currentTimeMillis()
    startrow := make([]byte, 4)
    binary.LittleEndian.PutUint32(startrow, 1)
    stoprow := make([]byte, 4)
    binary.LittleEndian.PutUint32(stoprow, 10)
 
    scanresultnum, err := client.OpenScanner([]byte(temptable), &hbase.TScan{
        StartRow: startrow,
        StopRow:  stoprow,
        //     FilterString: []byte("RowFilter(=, ‘regexstring:00[1-3]00‘)"),
        //     FilterString: []byte("PrefixFilter(‘1407658495588-‘)"),
        Columns: []*hbase.TColumn{
            &hbase.TColumn{
                Family:    []byte("name"),
                Qualifier: []byte("idoall.org")}}})
    if err != nil {
        fmt.Printf("OpenScanner err:%s\n", err)
    } else {
        fmt.Printf("OpenScanner %d done\n", scanresultnum)
 
        scanresult, err := client.GetScannerRows(scanresultnum, 100)
        if err != nil {
            fmt.Printf("GetScannerRows err:%s\n", err)
        } else {
            fmt.Printf("GetScannerRows %d done\n", len(scanresult))
            for _, k := range scanresult {
                fmt.Println("scan Rowkey:" + string(k.Row))
                for _, cv := range k.ColumnValues {
                    printscruct(cv)
                }
            }
        }
    }
    tmpendTime = currentTimeMillis()
    fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime - tmpstartTime))
 
    //--closescanner
    logformattitle = "调用CloseScanner方法"
    fmt.Printf(logformatstr_, logformattitle)
    tmpstartTime = currentTimeMillis()
    err = client.CloseScanner(scanresultnum)
    if err != nil {
        fmt.Printf("CloseScanner err:%s\n", err)
    }
    tmpendTime = currentTimeMillis()
    fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime - tmpstartTime))
 
    //-------------------GetScannerResults
 
    logformattitle = "调用GetScannerResults方法"
    fmt.Printf(logformatstr_, logformattitle)
    tmpstartTime = currentTimeMillis() //
    gsr, err := client.GetScannerResults([]byte(temptable), &hbase.TScan{
        StartRow: startrow,
        StopRow:  stoprow,
        //     FilterString: []byte("RowFilter(=, ‘regexstring:00[1-3]00‘)"),
        //     FilterString: []byte("PrefixFilter(‘1407658495588-‘)"),
        Columns: []*hbase.TColumn{
            &hbase.TColumn{
                Family:    []byte("name"),
                Qualifier: []byte("idoall.org")}}}, 100)
    if err != nil {
        fmt.Printf("GetScannerResults err:%s\n", err)
    } else {
        fmt.Printf("GetScannerResults %d done\n", len(gsr))
        for _, k := range gsr {
            fmt.Println("scan Rowkey:" + string(k.Row))
            for _, cv := range k.ColumnValues {
                printscruct(cv)
            }
        }
    }
 
    tmpendTime = currentTimeMillis()
    fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime - tmpstartTime))
 
    //---------------DeleteMultiple--------------
    logformattitle = "调用DeleteMultiple方法删除" + strconv.Itoa(TESTRECORD) + "数据"
    fmt.Printf(logformatstr_, logformattitle)
    tmpstartTime = currentTimeMillis()
    var tdelArr []*hbase.TDelete
    for i := 0; i < TESTRECORD; i++ {
        putrowkey := strconv.Itoa(i)
        tdelArr = append(tdelArr, &hbase.TDelete{
            Row: []byte(putrowkey)})
    }
    r, err := client.DeleteMultiple([]byte(temptable), tdelArr)
    if err != nil {
        fmt.Printf("DeleteMultiple err:%s\n", err)
    } else {
        fmt.Printf("DeleteMultiple %d done\n", TESTRECORD)
        fmt.Println(r)
    }
    tmpendTime = currentTimeMillis()
    fmt.Printf(logformatstr, logformattitle, tmpendTime, tmpstartTime, (tmpendTime - tmpstartTime))
 
    endTime := currentTimeMillis()
    fmt.Printf("\nGolang调用总计用时:%d-%d=%d毫秒\n", endTime, startTime, (endTime - startTime))
}
 
func printscruct(cv interface{}) {
    switch reflect.ValueOf(cv).Interface().(type) {
    case *hbase.TColumnValue:
        s := reflect.ValueOf(cv).Elem()
        typeOfT := s.Type()
        //获取Thrift2中struct的field
        for i := 0; i < s.NumField(); i++ {
            f := s.Field(i)
            fileldformatstr := "\t%d: %s(%s)= %v\n"
            switch f.Interface().(type) {
            case []uint8:
                fmt.Printf(fileldformatstr, i, typeOfT.Field(i).Name, f.Type(), string(f.Interface().([]uint8)))
            case *int64:
                var tempint64 int64
                if f.Interface().(*int64) == nil {
                    tempint64 = 0
                } else {
                    tempint64 = *f.Interface().(*int64)
                }
                fmt.Printf(fileldformatstr, i, typeOfT.Field(i).Name, f.Type(), tempint64)
            default:
                fmt.Printf("I don‘t know")
            }
        }
    default:
        fmt.Printf("I don‘t know")
        fmt.Print(reflect.ValueOf(cv))
    }
 
}
 
func currentTimeMillis() int64 {
    return time.Now().UnixNano() / 1000000
}
　　#在hbase创建一个测试表“test_idoall_org”,用于使用Golang基于Thrift2的客户端来测试，因为Thrift2并没有提供创建和删除表的方法
?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
root@m1:/home/hadoop# /home/hadoop/hbase-0.96.2-hadoop2/bin/hbase shell
2014-07-27 09:31:07,601 INFO  [main] Configuration.deprecation: hadoop.native.lib is deprecated. Instead, use io.native.lib.available
HBase Shell; enter ‘help<RETURN>‘ for list of supported commands.
Type "exit<RETURN>" to leave the HBase Shell
Version 0.96.2-hadoop2, r1581096, Mon Mar 24 16:03:18 PDT 2014
hbase(main):001:0> list
TABLE                                                                                                                                                                                                                 
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hadoop/hbase-0.96.2-hadoop2/lib/slf4j-log4j12-1.6.4.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hadoop/hadoop-2.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
0 row(s) in 2.8030 seconds
=> []
hbase(main):002:0> version
0.96.2-hadoop2, r1581096, Mon Mar 24 16:03:18 PDT 2014
hbase(main):003:0> status
2 servers, 0 dead, 1.0000 average load
hbase(main):004:0> create ‘test_idoall_org‘,‘uid‘,‘name‘
0 row(s) in 0.5800 seconds
=> Hbase::Table - test_idoall_org
hbase(main):005:0> list
TABLE                                                                                                                                                                                                                 
test_idoall_org                                                                                                                                                                                                       
1 row(s) in 0.0320 seconds
=> ["test_idoall_org"]
hbase(main):006:0> put ‘test_idoall_org‘,‘10086‘,‘name:idoall‘,‘idoallvalue‘
0 row(s) in 0.1090 seconds                 ^
hbase(main):009:0> get ‘test_idoall_org‘,‘10086‘
COLUMN                                                 CELL                                                                                                                                                           
 name:idoall                                           timestamp=1406424831473, value=idoallvalue                                                                                                                     
1 row(s) in 0.0450 seconds
hbase(main):010:0> scan ‘test_idoall_org‘
ROW                                                    COLUMN+CELL                                                                                                                                                    
 10086                                                 column=name:idoall, timestamp=1406424831473, value=idoallvalue                                                                                                 
1 row(s) in 0.0620 seconds
hbase(main):011:0>
 
　　#启动Hbase的Thrift2
?
1
2
3
4
5
6
7
8
9
10
11
12
13
root@m1:/home/hadoop/thrift_hbase# /home/hadoop/hbase-0.96.2-hadoop2/bin/hbase-daemon.sh start thrift2
#在这里可以看到thrift2的日志记录到下面的路径中
starting thrift2, logging to /home/hadoop/hbase-0.96.2-hadoop2/bin/../logs/hbase-root-thrift2-m1.out
root@m1:/home/hadoop/thrift_hbase# jps
5705 Main
3614 ResourceManager
9426 HMaster
9944 Jps
3844 DFSZKFailoverController
9679 ThriftServer
2773 QuorumPeerMain
2944 JournalNode
root@m1:/home/hadoop/thrift_hbase#
　　#运行基于Thrift2的Golang客户端，在之前要先确保hbase已经打开，在浏览器输入：http://m1:60010/master-status,可以看到下图效果，如果看不到，去调试下hbase吧，还没有启动~~~：

bubuko.com,布布扣

 
　　#运行hbase.go，如果你看到和我一样的提示信息，再次恭喜你我们的Golang也可以正常访问到Hbase了。
?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
root@m1:/home/hadoop/thrift_hbase# go run hbase.go
----建立连接 用时:1408639700869-1408639700867=2毫秒
 
----调用Exists方法
rowkey{1} in table{test_idoall_org} Exists:false
----调用Exists方法 用时:1408639700894-1408639700869=25毫秒
 
----调用Put方法写数据
Put done
----调用Put方法写数据 用时:1408639700905-1408639700894=11毫秒
 
----调用Get方法获取新增加的数据
Rowkey:1
        0: Family([]uint8)= name
        1: Qualifier([]uint8)= idoall.org
        2: Value([]uint8)= welcome idoall.org
        3: Timestamp(*int64)= 1408639700843
----调用Get方法获取新增加的数据 用时:1408639700909-1408639700905=4毫秒
 
----调用Put update方法‘修改‘数据
Put update done
----调用Put update方法‘修改‘数据 用时:1408639700916-1408639700909=7毫秒
 
----调用Get方法获取‘修改‘后的数据
update Rowkey:1
        0: Family([]uint8)= name
        1: Qualifier([]uint8)= idoall.org
        2: Value([]uint8)= welcome idoall.org---update
        3: Timestamp(*int64)= 1408639700858
----调用Get方法获取‘修改‘后的数据 用时:1408639700918-1408639700916=2毫秒
 
----调用DeleteSingle方法删除一条数据
DeleteSingel done
----调用DeleteSingle方法删除一条数据 用时:1408639700931-1408639700918=13毫秒
 
----调用PutMultiple方法添加10条数据
PutMultiple done
----调用PutMultiple方法添加10条数据 用时:1408639700941-1408639700931=10毫秒
 
----调用GetMultiple方法获取10数据
GetMultiple Count:10
Rowkey:0
        0: Family([]uint8)= name
        1: Qualifier([]uint8)= idoall.org
        2: Value([]uint8)= 2014-08-22 00:48:20.931138625 +0800 CST
        3: Timestamp(*int64)= 1408639700881
Rowkey:1
        0: Family([]uint8)= name
        1: Qualifier([]uint8)= idoall.org
        2: Value([]uint8)= 2014-08-22 00:48:20.931689108 +0800 CST
        3: Timestamp(*int64)= 1408639700881
Rowkey:2
        0: Family([]uint8)= name
        1: Qualifier([]uint8)= idoall.org
        2: Value([]uint8)= 2014-08-22 00:48:20.931691688 +0800 CST
        3: Timestamp(*int64)= 1408639700881
Rowkey:3
        0: Family([]uint8)= name
        1: Qualifier([]uint8)= idoall.org
        2: Value([]uint8)= 2014-08-22 00:48:20.931693392 +0800 CST
        3: Timestamp(*int64)= 1408639700881
Rowkey:4
        0: Family([]uint8)= name
        1: Qualifier([]uint8)= idoall.org
        2: Value([]uint8)= 2014-08-22 00:48:20.931694819 +0800 CST
        3: Timestamp(*int64)= 1408639700881
Rowkey:5
        0: Family([]uint8)= name
        1: Qualifier([]uint8)= idoall.org
        2: Value([]uint8)= 2014-08-22 00:48:20.931696588 +0800 CST
        3: Timestamp(*int64)= 1408639700881
Rowkey:6
        0: Family([]uint8)= name
        1: Qualifier([]uint8)= idoall.org
        2: Value([]uint8)= 2014-08-22 00:48:20.931698745 +0800 CST
        3: Timestamp(*int64)= 1408639700881
Rowkey:7
        0: Family([]uint8)= name
        1: Qualifier([]uint8)= idoall.org
        2: Value([]uint8)= 2014-08-22 00:48:20.931700279 +0800 CST
        3: Timestamp(*int64)= 1408639700881
Rowkey:8
        0: Family([]uint8)= name
        1: Qualifier([]uint8)= idoall.org
        2: Value([]uint8)= 2014-08-22 00:48:20.931702153 +0800 CST
        3: Timestamp(*int64)= 1408639700881
Rowkey:9
        0: Family([]uint8)= name
        1: Qualifier([]uint8)= idoall.org
        2: Value([]uint8)= 2014-08-22 00:48:20.931704203 +0800 CST
        3: Timestamp(*int64)= 1408639700881
----调用GetMultiple方法获取10数据 用时:1408639700950-1408639700941=9毫秒
 
----调用OpenScanner方法
OpenScanner 23 done
GetScannerRows 0 done
----调用OpenScanner方法 用时:1408639700956-1408639700950=6毫秒
 
----调用CloseScanner方法
----调用CloseScanner方法 用时:1408639700957-1408639700956=1毫秒
 
----调用GetScannerResults方法
GetScannerResults 0 done
----调用GetScannerResults方法 用时:1408639700961-1408639700958=3毫秒
 
----调用DeleteMultiple方法删除10数据
DeleteMultiple 10 done
[]
----调用DeleteMultiple方法删除10数据 用时:1408639700967-1408639700961=6毫秒
 
 
root@m1:/home/hadoop/thrift_hbase#
五、常见问题
　　　　1) 并发访问线程数
首先，为了尽可能减少由于网络传输带来的时间开销，HBase的Thrift Server最好和应用客户端部署在同一台机器上。Thrift Server启动时可以通过参数配置并发线程数，否则很容易导致Thrift Server线程满了不响应客户端的读写请求，具体命令：/home/hadoop/hbase-0.96.2-hadoop2/bin/hbase- daemon.sh start thrift –threadpool -m 200 -w 500（更多参数参考这里：/home/hadoop/hbase-0.96.2-hadoop2/bin/hbase-daemon.sh start thrift -h）。
 
　　　　2) 最大堆内存配置
如果客户端与Thrift Server进行scan操作顺序读取数据，而且设置了一定的cache记录条数（通过TScan的int32_t caching变量设置），那么这些被caching的记录数可能会占用Thrift Server相当部分的堆内存，尤其在多客户端并发访问时更明显。
　　因此，在Thrift Server启动前，可以调大最大堆内存，否则可 能由于java.lang.OutOfMemoryError异常而导致进程被杀掉，尤其是当Scan时设置了较大的caching记录条数的情况（默认为expor t HBASE_HEAPSIZE=1000MB，可以在/home/hadoop/hbase-0.96.2-hadoop2/hbase- env.sh中设置）。
 
　　六、扩展阅读


-------- -------------------------------
博文作者：迦壹
转载声明：可以转载, 但必须以超链接形式标 明文章原始出处和作者信息及版权声明，谢谢合作！
---------------------------------------
 
 
Golang调用总计用时:1408639700967-1408639700867=100毫秒

PHP和Golang使用Thrift1和Thrift2访问Hbase0.96.2(ubuntu12.04),布布扣,bubuko.com

PHP和Golang使用Thrift1和Thrift2访问Hbase0.96.2(ubuntu12.04)

标签：des   style   blog   http   color   java   使用   os   

原文地址：http://www.cnblogs.com/lion.net/p/3928453.html
踩
(0)
赞
(0)
   
举报
评论 一句话评论（0）
登录后才能评论！
分享档案
更多>
2021年07月29日 (22)
2021年07月28日 (40)
2021年07月27日 (32)
2021年07月26日 (79)
2021年07月23日 (29)
2021年07月22日 (30)
2021年07月21日 (42)
2021年07月20日 (16)
2021年07月19日 (90)
2021年07月16日 (35)
周排行
mamicode.com排行更多图片
更多
友情链接
兰亭集智   国之画   百度统计   站长统计   阿里云   chrome插件   新版天听网
关于我们 - 联系我们 - 留言反馈
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码！