实验4熟悉常用的HBase操作

1、实验4 熟悉常用的HBase操作实验4熟悉常用的HBase操作姓名：包生友 专业年级：软件143 学号：20140126991 .实验目的1 .理解HBase在Hadoop体系结构中的角色；2,熟练使用HBase操作常用的Shell命令；3 .熟悉HBase操作常用的Java API。2 .实验环境操作系统：LinuxHadoop版本：2.6.0或以上版本HBase版本：1.1.2或以上版本JDK版本：1.6或以上版本Java IDE： Eclipse3 .实验内容和完成情况1,编程实现以下指定功能，并用 Hadoop提供的HBase Shell命令完成相同任务:(完整可执行代码见 代码/Qu

2、estionOne.java)(1)列出HBase所有的表的相关信息，例如表名；Shell:ListTibaseCinatr)ListTABLE511 rowCs) tnseconds2 -Si-图1列出HBase所有表的相关信息编程：/(1)列出HBase所有的表的相关信息，例如表名、创建时间等public static void listTables() throws lOException init();/建立连接HTableDescriptor hTableDescriptors口 = admin.listTables();for(HTableDescriptor hTableDesc

3、riptor :hTableDescriptors)System.out.println("表名:"+hTableDescriptor.getNameAsString();close();关闭连接(2)在终端打印出指定的表的所有记录数据；Shell:scan 's1'hbase(ntdln):o02：0> scan ' ROMCOLUMN+CEL-La row(i) in 0.1390 seconds图2打印指定表的所有记录数据编程：/(2)在终端打印出指定的表的所有记录数据public static void getData(String t

4、ableName)throws IOExceptioninit();Table table = connection.getTable(TableName.valueOf(tableName);Scan scan = new Scan();ResultScanner scanner = table.getScanner(scan);for (Result result:scanner)printRecoder(result);close();打印一条记录的详情public static void printRecoder(Result result)throws IOExceptionfor(

5、Cell cell:result.rawCells()System.out.print("行健:"+new String(CellUtil.cloneRow(cell);System.out.print("列簇:"+new String(CellUtil.cloneFamily(cell);System.out.print("列:"+new String(CellUtil.cloneQualifier(cell);System.out.print("值:"+new String(CellUtil.cloneValu

6、e(cell);System.out.println("时间戳:"+cell.getTimestamp();(3)向已经创建好的表添加和删除指定的列族或列；p.s：此题请先在 Shell中创建si作为示例表：create 's1','score'a)在si表，添加数据：Shell:put 's1','zhangsan','score:Math','69'hbase(nain):093put 1 si1 / zhangsan111 seore:Math 1 *f 69'0

7、row(s) in 9,0663 secondshb己5t(ri0in); &曰4IsNaneError; undefined Xocal variable or nuthod 'Is' f&r jecthbase(naln>：0S5：Q> ListTABLESI1tn a.HSlB seconds图3给si添加数据编程：向表添加数据public static void insertRow(String tableName,String rowKey,String colFamily,String col,Stringval) throws lOEx

8、ception init()；Table table = connection.getTable(TableName.valueOf(tableName);Put put = new Put(rowKey.getBytes();put.addColumn(colFamily.getBytes(), col.getBytes(), val.getBytes();table.put(put);table.close();close();insertRow("s1",'zhangsan','score','Math','69

9、')b)在si表，删除指定的列：Shell:delete 's1','zhangsan','score:Math':01176 delate H«1h,'zhang&an'f score G row(s) In ft.0070 seconds图4删除数据编程：删除数据public static void deleteRow(String tableName,String rowKey,String colFamily,String col) throws lOException init();Table

10、table = connection.getTable(TableName.valueOf(tableName);Delete delete = new Delete(rowKey.getBytes();删除指定列族delete.addFamily(Bytes.toBytes(colFamily);删除指定列delete.addColumn(Bytes.toBytes(colFamily),Bytes.toBytes(col);table.delete(delete);table.close();close();deleteRow("s1",'zhangsan

11、9;,'score','Math')(4)清空指定的表的所有记录数据；Shell:truncate 's1hbase(ni3in) tbis : ei RONList wanghu i cjw(s) in o.ozin“an 1 si1COLUMN+CELLcolunn=5core:Matht tiriC5tanp=148Z235IflaiS, vlue=S5colunn-score;atht tiFiestanp-14g2Z35732643, veluu=75 seconds hba£e(nain)1e> truncate 1 si,T

12、runcating 'si1 tabic (it may tike a while):* Disdbllng tdblu*-Truncdttny table-.0 raw(s) tn 3.78/0 secondssen 1 si'COLUMN+CELL seronrlshbase(main):020:0> ROWfl row(s) tn图5清空指定表的所有记录数据编程：/(4)清空指定的表的所有记录数据public static void clearRows(String tableName)throws IOExceptioninit()；TableName table

13、name = TableName.valueOf(tableName);admin.disableTable(tablename);admin.deleteTable(tablename);HTableDescriptor hTableDescriptor = new HTableDescriptor(tableName);admin.createTable(hTableDescriptor);close();(5)统计表的行数。Shell:count 's1'hbase(nin) :021 ： count 飞 卓 row(s) in 6,03日日 seconds-* 3图6统

14、计表的行数编程：/(5)统计表的行数public static void countRows(String tableName)throws IOExceptioninit();Table table = connection.getTable(TableName.valueOf(tableName);Scan scan = new Scan();ResultScanner scanner = table.getScanner(scan);int num = 0;for (Result result = scanner.next();result!=null;result=scanner.ne

15、xt()num+;System.out.println("行数:"+ num);scanner.close();close();2.现有以下关系型数据库中的表和数据，要求将其转换为适合于HBase存储的表并插入数据：学生表（Student）学号(S_No)姓名(S_Name)2015001Zhangsan2015003Mary2015003LisiT生另 ij ( S_Sex)年龄(S_Age)male23female22male24课程表(Course)课程号（C_No）课程名（C_Name）学分(C_Credit)123001Math2.0123002Computer

16、Science5.0123003English3.0选课表（SC）学号（SC_Sno）课# ( SC_Cno)成绩(SC_Score)201500112300186201500112300369201500212300277201500212300399201500312300198201500312300295学生Student表p.s：主键的列名是随机分配的，因此无需创建主键列创建表：create 'Student','S_No','S_Name','S_Sex','S_Age'hbase(nain):crea

17、te "Student't 'S_No' i rS_Name1t * S_Sex', 'S_Age 0 row(s) in Ii3S6Q seconds=> ;Tble - student hbaqg(rwiln):list TABLE Student si2 on(s) in 0.0130 seconds=> "Student11, hbase(main) :024:0s* scan , student * noucolumn+cell6 row(5) in O.SllO seconds图7创建Student表插入数据插

18、入数据shell命令第一行数据put 'Student','s001','S_No','2015001'put 'Student','s001','S_Name','Zhangsan'put 'Student','s001','S_Sex','male'put 'Student','s001','S_Age','23'第二行数据put 

19、9;Student','s002','S_No','2015002'put 'Student','s002','S_Name','Mary'put 'Student','s002','S_Sex','female'put 'Student','s002','S_Age','22'第三行数据put 'Student','s003

20、','S_No','2015003'put 'Student','s003','S_Name','Lisi'put 'Student','s003','S_Sex','male'put 'Student','s003','S_Age','24'hbase(niatn) :326 : 0 row(s) in 3.6090hba£g(nain):日N7：0 r

21、ow(s) tn &.818Qhbdse(main);028;0> e row(s) tnhbdse(natn):O2916>0 row(s) in Q>608OhbaseCnaifi) ：9de：e ROUsoai5001SO01sOSl1 row(s) tn 3.6310put student(,1s091'f S_Nof,b2615961 seconds put T student' / sB01'f 's_NArp't 'Ehmngsan seconds put ' Studert 'J sOOl

22、' J S_Sex' p 'naleSPtond put 'Student',150611,SAge','23' seconds scan 'student'COLUhM+CELLcclunn=S Age:, tinestanp=14O2236406536, velur=J3cclunn=5_Hane:, tinestanp_14B2230306213vaYue=Zhangsan colunn=s_mo: p ttniestamp=ifl82236339580, value=zai5fiBi colunn=S_Se

23、x:t tinestamp=lB22363S564L, value=maleseconds图8添加数据并查看hb3se(nain):639:6>scan * Student*ROWCCLUMrt+CELLS001£口1111，m=3_含9尸：11|*1PSt48?2364BG35, VCl j?5S物colunFn-S_Nare: p ttnesta»ip-148223636&213, vdlue-Zhangsan写彻colurn-sluo:, tincstanp-H8223S339580, value-2015001S001colurn-S Sex:, ti

24、nestcnp-14B22363B50411 volue-naleSO02column=S_Age:( tlne5tanp=1482230S51Z74p Vdlue=22腔colunnsS Nar'e, ttniestanp=i482236522Zi2, value=Mar/械2coluniS Not, tinestarip=lS22154eee51, value=2&15902SO02colun-S Sex:, tine515-1482236504719, vdluc-fcnaLcS08column-S_Age：, tlmestanp-1482236672914, value

25、-24s血column=S_Mare: , tInes13148223(162145©, value=LtslS693eluEn=0二Mo tinestanp822366e7e7 2, value2&lSfl93SQ93colurn=S_Sex: t ti5tanp=；H822366383Bl, value=nale3 row(s) in 0.0520seconds图9添加3个学生课程Course表创建表：create 'Course','C_No','C_Name','C_Credit'hbase(nain):

26、049：e> create 'Course','C_No't1CNane','CCredit e row(s) in 1.269(3 seconds=> Mbase:iTiblt * Course图10创建Course表插入数据:插入数据shell命令第一行数据put 'Course','c001','C_No','123001'put 'Course','c001','C_Name','Math'put &

27、#39;Course','c001','C_Credit','2.0'第二行数据put 'Course','c002','C_No','123002'put 'Course','c002','C_Name','Computer'put 'Course','c002','C_Credit','5.0'第三行数据put 'Course',

28、'c003','C_No','123003'put 'Course','c003','C_Name','English'put 'Course','c003','C_Credit','3.0'=> Hbase：:Table - hbase(nain):941:0>0 row(s) in B.G15。Coursehbise(matn>:042i0>0W(S) In 0,0130hb3，u(fnbtn

29、"84T: 6% G row(s) inhbase(natn):056:G row(s) in 9.610hbasefmain):957;0>0 国(s) tn 9.6050hbose(ncin):。58:0”0 row(s) in 0»C090hbdse(nain):9£9: ROWcOOl C001 C091C002 c002 C033C0933 row(s) tn 9,G500put 1CQjr5er/c&01, /C_No' / 123031, seconds put 'Course*, 'c&Dl',

30、 'CJaHe1, 'Hath seconds put 'course 111ceoiH,'c Credtt1,rz.o seconds图11添加数据put b Course+,'COBS' ,dC_NOd ,* 125393( seconds put TCoi|jr5eT ,1 cBB，' JC_Name' p ' English , spcnr.rl put bourse1 /c003' /C Credit' , r 3,0, seconds scan 'Course' COLUMNiC

31、ELL colunn-C_Credit:, ttnestanp-1432Z3<?833022, value-2.0 colunn=C_hdnc;, ttrtest&mp=148236818175, value=Mdth colunn=c_No： , ttmestai<ip=W8223680e249r valuei23eei colunn=CCredtti, tine£tanp=143223767056 value=E,0 colunn=CNane:f tihestanp=1462237054701r value=Conputer cclunn=C_ho; , ti

32、nestdmp=l£182Z37042433. tfdlue=lZ3e02 colunn=C_C redttz: # ttnetanip=148223718t 559, value=3.9 colunn=C_Nane : t tinestamp=14S2237167697J value=Engli.sh calunn=C-No： , ttmestaFip=ne2237156637, valuc=123e03 seconds图12添加3个课程 选课表创建表：create 'SC','SC_Sno','SC_Cno','SC_Sco

33、re'hbase(main);d> create 1 SC 1 j'SC_Sno111$C_Cno','SC_Score 0 row(s) tn 1.2790 seconds=> Hbase:Table - SC图13创建表SC插入数据:插入数据shell命令第一行数据put 'SC','sc001','SC_Sno','2015001'put 'SC','sc001','SC_Cno','123001'put 'S

34、C','sc001','SC_Score','86'第二行数据put 'SC','sc002','SC_Sno','2015001'put 'SC','sc002','SC_Cno','123003'put 'SC','sc002','SC_Score','69'第三行数据put 'SC','sc003','S

35、C_Sno','2015002'put 'SC','sc003','SC_Cno','123002'put 'SC','sc003','SC_Score',77'第四行数据put 'SC','sc004','SC_Sno','2015002'put 'SC','sc004','SC_Cno','123003'put '

36、SC','sc004','SC_Score','99'第五行数据put 'SC','sc005','SC_Sno','2015003'put 'SC','sc005','SC_Cno','123001'put 'SC','sc005','SC_Score','98'第六行数据put 'SC','sc006','

37、SC_Sno','2015003'put 'SC','sc006','SC_Cno','123002'put 'SC','sc006','SC_Score','95'hbase(natn);0OO；O> create 'SC ,1SC_Sno1'5C_Cno11SC.Score o raw(sJ tn 1*77en seconds => Hbasc:tTable - £Chbdse(ndin):Ol:O&g

38、t; put hSC' /scOOlr / 5C_Snoh /201&0011Gin O*GIZO secondshbase(nain)r0b2: G row(£) in Q.©ISOput，SC( /SC901 'J SC_tno' /12300L secondshb己5U(noin);003：。7put 'SC' F1scOOlr(F SC_5corer,1B6 o rox(s) tn 0.0290 seconds图14插入数据hbase(rnatn) 1012; a> can ' SC1nowcolumn+

39、cell5C0O1colvnn=5C Cno: , tiFiestaiIip=：1432237297903 , valuc=1230Olst。cQlurin=SC_5cQre; tinestaFip=14S22373133941 vaue-0&scsejicolumn=c_snn：, t(nesrAnp=i4H27377S5573, ulue=2ei506isc502cclunn-Sclcno:, tinestanp-1492237433209, value=12303“时2colunn-SC_Scorc:, ti.F>estOHp-l/182237H3270( «lu

40、e-69scODZcolunn-sclsno;,tiriestdnpL4B22374Z2O49 Rvdlue=20150ClSC&D3Colunn=5C_Cno；,tlMB$SFip=1482237492242,vdluf=123O02SCS33colwnn=SC Score:ttiwstanp=1482237565037F value=r7sc&95ccluftn=£Csno:,timestanp-l4a2237475626fvalue-2615062±cB84tQlunn=SC_Cno:,tinrstarip=L482238002fiE3,vaLu

41、63;=1230O3sc004colunn=SC_Score: t tinestdnp-L48223S015244. v<alue=99$CBS4匚0111田门=三匚_三1口：, tlriestaFp=1482237753343 , value = 2015BB2scftOcolumn=SC_Cno:, tinestanp=149223fia9903value-12J001scODScolunn=5C Score; t ttr)estaFip=14e22381055031 V31uc=?3scSOScolunn=5cl5no;( tinestanp=1482Z3®061590,

42、 value=20150e3SC006coldMn=sc_cno:, Hnstaf(ip=i4ft2?38i49ii821 valufsi?ieo?SC0D6cclumn=£C_Score: F ttir>estaFip=14fi223ei5979if value-SS“目gcolunn-SC Srio:, tinestoi*ip-14B22381J7G94, voluc-ieiS0G36 。N(5)tn O.Q510 seconds图15数据显示Summary：Tdbl* hbdw.mvtd k ukayNumber Df regKjns! 1DtpluyM 斑 uU un

43、l Uj33307r 1 dS2233950M57>blf Student is skoy.Number *r晒ons: 1un: uLunc l>1307, 14B22339S0K45Tble M b oy.Numb4>f oh地垢rH! 1Deployed on: ubuncUhB%?,1482JJdSOtMbT曲世 U kokdy.Numbtrof rgian5： 1IX ploy KI on： ub un C u333D7r)4822339Mfr45Tdbk tuck 也 uk.Number of regtons: 1Dvpluy wj on： uLun 133307

44、,1 d<2233»50Ul>hle hbB*：MEHw” 片中心YUBaseFsck command line options:Versor: 1.Z2N .imber o( Ihe region servers: 1N jmaer of dead regie n 5erjer5； 0Master ubuntu 4431umnpr nf fi；waip marprs' oArrigp tnriii 7.0N jmber of reqi>«t: 0Njmbcrof rsgiart: 7Nunbhf »f ipgi

45、E：Fis mi，411*1心”：oNumber &f empty REG NNFO_QUALIFIER rows in htusojnet- 0Number afrabiK： bMLimb&r of regions： 1l>电 ployed on: ubu nc j,JJj04 l JizjjybOHdbTable Course 7 uhy.Number of regions: 1Do ployed on: ubu ni u J3307,14322339S0M5 0 ncor)sK«nrie£ detKiMl.Sins： ok图16 Question

46、One运行后控制台消息同时，请编程完成以下指定功能：(完整可执行代码见代码/QuestionTwo.java )(1) createTable(String tableName, String fields)创建表，参数tableName为表的名称，字符串数组fields为存储记录各个域名称的数组。要求当HBase已经存在名为tableName的表的时候，先删除原有的表，然后再创建新的表。代码: public static void createTable(String tableName,String口 fields) throws lOException init()；TableName

47、tablename = TableName.valueOf(tableName);if(admin.tableExists(tablename)System.out.println("table is exists!");admin.disableTable(tablename);admin.deleteTable(tablename);/ 删除原来的表HTableDescriptor hTableDescriptor = new HTableDescriptor(tablename);for(String str:fields)HColumnDescriptor hCol

48、umnDescriptor = new HColumnDescriptor(str);hTableDescriptor.addFamily(hColumnDescriptor);admin.createTable(hTableDescriptor);close();(2) addRecord(String tableName, String row, String fields, String口 values)向表tableName、行row (用S_Name表示)和字符串数组 files指定的单元格中添加对 应的数据values。其中fields中每个元素如果对应的列族下还有相应的列限定符的

49、话，用“columnFamily:column 表示。例如，同时向 “ Math"、 "Computer Science"、 "English 三列添加成绩时，字符串数组 fields 为 "Score:Math" ," Score; Computer Science"," Score:English” ,数组values存储这三门课的成绩。代码：public static void addRecord(String tableName,String row,String fields,String va

50、lues) throws IOException init();Table table = connection.getTable(TableName.valueOf(tableName);for(int i = 0;i != fields.length;i+)Put put = new Put(row.getBytes();String口 cols = fieldsi.split(":");put.addColumn(cols0.getBytes(), cols1.getBytes(), valuesi.getBytes();table.put(put);table.cl

51、ose();close();(3) scanColumn(String tableName, String column)浏览表tableName某一列的数据，如果某一行记录中该列数据不存在，则返回 null。要 求当参数column为某一列族名称时，如果底下有若干个列限定符，则要列出每个列限定符代表的列的数据；当参数 column为某一列具体名称(例如 “Score:Math)时，只需要列出 该列的数据。代码：public static void scanColumn(String tableName,String column)throws IOException init();Table

52、 table = connection.getTable(TableName.valueOf(tableName);Scan scan = new Scan();scan.addFamily(Bytes.toBytes(column);ResultScanner scanner = table.getScanner(scan);for (Result result = scanner.next(); result != null; result = scanner.next() showCell(result);table.close();close();/格式化输出public static

53、 void showCell(Result result)Cell cells = result.rawCells();for(Cell cell:cells)System.out.println("RowName:"+new String(CellUtil.cloneRow(cell)+"");System.out.println("Timetamp:"+cell.getTimestamp()+"");System.out.println("column Family:"+new String

54、(CellUtil.cloneFamily(cell)+"");System.out.println("row Name:"+new String(CellUtil.cloneQualifier(cell)+"");System.out.println("value:"+new String(CellUtil.cloneValue(cell)+"");(4) modifyData(String tableName, String row, String column)修改表tableName,行

55、row (可以用学生姓名 S_Name表示),歹U column指定的单元格 的数据。代码：public static void modifyData(String tableName,String row,String column,String val)throwsIOExceptioninit();Table table = connection.getTable(TableName.valueOf(tableName);Put put = new Put(row.getBytes();put.addColumn(column.getBytes(),null,val.getBytes()

56、;table.put(put);table.close();close(); deleteRow(String tableName, String row)删除表tableName中row指定的行的记录。public static void deleteRow(String tableName,String row)throws IOExceptioninit();Table table = connection.getTable(TableName.valueOf(tableName);Delete delete = new Delete(row.getBytes();删除指定列族delet

57、e.addFamily(Bytes.toBytes(colFamily);删除指定列/delete.addColumn(Bytes.toBytes(colFamily),Bytes.toBytes(col);table.delete(delete);table.close();close();幺 Probl flasks Java Map/ cons S3 ° D一 一1.一一 一 一 一 .<teminated>QuestionTwo fJava Application /usr/lib/jvm/java画- 曲向回J 士里B -log4j:WARN No appenders could bw found for logger org.apache.hado log4j:WARN Please inhialize (he Iog4j system properly.lag4j:WARI| See