试验4熟悉常用的HBase操作

1、实验4熟悉常用的HBase操作姓名：包生友 专业年级：软件143学号：6991. 实验目的1. 理解HBase在Hadoop体系结构中的角色；2. 熟练使用HBase操作常用的Shell命令；3. 熟悉HBase操作常用的Java API。2. 实验环境操作系统：Lin uxHadoop版本：2.6.0或以上版本HBase版本：1.1.2或以上版本JDK版本：1.6或以上版本Java IDE: Eclipse3. 实验内容和完成情况1.编程实现以下指定功能，并用Hadoop提供的HBase Shell命令完成相同任务:(完整可执行代码见代码/QuestionOne.java)(1) 列出HBa

2、se所有的表的相关信息，例如表名；Shell:ListhbaseCinatn)ListTABLE511 rowCs) tn &.seconds-【" 图1列岀HBase所有表的相关信息编程：列出HBase所有的表的相关信息，例如表名、创建时间等public static void listTables() throws IOException init();建立连接HTableDescriptor hTableDescriptors = adm in.li stTables();for(HTableDescriptor hTableDescriptor :hTableDesc

3、riptors)System.out.println(” 表名:"+hTableDescriptor.getNameAsString();close();关闭连接(2) 在终端打印出指定的表的所有记录数据；Shell:scan 's1':e> scan 1ROWCOLUMN+CEILLa row(£) tn 0.1380 seconds图2打印指定表的所有记录数据编程：(2)在终端打印出指定的表的所有记录数据public static void getData(Stri ng tableName)throwsIOExcepti onini t();Ta

4、ble table = conn ecti on. getTable(TableName.valueOf(tableName);Scan sca n = new Scan();ResultSca nner sca nner = table.getSca nn er(sca n);for (Result result:sca nner)prin tRecoder(result);close();/打印一条记录的详情public static void printRecoder(Result result)throws IOExceptionfor(Cell cell:result.rawCell

5、s()System.out.print("行健:"+new String(CellUtil.cloneRow(cell);System.out.print("列簇:"+new String(CellUtil.cloneFamily(cell);System.out.print("列:"+new String(CellUtil.cloneQualifier(cell);System.out.print("值:"+new String(CellUtil.cloneValue(cell);System.out.print

6、ln(” 时间戳:"+cell.getTimestamp();(3) 向已经创建好的表添加和删除指定的列族或列；p.s :此题请先在 Shell中创建si作为示例表：create 's1','score'a)在si表,添加数据：Shell:put 'si','zha ngsa n','score:Math','69'hbase(nain):933pjt 1 si1 / zhangsan',1 score:Math r *T69' 0 rt>w(s) tn 9,0609

7、 secondshba5e(nein);&D4IsNaneError; undefined local varxble or nethod 'Xs' f&r jecthbase(nalft):&05：0> ListtableSI1 row(sJ i.n 6516 seconds图3给si添加数据编程：/向表添加数据public static void insertRow(String tableName,String rowKey,String colFamily,String col,String val) throws lOException

8、ini t();Table table = conn ecti on. getTable(TableName.valueOf(tableName);Put put = new Put(rowKey.getBytes();put.addColu mn( colFamily.getBytes(), col.getBytes(), val.getBytes();table.put(put);table.close();close();in sertRow("s1",'zha ngsa n','score','Math','6

9、9')b)在si表，删除指定的列：Shell:delete 's1','zha ngsa n','score:Math':011:6 delete H«11'2hangan't score:Math row(s) in ft.9070 seconds图4删除数据编程：删除数据public static void deleteRow(Stri ng tableName,Stri ng rowKey,Stri ng colFamily,Stri ng col) throws lOExcepti on ini t();

10、Table table = conn ecti on. getTable(TableName.valueOf(tableName);Delete delete = new Delete(rowKey.getBytes();删除指定列族delete.addFamily(Bytes.toBytes(colFamily);删除指定列delete.addColu mn( Bytes.toBytes(colFamily),Bytes.toBytes(col);table.delete(delete);table.close();close();deleteRow("s1",'

11、zha ngsa n','score','Math')(4) 清空指定的表的所有记录数据；Shell:trun cate 's1hbasefnatn) ：BJ8:0-> RONLisiwanghu2tn o.ezin1 si1COLUMN+CELLCQlunn=5core:Mfltht tiricstmnpul斗822弓5后 13818 value=S5 colunn-score；r*eth* ttmesta，Iipl482Z35732643, vue-75hba»(natn)19191truncate 1sl'Trunc

12、ating * si1 tabic (it may tike a while): * Disabling t曲blc*Truncdtlny table*-,0 row(s) tn 3.7870 secondssen ' si'COLUMN+CELL seconrlhbdse(main):026:O>ROWfl row(s tn图5清空指定表的所有记录数据编程：(4)清空指定的表的所有记录数据public static void clearRows(String tableName)throws lOExceptionini t();TableName table name

13、 = TableName.valueOf(tableName);adm in. disableTable(table name);admi n.deleteTable(table name);HTableDescriptor hTableDescriptor = new HTableDescriptor(tableName);adm in. createTable(hTableDescriptor);close();(5 )统计表的行数。Shell:coun t 's1hbase(ndin):021：count *sl 炒 row(s) in 0,0300 seconds“ 0图6统计

14、表的行数编程：(5)统计表的行数public static void countRows(String tableName)throws IOExceptionini t();Table table = conn ectio n.getTable(TableName.valueOf(tableName);Scan sca n = new Scan();ResultSca nner sca nner = table.getSca nn er(sca n);int num = 0;for (Result result = sca nner.n ext();result!=nu ll;result=

15、sca nner.n ext()nu m+;System.out.println（"行数:"+ num）;sea nn er.close（）;close（）;2. 现有以下关系型数据库中的表和数据，要求将其转换为适合于HBase存储的表并插入数据：学生表（Student）学号（S_No）姓名（S_Name）性别（S_Sex）年龄（S_Age）Zhangsanmale23Maryfemale22Lisimale24课程表（Course）课程号（C No）课程名（C Name）学分（C Credit）123001Math2.0123002Computer Science5.01

16、23003En glish3.0选课表（SC）学号（SC_Sno）课程号（SC_Cno）成绩（SC_Score）123001861230036912300277123003991230019812300295学生Student表p.s：主键的列名是随机分配的，因此无需创建主键列创建表：create 'Stude nt','S_No','S_Name','S_Sex','S_Age'hbase(main) : O22r0> create 1 Student' , ' S_No' J S_

17、Narne' t 1' t ' S AgeG row(s) in 1*3S60 seconds=> Hb»E；:T©blu - studenthbAsetrwiln) :o?3：0> listTABLEStudentsi2 roM(s) tfi 0.0130 seconds二Student1*, 怙丁 hba5e(main) :024:0s* scan 1 Student * noucolumn+cellG row(5) in 0.9110 seconds图7创建Student表插入数据插入数据shell命令第一行数据put 'S

18、tude nt','s001','S_No',”put 'Stude nt','s001','S_Name','Zha ngsa n'put 'Stude nt','s001','S_Sex','male'put 'Stude nt','s001','S_Age','23'第二行数据put 'Stude nt','s002','

19、;S_No',”put 'Stude nt','s002','S_Name','Mary'put 'Stude nt','s002','S_Sex','female'put 'Stude nt','s002','S_Age','22'第三行数据put 'Stude nt','s003','S_No',”put 'Stude nt',&

20、#39;s003','S_Name','Lisi'put 'Stude nt','s003','S_Sex','male'put 'Stude nt','s003','S_Age','24'hbase(nain)put h Student(,1sD61'f S_Mor,b 2015961 6 row(s) in 3.6090 secondshba(niain)：o770 nw(s) tn B.61B6pur Tstude

21、nt'1saai' seccn<J$hbase(nain);0Z8;0> e row(s tn a.160hbse(natn):029:0> 0 row(s) tn 0.6080hbasetnatfi) ?9J6：0iROWsoai5601SO01sOSl1 row(s) in 3.6310put * Student'JsDOl'* p5_5ex','nale seconds put Student', '5031' /S_Age' /231 seconds scan 'student1C

22、OLUhM+CELLcclunn=S Age:t tinestanp=14fl2236406336, valur=J3 cclunn=5_hane:p tinestanp=1462230306213 value=Zhangsan colunn=s_mo:, ttnestampiBZJ36339580, value=zai5fiBi colunn=S_Sex:t tinestamp=14B22363S5641, value=nale seconds图8添加数据并查看hbase(nin):039iscan * Student*ROMCCLUMN+CELLCOlllF± 11meSTanp

23、=l48?23640G36, VA1 J&=?soeicolunFn-S_Nare:, ti.nesta<ip-1482236366213 , vlue-Zhangsan5051colurn=rsZNo:, tinc5tanp-18223S339580, volue-20150513001colurn-S Sex:T tincstcnp=lB22363B5Q41, volue-naleS002coLumn=S_Age: ( tline5tanp=1482230551Z74p value=22SO02colurrn=s_N3re, ttniesrafip=i斗3223旨522212

24、, value=Nar/£M2coLufrn=r£_No:, tinestarip=lS22364eeS51, vaLue=2G15O02column-5 Sex:, tSc航跡斗322弓65。斗718 vdlue-fcnaLesealcolumn-S_Age：, tlmestanp-1482236672914, value-24S003匸口5阿=£-脱呢：，ttnestanp=148223C62145OJI value=Ltsts693columnS1*51 > tinestanp822366e7e7 2, value2&15，fl93S093co

25、lumn=S_Scx： , time5tanp=H822166383Sl, valje=nale3 rw(3)in G.OSZaseconds图9添加3个学生课程Course表创建表：create 'Course','C_No','C_Name','C_Credit'hbaseCmain):O40iQ> create 'Course'»“C_No'11C Nane'f 'C_Credit 0 rows) in 1.3690 seconds" Hbase:Tblf -

26、 Course图10创建Course表插入数据:插入数据shell命令第一行数据put 'Course','c001','C_No','123001'put 'Course','c001','C_Name','Math'put 'Course','c001','C_Credit','2.0'第二行数据put 'Course','c002','C_No',&#

27、39;123002'put 'Course','c002','C_Name',' Computer'put'Course','c002','C_Credit','5.0'第三行数据put 'Course','c003','C_No','123003'put 'Course','c003','C_Name',' English 'put

28、 'Course','c003','C_Credit','3.0'=> Hbase:Tble - Coursehbase(Hain):941:0> put bourse1 #'ceoi1 (t'123001 0 row(s tn 9,Gl&0 secondshbise(matn);042;0>a rowts) tn 0,0150tn) J943: 0、0 row(s) tn &,e&80put Xourse1 /c&Ol' /C_Nane 'Math

29、 seconds put 'course 1 'cooiH /c credtt1， 2.0 seconds图11添加数据hbase(natn)：056!o>put *toure*cobs*."c_no,HI23ae3'G row(s) in 9,610secondshbae(nain) ;t>57; 0>put 'Course' *'*1CName',1 English'fl row(s 1n a+B65DserondhbaseCnoin):O5G:0>put bourse* /C0031 /C_

30、Credit'/3.O*0 row(s) in 0»C090secondshbase(nain):O59:scan 'Course'ROMCOLUMNiCELLcOOlcolunn-C_Credtt: t tinestanp=143223<j8330Z2, value-2.0cOOlC01Dfln=C_Napie; p ttwestamp= 148236818175, value=MathC091colunn=c_No: ( ttmestamp=iJq822368e6ii49, valuc=L23eaic6B2colunnCCredit:, tinest

31、anp=14a2237671S&6, value=5,Gc002colunnC hcnc:, tinestamp=14S2237GS47S1r value=COflputerc002cclunn=C_No: , tinestaFip=1162237&424S3, value=LZ3O02C003colunn=C_CredIt: # ttnestanip=lflr322 571Sl559, valued.0c093colunn-C_Nane : t tine£tamp=14S2237167697f value=Engli.shC093colunn=C_No： . ttm

32、estaFip=H&223715e637, valuc=123e033 ruw(s) in 0*0500seconds图12添加3个课程选课表创建表：create 'SC','SC_S no ','SC_C no ','SC_Score'hbase(nain):G66;0> create 1 SC ,'SC_Sno111SC_Cno','SC_Score 0 row(s) tn 1.2790 seconds=> Hbase:Table - SC图13创建表SC插入数据:插入数据shell

33、命令第一行数据put'SC','sc001','SC_S no',”put 'SC','sc001','SC_C no','123001'put 'SC','sc001','SC_Score','86'第二行数据put 'SC','sc002','SC_S no',”put 'SC','sc002','SC_C no',

34、9;123003'put 'SC','sc002','SC_Score','69'第三行数据put 'SC','sc003','SC_S no',”put 'SC','sc003','SC_C no','123002'put 'SC','sc003','SC_Score','77'第四行数据put 'SC','sc004

35、9;,'SC_S no',”put 'SC','sc004','SC_C no','123003'put 'SC','sc004','SC_Score','99'第五行数据put 'SC','sc005','SC_S no',”put 'SC','sc005','SC_C no','123001'put 'SC','s

36、c005','SC_Score','98'第六行数据put 'SC','sc006','SC_S no',”put 'SC','sc006','SC_C no','123002'put 'SC','sc006','SC_Score','95'hbase(natn);0OO；O> create rSC',rSC_5no',SCjCno'f *5C_Sco

37、re o raw(s tn 1 ,?7en seconris=> Hbasc:tTable - SChb»e(natn>：e61> put 15C,1 scBM' x rSC_5nor f r2fll5091T O rin 0 *©120 secondshbase(niin)r0b2: G r ow(£) in Q,6160put SC( /SC901 * i SC_tno' /12300L secondshbaie(F«in);0C3;0> put '5C' F1scOOl1('SC.Sc

38、orer,1B6 o ro»(s) In fi.0290 seconds图14插入数据hba&e(matn)：O12:0> ROU5C001atOOl5caniSCSB2SC&02C0D2SC 003SC033sc&OaS&043C5D4SC 8 343C0&SKOOSSC0066 row(sin O.C510can 'sc1COLUMN+CELLcolunn=SC Cno:, tinestaFpl432237297903, value=l23O01 colurin=SC_5cQre; tine5tflFip=1482237313

39、394, value=BG calumn=S匚-Eno: , tlmestiinip=14H77377H5573 , vlue=7ei506 c©Iunn-SC_Cno: , tinestanp-?1492237433269, value-12593 colunn-SC Scorc:, tXF>e5torip-H82237H3270, voluc-69 colunn-SCIsno:, tinestanp-14B22374Z2O49, Vdlue-Z0150Cl coldflr)=sc_cno； F Ctnestanp=1482237492242 # vdlue=l?3O02 c

40、olumnscscore:, ttmestanp=i48223S0be37r “alu电二TF cclurtri=£C_£no： , tinestanp-14B223747S26, value=615062 colunn=SC_ChO：, tinrstanp=L482238&G2flS3, vaLu«=123963 colunn=SC_Score: t tinestdFip=1482233015244, v1ue=99 column=sc_sno: , tirie&taFp=i482237753343, value=zei5BB2 column=S

41、C_Cno: , tineitaFip=14B223SE)99e2 V3Lue=123301 column=SC core: , tir)estaFip=14S223810990gt V01ue=9B colunn-SC Who: ti.riestanp-14B22380Q159Ovalue=Z015003 c ol iimn=sc_cno：, t ti'iPStanp=14B2?814918； f wslusl? colunn=£C_Score:F tinestanp=14822281S9793r value=95 colunn-SCSno:, tinestaFip-L43

42、2238157e94 j valuc-2eiS6e3 seconds图15数据显示Summary:Tdbl* hbdsr.mld kNumber of regions! 1Epi邯创 oik uUuiiId,3i3O7r 1 d>2233930M5 ThJf Stu c*t nr isokay.Numbnr of cgtons! 1Dvplid an-.讪11皿耳站307482铀菊丈1&45 旳城就n otiay.itlumtef of r测qx 1Deployed on： unuiniuj3JD?P 14S?2JJ9WB46Ta Na &1 h uky.Number o

43、f ff ginn! 1IXployed OH： UHunCu333O7r)4822J3950M5 Tbl» twok Hoh 曲.Number of fDgtons: 1Wpluyedon： uUunLjlW?, 1 d<2233950U T>h甘 hhw:ru wpflf R QkyUBaseFsck command line optionsVersiar: 1.Z2Number of ir region sp(wrs- iN jmlbEr of d?ad regie n servers; DMaster： ubj nt u 44-! 31u

44、mhpr nf harkiip imarprs' aArrigp 7.0N jmocr of reqxst,: 0Number of r«giant: 7n jnbhf »f i ipgii ns m v <tns4仍q： rNumber of &mpty REGQNNFO_QLALIFIER ro1 in bitu强:met凯 0Number af fa bl«: bNLimb&r of regtons： 1De ployed on: ubu nc u, J J ju< lj vboH45Table Course?恪 ukay

45、.Number of rtgions: 1Do ployed on: ubu nr UJ33OZ14B2233950&45 0 ncor)sH«n:ie£ detKiMl.Sw* Ok图16 QuestionOne运行后控制台消息同时，请编程完成以下指定功能：(完整可执行代码见代码/QuestionTwo.java )(1) createTable(String tableName, String fields)创建表，参数tableName为表的名称，字符串数组fields为存储记录各个域名称的数组。要求当HBase已经存在名为tableName的表的时候，先删除

46、原有的表，然后再创建新的表。代码: public static void createTable(String tableName,String fields) throws IOException init();TableName tablename = TableName.valueOf(tableName);if(admin.tableExists(tablename)System.out.println("table is exists!");admin.disableTable(tablename);admin.deleteTable(tablename);/ 删

47、除原来的表HTableDescriptor hTableDescriptor = new HTableDescriptor(tablename);for(String str:fields)HColumnDescriptor hColumnDescriptor = new HColumnDescriptor(str); hTableDescriptor.addFamily(hColumnDescriptor);admin.createTable(hTableDescriptor);close();(2) addRecord(String tableName, String row, Strin

48、g fields, String values)向表tableName、行row (用S_Name表示)和字符串数组 files指定的单元格中添加对应的数据values。其中fields中每个元素如果对应的列族下还有相应的列限定符的话，用“ columnFamily:column 表示”。例如，同时向“ Math”、“ Computer Science”、“ English 三列添加成绩时，字符串数组 fields 为 “Score:Math ”,”Score；Computer Science”,”Score:English ” ，数组values存储这三门课的成绩。代码：public sta

49、tic void addRecord(String tableName,String row,String fields,String values) throwsIOException init();Table table = connection.getTable(TableName.valueOf(tableName);for(int i = 0;i != fields.length;i+)Put put = new Put(row.getBytes();String cols = fieldsi.split(":");put.addColumn(cols0.getB

50、ytes(), cols1.getBytes(), valuesi.getBytes(); table.put(put);table.close();close();(3) scanColumn(String tableName, String column)浏览表 tableName 某一列的数据，如果某一行记录中该列数据不存在，则返回null 。要求当参数 column 为某一列族名称时，如果底下有若干个列限定符，则要列出每个列限定符 代表的列的数据；当参数 column 为某一列具体名称(例如 “Score:Math )”时，只需要列出 该列的数据。代码： public static v

51、oid scanColumn(String tableName,String column)throws IOExceptioninit();Table table = connection.getTable(TableName.valueOf(tableName);Scan scan = new Scan();scan.addFamily(Bytes.toBytes(column);ResultScanner scanner = table.getScanner(scan);for (Result result = scanner.next(); result != null; result

52、 = scanner.next() showCell(result);table.close();close(); /格式化输出 public static void showCell(Result result)Cell cells = result.rawCells();for(Cell cell:cells)System.out.println("RowName:"+new String(CellUtil.cloneRow(cell)+" ");System.out.println("Timetamp:"+cell.getTim

53、estamp()+" ");System.out.println("column Family:"+new String(CellUtil.cloneFamily(cell)+" ");System.out.println("row Name:"+new String(CellUtil.cloneQualifier(cell)+" ");System.out.println("value:"+new String(CellUtil.cloneValue(cell)+"

54、; ");(4) modifyData(String tableName, String row, String column)修改表tableName，行row (可以用学生姓名 S_Name表示)，列column指定的单元格的数据。代码：public static void modifyData(String tableName,String row,String column,String val)throwsIOExceptioninit();Table table = connection.getTable(TableName.valueOf(tableName);Put

55、put = new Put(row.getBytes();put.addColumn(column.getBytes(),null,val.getBytes();table.put(put);table.close();close();( 5) deleteRow(String tableName, String row)删除表 tableName 中 row 指定的行的记录。public static void deleteRow(String tableName,String row)throws IOExceptioninit();Table table = connection.get

56、Table(TableName.valueOf(tableName);Delete delete = new Delete(row.getBytes();删除指定列族delete.addFamily(Bytes.toBytes(colFamily);删除指定列/delete.addColu mn (Bytes.toBytes(colFamily),Bytes.toBytes(col);table.delete(delete);table.close();close();£百f了f化 ProblTasks Javd3 Cons S3° <terminated>QuestionTwo fJava Application /usr/lib/jvm/javaX曙丨園朝區画兰冃审旁甲 log*lj:WARN No appenders could b found for Iogger org,apache.hado lag4j:WARN Please Initialize the k)g4| system properly, log4j:WARN See http:/logging, /lDg4|/1x2/faq»hirnl#noconfi