centos+raid0+lvm+iscsi配置详解

1、centos 5.2 + raid0+lvm+iscsi 配置详解zeng lingping http:/ 2008/12/06 page 1 of 12centos 5.2+raid 0+lvm+iscsi配置详解系统环境： centos 5.2 硬件配置： asus p5gc-mx +intel e2160+kinston 1gb 667*2+d-link 530 1gbps 硬盘 1：seagate 80gb/7200s/8mb/sata 硬盘 2：seagate 1tb/7200s/32mb/sata 硬盘 3：seagate 1tb/7200s/32mb/sata 场景：一台用于备份

2、文件的服务器目前空间不足（使用symantec backup exec 10d for windows servers备份软件，备份文件服务器的所有数据，保留15 天，每周六完整备份，周一至周五增量备份，现有2tb的空间已用完），该服务器已安装8 个（容量与规格不一，有scsi/ide/sata ）硬盘，机箱内已无空间再增加硬盘，经过考虑，决定重新安装一台linux服务器配置iscsi 服务（提供target server角色），在windows server 2003 上安装 microsoft iscsi initiator 驱动，添加iscsi硬盘，解决目前的空间问题；target se

3、rver 要求系统安装在st 80gb的硬盘上，两块1tb的硬盘配置为raid 0,然后在 raild 0 硬盘上配置 lvm，在 lvm 上创建两个逻辑卷，lv1（data1）为 1.5tb，lv2(data2)为 325gb；target server 建立两个 target device ，将 lvm 创建的两个逻辑卷分别加入两个target device,node1 连接到target device 1(iqn.2008-.lanexpert:iscsi-data1) ，node2 连接到 target device 2（iqn.2008-.lanexpert:iscsi-data2）

4、功能操作系统主机名称ip地址iscsi-target server centos 5.2 f 9 iscsi-initiator(node1) windows server 2003 std r2 with sp2 f 8 iscsi-initiator(node2) windows xp pro with sp3 c 08 target server 系统安装在 seagate 80gb的硬盘上，在安装系统时，根据以下要求分区（仅为个人习惯，供参考），其它两个1tb的硬盘暂时不分区；/boot 100mb / 10gb /u

5、sr 20gb /home 10gb /var 30gb swap 3gb 系统选择最小化安装，仅安装最基本的服务；一、 配置 raid 0 #fdisk -l ( 显示所有连接在主机上的硬盘) disk /dev/sda: 80.0 gb, 80025280000 bytes 255 heads, 63 sectors/track, 9729 cylinders units = cylinders of 16065 * 512 = 8225280 bytes device boot start end blocks id system /dev/sda1 * 1 13 104391 83 l

6、inux centos 5.2 + raid0+lvm+iscsi 配置详解zeng lingping http:/ 2008/12/06 page 2 of 12/dev/sda2 14 2624 20972857+ 83 linux /dev/sda3 2625 3929 10482412+ 83 linux /dev/sda4 3930 9729 46588500 5 extended /dev/sda5 3930 5234 10482381 83 linux /dev/sda6 5235 5708 3807373+ 82 linux swap / solaris /dev/sda7 5

7、709 9729 32298651 83 linux disk /dev/sdb: 1000.2 gb, 1000204886016 bytes 255 heads, 63 sectors/track, 121601 cylinders units = cylinders of 16065 * 512 = 8225280 bytes disk /dev/sdc: 1000.2 gb, 1000204886016 bytes 255 heads, 63 sectors/track, 121601 cylinders units = cylinders of 16065 * 512 = 82252

8、80 bytes 从以上信息可以看到，/dev/sda 连接的是我们的系统分区，其它两个硬盘（sdb and sdc）没有分区表；现在，我们要为另外两个硬盘分别创建分区，参考如下：#fdisk /dev/sdb the number of cylinders for this disk is set to 121601. there is nothing wrong with that, but this is larger than 1024, and could in certain setups cause problems with: 1) software that runs at

9、boot time (e.g., old versions of lilo) 2) booting and partitioning software from other oss (e.g., dos fdisk, os/2 fdisk) #进入 fdisk 程序的工作画面后，如果硬盘太大，就会出现上述信息，这个信息表明，某些旧版本的软件与操作系统将无法支持大于1024 柱面后的扇区使用command (m for help):m # 可以通过此命令查看所有参数command action a toggle a bootable flag b edit bsd disklabel c tog

10、gle the dos compatibility flag d delete a partition l list known partition types m print this menu n add a new partition o create a new empty dos partition table p print the partition table q quit without saving changes s create a new empty sun disklabel t change a partitions system id u change disp

11、lay/entry units v verify the partition table w write table to disk and exit x extra functionality (experts only) command (m for help): new command action e extended centos 5.2 + raid0+lvm+iscsi 配置详解zeng lingping http:/ 2008/12/06 page 3 of 12 p primary partition (1-4) ppartition number (1-4): 1first

12、 cylinder (1-121601, default 1): using default value 1 last cylinder or +size or +sizem or +sizek (1-121601, default 121601): using default value 121601 command (m for help): type selected partition 1 hex code (type l to list codes): fd changed system type of partition 1 to fd (linux raid autodetect

13、) command (m for help): write the partition table has been altered! calling ioctl() to re-read partition table. syncing disks. 重复以上命令对sdc进行分区；完毕后使用fdisk l 查看分区信息：#fdisk -l isk /dev/sda: 80.0 gb, 80025280000 bytes 255 heads, 63 sectors/track, 9729 cylinders units = cylinders of 16065 * 512 = 8225280

14、bytes device boot start end blocks id system /dev/sda1 * 1 13 104391 83 linux /dev/sda2 14 2624 20972857+ 83 linux /dev/sda3 2625 3929 10482412+ 83 linux /dev/sda4 3930 9729 46588500 5 extended /dev/sda5 3930 5234 10482381 83 linux /dev/sda6 5235 5708 3807373+ 82 linux swap / solaris /dev/sda7 5709

15、9729 32298651 83 linux disk /dev/sdb: 1000.2 gb, 1000204886016 bytes 255 heads, 63 sectors/track, 121601 cylinders units = cylinders of 16065 * 512 = 8225280 bytes device boot start end blocks id system /dev/sdb1 1 121601 976760001 fd linux raid autodetect disk /dev/sdc: 1000.2 gb, 1000204886016 byt

16、es 255 heads, 63 sectors/track, 121601 cylinders units = cylinders of 16065 * 512 = 8225280 bytes device boot start end blocks id system /dev/sdc1 1 121601 976760001 fd linux raid autodetect 配置 raid 使用 mdadm 命令，有关该命令的具体参数参考man mdadm; centos 5.2 + raid0+lvm+iscsi 配置详解zeng lingping http:/ 2008/12/06 p

17、age 4 of 12# mdadm -create /dev/md0 -level=0 -raid-devices=2 /dev/sdb1 /dev/sdc1 mdadm: array /dev/md0 started. # cat /proc/mdstat # 查看 raid信息personalities : raid0 md0 : active raid0 sdc11 sdb10 1953519872 blocks 64k chunks unused devices: # fdisk -l disk /dev/sda: 80.0 gb, 80025280000 bytes 255 hea

18、ds, 63 sectors/track, 9729 cylinders units = cylinders of 16065 * 512 = 8225280 bytes device boot start end blocks id system /dev/sda1 * 1 13 104391 83 linux /dev/sda2 14 2624 20972857+ 83 linux /dev/sda3 2625 3929 10482412+ 83 linux /dev/sda4 3930 9729 46588500 5 extended /dev/sda5 3930 5234 104823

19、81 83 linux /dev/sda6 5235 5708 3807373+ 82 linux swap / solaris /dev/sda7 5709 9729 32298651 83 linux disk /dev/sdb: 1000.2 gb, 1000204886016 bytes 255 heads, 63 sectors/track, 121601 cylinders units = cylinders of 16065 * 512 = 8225280 bytes device boot start end blocks id system /dev/sdb1 1 12160

20、1 976760001 fd linux raid autodetect disk /dev/sdc: 1000.2 gb, 1000204886016 bytes 255 heads, 63 sectors/track, 121601 cylinders units = cylinders of 16065 * 512 = 8225280 bytes device boot start end blocks id system /dev/sdc1 1 121601 976760001 fd linux raid autodetect disk /dev/md0: 2000.4 gb, 200

21、0404348928 bytes 2 heads, 4 sectors/track, 488379968 cylinders units = cylinders of 8 * 512 = 4096 bytes disk /dev/md0 doesnt contain a valid partition table # mdadm -detail -scan array /dev/md0 level=raid0 num-devices=2 uuid=bf5eb6f7:da2e7196:dc2b05b5:fa5903c1 centos 5.2 + raid0+lvm+iscsi 配置详解zeng

22、lingping http:/ 2008/12/06 page 5 of 12为了让系统开机启动时自动加载raid 信息，可以使用以下命令将配置信息写入/etc/mdadm.conf文件，该文件默认没有，可自行创建；#echo device /dev/sdb1 /dev/sdc1 /etc/mdadm.conf #mdadm detail scan /etc/mdadm.conf (- 为两个中短线 ) # cat /etc/mdadm.conf device /dev/sdb1 /dev/sdc1 array /dev/md0 level=raid0 num-devices=2 uuid=b

23、f5eb6f7:da2e7196:dc2b05b5:fa5903c1 二、 配置 lvm；要配置 lvm，可以按以下步骤进行：1.创建和初始化物理卷（physical volume），通过 pvcreate 建立 pv，即 pv 阶段；2.添加物理卷到卷组（volume group），使用 vgcreate 加入多个 pv 成为 vg，即 vg 阶段；3.在卷组上创建逻辑卷（logical volume），使用 lvcreate 划分 vg,成为一个或多个lv,即 lv 阶段；# pvcreate /dev/md0 # 创建 pv physical volume /dev/md0 succes

24、sfully created # pvscan pv /dev/md0 vg lvm1 lvm2 1.82 tb / 1.82 gb free total: 1 1.82 tb / in use: 1 1.82 tb / in no vg: 0 0 # vgcreate lvm1 /dev/md0 # 创建 vg volume group lvm1 successfully created # vgdisplay lvm1 - volume group - vg name lvm1 system id format lvm2 metadata areas 1 metadata sequence

25、 no 1 vg access read/write vg status resizable max lv 0 cur lv 0 open lv 0 max pv 0 cur pv 1 act pv 1 vg size 1.82 tb pe size 4.00 mb total pe 476933 alloc pe / size 0 / 0 free pe / size 476933 / 1.82 tb vg uuid 97fecv-lbfp-ob19-dfrf-wj7y-9kj9-9sikyg centos 5.2 + raid0+lvm+iscsi 配置详解zeng lingping ht

26、tp:/ 2008/12/06 page 6 of 12# lvcreate -l 1.5tb -n data1 lvm1 # 创建 lv /dev/cdrom: open failed: read-only file system logical volume data1 created # lvcreate -l 325gb -n data2 lvm1 # 创建 lv /dev/cdrom: open failed: read-only file system logical volume data2 created # lvscan # 查看 lv信息 active /dev/lvm1/

27、data1 1.50 tb inherit active /dev/lvm1/data2 325.00 gb inherit # pvscan # 再次查看 pv信息 pv /dev/md0 vg lvm1 lvm2 1.82 tb / 2.02 gb free total: 1 1.82 tb / in use: 1 1.82 tb / in no vg: 0 0 # vgdisplay lvm1 # 再次查看 vg信息 - volume group - vg name lvm1 system id format lvm2 metadata areas 1 metadata sequence

28、 no 5 vg access read/write vg status resizable max lv 0 cur lv 2 open lv 2 max pv 0 cur pv 1 act pv 1 vg size 1.82 tb pe size 4.00 mb total pe 476933 alloc pe / size 476416 / 1.82 tb free pe / size 517 / 2.02 gb vg uuid 97fecv-lbfp-ob19-dfrf-wj7y-9kj9-9sikyg 到这里，本实例的lvm配置完成；注意：由于要将lv 挂载到 iscsi，所以，在这

29、里不必格式化，也不必将lv挂载到系统中，以下是挂载方法，如应用于其它地方，可以参考；格式化逻辑卷#mke2fs -j /dev/lvm1/data1 #mke2fs -j /dev/lvm1/data2 参数说明：-j ：本来 mke2fs 是 ext2,加上 -j 后，会主动加入日志而成为ext3 centos 5.2 + raid0+lvm+iscsi 配置详解zeng lingping http:/ 2008/12/06 page 7 of 12#新增两个挂载点#mkdir -pv /data1 /data2 挂载命令，本例不需要将以下两个lv挂载到系统中；#mount /dev/lvm

30、1/data1 /data1 #mount /dev/lvm1/data2 /data2 设置开机自动挂载：编辑 /etc/fstab #raid 0 and lvm1 /dev/lvm1/data1 /data1 ext3 defaults 2 2 /dev/lvm1/data2 /data2 ext3 defaults 2 2 有关挂载参数说明，请参考鸟哥私房菜基础学习篇三、 配置 iscsi 1、 安装 iscsi-target，iscsi-target 可以从以下网址下载: #wget ftp:/ftp.muug.mb.ca/mirror/centos/5.2/os/i386/cent

31、os/scsi-target-utils-0.0-0.20070620snap.el5.i386.rpm #rpm -ivh scsi-target-utils-0.0-0.20070620snap.el5.i386.rpm #启动 tgtd 服务#service tgtd start #设置开机自启动tgtd 服务#chkconfig -level 345 tgtd on #服务启动后，可以查看监听端口：# netstat -tunlp |grep tgtd tcp 0 0 :3260 :* listen 2295/tgtd tcp 0 0 :3260 :* li

32、sten 2295/tgtd 可以看到，默认target server 使用的服务端口是tcp 3260; 2、 建立 target device centos 5.0 以上是利用 tgtadm 命令来设置而不是编写/etc/ietd.conf( 据台湾朋友alex lin介绍， 4.0 是通过编写 /etc/ietd.conf来配置的，本节也是参考alex的文章测试的 )，个人认为 tgtadm 的命令相关参数真的较难记，幸好scsi-target-utils 软件中包含如何利用tagadm 命令建立 target device 的步骤，建议读者一定要把 /usr/share/doc/scs

33、i-target-utils-0.0/readme.iscs 打开阅读，里面有举例说明可参考；现准备新增iqn 名为 iqn.2008-.lanexpert:iscsi-data1 的 target device iscsi qualified name (简称 iqn)，格式通常如下：格式意 义范 例yyyy-mm 年月2008-12 reversed domain name 把域名反过来写cn.lanexpert identifier 识别字，通常描述这个储存空间的用途iscsi-data1 centos 5.2 + raid0+lvm+iscsi 配置详解zeng lingping ht

34、tp:/ 2008/12/06 page 8 of 12新增 target device 命令如下：#tgtadm -lld iscsi -op new -mode target -tid 1 -t iqn.2008-.lanexpert:iscsi-data1 执行后，可以利用以下命令查看target device: # tgtadm -lld iscsi -op show -mode target target 1: iqn.2008-12. cn.lanexpert:iscsi-data1 system information: driver: iscsi status: running

35、 lun information: lun: 0 type: controller scsi id: deadbeaf1:0 scsi sn: beaf10 size: 0 backing store: no backing store account information: acl information: 3、 将 target server 新建的 lv1（/dev/lvm1/data1 ）加入 target device #tgtadm -lld iscsi -op new -mode logicalunit -tid 1 -lun 1 -b /dev/lvm1/data1 将逻辑卷

36、加入target device 后，利用下列命令查看target device 的信息：# tgtadm -lld iscsi -op show -mode target target 1: iqn.2008-12. cn.lanexpert:iscsi-data1 system information: driver: iscsi status: running i_t nexus information: lun information: lun: 0 type: controller scsi id: deadbeaf1:0 scsi sn: beaf10 size: 0 backing

37、 store: no backing store lun: 1 type: disk scsi id: deadbeaf1:1 scsi sn: beaf11 size: 1t #这里有些不解，实际上的容量是1.5tb，后来我在win2k3 上连接并分区时也是1.5tb backing store: /dev/lvm1/data1 account information: acl information: 4、 允许 initiator 主机 node1(8)可以存取此target device #tgtadm -lld iscsi -op bind -mode targ

38、et -tid 1 -i 8 将逻辑卷加入target device 后，利用下面的命令查看target device 的信息：# tgtadm -lld iscsi -op show -mode target centos 5.2 + raid0+lvm+iscsi 配置详解zeng lingping http:/ 2008/12/06 page 9 of 12target 1: iqn.2008-12.local.superlgroup.xt:iscsi-data1 system information: driver: iscsi status: running i

39、_t nexus information: lun information: lun: 0 type: controller scsi id: deadbeaf1:0 scsi sn: beaf10 size: 0 backing store: no backing store lun: 1 type: disk scsi id: deadbeaf1:1 scsi sn: beaf11 size: 1t backing store: /dev/lvm1/data1 account information: acl information: 8 再使用以下命令创建另一个ta

40、rget device: tgtadm -lld iscsi -op new -mode target -tid 2 -t iqn.2008-.lanexpert:iscsi-data2 tgtadm -lld iscsi -op new -mode logicalunit -tid 2 -lun 1 -b /dev/lvm1/data2 tgtadm -lld iscsi -op bind -mode target -tid 2 -i 08 创建完毕后，再利用下面的命令查看target device: # tgtadm -lld iscsi -op show -mod

41、e target target 1: iqn.2008-.lanexpert:iscsi-data1 system information: driver: iscsi status: running i_t nexus information: lun information: lun: 0 type: controller scsi id: deadbeaf1:0 scsi sn: beaf10 size: 0 backing store: no backing store lun: 1 type: disk scsi id: deadbeaf1:1 scsi sn: beaf11 siz

42、e: 1t backing store: /dev/lvm1/data1 account information: acl information: 8 centos 5.2 + raid0+lvm+iscsi 配置详解zeng lingping http:/ 2008/12/06 page 10 of 12target 2: iqn.2008-.lanexpert:iscsi-data2 system information: driver: iscsi status: running i_t nexus information: lun information: lu

43、n: 0 type: controller scsi id: deadbeaf2:0 scsi sn: beaf20 size: 0 backing store: no backing store lun: 1 type: disk scsi id: deadbeaf2:1 scsi sn: beaf21 size: 325g backing store: /dev/lvm1/data2 account information: acl information: 08 5、 修改 /etc/rc.local ，设置开机自动加载target device 配置因为以上命令的配置结果，重新开机后便会消失，而且现版本的target server,redhat 尚未修改成system v 服务，如果需要系统重新启动后上述配置仍然有效，只能将以上配置写入/etc/rc.local ，让系统每次开机都自动执行一次：# iscsi target device setting -lingping 2008