DellRLinu+MDf+Rose+Oracleg双机热备

1、Linux6. 5 x64+oracle 11. 2. 0. 4+RoseHA9. 0一、设备信息及规划Dell R730服务器：2*E5-2643 v3 3. 4GHz 6C/4*16G DDR4 memory/3* 300GB 15K RPM SAS/Raid /2*2Port1000M/dual power supp1y/DVD/2*sing1e port FC HBA card/ 3year overseas warranty其中磁盘0、1做Raidl,磁盘2做热备Dell MD3800f：7*600GB 15K RPM SAS/2*16G FC, 2U MD38xxF, 8G CF

2、cache,4*5m fiber/ 3year overseas warranty其中磁盘0、1、2、3、4、5做RaidlO,磁盘6做热备Lunl=l. 7T做数据存储,Lun2=3G做oracle仲裁,Lun3二2G预留BOSS仲裁。二、R730安装配置K开机设置RaidlF2 = System Setup F16 = Lif ecyic )e Control ler Fll = BIOS Boot Manager F12 = rx Boot Two Z.ZO Gllz Eight-core Processors Bus SpeedGT/s, LZ/L3 Cache:Z HB/2O HBS

3、ystem runnin? at 2.20 GHz System Memory Size* 61.9 GB Systcn Mcnory Speed： 1333 nHs Vo 1 tagc 1 3SUBroadcom HetXtree Ethernet Boot Agent Copyright (C) 2090-2911 Broftdcoto Corporation AI rights reserued.Pre&s Ctrl-S to enter Configuration MenurowcrEdgc Expandable RAID Contro1ler D10$ Co|)yr ighttc )

4、 Z011 LSI CorporationPress to Run Conf igurat ion Utility服务器开机，系统自检，加载到PERC卡门检界面的时候，按CTRL+R进入PERC BIOS管理界面PERC H71GP Mini BIOS ConP iurat i on Uti 1 fitV 4.00-0014 EQud Mg 4U3G3E3SSK2EQHXETZ3SXS3HHHHHHI -(-PERC H719P Mini CBus 6x03 Dev 0x60)Ho Confion Present !Unconf igured Phys ica1 Disks1lAOi! Al

5、: AA :it tZVIIGBee：ei：ei： Ready：ee：i：ez：Ready:Z788?ee：ei：e3：Ready:278 8?G：ei：e4：Ready：278.87oooi；es：Re ad y Z7O.O7oe：ei：e6：Ready：Z78.87Aftrfit :A7：Rey :?7 R700:01:68：Ready:278.87|pHTF2-0peionFRerreslCtMexPiL9Ctr-PPrev Page FIZ-Ct 1 r査看磁盘列表，各个磁盘状态为“Ready”(-)PERC H719P Mini CBus 0x03, Dev 0x00)t- 1 Di

6、sk Group : 6 RA I D 1 1 3 U i rtua 1 Divlcs1 IP： 6 Uirtual Disk 5 Z7B.07 【】Pliys ica 1 D Lsks【】Tota I Free Capac i ty : 090 CB Hot spares|C- Spanned Disk Group： RAID 16 l-J Uirtua DisksID： 1. Uirtual Disk 1, 557.75 【】Phys ic1 DisksI J To tei 1 Free Capac i ty : 0.80 CB Hot sparer【】 Unconf igured Phy

7、s ica1 Disks Ready： Z78 ” GBlcar ConfigForeign Conf 1gCreate CatcheCade Vi r tua 1 Diskte H-eu UDManage Preserved CftcheI oe：ei：e60G!01：e?： Readii： 278 B7 GB1 9o：ei：e8Ready； 2?d 278 87 GBI 3 Physical DisksTota I Free Cap:0於2:#;:t:I=ii:实-B:l:x=毎象 31丁 了31丁 mlFTPr 丿丁 旦 FT 了JlnJlnpTa习 m 习-k-s JH cs BBRH

8、R0OR BhMBfin BBftpMnuflBR DuIB 间 IB 间1 IB 间IB1rc 1 mx lms1 ma1rc 1 mx 1RO 1fiv 1 rr花 lnclmo 1piv 1ma 1rc 2mx lms 12 lma lnc 1 wx 1R31rcTTL=64ITL-64IIL=4IIL-64IIL-t4IIL-64TTL-64IIL-64ITL-64HL-4IIL=64TTL-64IIL-64TIL 64TT，如 ITL=64 TTL-64 UL=64 ITL-64ITIIL64TTL-64IIL-64S性纯多个存糜列上升绑 丄级庖匸业管谨笛口任惋V）中炭上寺走升级任务

9、。62 (92.,6B129,R2 192 168.129.1152 02 H2 02”亘邑nu 畐的2 2 2 2 2 2 m-3T7 2节匕琴 Ar-r工(EjClns(0lm 9t|BlnoITL-61TTL-64 1TL=64ITL-64ITL-64ITL-61RAID控制器模块网络配置可使用DHCP服务器（默认设豐）进行分配。但是，如果DHCP服 务器在150秒内不可用，RAID控制器模块会分配静态IP地址。初次连接或每次断开后, 都需要等待150秒才能ping通。2、查找添加存储

10、D0LLPOWERVAULT MODULAR DISK STORAGE MANAGERaE 一旦房既初始设置任务管理域擁ais细以友空左天刁施刖屯子佛件:猝列管理H才级亦c :笛嚴桶妖固（牛 冃时在多个初満列上升锻刃件。 注必派存金“它幕底口（EMM中导| 5107心*.戒更#扳和 不得ti井 此持竜齐纪任芳。可访间性匸；牛虽无;號?餐-山二糸承了质第P宇洙的程农舌，Sl md sroraao “初臥町牧件可诸尸和加作惑克就饰希。在软件的一设置”菜单中添加存储阵列，可以使用口动搜索也可以使用带外管理手动输入主 备控制器的IP地址进行添加。D0LLPOWERVAULT MODULAR DISIi

11、SS曰凰Wipadb Ei已查找到的存储阵列（1 “卫存储阵歹ij Burundi右键点击添加的存储，选择“管理存储阵列S将打开新的窗I I对存储进行管理。3、创建磁盘组点击“存储和备份服务S 7块彳更盘情况下.使用前6块创建磁盘组，使用RaidlO.映射关 系为 0-1, 2*3, 4-5。4、创建并初始化虚拟磁盘存储阵存储主机映幻硬件鉴测升圾曙助Burundi D 最佳存跌和备伯服务二i刃存储傳列BurundiM虚拟磁盘“Lim3”磁盘组g Burundi (RAID 10) (1,660.228GB |-g Lun1 (1,650.000 GB)Lun2 (2.000 GB)n3 (2.

12、000 GB)8】空闲咨里(6.228 GB)创建Lunl用于数据存储，Lun2用于oracle仲裁, 不分配。新的虎拟磁盘初始化需耍人概2个小时，沓看关联的物理绢件虚拟碇盘状态: 楕简配蓋：匕最佳容量：虚拟磁盘全局标识符:60:0a:09:80:00:! 子系统ID (SSID):RAID级别：2.000 GB210Lun3用于jboss仲裁.选中虚拟磁盘后可以在右侧看到进度。留出一部分容鼠5、创建热备盘选择“硬件”标签.右键点击未使用的磁盘，选择“热备用容彊”，选择“手动分配单个物 理盘s为磁盘组分配热备用物理磁盘6、添加主机和主机组存砂冽曲主机创硬件监测fhfiBurundi 7 最佳主

13、机映射换件设羞在树中查栈科象存储蹲列旦音理主机蓟标识符“ 查看非关联三命口标识帘s己定义的映财I施则曲立a皿.存録分区无关联需毀功能;a*jbSW.实全选择存储阵列.在右键选项中选择定义主机填写主机名D0LL土仇世赳矣土忧加刃廷目CZSILD/V叹具RAzB局型zs I貝屮均八也逞W悯冃庇一的土仇炯LHTVt付丿勺仔1WP*刘15悟。住肌方疥 亠、丄匸十八14.、rvi 右Apr亠 eH-力 Mk 0、二丄-F-1 亠 Jr ccca。HWXAA 丁亠如（可恪主乱端口柢识祐m主机泪卩配？游主机瘙口类型，FC迭择将主观诱口标识符】忝加到主机的方法:。通过迭拴已知的尢关联壬札端口杯识符邊订疥加已知

14、非天联王机端口杯识花别名（磯多30个字苻）添加移除爺回-步II卿帮助选抒FC接口类型和主机端1标识.可通过下列命令确定FC HBA卡的WW号 rootBOSSl # cat /sys/class/fc_host/host 1/ 0x21000024ffllllcarootBOSSl # cat /sys/class/fc_host/host2/port_name0x21000024fflllld0S Burundi -主机且问惡（圣义三机）D0LL什么总午和纟冃？问题：您正右定义的主机是否厲于多个主机（这些主机将共享对存储阵列上存储区中的相同曲楓盘的访问）组成的群集（主机组）

15、？O i二逐圭黄旦包荃但圭址共亨胆担冋删越趙懣回否-该主机不能与其他主机共拿珀相同虚似琏盘的访问。如果迭择“是将转至指定主机爼名称的屏蘇。如果迭择烤”，则会转至预览屏帥 女瞎在以后走义主机纟臼可使用定义主机组”选项确定足主机组命名主机组7. 为主机组添加映射Burundi 定Bunrndi b 男任DLL三血帀健阵刁IBunj n5 帝标、的砂 汀出 Lun“LUN? ；帝 LUH2 二 LUN? 订誉 Lun3mLUN?呂題雷认姐使巨该违心J定:沖JIB?由阳盘到7您可以将支占巫盐眇钳制盘自e也$申处默zb如味?5卽逗湘皓 廿附 谓战用定文念侦区选Id自短阿I依息店参厲啊虑 -ifliDSR

16、丄札主视纽Onrie.HA倉B甲元号：LUN) :日主杭B0SS1 主杭BOSS?光天辰王机扁口标识75SJbirtSSLun111.650.000 GBLun22000 GBLun32000 00右键选择主机组“添加LUN映射”8、MD3800f存储摘要可Girt详附干斤倩伙态f无林斑I氏1120.0GOfl.0015割夏妳fltttRAJDI拎出帥固件务左0820 12 60m也上於直ct砒盘 o / o巴丽曲乩1R-AIC VlOififrJl - M5uS 3B 厅*庄軾細 3/512ai.3U豬SJ住拟就盘P弔衍浒：0/512写打储分巨C已用佑F) : 1/94 宅丄札殂：1日宓葩主

17、机2 “DEES複怙 2巫机也1dJW o.QU 希5. 介聒Q2!包 HDD. 扭二娈2!Fffi51 660 228 GB、茨昭: oo: r/EwV砸6 228 GF材 itUitiiat EOJS1 654.000 GBJ二处地咎如曲8页 SAS.dd績a用物的急.1 aL正矗怜用：0 噩行机： 1信总中心已亲用Q(|321UUtHIBi 弓5!FipT5l!1!Linux多路径软件DMMP安装配置砌左札惑昨牛 Q!升点辛StBuiundi J ftffwn i?8rootBOSSl 、# fdisk TDisk /dev/sdb: 1771.7 GB, 1771674009600 b

18、ytes255 heads, 63 sectors/track, 215393 cylindersDisk /dev/sdc: 3221 MB, 3221225472 bytes100 heads, 62 sectors/track, 1014 cylindersDisk /dev/sde: 1771.7 GBt 1771674009600 bytes255 heads, 63 sectors/track, 215393 cylindersDisk /dev/sdd: 2147 MB, 2147483618 bytes67 heads, 62 sectors/track, 1009 cylin

19、dersDisk /dev/sdg: 2147 MB, 2147483648 bytes67 heads, 62 sectors/track 1009 cylindersDisk /dev/sdf: 3221 MB, 3221225472 bytes100 heads, 62 sectors/track, 1014 cylinders将MD3800f的卷映射到主机上以后，可以看到六块盘，需要通过linux自带的多 路径软件聚合成三块。主要用到下面这两个包device-mapper-1. 02. 63-4 el5. x86_64 rpmdevice-mapper-multipath-0. 4 7

20、-46 el5 x86_64 rpm査看包的安装情况rootsms01 Server# rpm -q device-mapper-*package device-mapperT. 02. 63-4. el5. 1386 rpm is not installedpackage device-mapper-1. 0263-4. el5. x86_64. rpm is not installedpackage device-mapper-event-1.02.63-4.elo. x86_64. rpm is not installed package device一mapper-multipath-

21、0. 4. 7-46 el5 x86_64. rpm is not installed安装底层模块rootsms01 Server# rpm -Uvh device-mapper-1. 02. 63-4el5.x86_64. rpm检査安装是否正常rootsms01 Server# lsmod grep dm_multipat安装多路径工具包.rootsms01# rpm 一Uvh device一mapper一multipath一0. 4. 7-46. el5 x86_64. rpm 设置开机自启动rootsms01 Server# chkconfig multipathd on接着通过命令：

22、multipath -11查看多路径状态査看模块是否加载成功rootliujing # multipath - 11查看多路径状态Mar 10 19:18:28 | /etc/multipath. conf does not exist, blacklisting all devices.Mar 10 19:18:28 | A sample multipath .conf file is located atMar1019:18:28|/usr/share/doc/device-mapper-multipath-0. 4. 9/multipath. confMar 10 19:18:28 |

23、You can run /sbin/mpathconf to create or modify etc/multipath confMar 10 19:18:28 | DM multipath kernel driver not loaded DM 模块没有加载如果模块没有加载成功请使用下列命初始化DM,或重启系统Use the following commands to initialize and start DM for the firsttime：tt modprobe dm-multipath# modprobe dm-roundrob in tt service multipath

24、d start# multipath -v2初始化完了Z后再通过multipath -11命令査看是否加载成功rootliujing 、# multipath -11Mar 10 19:21:14 | /etc/multipath. conf does not exist, blacklisting all devices.Mar 10 19:21 ： 14 | A sample multipath, conf file is located atMar1019:21:14|/usr/share/doc/device-mapper-multipath-0. 4. 9/multipath con

25、fMar 10 19:21:14 | You can run /sbin/mpathconf to create or modify etc/multipath confDM multipath kernel driver not loaded 这个提示没了说明 DM 模块已加载成功。从上面的提示可以看到，DM模块是成功加载，但是/etc/下没有multipath, conf 配置文件编辑配置文件rootdsmsOl Servervi etc/multipath. conf 在 linux6. 5 中需要手动创建 此文件ttcp/usr/share/doc/device-mapper-mult

26、ipath-0. 4. 9/multipath. conf/etc/multipath conf将 blacklist devnode *注释掉并确保下面不被注释defaults user_friendly_names yes 一 开启服务rootsms01 Server# service multipathd startmultipath基本操作命令# /etc/init. d/multipathd startU multipath 一F# multipath -v2#开启mulitipath服务#删除现有没有使用的路径#格式化路径，检测路径，合并路径双机中，如果映射3个卷，可能在主机是dm

27、-0, dm-1, dm-2的顺序，而在另一 台是dm-1, dm-0, dm-2的顺序。为了保证双机的设备名一致，需在两台机器的 multipath, conf配置文件中增加wwid对应名称的配置。检测并查看WWN号rootBOSS1 Packagesift multipath -v2rootBOSS1 Packages# multipath -11wp=rw status=activempathd (3600a0980009883c8000001fd574c51d9) dm-2 DELL, MD38xxf size=2. OG features二O hwhandler=，0*Ipolicy

28、=，round-robin 0 prio=lrunning status=enabled running|- 1:0:0:3 sdd 8:48 active ready-+- policy二round-：robin 0 prio=l-2:0:0:3 sdg 8:96 active ready mpathc (3600a098000988461000002725747151e) dm-l DELL, MD38xxf size=3. OG featuresO hwhandler=，0 wp=rwstatus=activerunningstatus二enabled|-+- policy二round-

29、robin 0 prio=l | - 1:0:0:2 sdc 8:32 active ready 一+- policy=，round-robin 0 prio=l一 2:0:0:2 sdf 8:80 active ready runningsize=1. 6T features二0 hwhandler=，0 wp=rw status=activempathb (3600a0980009884610000026f574714fd) dm-0 DELL, MD38xxfIpolicy二round-robin 0 prio=lrunning status=enabledrunning- 1:0:0:

30、1 sdb 8:16 active ready-+- policy二round-:robin 0 prio=l -2:0:0:1 sde 8:64 active readymultipaths multipath wwidaliasmultipath wwidaliasmultipath wwidalias配置文件中增加对应3600a0980009884610000026f574714fd mpathb3600a098000988461000002725747151e mpathc3600a0980009883c8000001fd574c51d9mpathd在使用fdisk -1査看的时候，会

31、看到多出的聚合后设备Disk dev, mapper； mpathb: 1771. 7 GB, 1771674009600 bytes 255 heads, 63 sectors/track, 215393 cylindersUnits = cylinders of 16065 * 512 = 82252S0 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal) : 512 bytes / 512 bytesDisk identifier: 0x00000000Disk dev,

32、 mapper/mpathc: 3221 MB, 3221225472 bytes 255 heads, 63 sectors/track, 391 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical) : 512 bytes / 512 bytes 1/0 size (minimum/optimal) : 512 bytes / 512 bytes Disk identifier: 0x00000000Disk /dev, mapper/mpathd: 2147 MB,

33、 2147483648 bytes 255 heads, 63 sectors/track, 261 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical) : 512 bytes / 512 bytes 1/0 size (minimum/optimal) : 512 bytes / 512 bytes Disk identifier： 0x00000000使用 fdisk 对/dev/mapper/mpathb 进行分区，Lrootsms02 starboss# fdi

34、sk /dev mapper mpathbDevice contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabelBuilding a new DOS disklabel with disk identifier 0xac956c3aChanges will remain in memory only, until you decide to write them. After that, of course, the previous content won t be recoverable Warn

35、ing: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)WARNING: DOS-compatible mode is deprecated Its strongly recommended to switch off the mode (command c ) and change display units to sectors (command u)Command (m for help) : n新建分区Command actione extendedp primary partition (1-

36、4)p主分区Partition number (1-4): 1First cylinder (1-1016, default 1):Using default value 1Last cylinder, +cylinders or +sizeK, M, G (1-1016, default 1016):Using default value 1016Command (m for help) : w写入歹lj表相当于保存The partition table has been altered!Calling ioctl() to re-read partition tableSyncing di

37、sks分区后会出现/dev/mapper/ mpathbpl如果是超过2T需要用GPT进行分区，oracle不能用超过2T的分区rootsms02 starboss# parted /dev/mapper/mpathb/用 parted 命令分区GNU Parted 2. 1Using /dev/mapper/mpathbWelcome to GNU Parted! Type help to view a list of commands.(parted)mk labelGPT/ 建立磁盘 labelWarning: The existing disk label on /dev/mapper

38、/mpathb will be destroyed and alldata on this disk will be lost Do you wantto continue?Yes/No?y(parted)mkpart/仓Ij建分区Partitionname?oraOl分区名称File system type?ext2? ext4文件系统Start?1End?2577GB/ 大小(parted) p/査看是 否成功Model: Linux device-mapper (multipath) (dm)Disk /dev/mapper/mpathb: 2577GBSector size (logical/physical): 512B/512B Partition Table： gptNumber StartEndSizeFile system Name Flags11049kB2577GB2577GBoraOl(parted)quitInformation： You may need to update