HACMP54OracleRac安装配置手册

1、HACMP Oracle Rac安装设置手册版本v1.0零一二年十月北京华胜天成科技股份有限公司TCU451JN华J®天咸文档控制更改记录版本创建/修改时间编制/修改者文件/修改内容审批人V1.02012.10审阅姓名职位发布姓名职位北京华胜天城科技股份有限公司第3页共58页7石产三北京华胜天城科技股份有限公司第5页共58页目录目录第1章HACMP v5.4安装配置IP规划1.2、 安装HACMP软件1.2.1、操作系统版本.1.2.2、安装依赖的软件包1.2.3、安装 HACMP v5.4 .1.3、1.4、HACMP配置规划1.3.1、配置IP和网络编辑/etc/hosts文件创

2、建共享卷组和文件系统1.3.2、1.3.3、 HACMP Standard 配置添加cluster和节点创建并配置资源组.同步HACMP的配置141、1.4.2、1.4.3、HACMP启动1.5.1、启动 HACMP 第2章 Oracle RAC安装配置 1.5、2.1、规划2.2、安装检查.2.3、安装前配置2.4、安装CRS .2.5、安装数据库2.6、创建数据库2.7、数据测试.8.91011111313131313151720394657ass第1章HAC MP V5.4安装配置1.1、IP规划NodeIn terface NameTy peIP AddressRegistered I

3、nibm03ibm03P ublic8Hosts fileibm03ibm03-vi pVirtual10Hosts fileibm03ibm03-privPrivate8Hosts fileibm04ibm04P ublic9Hosts fileibm04ibm04-vi pVirtual11Hosts fileibm04ibm04-privPrivate9Hosts file1.2、安装HACMP软件完成HACMP安装配置规划后，可以开始安装HACMP软件，安装版

4、本为HACMP v5.4。1.2.1、操作系统版本# oslevel -r5300-061.2.2、安装依赖的软件包检查系统的软件安装包，请参照IBM 红皮书中High Availability Cluster Multi-Processingfor AIX 5L Installation Guide 。这里我们建议安装软件束APP-DEV和SERVER，其中包括的红皮书要求的大部分软件包。安装rsct软件包:rsct.basicrsct.co mp at.basicrsct.co mp at.clie ntsUh "rFAMSUN华J®天展rsct.opt.storage

5、rm (可选)安装其它软件包:bos.clvm （使用并行资源组时需要，RAC下必须安装此包）bos.data#smitty in stall_latestIn stall SoftwareType or select values in entry fields.Press En ter AFTER making all desired cha nges.Entry FieldsINPUT device / directory for software/dev/cdOSOFTWARE to installLall_latestP REVIEW on ly? (in stall op erat

6、io n will NOT occur)noCOMMIT software up dates?yesSAVE rep laced files?noAUTOMA TICALL Y in stall requisite software?yesEXTEND file systems if sp ace needed?yesOVERWRITE same or newer versio ns?noVERIFY in stall and check file sizes?noIn clude corres pon di ng LANGUAGE filesets?yesDETAILED out put?n

7、oProcess mult iple volumes?yesACCE PT new license agreements?yesP REVIEW new LICENSE agreeme nts?no在"SOFTWARE to in stall ”一栏中选择要安装的软件包，“ ACCE PT new lice nse agreeme nts ”中接受许可协议，即完成安装软件包。或参考以下脚本命令:#安装软件包/usr/lib/i nstl/sm_ instin stallp_cmd-a-Q-d'/dev/cd0'-f'bos.clvmALLI:bos.clvm_

8、all_filesets,bos.dataALLI:bos.data_all_filesets,rsct.basicALLI:rsct.basic_all_filesets,rsct.co mp at.basicALLI:rsct.co mp at.basic_all_filesets,rsct.co mp at.clie ntsALLI:rsct.co mp at.clie nts_all_filesets,rsct .op t.storagermALL l:rsct.o pt.storagerm _all_filesets' '-c' '-N' &#

9、39;-g' '-X'#'-G''-Y'-TFAMSUN1.2.3、安装 HACMP V5.4除了 haview ， netwiew ( Tivoli),的包以外，所有的HACMP的软件包都要安装,rsct.opt.storagerm为可选安装包，对于基本包在安装光盘或Expansion光盘上可以查找到。中接受许可协议，即完成安装软件包，安装过程大约在1030分钟完成。#smitty in stall_latestIn stall SoftwareType or select values in entry fields.Press En

10、 ter AFTER making all desired cha nges.Entry FieldsINPUT device / directory for software/dev/cdOSOFTWARE to installLall_latestP REVIEW on ly? (in stall op eratio n will NOT occur)noCOMMIT software up dates?yesSAVE rep laced files?noAUTOMA TICALL Y in stall requisite software?yesEXTEND file systems i

11、f sp ace needed?yesOVERWRITE same or newer versio ns?noVERIFY in stall and check file sizes?noIn clude corres pon di ng LANGUAGE filesets?yesDETAILED out put?noP rocess multi pie volumes?yesACCE PT new license agreements?yesP REVIEW new LICENSE agreeme nts?no在"SOFTWARE to in stall ”一栏中选择要安装的软件包

12、，“ ACCE PT new lice nse agreeme nts ”或参考以下脚本命令:#安装HACMP/usr/lib/i nstl/sm_ instin stallp_cmd-a-Q-d'/dev/cdO'-f'cluster.adt.esALLl:cluster.adt.es_all_filesets,cluster.doc.e n_US.esALLl:cluster.doc.e n_US.es_all_filesets,cluster.esALLI:cluster.es_all_filesets,cluster.es.cfsALLI:cluster.es.

13、cfs_all_filesets,cluster.es.cs pocALLI:cluster.es.cs poc_all_filesets,cluster.es. nfsALLI:cluster.es .nfs_all_filesets,cluster.es .pluginsALLI:cluster.es .pl ugi ns_all_filesets,cluster.es.worksheetsALLI:cluster.es.worksheets_all_filesets,cluster.lice nseALLI:cluster.lice nseall filesets,cluster.man

14、.en US.es北京华胜天城科技股份有限公司第9页共58页ZA TCAMSUN华J®天咸I ALLl:cluster.man.en US.es all filesets''-c' '-N' '-g' '-X'系统提示成功后，完成HACMP补丁升级，将 HACMP补丁上传至系统目录下，我们假设'-G''-Y'上传目录为/tmp/hav54,本地目录为 d:hav54，上传命令如下:Windows :"开始”-> "运行”，输入"cmd”，根据以下

15、提示完成补丁上传:Microsoft Windows XP 版本 5.1.2600(C)版权所有 1985-2001 Microsoft Corp.C:Docume nts and Sett in gsAdmi nistrator>ft p remote-host-i p- address输入用户名和密码(此处需要给用户设置密码，如果为空，AIX拒绝登录)，成功登录后:ftp>cd /tmp/hav54ftp >lcd d:hav54ftp>promptftp >bi nftp>mp ut *.bff上传完成后退出ftp程序ftp> quit升级 HA

16、CMP v5.4 补丁#smitty up date_allUp date Installed Software to Latest Level (U pdate All) Type or select values in entry fields.Press Enter AFTER making all desired changes.Entry FieldsINPUT device / directory for software/tmp/hav54SOFTWARE to up date_up date_allP REVIEW only? (up date op eration will

17、NOT occur) no COMMIT software up dates?SAVE repl aced files?noAUTOMATICALLY install requisite software?EXTEND file systems if sp ace needed?VERIFY install and check file sizes?DETAILED out put?Process mult iple volumes?ACCE PT new lice nse agreeme nts?yesyes yes no noyesyesP REVIEW new LICENSE agree

18、me nts?回车确认以上命令，即可完成升级工作。注意:no如果安装程序提示“没有找到相关的fileset”等类似的提示，可在/tmp/hav54目录下执行命令“ #inutoc .”，重新生成软件包安装列表。重新启动AIX安装完成HACMP后，为避免各种问题，要求将 AIX系统重新启动，至此我们完成了所有的安装工作，可以开始配置 HACMP双机集群了。北京华胜天城科技股份有限公司第13页共58页#shutdown -Fr1.3、HACMP配置规划1.3.1、配置IP和网络AIX网卡IP地址配置: #smiitty mktcpipAvailable Network Interfaces Mov

19、e cursor to desired item and p ress Enter.en008-08en108-09en20A-08en30J-08Standard Ethernet Network InterfaceStandard Ethernet Network Interface Standard Ethernet Network Interface Standard Ethernet Network Interface选择想要配置IP地址的网卡，选择“ enO”回车确认，进入下一步配置:Minimum Configuration & Start upTo Delete exi

20、sting configuration data, pl ease use Further Configuration menusType or select values in entry fields.Press Enter AFTER making all desired changes.Entry Fields HOSTNAMEIn ternet ADDRESS (dotted decimal) Network MASK (dotted decimal) Network INTERFACENAMESERVERInternet ADDRESS (dotted decimal) DOMAI

21、N NameDefault GatewayAddress (dotted decimal or symbolic n ame)Cost0Do Active Dead Gateway Detection?Your CABLE TypeSTART TCP/IP daemons Nowora01en02noN/Ano使用同样的方法配置完成网卡的IP地址。1.3.2、编辑 /etc/hosts 文件对于两个节点ibm03和ibm04要配置相同的/etc/hosts文件，使用vi配置hosts文件，建议首先对原有的hosts文件进行备份，配置如下:#cp

22、/etc/hosts /etc/hosts.bak-TFAMSUN#vi /etc/hosts8 ibm039 ibm048 ibm03-priv9 ibm04-priv10 ibm03-vip11 ibm04-vip1.3.3、创建共享卷组和文件系统创建卷组#smitty _mkbvgAdd a Big Volume GroupType or select values in entry fields.Press Enter AFTER making all des

23、ired changes.Entry FieldsVOLUME GRO UP nameP hysical p artiti on SIZE in megabytesPH YSICAL VOLUME n amesFORCE the creati on of volume grou p? Activate volume group AUTOMATICALLY at system restart?Volume group MAJOR NUMBERCreate VG Concurrent Ca pable?vgdatahdisk2 yes no100no我们在建立 HACM洪享卷组时，要注意的参数“A

24、ctivate volume group AUTOMATICALlaY system restart? "该项，设置为“ no”，是使用HACMP勺方式将卷组 varyon，而不是系统启动时由操作系统 varyon 。Major NUMBER是指该设备（orvg ）的主设备号，此处设置为系统唯一即可，可通过命令查 询可用的 major number :# lvlstmajorI 43.59,62.或可通过ls - l /dev/命令查看当前所有设备的主设备号，第五列即是。创建逻辑卷和文件系统在ibm03节点上按照以下操作:#varyoffvg vgdata需将vgdata varyo

25、ff 下来，在ora02节点上才能正确的导入。在ibm04节点上按照一下操作:4Ihdisk000c2ef1fa38df28arootvgactivehdisk100c2ef1f38b81facrootvgactivehdisk200c2ecef3846c20eNo ne#ls pv查看hdisk2是否为物理卷，否则需要使用命令将其属性更改:#chdev - l hdisk2 - a pv=yes导入卷组和文件系统的定义#smitty importvgImport a Volume GroupType or select values in entry fields.Press Enter A

26、FTER making all desired changes. Entry FieldsVOLUME GROU P name* PH YSICAL VOLUME nameVolume group MAJOR NUMBER#vary onvg vgdata#varyoffvg vgdatavgdata hdisk2 1001.4、HACMP Standard 配置HACMP配置可采用Standard和Extended两种方式，其中 Standard配置可以配置Cluter名称，节点，基本网络，应用服务器，各种资源和资源组等基本的内容，如果要配置永久IP，串口心跳，对资源组进行详细的配置等工作则

27、需要在Extended中完成，可以说 Standard菜单是Extended菜单的一个子集，可以在 Extended菜单中完成HACMP的所有配置工作。下面我们以Standard菜单完成HACMP的基本配置工作:添加cluster禾n节点配置cluster 资源创建并配置资源组同步HACMP的配置HACMP配置菜单如下: #smitty hacm pHAC MP for AIXMove cursor to desired item and p ress Enter.Initialization and Standard Configuration Extended ConfigurationS

28、ystem Management (C-S POC)P roblem Determination Tools选择 “ Initialization and Standard Configuration ” 女0下菜单:Initialization and Standard ConfigurationMove cursor to desired item and p ress Enter.Configuration AssistantsConfigure an HAC MP Cluster and NodesConfigure Resources to Make Highly Available

29、Configure HAC MP Resource Grou psVerify and Synchronize HAC MP Configuration Dis play HAC MP ConfigurationHAC MP Cluster Test Tool141、添加cluster和节点#smitty hacm p->ln itializati on and Stan dard Con figurati onCon figure an HAC MP Cluster and NodesConfigure Nodes to an HAC MP Cluster (standard)Type

30、 or select values in entry fields.Press Enter AFTER making all desired changes.Entry Fieldsora_clusteribm03 ibm04* Cluster NameNew Nodes (via selected communication p aths) Currently Configured Node(s)1.4.2、创建并配置资源组#smitty hacm p -> In itializatio n and Stan dard Con figurati onCon figure HAC MP

31、Resource GroupsConfigure HAC MP Resource GroupsMove cursor to desired item and p ress Enter.Add a Resource GroupChange/Show a Resource GroupRemove a Resource GroupChange/Show Resources for a Resource Group (standard)北京华胜天城科技股份有限公司第15页共58页TCU451JN邸L为籤魁皋秦J二：-Add a Resource GroupType or select values i

32、n entry fields.Press Enter AFTER making all desired changes.Entry Fields* Resource Group Nameora_rg* P artic ip ating Nodes (Default Node Priority) ibm03 ibm04Startup Policy Fallover P olicy Fallback P olicyOnline On All AvailabBring Offline (On Err>Never Fallback添加资源至资源组中#smitty haem p-> In i

33、tializati on and Stan dard Con figurati onCon figure HAC MP Resource GroupsConfigure HAC MP Resource GroupsMove cursor to desired item and p ress Enter.Add a Resource GroupChange/Show a Resource Group Remove a Resource GroupChan ge/Show Resources for a Resource Grou p (sta ndard)Change/Show All Reso

34、urces and Attributes for a Resource GroupType or select values in entry fields.Press Enter AFTER making all desired changes.Entry FieldsResource Group Nameora_rgP artic ip ating Nodes (Default Node Priority) ibm03 ibm04Startup Policy Fallover P olicy Fallback Policy ListOnline On Home Node OnlyFallo

35、verTo Next Priority Node In The ListFallback To Higher Priority Node In TheConcurrent Volume GroupsvgdataUse forced varyon of volume group s, if necessary false Automatically Import Volume GroupsfalseApp lication ServersTape Resources至此，我们完成了资源和资源组的关联。北京华胜天城科技股份有限公司第#页共58页£Jh 'TFAMSUNaam 严a

36、TCftMSUN华®天咸1.4.3、同步HACMP的配置#smitty haem p -> In itializati on and Stan dard Con figurati onVerify and Synchronize HAC MP Con figurati on回车确认该操作，这样 HACMP的配置已经同步到两个节点中了，也就是 ibm03和ibm04拥有同样的配置了。1.5、HACMP 启动1.5.1、启动 HACMP在节点ibm03上执行smitty clstart，回车确认即可启动 HACMP* Start now, on system restart or

37、bothnow+Start Cluster Services on these nodesibm03+BROADCAST message at startu p?true+Start up Cluster Information Daemon?false+Reacquire resources after forced down ?false+第2章Oracle RAC 安装配置2.1、规划数据库配置规划ORACLE BASE/dataCRS HOME/data/product /10.2.0/crsORACLE HOME/data/product /10.2.0/dbDB NAMEorclO

38、RACLE SIDorcl1， orcl2TNS ADMIN$ORACLE HOME/network/admi nOracle管理帐户的口令sysoracleSystemoracle数据库存放位置裸设备是否使用归档方式运行数据库YesRMAN/TSM备份方式说明裸设备规划设备名称数据库设施存储空间dev/rorcl_ocrOcr信息存放500Mbdev/rorcl_vdiskvdisk信息存放500Mbdev/rorcl_s pfilesp file信息存放100Mbdev/rorcl_srvc onfigsrvconfig信息存放100Mbdev/rorcl_systemSystem表空间1

39、0Gdev/rorcl_sysauxSysaux表空间10Gdev/rorcl_te mp1系统临时表空间20Gdev/rorcl_te mp2系统临时表空间20Gdev/rorcl_ un do1撤销表空间20Gdev/rorcl_ un do2撤销表空间20Gdev/rorcl_ un do3撤销表空间20Gdev/rorcl_ un do4撤销表空间20Gdev/rorcl_usersUser表空间10Gdev/rorcl_toolsTools表空间10Gdev/rorcl_ctl1控制文件1Gdev/rorcl_ctl2控制文件1Gdev/rorcl_ctl3控制文件1Gdev/ror

40、cl_log1a日志文件500Mbdev/rorcl_log1b日志文件500Mb北京华胜天城科技股份有限公司第#页共58页/dev/rorcl_log2a日志文件500Mb/dev/rorcl_log2b日志文件500Mb/dev/rorcl_log3a日志文件500Mb/dev/rorcl_log3b日志文件500Mb/dev/rorcl_log4a日志文件500Mb/dev/rorcl_log4b日志文件500Mb/dev/rorcl_data1用户数据表空间20G/dev/rorcl_data2用户数据表空间20G/dev/rorcl_data3用户数据表空间20G/dev/rorcl

41、_data4用户数据表空间20G2.2、安装检查操作系统版本:# oslevel -r 5300-06物理内存:# Isattr -El memosizegoodsize 7744 Amount of usable p hysical memory in Mbytes False7744 Total amount of p hysical memory in Mbytes False交换区:# Isps -aPage SpacePhysical VolumeVolume GroupSize %Used Active Auto Typehd6hdisk0rootvg512MB3 yes yes

42、lv北京华胜天城科技股份有限公司第19页共58页7石产三系统空间：# df -gFilesystemGB blocksFree %UsedIused %Iused Moun ted on/dev/hd40.500.4413%47885% /dev/hd25.003.1038%412406% /usr/dev/hd9var1.000.956%7471% /var/dev/hd31.000.6536%10821% /tmp/dev/fwdu mp0.380.371%41% /var/adm/ras/pl atform/dev/hd10.500.501%711% /home/proc-/proc/d

43、ev/hd10o pt2.001.867%34611% /opt/dev/fslv0015.004.4271%292433% /data/dev/lv000.120.124%181% /var/adm/csd文件集:#lsl PP-lbos.adt.basebos.adt.lib bos.adt.libmbos .p eif.lib perfstatbos .perf.p erfstat北京华胜天城科技股份有限公司第25页共58页bos .perf.p roctools rsct.basic.rte rsct.co mp at.clie nts.rte xlC.aix50.rte xlC.rt

44、eFilesetLevel StateDescri ptio nPath: /usr/lib/objre posbos.adt.baseCOMMITTEDBase App licati on DevelopmentToolkitbos.adt.libCOMMITTEDBase App licati on Devel opmentLibrariesbos.adt.libmCOMMITTEDBase App licati on DevelopmentMath Librarybos .p erf.lib perfstat COMMITTEDP

45、erforma nee Statistics LibraryIn terfacebos .pef.p efstatCOMMITTEDPerforma nee StatisticsIn terfacebos .pef.p roctoolsCOMMITTEDProc Filesystem Toolsrsct.basic.rte249.0COMMITTEDRSCT Basic Fun ctionrsct.co mp at.clie nts.rte COMMITTEDRSCT Event Man ageme nt Clie ntFunctionxlC.aix5

46、0.rte COMMITTED XL C/C+ Run time for AIX 5.2xlC.rte COMMITTED XL C/C+ Run timePath: /etc/objre posbos .p erf.lib pefstatCOMMITTEDP erforma nee Statistics LibraryIn terfacebos .pef.p erfstatCOMMITTEDPerforma nee StatisticsIn terfacersct.basic.rteCOMMITTEDRSCT Basic

47、Fun ction2.3、安装前配置为确保oracle rac安装成功，系统管理员root用户需创建相应的系统用户、系统组，以便安rac环装管理oracle软件。Root用户还要修改相关系统参数，打破默认值对实际应用的限制。调试rac节点之间的高级功能：rep/rsh/rlogin，以确定两节点之间可以通信正常，可以在境下进行相关同步操作和心跳监控。创建dba组，id为500# smitty mkgro up创建oracle用户，id为601# smitty mkuser修改oracle用户密码# P asswd oracle在oracle用户下创建.rhosts文件，内容为"+ ”

48、。# su -oracle$ vi .rhosts修改oracle用户限制# smitty chuserSoft FILE size-1Soft CPU time-1Soft DATA segme nt-1Soft STACK size-1Soft CORE file size-1修改系统配置参数，增加系统进程数上限# smitty chgsysMaximum number of P ROCESSES allowed per user2048修改网络参数no -p -0 udp_sendsp ace=65536no -p -0 udp _recvs pace=65536no -p -0 tcp

49、_sendsp ace=65536no -p -0 tcp _recvs pace=65536no -p -o rfc1323=1no -p -o sb_max=1310720no -r -o ipq maxle n=512创建裸设备mkiv -y rac_system_raw_500m -T O -w n -s n -r n vgdata 8依次创建以下裸设备:/dev/rorci ocr /dev/rorci vdisk /dev/rorcl_s pfile /dev/rorcl_srvc onfig /dev/rorcl_system /dev/rorcl_sysaux /dev/ror

50、cl_te mp1 /dev/rorcl_te mp2 /dev/rorcl undo1 /dev/rorcl undo2 /dev/rorcl undo3 /dev/rorcl undo4 /dev/rorcl users /dev/rorcl tools /dev/rorcl ctl1 /dev/rorcl ctl2 /dev/rorcl ctl3 /dev/rorcl_log1a /dev/rorcl_log1b /dev/rorcl_log2a /dev/rorcl_log2b /dev/rorcl_log3a /dev/rorcl_log3b /dev/rorcl_log4a /de

51、v/rorcl_log4b/dev/rorci datal /dev/rorci data2 /dev/rorcI data3 /dev/rorci data4更改裸设备权限# cd /dev # chown oracle:dba *orcl*修改oracle用户环境变量# vi /home/oracle/.profile umask 022 export ORACLE_BASE=/data export ORACLE_HOME=$ORACLE_BASE/product/10.2.0/db export ORA_CRS_HOME=$ORACLE_BASE/product/10.2.0/crs

52、export LD_LIBRARY_PA TH=$ORACLE_HOME/lib:$ORACLE_HOME/lib32 export ORACLE_SID=orcl export P ATH=$ORACLE_HOME/bi n:$ORA_CRS_HOME/bi n:$ PATH2.4、安装 CRS在ibm03节点上进入 CRS安装盘，执行 runInstaller启动CRS的安装;按提示在每个安装节点上以root用户执行root pre.sh，该脚本在 CRS安装盘上的root pre目录下;# ./root pre.sh ./root pre.sh out put will be logged in /tmp/root pre.out_06-04-13.12:33:04Savi ng the origi nal files in /etc/ora_save_06-04-13.12:3