HP MC ServicesGuaidA本安装手册

1、HP MC /ServicesGuaid A11.16版本安装手册文档版本 v1.4作者：*HP的MC软件是一个使用的比较广泛的CLUSTER成熟版本，以LICENSE核算，IBM的最高，下来就是HP的MC，但是下来的SUN的CLUSTER数量只相当于HP的七分之一。 做双机热备的时候需要提前准备： 准备工作A-1：编辑/.rhosts ,MC/ServiceGuard将使用/.rhosts 文件验证2机之间的信任关系。A-1-1命令 vi /.rhosts （oracle 用户需在安装oracle后 加入）示列：hp1 roothp2 root验证双方关系是否成立在HP1

2、号机上 rlogin hp2A-1-2 分发 /.rhosts到所有群集结点rcp /.rhosts hp2:/.rhostsA-2 在所有节点上编辑 /etc/nsswitch.conf 文件，确保有下面一行.A-2-1命令 vi /etc/nsswitch.confhosts: files NOTFOUND=continue dnsA-2-2分发 /etc/nsswitch.conf 到所有群集节点rcp /etc/nsswitch.conf hp2 /etc/nsswitch.confA-3在所有节点上编辑 /etc/hosts 文件，解析所有节点的IP和name.A-3-1命令 vi

3、/etc/hosts注意：在安腾的操作系统，要非常规范的写hosts文件，（主机的IP地址,主机名） 10001 hp110002 hp2 （注意：本处建议不要将心跳地址、浮动IP地址写入） 2：在HP主机上安装MC的步骤 首先，必须根据HP对所安装的软件提供的License(Customer Identifier) 在 上申请该软件的Codeword。 然后，将光盘(光盘的驱动是/dev/dsk/c3t2d0)放入驱动器中，MOUNT以后，在超级用户提示符下执行 # swinstall s&#

4、160;/dev/dsk/c3t2d0 (如果未建立CD-ROM文件系统则 mkdir /cdromIoscan funCdiskIssf -e找到对应硬件路径Mount /dev/dsk/c3t2d0 /cdrom (假设当前为c3t2d0)用ls 命令查看CDROM里面的文件cd cdromls进入交互式界面后，先加Codeword，才能在列表见到需安装的软件。最后，按其提示完成该软件的安装。需要注意，两台机器需要不同的密码。（MC的补丁请根据机型及HP-UX OS的版本进行选择） 3：网络准备  关于网络的准备，一定要仔细，有图纸，IP规划，对应的机器主板结

5、构示意图，如果网络有蹊跷，最好不要做MC 。 IP的网段要隔绝好，不要出现局域网有重名的IP地址。 推荐使用HP的三大底层法宝命令 #lanscan 看主机的底层物理状况，是否UP,（注意这个命令无法看到IP层）  #netstat in 看IP地址绑定是否正确 #nslookup hp1  看自己可不可以解析自己在配置机上的操作1、 在HP1（配置机）创建卷组、缩盘2、强调：做这个前要先复制/etc/lvmtab文件， cp /etc/lvmt

6、ab /etc/lvmtab.old  在HP2（非配置机）上一样先备份 lvmtab fstab请注意：在存储上的共享卷组是不需要加入到fstab。如果ORALCE建立在数据库服务器上的，其/oracle . /backup(如果有)应该在fstab。A：在主机hp1上创建卷组vgdata 和vglock #  mkdir /dev/vglock# mkdir  /dev/vgdata1# mkdir  /dev/vgdata2# ls -l /dev/*/group ( 查看group文件 确定使用 0x010

7、000中的X数值)# mknod  /dev/vglock/group  c 64 0x0A0000 # mknod  /dev/vgdata/group  c 64 0x0B0000 # mknod  /dev/vgdata/group  c 64 0x0C00002、格式化硬盘：#pvcreate /dev/rdsk/c5t0d0 创建VG #vgcreate  /dev/vglock  /dev/ds

8、k/c5t0d0  #vgcreate e 65534 s 16  /dev/vgdata  /dev/dsk/c5t0d1 #vgcreate e 65534 s 16  /dev/vgdata2  /dev/dsk/c5t1d13、在主机hnyb01上执行，创建逻辑卷。(如数据库建立在裸设备则不需要)#对于VGLOCK卷组来说， 我们一般不建议在VGLOCK 锁盘上划分卷组# lvcreate L 300000 n oradata1 s 16 /dev/vgda

9、ta1 （划分名字问oradata1大小为300000MB，属于VGDATA1的卷）# lvcreate L 300000 n oradata2 s 16 /dev/vgdata2  （划分名字问oradata2,大小为300000MB，属于VGDATA2的卷）4、创建文件系统 (如数据库建立在裸设备则不需要)# newfs F vxfs o largefiles  /dev/vgdata1/roradata (如果系统有超过2GB的文件 需要加-o的参数

10、，一般用于装ORACLE数据的卷 才加本参数)请注意 newfs命令一般用于RAW设备，请在前面加r# newfs F vxfs /dev/vgdata2/roralog15、挂载文件系统到目录 (如数据库建立在裸设备则不需要)# cd/# mkdir  /oradata # mkdir  /oralog1 # mount /dev/vgdata1/oradata /oradata# mount /dev/vgdata2/oralog1 /oralog1自动MOUNT 文件目录# vi

11、 /etc/fstab 加入 磁阵上所划的卷# mount a 挂起所有/etc/fstab 下的所有卷# mount v 显示所有的已经mount上的设备（或者用bdf查看）6、关闭vg属性（请关闭所有除开与VG00相关的卷组）在hp1上去激活卷组：# umount /oradata# umount /oralog1# vgchange -a n /dev/vglock# vgchange -a n /dev/vgdata1# vgchange -a n /dev/vgdata27、将主机hp1号的vg信息导出# vgexport p s m

12、 /tmp/vgdata.map  /dev/vgdata1# vgexport p s m /tmp/vglock.map  /dev/vgdata2# vgexport p s m /tmp/vglock.map  /dev/vglock 8、在HP1（配置机）上将VG信息文件复制到HP2（非配置机）上在hp1号机将文件复制到hp2上： # rcp /tmp/vgdata1.map  hnyb01:/t

13、mp/vgdata1.map # rcp /tmp/vglock.map  hnyb01:/tmp/vglock.map # rcp /tmp/vgdata2.map  hnyb01:/tmp/vgdata2.map在非配置机上的操作9、在HP2（非配置机）建立文件目录在主机hp2上创建group文件 # mkdir /dev/vglock# mkdir  /dev/vgdata1# mkdir  /dev/vgdata2# ls /dev/*/group ( 查看group文件 确定

14、使用 0x0F0000中的X数值)# mknod /dev/vglock/group c 64 0x010000 # mknod /dev/vgdata1/group  c 64 0x020000# mknod /dev/vgdata2/group  c 64 0x020000 这两个命令使用的0x0A0000，0x0B0000一定要和主机hny01要严格符合，否则会有错误。10、在HP2（非配置机）上将VG信息导入

15、将映射文件导入卷组数据，在HP2上输入： # vgimport s -v m /tmp/vgdata1.map  /dev/vgdata1# vgimport s v -m /tmp/vglock.map  /dev/vglock# vgimport s -v m /tmp/vgdata2.map  /dev/vgdata2注意：hp2上的cXtYdz可能会和hp1上的不同，检查一下，确保整个集群中physical volume nam

16、es是正确的。例如：#vgimports v -m /tmp/vgdata.map /dev/vgdata /dev/dsk/c1t2d0 (假设vgdata在磁阵上对应的PV名c1t2d0）请注意：本处可能也不需要本步11、在HP2（非配置机）上将VG激活（请不要激活除开锁磁盘以外的数据VG）# vgchange -a y /dev/vglock12、在HP2（非配置机）上建立mount点、文件系统等创建mount目录点# mkdir /oradata# mkdir  /oralog1 Mount文件系统以检查正确性# mount /dev/vgdata1/oradata

17、 /oradata # mount /dev/vgdata2/oralog1 /oralog1# mount a 挂起所有/etc/fstab 下的所有卷# mount v 显示所有的已经mount上的设备（或者用bdf查看）13、备份VG信息（可选择不要）# vgcfgbackup /dev/vgdata1# vgcfgbackup /dev/vglock# vgcfgbackup /dev/vgdata214、UMOUNT文件系统# umount /oradata# umount /oralog1=下面红色标记的不是太安全=15、改变VG的控制属性（锁磁盘不加了）#chmod 777 /d

18、ev/vgdata1#chmod 777 /dev/vgdata217、改变卷组的控制属性（数据lv）chmod 660 /dev/vgdata1/roradatachmod 660 /dev/vgdata2/roralog1可以参见实际系统：18、改变卷组的用户使用者（一般在ORACLE安装完毕后使用，目前不用）chown oracle:oracle /dev/vgdata1/roradatachown oracle:oracle /dev/vgdata2/roralog119、对卷组属性的更改编辑/etc/lvmrc文件，保证在系统启动时不自动激活与CLUSTER有关的卷组（将AUTO_V

19、G_ACTIVATE置为0，并确认root卷组包含在custom_vg_activation函数内。实际上，Vg00自动处于激活状态），这些卷组将在应用包的control.sh脚本中激活。实际上只需要更改AUTO_VG_ACTIVATE=020、在HP1上去激活所有卷组（请关闭所有除开VG00以外的VG）# vgchange -a n /dev/vglock# vgchange -a n /dev/vgdata1# vgchange -a n /dev/vgdata2到这里，我们所有的准备工作已经做完了；开始正式配置MC。请保证2套机器上的VGDATA VGLOCK都处于非激活状态。A.系统级

20、别的MC配置 A-1: 指定群集节点和生成群集配置模版文件并改动模版文件 # cmquerycl v C /etc/cmcluster/cmclconf.ascii n  hp1n hp2 注意:有时候系统的CLUSTER里面主机不止两个，要在-n跟上各个主机的名字. 两个主机版本不同，密码不同有时候会有问题 生成文件后,用vi改动,粽色表示需要人工干预的参数 #vi /etc/cmcluster/cmclconf.ascii 

21、;脚本# *# * HIGH AVAILABILITY CLUSTER CONFIGURATION FILE # * For complete details about cluster parameters and how to # * set them, consult the Serviceguard manual. # * # Enter a name for this cluster. This name will be used to identify the# cluster when viewing or manipulating it.CLUSTER_NAME cluster

22、1 (集群名字)# Cluster Lock Parameters# The cluster lock is used as a tie-breaker for situations# in which a running cluster fails, and then two equal-sized# sub-clusters are both trying to form a new cluster. The# cluster lock may be configured using only one of the# following alternatives on a cluster:

23、# the LVM lock disk# the quorom server# Consider the following when configuring a cluster.# For a two-node cluster, you must use a cluster lock. For# a cluster of three or four nodes, a cluster lock is strongly# recommended. For a cluster of more than four nodes, a# cluster lock is recommended. If y

24、ou decide to configure# a lock for a cluster of more than four nodes, it must be# a quorum server.# Lock Disk Parameters. Use the FIRST_CLUSTER_LOCK_VG and# FIRST_CLUSTER_LOCK_PV parameters to define a lock disk.# The FIRST_CLUSTER_LOCK_VG is the LVM volume group that# holds the cluster lock. This v

25、olume group should not be# used by any other cluster as a cluster lock device.# Quorum Server Parameters. Use the QS_HOST, QS_POLLING_INTERVAL,# and QS_TIMEOUT_EXTENSION parameters to define a quorum server.# The QS_HOST is the host name or IP address of the system# that is running the quorum server

26、 process. The# QS_POLLING_INTERVAL (microseconds) is the interval at which# Serviceguard checks to make sure the quorum server is running.# The optional QS_TIMEOUT_EXTENSION (microseconds) is used to increase# the time interval after which the quorum server is marked DOWN.# The default quorum server

27、 timeout is calculated from the# Serviceguard cluster parameters, including NODE_TIMEOUT and# HEARTBEAT_INTERVAL. If you are experiencing quorum server# timeouts, you can adjust these parameters, or you can include# the QS_TIMEOUT_EXTENSION parameter.# The value of QS_TIMEOUT_EXTENSION will directly

28、 effect the amount# of time it takes for cluster reformation in the event of failure.# For example, if QS_TIMEOUT_EXTENSION is set to 10 seconds, the cluster# reformation will take 10 seconds longer than if the QS_TIMEOUT_EXTENSION# was set to 0. This delay applies even if there is no delay in# cont

29、acting the Quorum Server. The recommended value for# QS_TIMEOUT_EXTENSION is 0, which is used as the default# and the maximum supported value is 30000000 (5 minutes).# For example, to configure a quorum server running on node# "qshost" with 120 seconds for the QS_POLLING_INTERVAL and to# a

30、dd 2 seconds to the system assigned value for the quorum server# timeout, enter:# QS_HOST qshost# QS_POLLING_INTERVAL 120000000# QS_TIMEOUT_EXTENSION 2000000FIRST_CLUSTER_LOCK_VG /dev/vglock（锁盘VG）# Definition of nodes in the cluster.# Repeat node definitions as necessary for additional nodes.# NODE_

31、NAME is the specified nodename in the cluster.# It must match the hostname and both cannot contain full domain name.# Each NETWORK_INTERFACE, if configured with IPv4 address,# must have ONLY one IPv4 address entry with it which could# be either HEARTBEAT_IP or STATIONARY_IP.# Each NETWORK_INTERFACE,

32、 if configured with IPv6 address(es)# can have multiple IPv6 address entries(up to a maximum of 2,# only one IPv6 address entry belonging to site-local scope# and only one belonging to global scope) which must be all# STATIONARY_IP. They cannot be HEARTBEAT_IP.NODE_NAME hp1 （1号机名字） NETWORK_INTERFACE

33、 lan0 （心跳网卡） HEARTBEAT_IP （心跳IP地址，请注意，请改成HEARTBEAT_IP） NETWORK_INTERFACE lan2 （业务网卡） HEARTBEAT_IP （1号机业务用IP地址） NETWORK_INTERFACE lan3（不需要配置） FIRST_CLUSTER_LOCK_PV /dev/dsk/c8t0d0 （LOCK VG 物理地址）# List of serial device file names# For example:# SERIAL_DEVICE_FILE /dev/tty0p0# Warning: There

34、 are no standby network interfaces for lan0.# Possible standby Network Interfaces for lan2: lan3.NODE_NAME hp2 （2号机名字） NETWORK_INTERFACE lan0 （心跳网卡） HEARTBEAT_IP （心跳IP地址，请注意，请改成HEARTBEAT_IP） NETWORK_INTERFACE lan2 （业务网卡） HEARTBEAT_IP （1号机业务用IP地址） NETWORK_INTERFACE lan3（不需要配置） FIRST_CLUSTER_LOCK_PV /

35、dev/dsk/c8t0d0 （LOCK VG 物理地址）#注意:物理路径要符合,不要把vgdb和vglock两个vg的物理地址混淆# List of serial device file names# For example:# SERIAL_DEVICE_FILE /dev/tty0p0# Warning: There are no standby network interfaces for lan0.# Possible standby Network Interfaces for lan2: lan3.# Cluster Timing Parameters (microseconds

36、).# The NODE_TIMEOUT parameter defaults to 2000000 (2 seconds).# This default setting yields the fastest cluster reformations.# However, the use of the default value increases the potential# for spurious reformations due to momentary system hangs or# network load spikes.# For a significant portion o

37、f installations, a setting of# 5000000 to 8000000 (5 to 8 seconds) is more appropriate.# The maximum value recommended for NODE_TIMEOUT is 30000000# (30 seconds).HEARTBEAT_INTERVAL 1000000NODE_TIMEOUT 2000000#注意:节点轮询时间和超时设置,一般不动,毫秒为单位# Configuration/Reconfiguration Timing Parameters (microseconds).A

38、UTO_START_TIMEOUT 600000000NETWORK_POLLING_INTERVAL 2000000#注意:网络启动时间,失败时候的顺序,一般不动, 毫秒为单位# Network Monitor Configuration Parameters.# The NETWORK_FAILURE_DETECTION parameter determines how LAN card failures are detected.# If set to INONLY_OR_INOUT, a LAN card will be considered down when its in

39、bound# message count stops increasing or when both inbound and outbound# message counts stop increasing.# If set to INOUT, both the inbound and outbound message counts must# stop increasing before the card is considered down.NETWORK_FAILURE_DETECTION INOUT# Package Configuration Parameters.# Enter t

40、he maximum number of packages which will be configured in the cluster.# You can not add packages beyond this limit.# This parameter is required.MAX_CONFIGURED_PACKAGES 3注意:MC里面需要预留几个程序包,有的环境是2个,3个,多个程序包多会耗费一定的内存   如果程序包只预留了一个,以后要加程序包,这个参数不可逆,所以要重新做MC生成模版 # Access Control Policy P

41、arameters.# Three entries set the access control policy for the cluster:# First line must be USER_NAME, second USER_HOST, and third USER_ROLE.# Enter a value after each.# 1. USER_NAME can either be ANY_USER, or a maximum of# 8 login names from the /etc/passwd file on user host.# 2. USER_HOST is wher

42、e the user can issue Serviceguard commands.# If using Serviceguard Manager, it is the COM server.# Choose one of these three values: ANY_SERVICEGUARD_NODE, or# (any) CLUSTER_MEMBER_NODE, or a specific node. For node,# use the official hostname from domain name server, and not# an IP addresses or ful

43、ly qualified name.# 3. USER_ROLE must be one of these three values:# * MONITOR: read-only capabilities for the cluster and packages# * PACKAGE_ADMIN: MONITOR, plus administrative commands for packages# in the cluster# * FULL_ADMIN: MONITOR and PACKAGE_ADMIN plus the administrative# commands for the

44、cluster.# Access control policy does not set a role for configuration# capability. To configure, a user must log on to one of the# cluster's nodes as root (UID=0). Access control# policy cannot limit root users' access.# MONITOR and FULL_ADMIN can only be set in the cluster configuration fil

45、e,# and they apply to the entire cluster. PACKAGE_ADMIN can be set in the# cluster or a package configuration file. If set in the cluster# configuration file, PACKAGE_ADMIN applies to all configured packages.# If set in a package configuration file, PACKAGE_ADMIN applies to that# package only.# Conf

46、licting or redundant policies will cause an error while applying# the configuration, and stop the process. The maximum number of access# policies that can be configured in the cluster is 200.# Example: to configure a role for user john from node noir to# administer a cluster and all its packages, en

47、ter:# USER_NAME john# USER_HOST noir# USER_ROLE FULL_ADMIN# List of cluster aware LVM Volume Groups. These volume groups will# be used by package applications via the vgchange -a e command.# Neither CVM or VxVM Disk Groups should be used here.# For example:# VOLUME_GROUP /dev/vgdatabase# VOLUME_GROU

48、P /dev/vg02VOLUME_GROUP /dev/vglock 与集群有关的VG请全部列出VOLUME_GROUP /dev/vgdata1VOLUME_GROUP /dev/vgdata21-2: 验正群集配置   # cmcheckconf k v C /etc/cmcluster/cmclconf.ascii  如果没有报错信息，显示完成信息，即表示通过。有的时候有一些有关CDROM的小警告,但是只要系统建议你可以做下一步,只要提示是complete就OK 1-

49、3:在节点间分发配置文件   # vgchange a y /dev/vglock （在HP1号机上激活VGLOCK，请检查一下HP2号机上VG的激活情况）。请不要激活所有与数据VG相关的VG 只激活VGLOCK因为分发是正式的要发二进制的控制文件,一定要提前激活vglock的属性,否则以后MC启动有小问题   # cmapplyconf k v C /etc/cmcluster/cmclconf.ascii 注意：设置自动启

50、动参数编辑文件，将参数AUTOSTART_CMCLD设为1，则在系统自举时自动启动CLUSTER。我们一般不自动启动群集，请将其设置为0，或者跳过这一步#vgchange a n /dev/vglock运行 群集Cmruncl v查看群集Cmviewcl v  B、应用级别的ORACLE程序包配置 B-1: 创建程序包配置模板, 编辑这些模板文件，以指定程序包名称、按优先级排序的节点列表、控制脚本的位置以及各个程序包的故障切换参数。 # mkdir /dev/cmcluster/pkg1# cmmakepkg

51、60;p /etc/cmcluster/pkg1/pkg1.conf#vi /etc/cmcluser/pkg1/pkg1.conf 脚本# * *# * HIGH AVAILABILITY PACKAGE CONFIGURATION FILE (template)# * *# * Note: This file MUST be edited before it can be used. # * For complete details about package parameters and how to set them, *# * consult the S

52、erviceguard Extension for RAC manuals.# *# Enter a name for this package. This name will be used to identify the# package when viewing or manipulating it. It must be different from# the other configured package names.PACKAGE_NAME pkg1 （程序包的名字）# Enter the package type for this package. PACKAGE_TYPE i

53、ndicates# whether this package is to run as a FAILOVER or SYSTEM_MULTI_NODE# package.# FAILOVER package runs on one node at a time and if a failure# occurs it can switch to an alternate node.# SYSTEM_MULTI_NODE# package runs on multiple nodes at the same time.# It can not be started and halted on in

54、dividual nodes.# Both NODE_FAIL_FAST_ENABLED and AUTO_RUN must be set# to YES for this type of package. All SERVICES must# have SERVICE_FAIL_FAST_ENABLED set to YES.# NOTE: Packages which have a PACKAGE_TYPE of SYSTEM_MULTI_NODE are# not failover packages and should only be used for applications# pr

55、ovided by Hewlett-Packard.# Since SYSTEM_MULTI_NODE packages run on multiple nodes at# one time, following parameters are ignored:# FAILOVER_POLICY# FAILBACK_POLICY# Since an IP address can not be assigned to more than node at a# time, relocatable IP addresses can not be assigned in the# package con

56、trol script for multiple node packages. If# volume groups are assigned to multiple node packages they must# activated in a shared mode and data integrity is left to the# application. Shared access requires a shared volume manager.# Examples : PACKAGE_TYPE FAILOVER (default)# PACKAGE_TYPE SYSTEM_MULTI_NODE#PACKAGE_TYPE FAILOVER（默认数值，不需更改）# Enter the