大型分布式网络课程第11周作业

Heartbeat项目是Linux-HA工程的一个组成部分，它实现了一个高可用集群系统。心跳服务和集群通信是高可用集群的两个关键组件，在Heartbeat项目里，由heartbeat模块实现了这两个功能。heartbeat最的包括两个动资源接管模块来接管运行在对方主机上的资源或者服务。[的配置文件来定义信息传递的方式和协议等。它是一个新兴的，2008年推出，但其实它并不是一个真正意义上的新，在2002年的时候有一个项目Corosync60%Openais.Corosync个完整的HA功能，但是要实现，更复杂的功能，那就需要使用Openaishb_guiHAcorosync+pacemaker+mysql+drbd实现y三、安装四、编译安装五、安装六、my与drbdyq数据的镜像七、利用sh配置yql的高可用 部 以附件的形式上传 的源码 ，网上很容易望各位博友带着下面的疑问去实现corosync+pacemaker+mysql+drbdmysql1、corosync是什么？pacemaker是什么？corosync与pacemaker的关系2、mysqldrbd3 corosync、pacemaker、y、drbd之间的关系立关系？各个节点会不会抢占资源使其出现脑裂(split-brain)？出现脑裂可以有fence设备自行解决corosynccorosyncOpenaisOpenais的一个子 ，相比Heartbeat这款很老很 corosync与HeartbeatcorosyncHeartbeatpacemaker：是众多集群资源管理器(ClusterResourceManager) 个，其主要功能是管理来着集群信息层发送来的信息。Pacemaker是集群的,它管理逻辑和集群信息.集群信息更新通过Corosync通知到各个节点.常见 有heartbeatv1-->haresourceshearbeatv2--->hearbeatv3--->pacemaker 如下图corosync与pacemaker之间的关系mysql：drbd：DRBD:(distributedreplicationblockdevice)即分布式块设行，，但是主从DRBD服务器之间是可以进行调换的。、hosts##把主机名改[root@jie2~]#sed-is/`grepHOSTNAME/etc/sysconfig/network|awk-'=''{print/g#把主机名改[root@jie3~]#sed-is/`grepHOSTNAME/etc/sysconfig/network|awk-'=''{print/g[root@jie2~]#cat>>/etc/hosts<<[root@jie3~]#cat>>/etc/hosts<<、ssh互[root@jie2[root@jie2~]#ssh-keygen-trsa-P[root@jie2~]#ssh-copy-id-i.ssh/id_rsa.pub[root@jie3~]#ssh-keygen-trsa-P[root@jie3~]#ssh-copy-id-i.ssh/id_rsa.pub、关闭[root@jie2[root@jie2~]#chkconfig--delNetworkManager[root@jie2~]#chkconfigNetworkManageroff[root@jie2~]#serviceNetworkManagerstop[root@jie3~]#chkconfig--delNetworkManager[root@jie3~]#chkconfigNetworkManager[root@jie3~]#serviceNetworkManager、时间同步(博主用的是自己的ntp[root@jie2[root@jie2~]#ntpdate[root@jie3~]#ntpdate三、安装corosync+pacemaker1）、安装corosync+pacemaker[root@jie2~]#yum-yinstallcorosync[root@jie2~]#yum-y--nogpgchecksh-[root@jie3~]#yum-yinstallcorosync[root@jie3~]#yum-y--nogpgchecksh-4.el6.x86_64.rpmpssh-2.3.1-#节#节的操[root@jie2~]#cd[root@jie2corosync]#mvcorosync.conf.example[root@jie2corosync]#vim#Pleasereadthecorosync.conf.5manualcompatibility:totem#心跳信息传递version:secauth:#认证信息threads:interface#定义心跳信息传递的接ringnumber:bindnetaddr: #绑定的网络地址，写网络地mcastaddr: #多播地mcastport:ttl:#多播的端#生存周}}loggingfileline:to_stderr:to_logfile:#是否输出在屏幕#定义自己的日to_syslog:#是否由syslog记录日logfile: #日志文件的存放路debug:timestamp:#时间戳是否关 AMF}}amfmode:}servicever:name: #pacemakercorosync}aisexecuser:root}[root@jie2corosync]#scpcorosync.conf##节的操[root@jie2corosync]#corosync-CorosyncClusterEngineAuthenticationkeygenerator.Gathering1024bitsforkeyfrom/dev/random.Presskeysonyourkeyboardtogenerateentropy(bits=#遇到这个情况，表示电脑的随机数不够，各位朋友可以不停的随盘，或者也可以生成随机[root@jie2corosync]#scpauthkey#把认证文件到主机[root@jie2~]#servicecorosyncStartingCorosyncClusterEngine[OK[root@jie2Lastupdated:ThuAug814:43:13Lastchange:SunSep116:41:182013_attributeStack:classicopenais(withCurrent-partitionwithVersion:..-7.el6-2Nodesconfigured,2expectedOnline:][root@jie3~]#servicecorosyncStartingCorosyncClusterEngine[OK[root@jie3Lastupdated:ThuAug814:43:13Lastchange:SunSep116:41:182013_attributeStack:classicopenais(withCurrent-partitionwithVersion:..-7.el6-2Nodesconfigured,2expectedOnline:]四、编译安装yl(##节的操#1)、解压编译安[root@jie2~]#tarxfmysql-PlatformDevelopment"[root@jie2~]#cdmysql-[root@jie2mysql-5.5.33]#yum-yinstallcmake[root@jie2mysql-5.5.33]#cmake.-DCMAKE_INSTALL_PREFIX=/usr/local/mysql\-DMYSQL_DATADIR=/mydata/data-DSYSCONFDIR=/etc-DWITH_INNOBASE_STORAGE_ENGINE=1-DWITH_ARCHIVE_STORAGE_ENGINE=1-DWITH_BLACKHOLE_STORAGE_ENGINE=1-DWITH_READLINE=1-\-DWITH_ZLIB=system-DWITH_LIBWRAP=0-\[root@jie2mysql-5.5.33]#make&&makeinstall [root@jie2mysql-5.5.33]#cp/usr/local/mysql/support-files/my- [root@jie2mysql-5.5.33]#cp/usr/local/mysql/support-files/mysql.server/etc/rc.d/init.d/mysqld[root@jie2mysql-5.5.33]#cd/usr/local/mysql/[root@jie2mysql]#useradd-r-u306mysql[root@jie2mysql]chownRroot:mysql*[root@jie2"PATH=/usr/local/mysql/bin:$PATH">/etc/profile.d/mysqld.sh[root@jie2mysql]#source/etc/profile.d/mysqld.sh[root@jie2mysql]#echo"/usr/local/mysql/lib"[root@jie2mysql]#ldconfig-v|grep[root@jie2[root@jie2mysql]#ln-sv/usr/local/mysql/include/先别初始化数据库，安装drbd把drbd挂载到 据库的数据存放到drbd挂载的 安装rpm包的drbd必须保证找相同内核版本的drbd-d、先划分一个分区，此分区做成drbd镜像(RHEL6.x的重新格式化一个新[root@jie2~]#fdiskCommand(mforhelp):nCommandaction primarypartition(1-pPartitionnumber(1-4):cylinder(7859-15665,default7859):Usingdefaultvalue7859Lastcylinder,+cylindersor+size{K,M,G}(7859-15665,15665):Command(mforhelp): [root@jie3~]#fdisk/dev/sdaCommand(mforhelp):nCommandaction primarypartition(1-pPartitionnumber(1-4):Partitionnumber(1-4):cylinder(7859-15665,default7859):Usingdefaultvalue7859Lastcylinder,+cylindersor+size{K,M,G}(7859-15665,15665):Command(mforhelp):、安装drbd和修改配置文#1)#1)、安装#节的操[root@jie2~]#rpm-ivhdrbd-kmdl-2.6.32-358.el6-warning:drbd-kmdl-2.6.32-358.el6-8.4.3-33.el6.x86_64.rpm:HeaderV4DSA/SHA1Signature,keyID66534c2b:NOKEY#################################[root@jie2~]#rpm-ivhdrbd-8.4.3-warning:drbd-8.4.3-33.el6.x86_64.rpm:HeaderV4DSA/SHA1Signature,keyID66534c2b:NOKEY###################################节的操[root@jie3~]#rpm-ivhdrbd-kmdl-2.6.32-358.el6-warning:drbd-kmdl-2.6.32-358.el6-8.4.3-33.el6.x86_64.rpm:HeaderV4DSA/SHA1Signature,keyID66534c2b:NOKEY#################################[root@jie3~]#rpm-ivhdrbd-8.4.3-warning:drbd-8.4.3-33.el6.x86_64.rpm:HeaderV4DSA/SHA1Signature,keyID66534c2b:NOKEY###################################2)、修改drbd的配置文#节 的操[root@jie2~]#cd[root@jie2drbd.d]# mon.conf#全局配置文globalusage-count#minor-countdialog-refreshdisable-ip-}common

handlers{pri-on-incon-degr"/usr/lib/drbd/notify-pri-on-degr.sh;/usr/lib/drbd/notify-emergency-reboot.sh;echob/proc/sysrq-trigger;reboot-after-sb.sh;/usr/lib/drbd/notify-emergency-reboot.sh;echob>/proc/sysrq-trigger;reboot-lclio-error"/usr/lib/drbd/notify-io-/usr/lib/drbd/notify-emergency-shutdown.sh;echoo>/proc/sysrq-trigger;halt-f";#fence-peer -fence-#split-brain"/usr/lib/drbd/notify-split-

#out-of-sync"/usr/lib/drbd/notify-out-of-sync.sh#before-resync- -lvm.sh-p15---c#after- -}startup#wfc-timeout#degr-wfc-timeout}disk}net}

on-io-error#fencingresource-cram-hmac-alg"sha1";syncer}}

rate[root@jie2drbd.d]#catmydata.res#资源配置文resource{ meta-disk} meta-diskmeta-disk}}copy[root@jie2drbd.d]#mon.conf3）drbd##节的操drbd资[root@jie2~]#drbdadmcreate-mdWritingmetadata...initializingactivitylogNOTinitializingbitmaplk_bdev_save(/var/lib/drbd/drbd-minor-0.lkbd)failed:NosuchfileNewdrbdmetadatablocksuccessfullycreated.#提示已经创建成功lk_bdev_save(/var/lib/drbd/drbd-minor-0.lkbd)failed:Nosuchfileor#启动服[root@jie2~]#servicedrbdStartingDRBDresources:[createres:drbdpreparedisk:drbdadjustdisk:adjustnet:]#节的操drbd[root@jie3~]#drbdadmcreate-mdWritingmetadata...initializingactivitylogNOTinitializingbitmaplk_bdev_save(/var/lib/drbd/drbd-minor-0.lkbd)failed:NosuchfileNewdrbdmetadatablocksuccessfullycreated.#提示已经创建成功lk_bdev_save(/var/lib/drbd/drbd-minor-0.lkbd)failed:Nosuchfileor[root@jie3~]#servicedrbdStartingDRBDresources:[createres:drbdpreparedisk:adjustdisk:drbdadjustnet:drbd]、设置一个主节点，然后同 的数据(此步骤只需在一个节点上操作##设drbd的主节[root@jie2~]#drbdadmprimary--force[root@jie2~]#cat#查看同步进version:8.4.3(api:1/proto:86-GIT-hash:89a294209144b68adb3ee85a73221f964d3ee515buildbygardner@,2013-05-2704:30:210:cs:SyncSourcero:Primary/Secondaryds:UpToDate/InconsistentCr--ns:1897624ns:1897624nr:0dw:0dr:1901216al:0bm:115lo:0pe:3ua:3ep:1wo:f[=================>..]sync'ed:90.3%finish:0:00:07speed:26,792(27,076)[root@jie2~]#watchn1'cat/proc/drbd'[root@jie2~]#cat/proc/drbdversion:8.4.3(api:1/proto:86-101)GIT-hash:89a294209144b68adb3ee85a73221f964d3ee515buildbygardner@,2013-05-2704:30:210:cs:Connectedro:Primary/Secondaryds:UpToDate/UpToDateCr----ns:120nr:354dw:435dr:5805al:6bm:9lo:0pe:0ua:0ap:0wo:fdrdb分区([root@jie2[root@jie2~]#mke2fs-text4六、myql与drbd实现yql数据的镜、在drbd的主节点上，挂载drbd的分区，然后初始化数据[root@jie2[root@jie2~]#mkdir/mydata#创建用于挂drbd[root@jie2~]#mount/dev/drbd0[root@jie2~]#mkdir[root@jie2~]#chownRmysql.mysql/mydata#把文件的属主和属组改成[root@jie2 #修改mysql的配置datadir=innodb_file_per_table[root@jie2~]#/usr/local/mysql/scripts/mysql_install_db----datadir=/mydata/data/--#初始化数据[root@jie2~]#servicemysqldStartingMySQLStartingMySQL [ 、验证drbd是否镜##节的操#1)、先在drbd的主节点上面创建一个数据[root@jie2~]#mysql>show++||++|information_schema|||performance_schema||++4rowsinset(0.00mysql>createdatabaseQueryOK,1rowaffected(0.01sec)mysql>showdatabases;++||++|information_schema|||||performance_schema||++5rowsinset(0.00#2)、停mysql服务，卸载drbd挂载[root@jie2~]#servicemysqld[root@jie2~]#umount/dev/drbd0#卸drbd的挂载[root@jie2~]drbdadmsecondarymydata#把此节点改为drbd#节 的操#3）、 变为drbd的主节[root@jie3~]#drbdadmprimary #把此节点改为drbd的主节[root@jie3~]#mkdir[root@jie3~]#mount/dev/drdb0/mydata[root@jie3~]#vim datadir=/mydata/datainnodb_file_per_table=1[root@jie3~]#servicemysqldstart#此节点上不用初始化数据库，直接StartingMySQL [ [root@jie3~]#mysql>show #可以看见jie2数据 | |information_schema| | |performance_schema| 55rowsinset(0.00七、利用sh配置yq的高可需要定义集群资源而yq、drbd都是集群的资源，由集群管理的资源开机是、关闭drbd的服务和关闭mysql的服[root@jie2[root@jie2~]#servicemysqld[root@jie2~]#servicedrbd[root@jie3~]#servicemysqld[root@jie3~]#umount/dev/drbd0#之drbd已经挂载节点了[root@jie3~]#servicedrdb定义drbd的资源(提供drbd的资源RA由OCF类别中的linbit提供[root@jie2~][root@jie2~]#(live)configure#propertystonith-enabled=false(live)configure#propertyno-quorum-policy=ignore(live)configure#primitivemysqldrbdocf:linbit:drbdparamsdrbd_resource=mydataopmonitorrole=Masterinterval=10timeout=20opmonitorrole=Slaveinterval=20timeout=20opstarttimeout=240opstoptimeout=100#定义drbd的主从资(live)configure(live)configure#msms_mysqldrbdmysqldrbdmetamaster-max=1master-node-max=1clone-max=2clone-node-max=1notify=true(live)configure#(live)configure(live)configure#primitivemystoreocf:heartbeat:Filesystemparamsdevice="/dev/drbd0"directory="/mydata"fstype="ext4"opmonitorinterval=40timeout=40opstarttimeout=60opstoptimeout=60(live)configure#(live)configure#colocationmystore_with_ms_mysqldrbdinf:mystore(live)configure#orderms_mysqldrbd_before_mystoremandatory:ms_mysqldrbd:promotemystore:start(live)configure#定义vip资源、mysql服务的资源约束关(live)configure(live)configure#primitivemyvipocf:heartbeat:IPaddrparamsip="172.16.22.100"opmonitorinterval=20timeout=20on-fail=restart(live)configure#primitivemyserverlsb:mysqldopmonitorinterval=20timeout=20on-fail=restart(live)configure#(live)configure#colocationmyserver_with_mystoreinf:myserver(live)configure#ordermystore_before_myservermandatory:mystore:startmyserver:start(live)configure#(live)configure#colocationmyvip_with_myserverinf:myvip(live)configure(live)configure#ordermyvip_before_myservermandatory:myvip(live)configure#(live)configure#\attributes\attributesprimitivemyserverlsb:mysqldopmonitorinterval="20"timeout="20"on-fail="restart"primitivemysqldrbdocf:linbit:drbd\paramsdrbd_resource="mydata"opmonitorrole="Master"interval="10"timeout="20"\opmonitorrole="Slave"interval="20"timeout="20"\opstarttimeout="240"interval="0"\opstoptimeout="100"interval="0"primitivemystoreocf:heartbeat:Filesystem\paramsdevice="/dev/drbd0"directory="/mydata"\opmonitorinterval="40"timeout="40"\opstarttimeout="60"interval="0"\opstoptimeout="60"interval="0"primitivemyvipocf:heartbeat:IPaddr\paramsip="172.16.22.100"\opmonitorinterval="20"timeout="20"on-fail="restart" -msms_mysqldrbdmysqldrbdmsms_mysqldrbdmysqldrbdmetamaster-max="1"master-node-max="1"clone-max="2"clone-node-max="1"notify="true"colocationmyserver_with_mystoreinf:myservercolocationmystore_with_ms_mysqldrbdinf:mystorems_mysqldrbd:Mastercolocationmyvip_with_myserverinf:myvipmyserverorderms_mysqldrbd_before_mystoreinf:ms_mysqldrbd:promoteordermystore_before_myserverinf:mystore:startmyserver:startordermyvip_before_myserverinf:myvipmyserverproperty$id="cib-bootstrap-options"dc-version="1.1.8-7.el6-394e906"cluster-infrastructure="classicopenais(withplugin)"\expected-quorum-votes="2"\stonith-enabled="false"no-quorum-查看资源运行的状态运行 [root@jie2[root@jie2Lastupdated:ThuAug817:55:30Lastchange:SunSep116:41:182013via _attributeonStack:classicopenais(withplugin)Current-partitionwithVersion:1.1.8-7.el6-2Nodesconfigured,2expected5ResourcesOnline:]Master/SlaveSet:ms_mysqldrbdMasters:]Slaves:Slaves:]myvip [root@jie3 node #把此节点设置为备用节[root@jie3 Lastupdated:MonSep201:45:07Lastchange:MonSep201:44:592013via _attributeonStack:classicopenais(withplugin)CurrentDC: -partitionwithquorumVersion:1.1.8-7.el6-394e9062Nodesconfigured,2expected5Resourcesconfigured. :standbyOnline:[ Master/SlaveSet:ms_mysqldrbdMasters: #资源已然转 上Stopped:[mysqldrbd:1 myvip(ocf::heartbeat:IPaddr): 由于定义了drbd的资源约束，Masters运行在那个节点，则此节点不可能成drbd[root@jie3[root@jie3~]#catversion:8.4.3(api:1/proto:86-GIT-hash:GIT-hash:89a294209144b68adb3ee85a73221f964d3ee515buildbygardner@,2013-05-2704:30:210:cs:Connectedro:Primary/Secondaryds:UpToDate/UpToDateCr-----ns:426nr:354dw:741dr:6528al:8bm:9lo:0pe:0ua:0ap:0wo:f[root@jie3~]#drbdadmsecondary0:Statechangefailed:(-12)DeviceisheldopenbysomeoneCommand'drbdsetupsecondary0'terminatedwithexitcode手动的停掉myvip还是会启动（因为定义资源是指on-[root@jie2[root@jie2~]#ifconfig|grepLinkinetHWaddrinet6addr:fe80::20c:29ff:fe1f:74cf/64Scope:LinkUPBROADCASTRUNNINGMULTICASTMTU:1500RXpackets:216