HPC解决方案实施服务 交付指导书

HPC解决方案实施服务V100R001交付指导书华为机密，未经许可不得扩散第图410所示图STYLEREF1\s4SEQ图\*ARABIC\s110node001配置信息配置项值Hostnamenode001MACAddress80:38:BC:07:63:D0Rack1Position1Height2UCategoryluster-clientipmi0IPaddress01BOOTIFipaddress01ib0IPaddress01至此，节点信息配置完成计算节点部署重启开始部署计算节点的配置信息填写完毕后，重启计算节点，从PXE启动，OS部署自动开始进行查看部署状态点击在“SoftwareImages”菜单项，在“ProvisioningStatus”Tab菜单页面看到节点的部署状态等待所有主机部署完成编译安装IB驱动集群节点需要安装Lustreclient用于访问Lustre文件系统，当采用Infiniband网络通信时，可以选择通过IPoIB或RDMA访问数据，通常会选择性能较优的RDMA模式。从Mellanox官网下载的驱动默认不支持通过RDMA模式访问Lustre文件系统（支持节点间通过RDMA模式通信），需要重新编译该驱动。编译安装操作只需要在其中一个计算节点例如node001上进行操作即可，后续通过同步镜像更新到其他所有节点。配置yum源上传OS镜像文件将CentosOS镜像文件CentOS-6.6-x86_64-bin-DVD1.iso上传到node001/home目录下执行命令挂载镜像mkdir/home/iso/centos6.6-pmv/home/CentOS-6.6-x86_64-bin-DVD1.iso/home/isomount-oloop/home/iso/CentOS-6.6-x86_64-bin-DVD1.iso/home/iso/centos6.6mkdir/etc/yum.repos.d.bakmv/etc/yum.repos.d/*/etc/yum.repos.d.bak/配置repo文件创建/etc/yum.repos.d/CentOS-dvd.repo文件，内容如下：[base]name=CentOS-Repobaseurl=file:///home/iso/centos6.6enabled=1gpgcheck=0gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6在node001上执行以下命令安装必要的软件包：yuminstall-ypython-devel由于版本问题安装失败的rpm包到如下地址下载/centos/6/updates/x86_64/Packages/将下载好的IB驱动上传到node001节点/home目录下在node001执行以下命令编译Infiniband驱动：cd/home/tar-zxvfMLNX_OFED_LINUX-2.4-1.0.4-rhel6.6-x86_64.tgzcd/home/MLNX_OFED_LINUX-2.4-1.0.4-rhel6.6-x86_64./mlnx_add_kernel_support.sh-m/home/MLNX_OFED_LINUX-2.4-1.0.4-rhel6.6-x86_64--make-tgz安装Infiniband驱动重新编译mellanox驱动后，新生成的驱动包存放在/tmp目录下，执行以下命令将驱动包移动到/home目录并解压：mkdir/home/Support_o2ib/mv/tmp/MLNX_OFED_LINUX-*-ext.tgz/home/Support_o2ib/在node001执行以下命令安装必要的软件包：yuminstall-ylsofnumactlgcc-gfortranlibxml2-pythonbctcltk在node001执行以下命令安装驱动包：cd/home/Support_o2ib/tar-zxvfMLNX_OFED_LINUX-2.4-1.0.4-rhel6.6-x86_64-ext.tgz./MLNX_OFED_LINUX-*-ext/mlnxofedinstall完成后不重启，继续进行下个小节配置编译安装Lustre客户端编译安装操作只需要在其中一个计算节点例如node001上进行操作即可，后续通过同步镜像更新到其他所有节点获取Lustre客户端安装包将下载的ieel安装包上传到node001节点/home目录并解压：cd/hometarxvfieel-.tar解压lustre-client安装包cd/homemkdir./lustre_clientcpieel-/lustre-client-2.5.34-bundle.tar.gzlustre_client/cdlustre_client/tarxvzflustre-client-2.5.34-bundle.tar.gz解压后lustre-client-source-2.5.34-2.6.32_504.12.2.el6.x86_64.x86_64.rpm即为所需源码包重新编译Lustre客户端Lustre客户端默认不支持RDMA方式访问Lustre文件系统，需要重新编译。在node001节点上通过以下命令安装lustre-client-source源码包：rpm-ivhlustre-client-source-*.rpm在node001上通过以下命令重新编译Lustre客户端：cd/usr/src/lustre-2.5.34/./configure--with-o2ib=/usr/src/ofa_kernel/default/makerpms在node001上将编译完成的Lustre客户端软件包拷贝至/home目录下cd/usr/src/redhat/RPMS/x86_64cp*/home/安装Lustre客户端在node001上执行以下命令安装lustre客户端：cd/homerpm-ivhlustre-client-modules-*.rpmrpm-ivhlustre-client-2.5.34-*.rpm同步镜像到所有节点将node001状态抓取成镜像打开“Nodes”菜单，选中node001节点；在右方“Tasks”页面下点击“Grabtoimage”按钮；图STYLEREF1\s4SEQ图\*ARABIC\s111抓取节点状态到镜像在弹出选择框中选择“lustre-client-image”；取消勾选“Dryrun”；镜像点击“Yes”。图STYLEREF1\s4SEQ图\*ARABIC\s112注意界面下端日志栏的提示信息，等待抓取完成下发Lustre客户端到其他所有节点点击“Nodes”菜单，在右侧“Tasks”目录下，点击“Updatenodes”。图STYLEREF1\s4SEQ图\*ARABIC\s113更新节点界面取消勾选“Dryrun”；在弹出确认框点击“Yes”，确认更新图STYLEREF1\s4SEQ图\*ARABIC\s114确认更新等待更新完成Lustre并行文件系统部署说明本章节描述IntelEnterpriseEditionforLustrev的安装过程，涉及IMLserver（IntelManagerforLustreserver）、MDS、OSS的安装和配置，存储网络基于MellanoxInfiniband。安装和配置Lustre文件系统详细部署步骤请参考《IntelLustreInfiniband安装部署指导书.docx》配置ldap为保证BCM集群普通用户能够正常访问Lustre文件系统，需要在MDS和OSS节点配置ldap，设定ldapserver为管理节点。在MDS和OSS节点执行以下命令安装ldap：[root@mds01~]#mount-oloop/iso/CentOS-6.6-x86_64-bin-DVD1.iso/iso/centos6.6[root@mds01~]#yuminstall-ynss-pam-ldapdLoadedplugins:fastestmirror,securitySettingupInstallProcessLoadingmirrorspeedsfromcachedhostfileResolvingDependencies-->Runningtransactioncheck>Packagenss-pam-ldapd.x86_640:0.7.5-18.2.el6_4willbeinstalledbase/filelists_db|3.3MB00:00...-->ProcessingDependency:/lib64/security/pam_ldap.soforpackage:nss-pam-ldapd-0.7.5-18.2.el6_4.x86_64HighAvailiable/filelists_db|55kB00:00...-->ProcessingDependency:nscdforpackage:nss-pam-ldapd-0.7.5-18.2.el6_4.x86_64-->Runningtransactioncheck>Packagenscd.x86_640:2.12-1.149.el6willbeinstalled>Packagepam_ldap.x86_640:185-11.el6willbeinstalled-->FinishedDependencyResolutionDependenciesResolved=============================================================================================PackageArchVersionRepositorySize=============================================================================================Installing:nss-pam-ldapdx86_640.7.5-18.2.el6_4base152kInstallingfordependencies:nscdx86_642.12-1.149.el6base223kpam_ldapx86_64185-11.el6base88kTransactionSummary=============================================================================================Install3Package(s)Totaldownloadsize:462kInstalledsize:785kDownloadingPackages:Total80MB/s|462kB00:00Runningrpm_check_debugRunningTransactionTestTransactionTestSucceededRunningTransactionInstalling:pam_ldap-185-11.el6.x86_641/3Installing:nscd-2.12-1.149.el6.x86_642/3Installing:nss-pam-ldapd-0.7.5-18.2.el6_4.x86_643/3Verifying:nss-pam-ldapd-0.7.5-18.2.el6_4.x86_641/3Verifying:nscd-2.12-1.149.el6.x86_642/3Verifying:pam_ldap-185-11.el6.x86_643/3Installed:nss-pam-ldapd.x86_640:0.7.5-18.2.el6_4DependencyInstalled:nscd.x86_640:2.12-1.149.el6pam_ldap.x86_640:185-11.el6Complete!将headnodeIP信息加入所有MDS和OSS节点的hosts文件：[root@mds1~]#cat/etc/hostslocalhostlocalhost.localdomainlocalhost4localhost4.localdomain4::1localhostlocalhost.localdomainlocalhost6localhost6.localdomain61IML2mds13mds24oss15oss2 master.cm.clustermasterlocalmaster.cm.clusterlocalmaster将任意计算节点（例如node001）的ldap配置文件拷贝至MDS和OSS节点在所有MDS和OSS节点执行如下命令：cp/etc/nsswitch.conf/etc/nsswitch.conf.bakcp/etc/pam_ldap.conf/etc/pam_ldap.conf.bakcp/etc/nslcd.conf/etc/nslcd.conf.bakscp01:/cm/conf/etc/nsswitch.conf/etc/nsswitch.confscp01:/etc/pam_ldap.conf/etc/scp01:/etc/nslcd.conf/etc/重启ldap服务并设置开机启动：[root@mds01~]#/etc/init.d/nslcdstartStartingnslcd:[OK][root@mds02~]#chkconfignslcdon客户端配置完成mdt及osst的配置，即可在client端挂载lustre文件系统，并对其进行读写访问。在CMGUI中配置客户端Lustre文件系统的挂载。获取挂载点步骤1在浏览器查看。点击LustreWeb界面，文件系统页面，“ViewClientMountInformation”按钮，看到挂载点信息图STYLEREF1\s5SEQ图\*ARABIC\s11挂载点信息其中“mount-tlustre3@o2ib0:2@o2ib0:/lfs-new/mnt/lfs-new”即为示例的挂载命令进入CMGUI“luster-client”NodeCategory，点击“FSMounts”Tab菜单图STYLEREF1\s5SEQ图\*ARABIC\s12如REF_Ref418174134\h图52所示点击“Add”按钮增加挂载项点击“Add”后弹窗如REF_Ref419358852\h图53所示图STYLEREF1\s5SEQ图\*ARABIC\s13参照《项目实施规划》填写，示例：表项描述值Device被挂载设备3@o2ib0:2@o2ib0:/lfs-newFilesystemtype文件系统类型lustreMountpoint挂载点/mnt/lfs-newRevision不需填写Extraoptions挂载选项defaultsFilesystemcheck文件系统检查0填写完成点击“Ok”确认。3.点击“Save”保存景行集群管理软件实施方案安装准备这里主要罗列出需要准备的软件，硬件按照第三章的配置准备，并完成组网CentOS完整的安装ISO包，CentOS-6.4-x86_64-bin-DVD1.iso和CentOS-6.4-x86_64-bin-DVD2.iso景行集群调度软件：jh-unischeduler-3.0-install-03073.tar.gzJDK1.7：jdk-7u51-linux-x64.gzEnginFrame2013：enginframe-2013.0-r28282.jar景行JHINNO集群管理软件：jhinno_ef_package_2.1.2-r40311.tar.gz安装配置OS我们选择的OS为CentOS6.4x64系统，安装过程较为简单，这里就不详细讲解。主要注意如下选择OS的安装时，选择SoftwareDevelopmentworkstation版本，并勾选Customizenow。在BaseSystem中勾选Compatililitylibraries在数据库中勾选PostgreSQLDatabaseclient和Server，注意需要勾选所有packages。勾选Developmenttools，需要勾选所有安装包勾选NFSServer在SystemManagement中勾选SystemManagement，注意勾选所有安装包以上勾选完成后，安装操作系统。关闭selinux[root@localhostrepodata]#grep^SELINUX=/etc/selinux/configSELINUX=enforcing[root@localhostrepodata]#sed-i's/^SELINUX=.*/SELINUX=disabled/g'/etc/selinux/config[root@localhostrepodata]#grep^SELINUX=/etc/selinux/configSELINUX=disabled关闭防火墙配置业务网络IP地址，重启network服务配置hosts文件配置YUM，详细过程请参考附录重新启动OS配置NIS挂载OS镜像。[root@JH02~]#mount-oloop/home/CentOS-6.4-x86_64-bin-DVD1.iso/media/CD1[root@JH02~]#mount-oloop/home/CentOS-6.4-x86_64-bin-DVD2.iso/media/CD2安装ypserv[root@JH02~]#yum-yinstallypserv安装simplejson[root@JH02~]#yum-yinstallpython-simplejson-2.0.9-3.1.el6.x86_64执行authconfig-tui，选择NIS，点击Next。输入域名和本地IP地址，点击OK重启NIS服务[root@JH02~]#chkconfigypservon[root@JH02~]#serviceypservrestartStoppingYPserverservices:[FAILED]StartingYPserverservices:[OK][root@JH02~]#配置NISMaster，输入Ctrl+D确认Master[root@JH02~]#/usr/lib64/yp/ypinit–m重启ypserv和ypbind服务[root@JH02~]#serviceypservrestartStoppingYPserverservices:[OK]StartingYPserverservices:[OK][root@JH02~]#serviceypbindrestartShuttingdownNISservice:[OK]StartingNISservice:[OK]BindingNISservice:.[OK][root@JH02~]#如何添加用户，需要在NIS服务器上做如下动作#adduser<username>#passwd<username>#cd/var/yp/#make配置NISClientClient不用安装ypserv安装包，直接通过authconfig-tui加入域即可。启用NIS支持，在NIS配置中填写NIS域:hpcNIS服务器：<NISMaster的IP地址>[root@JH01~]#[root@JH01~]#authconfig-tui[root@JH01~]#配置NFSServer新建安装目录/apps和用户的home目录/apps/users[root@JH02~]#mkdir/apps[root@JH02~]#mkdir/apps/users修改NFSServer权限[root@JH02~]#vi/etc/exports[root@JH02~]#cat/etc/exports/apps*(rw,no_root_squash)在NSFServer输入一下命令[root@JH02~]#cat/etc/exports/apps*(rw,no_root_squash)[root@JH02~]#exportfs-a[root@JH02~]#chkconfignfson[root@JH02~]#servicenfsrestartShuttingdownNFSdaemon:[FAILED]ShuttingdownNFSmountd:[FAILED]ShuttingdownNFSquotas:[FAILED]ShuttingdownNFSservices:[OK]StartingNFSservices:[OK]StartingNFSquotas:[OK]StartingNFSmountd:[OK]StoppingRPCidmapd:[OK]StartingRPCidmapd:[OK]StartingNFSdaemon:[OK][root@JH02~]#在NFS客户端上，做如下配置可以挂载NFSServer的/apps目录mkdir/appsvi/etc/fstab...<NFSSever的IP地址>：/apps/appsnfsdefaults00...mount-a安装JDK新建jhadmin用户[root@JH02~]#useraddjhadmin-b/apps/users/[root@JH02~]#passwdjhadminChangingpasswordforuserjhadmin.Newpassword:BADPASSWORD:itisbasedonadictionarywordBADPASSWORD:istoosimpleRetypenewpassword:passwd:allauthenticationtokensupdatedsuccessfully.[root@JH02~]#[root@JH02~]#cd/var/yp/[root@JH02yp]#makegmake[1]:Enteringdirectory`/var/yp/'Updatingpasswd.byname...Updatingpasswd.byuid...Updatinggroup.byname...Updatinggroup.bygid...Updatingnetid.byname...gmake[1]:Leavingdirectory`/var/yp/'[root@JH02yp]#把jdk-7u51-linux-x64.gz拷贝的/apps目录下，进入/apps目录，解压安装包到/opt目录tar-zxvfjdk-7u51-linux-x64.gz-C/opt/在/etc/profile最后添加如下命令exportJAVA_HOME=/opt/jdk1.7.0_51exportPATH=$JAVA_HOME/bin:$PATHexportCLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar验证JDK是否安装成功[root@JH02apps]#[root@JH02apps]#source/etc/profile[root@JH02apps]#java-versionjavaversion"1.7.0_51"Java(TM)SERuntimeEnvironment(build1.7.0_51-b13)JavaHotSpot(TM)64-BitServerVM(build24.51-b03,mixedmode)[root@JH02apps]#安装jh-unischeduler把jh-unischeduler-3.0-install-03073.tar.gz解压安装包[root@JH02apps]#tar-zxvfjh-unischeduler-3.0-install-03073.tar.gz进入解压目录，编辑install.conf[root@JH02jh-unischeduler-3.0-install]#viinstall.conf把license目录添加进去执行./install.py安装unischeduler[root@JH02jh-unischeduler-3.0-install]#./install.py14-03-2622:34:42INFOinstaller(606):瑙ｅ.缂?./linux/x86_64/jh-unischeduler-3.0-linux-x86_64-0307-1624.tar.gz...14-03-2622:34:45INFOinstaller(517):?..?..postgresql-jhscheduler锛...14-03-2622:34:49INFOinstall(160):瀹.?瀹..[root@JH02jh-unischeduler-3.0-install]#启动jhscheduler[root@JH02jh-unischeduler-3.0-install]#source/apps/unischeduler/conf/profile.jhscheduler[root@JH02jh-unischeduler-3.0-install]#jhschedulerstartStartingdaemons...limstartedresstartedsbatchdstarted[root@JH02jh-unischeduler-3.0-install]#检查服务是否正常启动[root@JH02jh-unischeduler-3.0-install]#jhschedulerstatuslimpid:<2989>respid:<2991>sbatchdpid:<2993>mbatchdpid:<3007>schedpid:<3433>[root@JH02jh-unischeduler-3.0-install]#安装Enginframe上传enginframe-2013.0-r28282.jar到/apps目录下运行Java–jarenginframe-2013.0-r28282.jar启动Enginframe的安装。java-jarenginframe-2013.0-r28282.jarWritinglogto(/tmp/efinstall.2014-03-26-22.41.58.log)点击Next勾选Iaccept…导入License输入安装目录/apps/nice，点击Next点击yes，创建目录确认jre目录点击Next。点击yes，创建目录选择ServerandAgent输入Enginframe管理员jhadmin，需要和jh-unischeduler的管理员一致。配置OSuserowning和Webapplication。其中OSuser是enginframe管理员，ContextoftheEnginframewebapplication设置为cloud，其他使用默认配置。点击Next按照默认值，点击Next是否开机运行，选择yes勾选InstalltheEnginFramedemo选择OperatingSystem设置enginframe的用户名和密码，用户名设置为Enginframe管理员名称，即第8步中设置的用户，用户名密码必须和OS密码一致选择IBMLSForOpenLava选择jh-unischeduler的profile文件点击yes，继续安装点击Install进行安装点击Finish，安装完成修改/apps/nice/enginframe/conf/server.conf中的ef.triggers.db.url=jdbc:derby://localhost:1527/EFTriggersDB的1527端口改为51527端口，如下所示，保存退出。修改/apps/nice/conf/enginframe.conf文件中的DERBY_PORT端口，把1527改为51527，如下图所示，保存退出。启动Enginframe[root@JH02apps]#serviceenginframestartEnginFrameControlScriptStartingDerbyDB[PID:43460]...[OK]DerbyDatabasestarted[OK]EnginFrameServerstarted[OK]EnginFrameAgentstarted[root@JH02apps]#安装jhinno把jhinno_ef_package_2.1.2-r40311.tar.gz上传到/home目录，并解压tar-zxvfjhinno_ef_package_2.1.2-r40311.tar.g切换到su–jhadmin[root@JH02home]#sujhadmin[jhadmin@JH02home]$cdjhinno_ef_package把jobstarter拷贝到/apps/unischeduler/[jhadmin@JH02jhinno_ef_package]$cp-rjhscheduler/jobstarter/apps/unischeduler/把lsb.users和lsb.quenes覆盖/apps/unischeduler/conf/users.conf和queues.conf[jhadmin@JH02jhinno_ef_package]$cpjhscheduler/conf/lsb.users/apps/unischeduler/conf/users.conf[jhadmin@JH02jhinno_ef_package]$cpjhscheduler/conf/lsb.queues/apps/unischeduler/conf/queues.conf修改queues.conf文件[root@JH02apps]#vi/apps/unischeduler/conf/queues.conf把JOB_STARTER=/apps/jhscheduler/jobstarter/fluent_starterJOB_STARTER=/apps/jhscheduler/jobstarter/fluent_starterJOB_STARTER=/apps/jhscheduler/jobstarter/cfx_starterJOB_STARTER=/apps/jhscheduler/jobstarter/cfx_starter改为JOB_STARTER=/apps/jhscheduler/jobstarter/fluent_starterJOB_STARTER=/apps/unischeduler/jobstarter/fluent_starterJOB_STARTER=/apps/unischeduler/jobstarter/cfx_starterJOB_STARTER=/apps/unischeduler/jobstarter/cfx_starter切换到root[jhadmin@JH02jhinno_ef_package]$surootPassword:[root@JH02jhinno_ef_package]#修改/root/.bashrc文件，注释掉aliascp=’cp-i’使用root用户重新登陆把jhinno_ef_package中的Enginframe和tomcat拷贝到/apps/nice下[root@JH02jhinno_ef_package]#cp-renginframe/apps/nice/[root@JH02jhinno_ef_package]#cp-rtomcat/apps/nice/增加jhadmin的权限[root@JH02jhinno_ef_package]#chown-Rjhadmin:jhadmin/apps/nice/tomcat/webapps/customer[root@JH02jhinno_ef_package]#chownjhadmin:jhadmin/apps/nice/enginframe/conf/authorization.xconf[root@JH02jhinno_ef_package]#chown-Rjhadmin:jhadmin/apps/nice/enginframe/conf/jhlogon在/apps/nice/tomcat/bin/setenv.ef.sh文件头上加入一个参数:exportJHCUSTOMER_CONFDIR=/apps/nice/tomcat/webapps/customer/conf[root@JH02enginframe]#cat/apps/nice/tomcat/bin/setenv.ef.shexportJHCUSTOMER_CONFDIR=/apps/nice/tomcat/webapps/customer/confCATALINA_OPTS="-Dtocol.handler.pkgs=mon.utils.xml.handlers$CATALINA_OPTS"CLASSPATH="$CLASSPATH":"$CATALINA_HOME"/lib/sdftree-handler.jar修改/apps/nice/enginframe/plugins/lsf/bin/ef.opendialog.sh将以下环境变量JHSCHEDULER_ENV=/apps/jhscheduler/conf/profile.jhscheduler改为JHSCHEDULER_ENV=/apps/unischeduler/conf/profile.lsf修改/apps/nice/enginframe/plugins/lsf/conf/ef.lsf.conf文件将以下环境变量LSF_PROFILE="/apps/jhscheduler/conf/profile.jhscheduler"改为LSF_PROFILE="/apps/unischeduler/conf/profile.jhscheduler"在/apps/nice/enginframe/plugins/vnc/conf/ef.vnc.conf文件修改VNC_HOSTNAME=””VNC_HOSTNAME_INT=””VNC_LSGRUN=”lsrun–m<vnc服务器名>”。执行如下命令chmod777/apps/nice/enginframe/plugins/vnc/etc/*修改enginframe/plugins/hpc/WEBAPP/application.xml文件[root@JH02nice]#vimenginframe/plugins/hpc/WEBAPP/application.xml把<ef:optionid="6.3.26">6.3.26</ef:option>改为<ef:optionid="14.5.0">14.5.0</ef:option>注意需要修改两个地方修改/apps/unischeduler/jobstarter/fluent_starter[root@JH02.lsbatch]#vim/apps/unischeduler/jobstarter/fluent_starter把文件头的PATH改为fluent的安装路径，如下所示exportPATH=/apps/ansys_inc/v145/fluent/bin:$PATH重新启动Enginframe服务[root@JH02tomcat]#serviceenginframestopEnginFrameControlScriptKillingTomcatwiththePID:44642WedMar2623:44:34CST2014:ApacheDerbyNetworkServer--(1344872)shutdown[OK]DerbyDatabaseisdown[OK]EnginFrameAgentisdown[root@JH02tomcat]#[root@JH02tomcat]#[root@JH02tomcat]#serviceenginframestartEnginFrameControlScriptStartingDerbyDB[PID:50413]...[OK]DerbyDatabasestarted[OK]EnginFrameServerstarted[OK]EnginFrameAgentstarted[root@JH02tomcat]#通过浏览器登陆景行高性能集群仿真平台http://主机IP:8080/通过jhadmin账户登陆安装VNC上传nice-dcv-2013.0-9073.iso到/home目录中新建/nice目录[root@JH02home]#mkdir/nice挂载nice-dcv-2013.0-9073.iso镜像文件[root@JH02home]#mount-oloop/home/nice-dcv-2013.0-9073.iso/nice在图形界面中安装VNC，安装包为：vnc-VE4_6_3-x64_linux.rpm，其路径为/nice/NICE/linux/rpms/双击vnc-VE4_6_3-x64_linux.rpm，点击Install安装完成后，导入License。[root@JH02home]#vnclicense-addHABA2-34RXF-4Q4PH-24N4R-AVYEB[root@JH02home]#启动vncserver[root@JH02home]#vncserverVNC(R)ServerVisualizationEditionVE4.6.3(r99394)BuiltonNov8201216:40:29Copyright(C)2002-2012RealVNCLtd.VNCisaregisteredtrademarkofRealVNCLheU.S.andinothercountries.SeeforinformationonVNC.Forthirdpartyacknowledgementssee:/products/enterprise/4.6/acknowledgements.htmlWarning:1istakenbecauseof/tmp/.X1-lockRemovethisfileifthereisnoXServerrunningas:1Runningapplicationsin/etc/vnc/xstartupVNCServersignature:c4-54-b3-b5-97-bc-2a-d7Logfileis/root/.vnc/JH02:2.logNewdesktopisJH02:2(02:2)[root@JH02home]#景行DCV方案实施流程安装准备这里主要罗列出需要准备的软件，硬件按照第三章的配置准备，并完成组网CentOS完整的安装ISO包，CentOS-6.4-x86_64-bin-DVD1.iso和CentOS-6.4-x86_64-bin-DVD2.iso景行DCV安装包：nice-dcv-2013.0-9073.isoNvidiaGT610驱动：NVIDIA-Linux-x86_64-331.49.runWin732位操作系统：X17-24280_Win7_English_professional_sp1-32bit.iso安装配置OS我们选择的OS为CentOS6.4x64系统，安装过程较为简单，这里就不详细讲解。主要注意如下，注意插上显卡可能无法安装系统，需要把显卡拔掉，安装完系统后，在插上显卡。选择OS的安装时，选择SoftwareDevelopmentworkstation版本，并勾选Customizenow。在BaseSystem中勾选Compatililitylibraries在数据库中勾选PostgreSQLDatabaseclient和Server，注意需要勾选所有packages。勾选Developmenttools，需要勾选所有安装包在SystemManagement中勾选SystemManagement，注意勾选所有安装包在Virtualization中勾选Virtualization、VirtualizationClient和VirtualizationPlatform，注意需要勾选所有安装包。以上勾选完成后，安装操作系统。关闭selinux[root@localhostrepodata]#grep^SELINUX=/etc/selinux/configSELINUX=enforcing[root@localhostrepodata]#sed-i's/^SELINUX=.*/SELINUX=disabled/g'/etc/selinux/config[root@localhostrepodata]#grep^SELINUX=/etc/selinux/configSELINUX=disabled关闭防火墙配置业务网络IP地址，重启network服务配置hosts文件配置YUM，详细过程请参考附录修改grub启动文件[root@JH01yum.repos.d]#vim/boot/grub/grub.conf在kernel行最后加上acpi=offnoapic下电系统，安装显卡，在重新启动系统安装显卡驱动输入lspci验证是否发现硬件设备04:00.0VGAcompatiblecontroller:NVIDIACorporationGF119[GeForceGT610](reva1)上传显卡驱动包NVIDIA-Linux-x86_64-331.49.run到系统中退出图形界面[root@JH01~]#init3[root@JH01~]#屏蔽nouveau第三方驱动，因为nouveau这个第三方nvidia驱动仍然在运行，且与官方驱动冲突。禁用nouveau，将nouveau加入黑名单，编辑黑名单文件：

vi/etc/modprobe.d/blacklist.conf在最后一行后加上blacklistnouveau并保存。修改/etc/grub.conf中的kernel参数，再该参数后面添加rdblacklist=nouveau来禁止加载该nouveau内核安装驱动shNVIDIA-Linux-x86_64-331.49.run-k$(uname–r)选择accept，按Enter选择yes，点击Enter点击yes点击yes点击OK，完成显卡驱动安装执行如下命令nvidia-xconfig--add-argb-glx-visuals--allow-glx-with-composite--busid="YOUR_PCI_ID"--damage-events--disable-glx-root-clipping-a--no-logo--overlay--cioverlay--render-accel--registry-dwords="PowerMizerEnable=0x1;PerfLevelSrc=0x2222;PowerMizerDefault=0x1;PowerMizerDefaultAC=0x1"--no-use-edid--no-use-edid-freqs--use-display-device="None"执行如下命令nvidia-xconfig-enable-all-gpus重启启动系统，在BIOS中，VideoCardSelected中选择OptionalVideoCard，按F10保存退出。Linux主机上安装配置VNCServer上传nice-dcv-2013.0-9073.iso到/home目录中新建/nice目录[root@JH01~]#mkdir/nice挂载nice-dcv-2013.0-9073.iso到/nice目录上[root@JH01~]#mount–oloop/home/nice-dcv-2013.0-9073.iso/nice在图形界面中安装VNCServer，需要在本地KVM上操作。安装包为：vnc-VE4_6_3-x64_linux.rpm，其路径为/nice/NICE/linux/rpms/双击vnc-VE4_6_3-x64_linux.rpm，点击Install安装完成后，导入License。[root@JH02home]#vnclicense-addHABA2-34RXF-4Q4PH-24N4R-AVYEB[root@JH02home]#启动vncserver[root@JH01~]#vncserverVNC(R)ServerVisualizationEditionVE4.6.3(r99394)BuiltonNov8201216:40:29Copyright(C)2002-2012RealVNCLtd.VNCisaregisteredtrademarkofRealVNCLheU.S.andinothercountries.SeeforinformationonVNC.Forthirdpartyacknowledgementssee:/products/enterprise/4.6/acknowledgements.htmlRunningapplicationsin/etc/vnc/xstartupVNCServersignature:ed-90-f7-32-65-68-10-0cLogfileis/root/.vnc/JH01:1.logNewdesktopisJH01:1(01:1)[root@JH01~]#修改vnc系统配置文件，加入x11字体路径和depth等选项，更新完成后，保存退出。vim/etc/vnc/config文件，修改-fp参数，保证如下配置。-fp"unix/:7100,built-ins,/usr/share/X11/fonts/100dpi,/usr/share/X11/fonts/75dpi,/usr/share/X11/fonts/Type1" 同时，添加如下选项 -depth24Config文件如下所示：#DefaultXServercommand-lineparametersforVNCEnterpriseEdition.##Thisfileisautomaticallygenerated.DONOTEDIT.#Tooverridesettingsinthisfile,createoredit/etc/vnc/config.custom.#Continueevenifstandardportsfail-pn-fp"unix/:7100,built-ins,/usr/share/X11/fonts/100dpi,/usr/share/X11/fonts/75dpi,/usr/share/X11/fonts/Type1"-depth24Linux主机上安装配置RenderingServer挂载OS镜像文件[root@JH01~]#mount-oloop/home/CentOS-6.4-x86_64-bin-DVD1.iso/media/CD1[root@JH01~]#mount-oloop/home/CentOS-6.4-x86_64-bin-DVD2.iso/media/CD2进入dcvserveringserver安装包目录[root@JH01~]#cd/nice/NICE/linux/rpms/rhel_5_x86_64/安装nice-dcv-server-2013.0-9073.i686.rpm和nice-dcv-server-2013.0-9073.x86_64.rpm安装包[root@JH01rhel_5_x86_64]#yum-yinstallnice-dcv-server-2013.0-9073.x86_64.rpm[root@JH01rhel_5_x86_64]#yum-yinstallnice-dcv-server-2013.0-9073.i686.rpm修改DCV的配置文件：vim/opt/nice/dcv/conf/dcv.conf将其中的[Remotization]section的host修改为本机的ip地址host=01将[RenderingServer]section的host修改成为KVM内部的网关地址，一般也为主机IP，如host=01拷贝/opt/nice/dcv/etc/init.d/dcvrenderingserver到系统的/etc/init.d/目录下[root@JH01rhel_5_x86_64]#cp/opt/nice/dcv/etc/init.d/dcvrenderingserver/etc/init.d/添加、运行启动dcvrenderingserver服务chkconfig--adddcvrenderingserverchkconfigdcvrenderingserveron/etc/init.d/dcvrenderingserverstart[root@JH01rhel_5_x86_64]#chkconfig--adddcvrenderingserver[root@JH01rhel_5_x86_64]#chkconfigdcvrenderingserveron[root@JH01rhel_5_x86_64]#/etc/init.d/dcvrenderingserverstartStartingdcvrenderingserver:NICEDCV:Grantingaccessto3DacceleratedXservertouser'dcv'...localuser:dcvbeingaddedtoaccesscontrollistOK导入DCVLicense，上传license.lic文件到/home目录下把License文件拷贝到/opt/nice/dcv/license/目录[root@JH01rhel_5_x86_64]#cp/home/license.lic/opt/nice/dcv/license/添加启动rlm服务[root@JH01rhel_5_x86_64]#cp/opt/nice/dcv/etc/init.d/rlm/etc/init.d/[root@JH01rhel_5_x86_64]#chkconfig--addrlm[root@JH01rhel_5_x86_64]#chkconfigrlmon修改rlm脚本，删除-o%PPID参数启动rlm服务[root@JH01rhel_5_x86_64]#servicerlmstartStartingrlm:OK安装配置KVM网桥创建桥接器在/etc/sysconfig/network-scripts目录下，创建一个ifcfg-br0文件，其类型设为Bridge[root@JH01network-scripts]#catifcfg-br0DEVICE=br0TYPE=BridgeONBOOT=yesIPADDR=01NETMASK=GATEWAY=NETWORK=将物理网口桥接到桥接器上修改eth0的内容（本服务器是用eth0上网的），去掉其IP相关信息，加上“BRIDGE=br0”，将其桥接到br0上；如果是双网卡或是多网卡，照此过程修改DEVICE=eth0TYPE=EthernetONBOOT=yesBRIDGE=br0停止NetworkManager服务，重启启动网络服务安装配置Win7虚拟机上传virtio-win-1.6.3-3.el6.noarch.rpm到/home目录上，安装此文件[root@JH01home]#rpm-ivhvirtio-win-1.6.3-3.el6.noarch.rpm安装完成后，virtio-win相关驱动放在/usr/share/virtio-win下，包括软驱格式的驱动：virtio-win-1.5.2.vfd，光驱格式的驱动：virtio-win-1.5.2.iso上传Win732位的OS到Linux主机上执行virt-manager点击New，输入虚拟机名称，选择Localinstallmedia，点击Forward载入Win732位的操作系统ISO文件，OStype选择Windows，Version选择MicrosoftWindows7，点击Forward。给虚拟机分配内存和CPU资源给虚拟机分配硬盘容量，勾选Allocateentiredisknow勾选Customizeconfigurationbeforeinstall，点击Finish处理器“Configuration”，最好点击“CopyhostCPUconfiguration”，保证虚拟机CPU支持的指令集与物理CPU一致。定义处理器“Topology”和“Pin”，对性能有少量提升。特别是XP和Windows7最大只能支持2个物理CPU，如果你需要在虚拟机中分配更多的core给虚拟机，就必须定义Topology。点击Apply在虚拟磁盘配置，选择”DiskBus”为：virtio。点击Apply虚拟网络接口配置中，选择“DeviceModel”为：virtio，点击Apply点击左下方的AddHardware添加软驱设备，如下图所示点击BeginInstallation开始安装Win7系统安装系统过程比较简单，在如下界面中点击LoadDriver选择RedHatVirtIOSCSIcontroller(A:\i38\Win7\viostor.inf)，点击Next。安装硬盘驱动。硬盘驱动安装完成后，如下所示，可以进行分区操作，点击Next。Win7系统开始安装系统安装完成安装网卡驱动，打开设备管理器点击EthernetController点击UpdateDriver..，选择软驱目录安装完成后点击Close配置IP地址，注意虚拟机的键盘和真实的键盘并不对应。VMWin7安装配置DCVServer和VNCServer挂载nice-dcv-2013.0-9073.iso到虚拟机上点击Runsetup.exe点击Next勾选Iaccept…点击Next选择安装路径勾选Iaccept…，点击Next勾选UseanExternalRenderingServer，输入Linux主机IP，勾选InstallVirtIOnetworkdrivers勾选Iaccept…添加License文件点击Next点击Install点击Finish安装完成重启系统导入VNCServerLicenseKey输入LicenseKeyClient端安装配置DCVEndstation解压nice-dcv-2013.0-9073.iso文件运行nice-dcv-endstation-2013.0-9073-Release.msi文件，其目录为nice-dcv-2013.0-9073\NICE\win32按照提示安装重新启动系统双击setup-niceviewer.bat文件，其目录为nice-dcv-2013.0-9073\Portable-Endstation-Windows双击niceviewer.bat，开启一个VNCView实例，其路径为nice-dcv-2013.0-9073\Portable-Endstation-Windows输入VMWin7的IP地址，点击Connect点击yes输入VMWin7的用户名和密码可以正常通过VNC远程登陆虚拟机打开CMD命令行输入dcvadmin输入dcvadminenable，开启DCV输入dcvon打开DCV，输入dcvtest验证是否安装成功出现DCV图形界面说明安装成功。方案验收集群性能验收计算能力验收Linpack现在在国际上已经成为最流行的用于测试高性能计算机系统浮点性能的benchmark。通过利用高性能计算机，用高斯消元法求解N元一次稠密线性代数方程组的测试，评价高性能计算机的浮点性能。HPL即HighPerformanceLinpack，也叫高度HYPE