[计算机软件及应用]EMCGreenplum安装手册_redhat.doc

3.操作系统安装服务器启动时根据提示使用ctrl+f键进入Raid卡设置，使用strip size 128KB, 启用读缓存, 启用写缓存, 将6块SAS划为一个磁盘组。服务器重启后进入操作系统安装：3.1.分区依据：/bootext3100MBSwapswap内存大小/ext3100GB不作处理不作处理剩余空间*注：剩余空间约1.3TB不处理，用于操作系统安装后使用xfs格式化。系统安装完毕使用xfs格式化如下：mkfs.xfs /dev/sda43.2.文件包依据： 不要安装防火墙, selinux, LVM 不要安装办公工具，例如office，浏览器，游戏，音视频工具 必须安装开发工具，例如glibc4.操作系统配置修改内核调度/boot/grub/menu.lst文件kernal添加elevator=deadline如下：rootlocalhost # vi /boot/grub/menu.lstdefault=0timeout=5splashimage=(hd0,0)/grub/splash.xpm.gzhiddenmenutitle Red Hat Enterprise Linux Server (2.6.18-308.el5) root (hd0,0) kernel /vmlinuz-2.6.18-308.el5 ro root=LABEL=/ rhgb quiet elevator=deadline initrd /initrd-2.6.18-308.el5.img修改/etc/rc.d/rc.local文件中所有磁盘预读为16384 （）NO如下：rootlocalhost # blockdev -setra 16384 /dev/sd*修改/etc/sysctl.conf添加如下内容：net.ipv4.ip_forward = 0net.ipv4.conf.default.accept_source_route = 0kernel.sysrq = 0kernel.core_uses_pid = 1net.ipv4.tcp_syncookies = 1kernel.msgmnb = 65536kernel.msgmax = 65536kernel.sem = 250 64000 100 2048kernel.shmmax = 5000000000kernel.shmmni = 40960kernel.shmall = 40000000000net.ipv4.tcp_tw_recycle=1net.ipv4.tcp_max_syn_backlog=4096dev_max_backlog=10000vm.overcommit_memory=2net.ipv4.conf.all.arp_filter = 1修改/etc/security/limits.conf添加如下内容：* soft nofile 1048576* hard nofile 1048576* soft nproc 1048576* hard nproc 1048576修改/etc/sysconfig/network 文件中hostname如下：rootlocalhost # vi /etc/sysconfig/networkNETWORKING=yesNETWORKING_IPV6=noHOSTNAME=crsz6sbidw01修改所有服务器/etc/hosts建立集群通信（NO，与此不同）rootlocalhost # vi /etc/hosts localhost7 mdw-1 mdw crsz6sbidw01 sdw1-1 sdw1 mdw-2 sdw1-2 mdw-3 sdw1-3 mdw-4 sdw1-48 smdw-1 smdw crsz6sbidw02 sdw2-1 sdw2 smdw-2 sdw2-2 smdw-3 sdw2-3 smdw-4 sdw2-4修改/data挂载xfs设备（NO）rootlocalhost # vi /etc/fstabLABEL=/ / ext3 defaults 1 1LABEL=/boot1 /boot ext3 defaults 1 2tmpfs /dev/shm tmpfs defaults 0 0devpts /dev/pts devpts gid=5,mode=620 0 0sysfs /sys sysfs defaults 0 0proc /proc proc defaults 0 0LABEL=SWAP-sda3 swap swap defaults 0 0/dev/sda4 /data xfs rw,noatime,inode64,allocsize=16m 0 0mount /data修改/data目录权限完全给gpadminrootlocalhost # chown -R gpadmin:gpadmin /data5.安装gpdb执行安装程序，根据提示选择yes确认即可安装完毕root localhost # bash greenplum-db--build-2-RHEL5-x86_64.bin根据安装目录定义gpadmin环境变量gpadmincrsz6sbidw01 $ cat .bashrcexport LANG=source /usr/local/greenplum-db/greenplum_path.sh建立集群信任：创建所有网口的列表文件如下：gpadmincrsz6sbidw01 $ cat /data/allhosts mdwmdw-1mdw-2mdw-3mdw-4smdwsmdw-1smdw-2smdw-3smdw-4sdw1sdw1-1sdw1-2sdw1-3sdw1-4sdw2sdw2-1sdw2-2sdw2-3sdw2-4gpadmincrsz6sbidw01 $ gpssh-exkeys -f /data/allhosts STEP 1 of 5 create local ID and authorize on local hostSTEP 2 of 5 keyscan all hosts and update known_hosts fileSTEP 3 of 5 authorize current user on remote hosts . send to mdw . send to mdw-1 . send to mdw-2 . send to mdw-3 . send to mdw-4 . send to smdw * * Enter password for smdw: . send to smdw-1 . send to smdw-2 . send to smdw-3 . send to smdw-4 . send to sdw1 . send to sdw1-1 . send to sdw1-2 . send to sdw1-3 . send to sdw1-4 . send to sdw2 . send to sdw2-1 . send to sdw2-2 . send to sdw2-3 . send to sdw2-4STEP 4 of 5 determine common authentication file contentSTEP 5 of 5 copy authentication files to all remote hosts . finished key exchange with mdw . finished key exchange with mdw-1 . finished key exchange with mdw-2 . finished key exchange with mdw-3 . finished key exchange with mdw-4 . finished key exchange with smdw . finished key exchange with smdw-1 . finished key exchange with smdw-2 . finished key exchange with smdw-3 . finished key exchange with smdw-4 . finished key exchange with sdw1 . finished key exchange with sdw1-1 . finished key exchange with sdw1-2 . finished key exchange with sdw1-3 . finished key exchange with sdw1-4 . finished key exchange with sdw2 . finished key exchange with sdw2-1 . finished key exchange with sdw2-2 . finished key exchange with sdw2-3 . finished key exchange with sdw2-4INFO completed successfully6.测试系统磁盘网络性能6.1.磁盘性能测试结果：nohup gpcheckperf -h mdw -h smdw -d /data/primary -d /data/mirror -r ds gpcheckperfio2.loggpadmincrsz6sbidw01 $ cat gpcheckperfio2.log /usr/local/greenplum-db/./bin/gpcheckperf -h mdw -h smdw -d /data/primary -d /data/mirror -r ds= disk write avg time (sec): 167.81 disk write tot bytes: 201875652608 disk write tot bandwidth (MB/s): 1147.25 disk write min bandwidth (MB/s): 571.69 smdw disk write max bandwidth (MB/s): 575.56 mdw disk read avg time (sec): 123.12 disk read tot bytes: 201875652608 disk read tot bandwidth (MB/s): 1563.71 disk read min bandwidth (MB/s): 780.46 mdw disk read max bandwidth (MB/s): 783.25 smdw stream tot bandwidth (MB/s): 16734.46 stream min bandwidth (MB/s): 8344.28 smdw stream max bandwidth (MB/s): 8390.18 mdw磁盘性能基本满足6块SAS盘正常水平。6.2.网络性能测试结果：创建4个子网IP的子文件，示例如下：gpadmincrsz6sbidw01 $ vi hostic1mdw-1smdw-1gpadmincrsz6sbidw01 $ nohup gpcheckperf -f hostic1 -r N -d /tmp gpcheckperfnet1.loggpadmincrsz6sbidw01 $ cat gpcheckperfnet1.log/usr/local/greenplum-db/./bin/gpcheckperf -f hostic1 -r N -d /tmp- NETPERF TEST-= RESULT=Netperf bisection bandwidth testmdw-1 - smdw-1 = 112.410000smdw-1 - mdw-1 = 112.400000Summary:sum = 224.81 MB/secmin = 112.40 MB/secmax = 112.41 MB/secavg = 112.41 MB/secmedian = 112.41 MB/sec网络性能基本满足千兆网络正常水平。7.初始化gpdb创建初始化配置文件：gpadmincrsz6sbidw01 $ vi gpinitsystem_config ARRAY_NAME=EMC Greenplum DWSEG_PREFIX=gpsegPORT_BASE=40000declare -a DATA_DIRECTORY=(/data/primary /data/primary /data/primary /data/primary /data/primary /data/primary /data/primary /data/primary)MASTER_HOSTNAME=mdwMASTER_DIRECTORY=/data/masterMASTER_PORT=5432TRUSTED_SHELL=sshCHECK_POINT_SEGMENTS=8ENCODING=UNICODEMIRROR_PORT_BASE=50000REPLICATION_PORT_BASE=41000MIRROR_REPLICATION_PORT_BASE=51000declare -a MIRROR_DATA_DIRECTORY=(/data/mirror /data/mirror /data/mirror /data/mirror /data/mirror /data/mirror /data/mirror /data/mirror)MACHINE_LIST_FILE=/home/gpadmin/hostfile_gpinitsystem初始化系统：gpadmincrsz6sbidw01 $ gpinitsystem -c gpinitsystem_config -s smdw20130117:22:47:34:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Checking configuration parameters, please wait.20130117:22:47:34:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Reading Greenplum configuration file gpinitsystem_config20130117:22:47:34:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Locale has not been set in gpinitsystem_config, will set to default value20130117:22:47:34:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Locale set to en_US.utf820130117:22:47:34:015705 gpinitsystem:crsz6sbidw01:gpadmin-WARN:-Master hostname mdw does not match hostname output20130117:22:47:34:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Checking to see if mdw can be resolved on this host20130117:22:47:34:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Can resolve mdw to this host20130117:22:47:34:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-No DATABASE_NAME set, will exit following template1 updates20130117:22:47:34:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-MASTER_MAX_CONNECT not set, will set to default value 25020130117:22:47:35:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Checking configuration parameters, Completed20130117:22:47:35:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Commencing multi-home checks, please wait.20130117:22:47:37:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Configuring build for multi-home array20130117:22:47:37:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Commencing multi-home checks, Completed20130117:22:47:37:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Building primary segment instance array, please wait.20130117:22:47:41:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Building group mirror array type , please wait.20130117:22:47:45:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Checking Master host20130117:22:47:45:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Checking new segment hosts, please wait.20130117:22:48:02:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Checking new segment hosts, Completed20130117:22:48:02:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Greenplum Database Creation Parameters20130117:22:48:02:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-20130117:22:48:02:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Master Configuration20130117:22:48:02:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-20130117:22:48:02:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Master instance name = EMC Greenplum DW20130117:22:48:02:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Master hostname = mdw20130117:22:48:02:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Master port = 543220130117:22:48:02:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Master instance dir = /data/master/gpseg-120130117:22:48:02:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Master LOCALE = en_US.utf820130117:22:48:02:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Greenplum segment prefix = gpseg20130117:22:48:02:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Master Database = 20130117:22:48:02:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Master connections = 25020130117:22:48:02:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Master buffers = 128000kB20130117:22:48:02:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Segment connections = 75020130117:22:48:02:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Segment buffers = 128000kB20130117:22:48:02:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Checkpoint segments = 820130117:22:48:02:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Encoding = UNICODE20130117:22:48:02:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Postgres param file = Off20130117:22:48:02:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Initdb to be used = /usr/local/greenplum-db/./bin/initdb20130117:22:48:02:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-GP_LIBRARY_PATH is = /usr/local/greenplum-db/./lib20130117:22:48:02:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Ulimit check = Passed20130117:22:48:02:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Array host connect type = Multi hostname per node20130117:22:48:02:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Master IP address 1 = 720130117:22:48:02:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Master IP address 2 = 2020130117:22:48:02:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Master IP address 3 = 20130117:22:48:02:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Master IP address 4 = 20130117:22:48:02:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Master IP address 5 = 20130117:22:48:02:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Standby Master = smdw20130117:22:48:02:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Primary segment # = 420130117:22:48:03:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Standby IP address = 820130117:22:48:03:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Standby IP address = 2020130117:22:48:03:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Standby IP address = 20130117:22:48:03:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Standby IP address = 20130117:22:48:03:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Standby IP address = 20130117:22:48:03:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Total Database segments = 820130117:22:48:03:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Trusted shell = ssh20130117:22:48:03:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Number segment hosts = 820130117:22:48:03:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Mirror port base = 5000020130117:22:48:03:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Replicaton port base = 4100020130117:22:48:03:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Mirror replicaton port base= 5100020130117:22:48:03:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Mirror segment # = 420130117:22:48:03:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Mirroring config = ON20130117:22:48:03:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Mirroring type = Group20130117:22:48:03:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-20130117:22:48:03:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Greenplum Primary Segment Configuration20130117:22:48:03:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-20130117:22:48:03:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-sdw1-1 /data/primary/gpseg0 40000 2 0 4100020130117:22:48:03:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-sdw1-2 /data/primary/gpseg1 40001 3 1 4100120130117:22:48:03:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-sdw1-3 /data/primary/gpseg2 40002 4 2 4100220130117:22:48:03:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-sdw1-4 /data/primary/gpseg3 40003 5 3 4100320130117:22:48:03:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-sdw2-1 /data/primary/gpseg4 40000 6 4 4100020130117:22:48:03:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-sdw2-2 /data/primary/gpseg5 40001 7 5 4100120130117:22:48:03:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-sdw2-3 /data/primary/gpseg6 40002 8 6 4100220130117:22:48:03:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-sdw2-4 /data/primary/gpseg7 40003 9 7 4100320130117:22:48:03:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-20130117:22:48:03:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Greenplum Mirror Segment Configuration20130117:22:48:03:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-20130117:22:48:03:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-sdw2-2 /data/mirror/gpseg0 50000 10 0 5100020130117:22:48:03:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-sdw2-3 /data/mirror/gpseg1 50001 11 1 5100120130117:22:48:03:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-sdw2-4 /data/mirror/gpseg2 50002 12 2 5100220130117:22:48:03:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-sdw2-1 /data/mirror/gpseg3 50003 13 3 5100320130117:22:48:03:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-sdw1-2 /data/mirror/gpseg4 50000 14 4 5100020130117:22:48:03:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-sdw1-3 /data/mirror/gpseg5 50001 15 5 5100120130117:22:48:03:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-sdw1-4 /data/mirror/gpseg6 50002 16 6 5100220130117:22:48:03:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-sdw1-1 /data/mirror/gpseg7 50003 17 7 51003Continue with Greenplum creation Yy/Nny20130117:22:52:37:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Building the Master instance database, please wait.20130117:22:53:05:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Starting the Master in admin mode20130117:22:53:19:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Commencing parallel build of primary segment instances20130117:22:53:19:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Spawning parallel processes batch 1, please wait.20130117:22:53:19:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Waiting for parallel processes batch 1, please wait.20130117:22:54:52:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-20130117:22:54:52:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Parallel process exit status20130117:22:54:52:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-20130117:22:54:52:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Total processes marked as completed = 820130117:22:54:52:015705 gpinitsystem:crsz6sbidw01:gpadmin-INFO:-Total processes marked as killed