2023学年完整公开课版Hadoop滚动升级

NeueduNeuEdu-Hadoop-Rolling-UpgradeHDFSrollingupgrade

allowsupgradingindividualHDFSdaemons.Forexamples,thedatanodescanbeupgradedindependentofthenamenodes.Anamenodecanbeupgradedindependentoftheothernamenodes.Thenamenodescanbeupgradedindependentofdatanodsandjournalnodes.简介官方文档：/docs/r2.7.3/hadoop-project-dist/hadoop-hdfs/HdfsRollingUpgrade.htmlInHadoopv2,HDFSsupportshighly-available(HA)namenodeservicesandwirecompatibility.ThesetwocapabilitiesmakeitfeasibletoupgradeHDFSwithoutincurringHDFSdowntime.InordertoupgradeaHDFSclusterwithoutdowntime,theclustermustbesetupwithHA.NotethatrollingupgradeissupportedonlyfromHadoop-2.4.0onwards.本次课程Hadoop集群环境版本2.7.3升级版本为Hadoop2.X的最新版本2.10.1环境准备1.准备待升级Hadoop集群环境，版本为2.4.0以上，3台节点2.启动集群保证具备HA3.下载hadoop-2.10.1.tar.gz上传至各个节点并解压检查服务jps环境校验主备情况hdfs

haadmin

-getServiceState

nn1hdfs

haadmin

-getServiceState

nn2WEB服务检查hadoop-up001:50070hadoop-up002:50070hadoop-up001:8088http://hadoop-up002:50070/dfshealth.html#tab-datanode检查高可用hadoop-daemon.sh

stop

namenodeyarn-daemon.sh

stop

resourcemanager恢复服务hadoop-daemon.sh

start

namenodeyarn-daemon.sh

start

resourcemanager高可用校验准备文件创建目录hdfsdfs-mkdir/neuedu上传文件hdfsdfs-put/usr/local/hadoop/etc/hadoop/hdfs-site.xml/neuedu检查文件hdfsdfs-ls/neuedu停止namenodestandby节点服务hadoop-daemon.sh

stop

namenoderoot用户切换hadoop安装目录mv/usr/local/hadoop/usr/local/hadoop-2.7.3mv/usr/local/hadoop-2.10.1/usr/local/hadoopchown-Rhadoop:hadoop/usr/local/hadoopHadoop用户复制配置文件及元数据cd/usr/local/hadoop-2.7.3/etc/hadoop/cpslavescore-site.xmlhdfs-site.xmlyarn-site.xmlmapred-site.xml/usr/local/hadoop/etc/hadoop/cp-r/usr/local/hadoop-2.7.3/tmp//usr/local/hadoop升级准备重启Journalnodehadoop-daemon.shstopjournalnodehadoop-daemon.shstartjournalnode重启DFSZKFailoverControllerhadoop-daemon.shstopzkfchadoop-daemon.shstartzkfc执行升级hdfsnamenode-rollingUpgradestartedhdfsdfsadmin-rollingUpgradequery升级NameNode启动NameNodehadoop-daemon.sh

start

namenode浏览器查看50070端口重复执行升级另一个NN节点升级DataNode准备root用户切换hadoop安装目录mv/usr/local/hadoop/usr/local/hadoop-2.7.3mv/usr/local/hadoop-2.10.1/usr/local/hadoopchown-Rhadoop:hadoop/usr/local/hadoopHadoop用户复制配置文件及元数据cd/usr/local/hadoop-2.7.3/etc/hadoop/cpslavescore-site.xmlhdfs-site.xmlyarn-site.xmlmapred-site.xml/usr/local/hadoop/etc/hadoop/cp-r/usr/local/hadoop-2.7.3/tmp//usr/local/hadoop仅在第三个节点执行即可，前两个节点已经在升级namenode过程执行过升级DataNode重启Journalnodehadoop-daemon.shstopjournalnodehadoop-daemon.shstartjournalnode执行升级hdfsdfsadmin-shutdownDatanodehadoop-up003:50020upgradehadoop-daemon.shstartdatanodehdfsdfsadmin-shutdownDatanodehadoop-up002:50020upgradehadoop-daemon.shstartdatanodehdfsdfsadmin-shutdownDatanodehadoop-up001:50020upgradehadoop-daemon.shstartdatanodehdfsdfsadmin-getDatanodeInfohadoop-up003:50020三台节点分别逐个进行升级浏览器查看50075端口重启resourcemanager节点服务（2台节点）yarn-daemon.shstopresourcemanageryarn-daemon.sh

start

resourcemanagerYARN升级