教学Hadoop平台搭建

主讲：侯磊目录01Hadoop安装Hadoop安装1Hadoop安装Master进入到软件包所在的目录/mnt/Rpms，使用yum命令安装hadoop软件包，进入安装目录：#cd/mnt/Rpms/#yuminstall-ybigdata-xiandian-hadoop-1.0-1.x86_64.rpm#cd/opt/hadoop_resources/hadoop-2.7.0/#cdetc/hadoop/配置Hadoop-env.sh在hadoop-env.sh中只需要添加JDK的路径：#cdhadoop-2.7.0/etc/hadoop#vihadoop-env.sh添加exportJAVA_HOME=/usr/java/jdk1.7.0_67配置yarn-env.sh在yarn-env.sh中只需要添加JDK的路径：#cdhadoop-2.7.0/etc/hadoop#viyarn-env.sh添加exportJAVA_HOME=/usr/java/jdk1.7.0_67Hadoop安装用下面的代码替换core-site.xml中的内容：<?xmlversion="1.0"encoding="UTF-8"?><?xml-stylesheettype="text/xsl"href="configuration.xsl"?><configuration><property><name>fs.defaultFS</name><value>hdfs://master:9000</value></property><property><name>io.file.buffer.size</name><value>131072</value></property><property><name>hadoop.tmp.dir</name><value>file:/home/hadoop/tmp</value><description>Abaseforothertemporarydirectories.</description></property><property><name>xyuser.aboutyun.hosts</name><value>*</value></property><property><name>xyuser.aboutyun.groups</name><value>*</value></property></configuration>MasterHadoop安装Master用下面的代码替换hdfs-site.xml中的内容：<?xmlversion="1.0"encoding="UTF-8"?><?xml-stylesheettype="text/xsl"href="configuration.xsl"?><configuration><property><name>.dir</name><value>file:///usr/hadoop/dfs/name</value></property><property><name>dfs.datanode.data.dir</name><value>file:///usr/hadoop/dfs/data</value></property><property><name>dfs.replication</name><value>3</value></property><property><name>services</name><value>hadoop-cluster1</value></property><property><name>node.secondary.http-address</name><value>master:50090</value></property><property><name>dfs.webhdfs.enabled</name><value>true</value></property><property><name>dfs.permissions</name><value>false</value></property></configuration>Hadoop安装Master用下面的代码替换yarn-site.xml中的内容：<?xmlversion="1.0"encoding="UTF-8"?><?xml-stylesheettype="text/xsl"href="configuration.xsl"?><configuration><!--SitespecificYARNconfigurationproperties--><property><name>yarn.resourcemanager.hostname</name><value>master</value></property>

<property><name>yarn.nodemanager.aux-services</name><value>mapreduce_shuffle</value></property><property><name>yarn.resourcemanager.address</name><value>master:8032</value></property><property><name>yarn.resourcemanager.scheduler.address</name><value>master:8030</value></property><property><name>yarn.resourcemanager.resource-tracker.address</name><value>master:8031</value></property><property><name>yarn.resourcemanager.admin.address</name><value>master:8033</value></property><property><name>yarn.resourcemanager.webapp.address</name><value>master:8088</value></property></configuration>Hadoop安装Master用下面的代码替换mapred-site.xml中的内容：<?xmlversion="1.0"?><?xml-stylesheettype="text/xsl"href="configuration.xsl"?><configuration><property><name></name><value>yarn</value><final>true</final></property><property><name>mapreduce.jobtracker.http.address</name><value>master:50030</value></property><property><name>mapreduce.jobhistory.address</name><value>master:10020</value></property><property><name>mapreduce.jobhistory.webapp.address</name><value>master:19888</value></property><property><name>mapred.job.tracker</name><value>http://master:9001</value></property></configuration>Hadoop安装Master编辑slaves文件，添加节点#visla