教学WordCount第一个MapReduce程序

主讲：张义010203目录WordCount简介所需Jar包程序编写与运行WordCount简介1WordCount简介WordCount用于统计每一个单词在整个数据集中出现的总次数。所需Jar包2所需Jar包Hadoop-2.4.1\share\hadoop\hdfs\hadoop-hdfs-2.4.1.jarhadoop-2.4.1\share\hadoop\hdfs\lib\所有jar包hadoop-2.4.1\share\hadoop\common\hadoop-common-2.4.1.jarhadoop-2.4.1\share\hadoop\common\lib\所有jar包hadoop-2.4.1\share\hadoop\mapreduce\除hadoop-mapreduce-examples-2.4.1.jar之外的jar包hadoop-2.4.1\share\hadoop\mapreduce\lib\所有jar包。程序编写与运行3程序编写与运行mapper类实现：public

class

WordCountMapper

extends

Mapper<LongWritable,

Text,

Text,

IntWritable>{

@Overrideprotectedvoidmap(LongWritablekey,Textvalue,Contextcontext)throwsIOException,InterruptedException{//拿到一行文本内容，转换成String类型

Stringline=value.toString();//将这行文本切分成单词

String[]words=line.split("");//输出<单词，1>for(Stringword:words){context.write(newText(word),newIntWritable(1));}}}程序编写与运行reducer类实现：public

class

WordCountReducer

extends

Reducer<Text,

IntWritable,

Text,

IntWritable>{

protectedvoidreduce(Textkey,Iterable<IntWritable>values,Contextcontext)throwsIOException,InterruptedException{//定义一个计数器

intcount=0;//通过value这个迭代器，遍历这一组kv中所有的value，进行累加

for(IntWritablevalue:values){count+=value.get();}//输出这个单词的统计结果

context.write(key,newIntWritable(count));}}程序编写与运行job提交客户端实现：publicclassWordCountJobSubmitter{publicstaticvoidmain(String[]args)throwsIOException,ClassNotFoundException,InterruptedException{Configurationconf=newConfiguration();JobwordCountJob=Job.getInstance(conf);wordCountJob.setJarByClass(WordCountJobSubmitter.class);wordCountJob.setMapperClass(WordCountMapper.class);wordCountJob.setReducerClass(WordCountReducer.class);wordCountJob.setMapOutputKeyClass(Text.class);wordCountJob.setMapOutputValueClass(IntWritable.class);wordCountJob.setOutputKeyClass(Text.class);wordCountJob.setOutputValueClass(IntWritable.class);FileInputFormat.setInputPaths(wordCountJob,"hdfs://192.168.8.245:9000/data/");FileOutputFormat.setOutputPath(wordCountJob,newPath("hdfs://192.