大数据实时计算系统实践2datastreamapi介绍与实战

1ØØ订单按照拆分，如：订单创建流、订单开始计Ø ØØØ数据源(DataSource)Ø数据Sinkpublicstaticvoidmain(String[]args)throwsException{StreamExecutionEnvironmentenv=StreamExecutionEnvironment.DataStream<Tuple2<String,Integer>>=.socketTextStream("localhost",.flatMap(new

env.execute("Window} 优化 包含SourceTransformation/SinkTransformation/ StreamNode-> 上下游并行度一致，且下游入度为属于同一个slot节点之间数据分区方式为没有主 JobGraph->Ø JobGraph2 casefromElements(T..privatelongnum=0L;

privatevolatilebooleanisRunning=true;while(isRunning){

throwsException}}publicvoidcancel(){isRunning= DataStream<Integer>dataStream=env.fromElements(1,2,3,DataStream<Integer>filteredStream=dataStream.filter(newFilterFunction<Integer>(){publicbooleanfilter(Integervalue){returnvalue%2==0;}Split含SplitselectDataStream->指定keys的方式Keyby可能出现数据倾斜的问题从dataStream提取key，不相同的key的值shuffleKeyedStream可以转换为DataStream和Tuple DataStream<Tuple3<Integer,String,Double>>d=...Pojo&caseclass指定指定字段名为

d=o实现KeySelectorrescale&shuffle&rescale针对并行度ShuffleRebalance针对数据，round-Join是cogroupCogroup偏向于 Connect&时,会非常有用;union合并多个流，新的流包含所有流的DataStream<MyType>input=...DataStream<AnotherType>other=...ConnectedStreams<MyType,AnotherType>connected=DataStream[ResultType]connected.map(newCoMapFunction<MyType,AnotherType,ResultType>(){publicResultTypemap1(value:MyT