高效视频编码技术标准及其演化

1、高效视频编码技高效视频编码技术标准及其演化术标准及其演化61stMPEG-2 Encoder2ND GenerationEncoder3rd GenerationEncoder4th GenerationEncoder5th GenerationEncoderAdvanced Codec Encoder仅标准化解码器的好处o MPEG-2编码器进化的证据77混合视频编码基本框架EntropyCodingInv Quant/Inv TransformMotion-CompensationControlDatacoeffsMotionDataIntra/InterCoderControlMotio

2、nEstimationTransform/Quant-InputVideoSignalSplit intoMacroblocks16x16 pixels Intra-frame PredictionDe-blockingFilterOutputVideoSignalTransform codingPrediction codingEntropy coding8Spatial Coding ToolsFourier TransformBlock QuantizationAdaptive Block Coding Slant TransformHadamard TransformScene Ada

3、ptive Coder Zonal Coding Threshold Coding Zigzag Scan ZZ-Run-Length Coding Non-Uniform QuantizerUniform QuantizerBlock Classification Cosine TransformDead-Zone QuantizerRate Buffer 1960s1970sStatistical Coding Andrews, 1968Ahmed, 1974Pratt, 1969Max, 1960Tescher, 1975VLC VideoVariable-Length Coding V

4、LCHuffman 1952Tescher, 1977Fano 1949Chen, 1973Habibi, 1971Chen 1981Tasto 1971Huang 1963Seyler 1962Pratt 1969Pratt 1969Woods 1969Tescher, 1976Huang 1966Run-Length Coding Pratt, 1970C Reader, 20029Temporal Coding ToolsConditional Update(Spatial; Pixel)ConditionalUpdate(Transform)3D (Temporal)DPCMTrans

5、formCoding(Intra)DPCMHybridCodingHybridCoding(Transform)Harrison 1952Pratt 1968Reader 1973Schroeder 1970Roese 1975TemporalStatisticsSeyler 1962MotionModelingBrofferio 1970InterlacedField/FrameCodingJones 1978Transform inDPCM Loop with MCJain 1979PRA MotionEstimationNetravali 1979Variable Size BMAMot

6、ion Est.Giorda 1975MotionCompensatedDPCM (Pixel)MotionMeasurementLimb 1975Rocca 1969Conditional Update(Spatial; Block)Seyler 1962Candy 1971MotionCompensatedTransformMotionCompensatedDPCM (Block)Koga 1981Netravali 1979Netravali 1979BMAMotionEstimationTaki 1974MPEG2H.261MPEG1MPEG2Etc. H.120C Reader, 2

7、0021960s1970s10If we can find a ideal transform XXXXXX1920192010801920YXXNNyyyY.00.0.00.02211owe can have 1280/(1280*1920)=1/1920, 1920:1SVD11网址网址 http:/12相关视频标准收费概要o MPEG-2视频和系统n 每台设备：4$ (2002前)/2.5$/2$(2010年后)n 每个封装媒体(例如DVD光盘)：约0.02美分o MPEG-4 AVC/H.264/VC-1n 编解码设备：500万台以内每台0.2$，超过的每台0.1$，封顶费每年650万

8、$n 运营商：每次点播0.02美元或销售价的2%，封顶费每年650万$13131st Generation, H.261, MPEG-1/2EntropyCodingInv Quant/Inv TransformMotion-CompensationControlDatacoeffsMotionDataIntra/InterCoderControlMotionEstimationTransform/Quant-InputVideoSignalSplit intoMacroblocks16x16 pixels Intra-frame PredictionDe-blockingFilterOutp

9、utVideoSignal14142nd Generation, H.264, AVS,VC-1EntropyCodingInv Quant/Inv TransformMotion-CompensationControlDatacoeffsMotionDataIntra/InterCoderControlMotionEstimationTransform/Quant-InputVideoSignalSplit intoMacroblocks16x16 pixels Intra-frame PredictionDe-blockingFilterOutputVideoSignal15 Key vi

10、deo coding tools contributedVariable Block-size Motion CompensationVideo Object PlanesMultiple Reference 20001950DPCMInterlaceB-framesP-framesScene Adaptive CoderBlock Motion EstimationDCT MacroblocksHybrid CodingHuffman CodingTransform CodingMotion Vector PredictionObject-Based ScalabilityError Res

11、ilienceGeneric B-picturesAdvanced Deblocking FilterModified from C Reader, 2002Integer Transform1990Using for freeNeed to pay16 AVS Working Groupo Set up by MII (Ministry of Information Industry) in June of 2002o In charge of national standardization of audio-video coding for Chinese A/V industryo A

12、lso in charge of organizing the delegation of China National Body to MPEG(ISO/IEC JTC1/SC29/WG11) 17 17AVS video standardo In Feb. 2006, AVS video issued as national standardn GB/T 20090.2-2006n Effective from March 2006AVS被国际电联接受为视频编码格式o 2007年5月，ITU-T IPTV FG推荐下列视频格式n H.264/AVC videon VC-1 videon A

13、VS videoo 2009年7月10日，上述文本正式发布，“Media coding toolbox for IPTV”AVS VideoAVS专辑在欧洲信号处理学会学报发表o2003年以前，ISO与ITU制定的MPEG-2、MPEG-4与H.264等独步天下o2006年2月，AVS成为中国国家标准GB/T20090o2006年4月，微软公司成功推动VC-1成为美国SMPTE标准Volume 24, Issue 4 (2009), pages 245-344Special Issue on AVS and its Applicationedited by Wen Gao, King N.Ng

14、an, Lu YuVolume 19, Number 9 (2004), 150 pagesTechnologies Enabling Moies on Internet, HD DVD, and DCinema edited by A. Puri, J. Ribas-Corbera, W. Husak et al. 专辑有两篇文章分别综述了MPEG-4 AVC/H.264，以及微软公司的VC-9该专辑用了全部版面，刊登了八篇AVS文章AVS video profile2009版2006AVS 3D ProfileAVS System Specification for 3Do view_or

15、ganizing_typen 1 bit, 0 for multiview independent video, 1 for multiview dependent video o stereo_packing_moden 2bits, 01 for left/right, 10 for top/bottomAVS Video Specification for 3D o stereo_packing_moden 2bits, 01 for left/right, 10 for top/bottom24 The system for 2010 Asia Game2010 Asia Game 3

16、D 2010 Asia Game 3D2010 Asia Game 3D showCNTV websiteCNTV website http:/ AVS3D 视频播放器一体机国家标准发布o 地面数字一体机国标，2011年6月16日发布nGB/T 26683-2011 地面数字电视接收器通用规范（即机顶盒）nGB/T 26684-2011 地面数字电视接收器测量方法nGB/T 26685-2011 地面数字电视接收机测量方法nGB/T 26686-2011 地面数字电视接收机通用规范（即电视一体机）o 2011年11月1日开始实施；（一年过渡期，其后AVS必选，地面数字电视接收器通用规范GB/T

17、 26683-2011、地面数字电视接收机通用规范GB/T 26686-2011）AVS解码芯片产业成熟o 中国大陆(9)：n北京：海尔集成、展讯、芯晟n上海：龙晶、上海高清n杭州国芯、深圳海思、福州瑞芯、镇江唐桥o 中国台湾(4)：n晨星mStar、联发科MTK、杨智Ali、瑞昱Realteko 美国(5)：nBroadcom、Sigma Design、C2、Trident、Verisilicono 欧洲(2)：ST、NXPo 日本(2)：富士通、NEC、o 韩国(1)：Chips&MediaAVS专业编码器 (国内供货商10家)o 上海国茂o 北京联合信源o 博雅华录o 数码视讯o

18、 广州高清o 典视睿通o 普天o 汉辰o 泰宇科技o 广州科维新AVS专业编码器(国际供货商3家) Telairity (美国) BE7110（SD） BE9100（SD） BE7400（SD） BE8100（HD） BE8500（HD/SD） BE9400（HD/SD） SES3200 Envivio (美国) 4Caster C4 EDBOX (德国) AVS地面数字电视应用商业应用运营前测试株洲株洲AVS海外应用AVS十周年o 2012年3月18日，AVS成立十周年庆典在北京大学举行o 工业与信息化部、国家广电总局、国家发改委、科学技术部、国标委、国知局领导到会祝贺，表示将继续加大力度支

19、持AVS发展o 国家广电总局和工业与信息化部共同发文并签署协议，推进AVS在立体电视和高清电视中的应用 38AVS高清与3D广播应用推动签约3rd Generation, HEVC and AVS 2.0o JVCT HEVCn MPEG & VCEGn CFP 2010 Oct， MPEG Xian Meetingn Will be down around 2013 Dec.o AVS 2.0n CFP 2010 Dec.n Still open40HEVC coding flow diagramEntropyCodingScaling & Inv. TransformMot

20、ion-CompensationIntra/InterDecoderMotionEstimationTransform/Scal./Quant.- Intra-frame PredictionAdaptive Loop FilterLCU SplittingRef. BufferInternal Bit Depth IncreaseDe-block FilterSample Adaptive Offset FilterHEVCHigh EfficiencyLow ComplexityCoding Unit 8x8 up to 64x64 in tree structurePrediction

21、UnitsTransform unit tree (3 level max)Transform unit tree (2 level max)Transform block size of 4x4 to 32x32 samplesAngular Intra Prediction (Max. 34 directions)DCT-based interpolation filter 12 tapDirectional interpolation filter 6 tapAdvanced motion vector predictionCABAC entropy codingLCEC Phase 2

22、Internal bit-depth increase (4 bits)XXTransform precision extension (4 bits)Deblocking filterAdaptive loop filterXStructure of tools forming the HM42HEVC Coding Unit (CU)oCU is the basic unit for codingUsed for quad-tree based segmentation of regionsPlays a similar role to 16x16 ad 8x8 blocks in AVC

23、Can take various sizespAlways power of 2 sizepAlways square shapeoRange of allowed CU sizes is specified in Sequence Parameter Set (SPS)Largest CU (LCU) Maximum hierarchical depthEasily adapted for various applicationsoRecursive structure with split flagOne 2Nx2N or four NxN 43HEVC Prediction Unit (

24、PU)43nPU is the basic unit for prediction of a CUpLike 16x16, 16x8, 8x16, 8x8, 8x4, 4x8 and 4x4 in AVCnAllowed PU partitions are depending on prediction type Example: Available PU for 64x64 CU Skip: PU = 64x64 Intra: PU = 64x64 Inter: PU = 64x64, 64x32, 32x64, 64x16, 64x48, 16x64, 48x64Note: 1. Curr

25、ently, NxN partition is only available for min. CU (and 4x4 for inter is disabled in SPS by default). 2. AMP is not available for 8x8 CU44Relationship of CU, PU & TU 3-level quad-tree split of TU sizes Flags to indicate the split status of each TU Max TU is 32x32 Min TU is 4x4For NSQT, further s

26、plit of 16x4 or 4x16 TU is four 4x4 TU (not 8x2 or 2x8)HEVC 帧内预测模式Angular Intra Prediction46Arbitrary Direction Intra (ADI)oADI provides 33 prediction directionso# of pred. modes depends on PU sizepUp to 35 modes (including DC & Planar)nIntra mode prediction orderpFrom left to right, from top to

27、 bottomintraPredMode01234567891011121314151617intraPredOrder-151317212933371115192327intraPredMode1819202122232425262728293031323334intraPredOrder312468101214161820222426283032PU sizesIntraPredModeNumIndex4x4180-178x8350-3416x16350-3432x32350-3464x6440-347Unified directional intra prediction (mode 4

28、34)nThe inverse angle table is defined for prediction calculation invAngle = 32 * 256 / intraPredAnglenDerive predict angle for the mode of intraPredOder intraAngleIdx = modeVer ? intraPredOrder 9 : modeHor ? intraPredOrder 25 : 0signAngle = intraAngleIdx 0 ? -1 : 1; absAngle = invAngle abs( intraAn

29、gleIdx ) intraPredAngle = signAngle * absAngleintraPredOrder012345678910111213141516intraPredAngle-32-26-21-17-13-9-5-2025913172126intraPredOrder 1718192021222324252627282930313233intraPredAngle32-26-21-17-13-9-5-202591317212632intraAngleIdx012345678intra PredAngle02591317212632invAngle0409616389106

30、30482390315256modeDC = ( intraPredOrder = 0 )modeVer = ( !modeDC & intraPredOrder 18 )modeHor = ( !modeDC & !modeVer )The real angle =arcctan(intraPredAngle / 32) 484th Generationo Internet Video Codingn rich data in the webn join video coding and multimedia searchn machine learning basedo I

31、EEE SA AVS project (P1857)n Will catch up the 4th generation standard在视频编码中引入背景模型o 刚开始时场景模型为空白，随着时间推移场景模型逐渐完善，数据总量趋向稳定o 开始视频码率与常规编码相同，但很快下降逐渐并趋向稳定，只在出现异常时码率脉冲式增高场景模型数据(基于场景建模)的监控编码(无场景建模)的常规编码t t码率AVS-S+监控视频编转码方法o 压缩效率评估结果o 转码RD曲线o 同等码率下AVS-S+性能提高1.96dBo 同等质量下AVS-S+码率节省62.24%o 转码PSNR39.5dB时o H.264每天要11.48G存储空间o AVS-S+每天仅需要4.42G空间o 转码实时性o 4核服务器8路标清实时o 未作条带等并行优化o 未作低性能损失复杂度优化AVS-S+监控视频编转码方法o 压缩效率评估结果o 转码RD曲线o 同等码率下AVS-S+性能提高1.68dBo 同等质量下AVS-S+码率节省55.04