HEVC编码技术概览

HEVC编码技术概览■

ISO-IEC/MPEG和ITU-T/VCEG成立了一个研究视频编码的联合协作小组JCT-VC(thejointcollaborativeteamonvideocoding)，其宗旨是建立新一代的视频编码标准。

■

2010年4月在德国德雷斯顿召开了JCT-VC(JointCollaborativeTeamonVideoCoding)第一次会议，确定了新一代视频编码标准的名称：HEVC(HighEfficiencyVideoCoding)。并且建立了测试模型TMuC(TestModelunderConsideration)，分领域搜集和审阅技术提案。初步定于2012年7月完成标准的最终稿。

■

2010年10月在广州召开了JCT-VC第三次会议，会上确立了HEVC的第一个试验模型HM1。

■

新一代视频压缩标准的核心目标，是在H.264的基础上将压缩效率提高一倍。即在保证相同视频图像质量的前提下，视频流的码率减少50%。在提高压缩效率的同时，可以允许编码端适当提高复杂度。

HMcontainsonlyaminimumsetofwell-testedtoolsthattogetherfromacoherentdesignthatisconfirmedtoshowgoodcapability.

Twosettingshavebeendefined：1)highefficiency;2)low-complexity.HMHMConfigurationHighEfficiencyConfigurationLowcomplexityConfigurationCodingUnittreestructure(8×8upto64×64lumasamples)PredictionUnitsTransformunittreestructure(3levelmax.)Transformunittreestructure(2levelmax.)Transformblocksizeof4x4to32x32samples(alwayssquare)AngularIntraPrediction(34directionsmax.)DCT-basedinterpolationfilterforlumasamples(1/4-sample,12-tap)Directionalinterpolationfilterforlumasamples(1/4-sample,6-tap)Bi-linearinterpolationfilterforchromasamples(1/8-sample)AdvancedmotionvectorpredictionContextadaptivebinaryarithmeticentropycodingLowcomplexityentropycodingphase2Internalbit-depthincrease(4bits)XXTransformprecisionextension(4bits)DeblockingfilterAdaptiveloopfilterX1、PicturePartitioning2、IntraPrediction3、InterPrediction4、TransformandQuantization5、EntropyCoding6、LoopFiltering7、CostFunction1、PicturePartitioning1.1.Treeblock(TB)PartitioningPicturesaredividedintoslicesandslicesaredividedintoasequenceoftreeblocks.AtreeblockisanNxNblock

oflumasamplestogetherwiththetwocorrespondingblocksofchromasamplesforapicturethathasthreesamplearrays.ThetreeblockconceptisbroadlyanalogoustothatofthemacroblockinpreviousstandardssuchasH.264/AVC[6].Themaximumallowedsizeofthetreeblockis64x64lumasamples.

TheCodingUnit(CU)isthebasicunitofregionsplittingusedforinter/intraprediction.

Codingunit(CU)thebasiccodingunitlikethemacroblock.CUscanbefurthersplitintopredictionunits(PUs).transformunit(TU)isdefinedfortransformandquantization.Itisalwayssquareanditmaytakeasizefrom8x8lumasamplesuptothesizeofthetreeblock.(8x8,16x16,32x32,64x64)TheCUconceptallowsrecursivesplittingintofourequallysizedblocks,startingfromthetreeblock.Thisprocessgivesacontent-adaptivecodingtreestructurecomprisedofCUblocks.1.2.CodingUnit(CU)structureGiventhattheregionofinterestishomogeneousandthelargesizesaCUcanhave,arelativelylargeCUcanrepresentthisregioninasmallernumberofsymbols.ThewiderangeofsizesforLCUqualitiesthecodectobeefficientandcarefullyoptimizedaccordingtousageorapplication

ThePredictionUnit(PU)isthebasicunitusedforcarryingtheinformationrelatedtothepredictionprocesses.PUsaredefinedafterthelastlevelofCUsplittingIngeneral,itisnotrestrictedtobeingsquareinshape,inordertofacilitatepartitioningwhichmatchestheboundariesofrealobjectsinthepicture.EachCUmaycontainoneormorePUs.1.3.PredictionUnit(PU)structureFigure1.3.FourtypesofPredictionUnitstructurePredictiontypecanhavethefollowingvalues:SkipIntra-prediction

Inter-prediction

illustratesallpossiblePUsplittingaccordingtodifferent

prediction

types.从上图中可以看到一种新的不对称运动分割预测（AsymmetricMotionPartition,AMP）,这也是HEVC与H.264在分块预测技术中最为不同之处。所谓AMP即将编码单元分割成两个大小不一致的预测块其中一个PU单元的宽/长为CU单元的3/4,这种预测考虑了大尺寸可能的纹理分布可以有效提高大尺寸块的预测效率

TheTransformUnit(TU)isthebasicunitusedforthetransformandquantizationprocesses.Itisalwayssquareanditmaytakeasizefrom4x4upto32x32lumasamples.EachCUmaycontainoneormoreTUs,wheremultipleTUsmaybearrangedinaquadtreestructure,asillustratedinFigure1.4below.3-levelquadtreeforhighefficiencyconfiguration.Samemaximumquadtreedepthforlumaandchroma.1.4.TransformUnit(TU)structureFigure1.4.ExampleofTransformUnitstructure配合不对称预测单元以及矩形预测单元，新的HM4.0模型还采纳了相应的矩形四叉数TU结构突破了方块变换的限制，下图展示了3级矩形四叉树变换水平TU结构，同理可以有垂直分割结构SummeryofCU,PU,TUthreeindependentblockconcepts:CU(Codingunit):thebasicunitofregionsplitting;PU(Predictionunit):thebasicunitofinter/intraprediction;TU(Transformunit):thebasicunitoftransform.CUisanalogoustotheconceptofmacroblock,butitdoesnotrestrictthemaximumsizeanditallowsrecursivesplittingintofourequalsizeCUstoimprovethecontentsadaptivity.EachCUmaycontainoneormorePUs.PUmaycontainmultiplearbitraryshapepartitionsinasinglePUtoeffectivelycodeirregularimagepatterns.TUcanbedefinedindependentlyfromthePU;however,itssizeislimitedtotheCUwhichtheTUbelongsto.Thisseparationoftheblockstructureintothreedifferentconceptsallowseachtobeoptimizedaccordingtoitsrole,whichresultsintheimprovedcodingefficiency.Codingunittreestructureislimitedfrom8×8to64×64forluma,thatistosay,nosplittingisallowedforCU3.IftheLCUsize=16andhierarchicaldepth=2,thenthisisasimilarcodingstructuretomacro-blockandsub-macro-blockinH.264/AVC.TospecifythepredictionmethodusePUtypeandPUsplitting.PUtypescanbeskip,intraandinter.ItshouldbenotedthatthesizeofTUmaybelargerthanthatofthePUbutnotexceedthatoftheCU.TUhavedifferentsplittingforlowcomplexity(LC)andhighefficiency(HE)configurations.2、IntraPredictionTheunifiedintrapredictioncodingtoolprovidesupto33directionalpredictionmodesandoneDCpredictionmodeforeachPU.Figure2.1.The33intrapredictiondirectionsFigure2.2.MappingbetweenintrapredictiondirectionandintrapredictionmodeTable2.1.NumberofsupportedintramodesaccordingtoPUsizePUsizeNumberofintramodes41783416343234645ThetotalnumberofavailablepredictionmodesisdependentonthesizeofthecorrespondingPU,asshowninTable2.1below.Thepixelisreconstructedusingthelinearinterpolationofthereferencetoporleftsamplesat1/32thpixelaccuracyforallblocksizes,insteadofdifferentaccuraciesfordifferentsizes.Twoarraysofreferencesamplesareused,correspondingtotherowofsampleslyingabovethecurrentPUtobepredicted,andthecolumnofsampleslyingtotheleftofthesamePU.Givenadominantpredictiondirection(horizontalorvertical),oneofthereferencearraysisdefinedtobethemainarrayandtheotherarraythesidearray.ThereferencerowabovethePUiscalledthemainarrayandthereferencecolumntotheleftofthesamePUiscalledthesidearray,incaseofverticalprediction.WhilethereferencecolumntotheleftofthePUiscalledthemainarrayandthereferencerowabovethePUiscalledthesidearray,incaseofhorizontalprediction.Onlythemainarraysamplesareusedforpredictionwhentheintrapredictionangleispositive,Whentheintrapredictionangleisnegative,aper-sampletestisperformedtodeterminewhethersamplesfromthemainorthesidearrayshouldbeusedforprediction,asshowninfigure.Whenthesidearrayisused,thecomputationoftheindexofthesidearrayrequiresadivisionoperation.Thedivisionoperationisremovedbyusingalook-uptable(LUT)technique.Thistechniqueisusedforthenegativeangularpredictionprocessinthecalculationofthey-interceptincaseofverticalpredictionorthex-interceptin

caseofhorizontalprediction.Theintegerandfractionalpartsoftheinterceptarecalculatedusingthefollowingequations:deltaIntSide=(256*32*(l+1)/absAng)>>8deltaFractSide=(256*32*(l+1)/absAng)%256absAngistheabsolutevalueofintrapredictionangle(=[2,5,9,13,17,21,26,32])andlisx/ypixellocationforvertical/horizontalprediction.UsingtheLUTtechniques,theaboveequationscanbereplacedwiththefollowingequations:deltaIntSide=(invAbsAngTable[absAngIndex]*(l+1))>>8deltaFractSide=(invAbsAngTable[absAngIndex]*(l+1))%256whereinvAbsAngTable=[4096,1638,910,630,482,390,315,256].ByusingLUT,thereisnodivisionduringthecomputationoftheindexofthesidearray.However,therestillexiststheper-sampletesttodeterminethemainorthesidearrayforprediction.Tosimplifythisprocess,themainarrayisextendedbyprojectingsamplesfromthesidearrayontoitaccordingtothepredictiondirection,asshowninFigure.Duringtheprojection,thefractionalpartoftheinterceptisomittedandtheinterceptisroundedtothenearestinteger:deltaInteger=(invAbsAngTable[absAngIndex]*(l+1)+128)>>8.Atlast,thepredictionprocessonlyusestheextendedmainarrayandthesamesimplelinearinterpolationformulatopredictallsamplesinthePU.

如果经过DC预测后，当前块的左侧和上侧边缘跟周围的重构像素可能会不大一致，HEVC对经过DC模式预测后的变换单元的边缘像素进行滤波以使变换块的边缘跟参考像素之间更加的平滑一些。滤波策略如图从上图中我们可以看到，左侧和上侧的灰绿色区域是已经解码的重构像素，当前块中被红色标注的像素是当前块与已重构像素相邻接的边缘像素，需要对这些像素进行滤波处理。根据这些边缘像素与已重构的像素相邻接的像素个数来决定是使用2-tap滤波器还是3-tap滤波器。其中左上角那个桃红色像素因为和周围两个像素相邻，所以使用3-tap滤波器；别的边缘像素只跟周围一个像素相邻，所以只使用2-tap滤波器。HEVC对不同大小的变换单元使用不同强度的滤波器。对于4x4大小的变换单元使用FilterA所指的滤波器，对于8x8大小的变换单元使用FilterB所指的滤波器，对于16x16大小的变换单元使用FilterC所指的滤波器。3、InterPrediction3.1.MotionVectorPridictionForeachCUorPU,thebestmotionvectorpredictoriscomputedwiththeprocessspecifiedasfollows.Firstly,asetofmotionvectorpredictorcandidatesshall

be

normativeprocess,byreferringtomotionparametersofneighboringpartitions.Then,thebestonefromthecandidatesetisdeterminedbyacriterionthatselectsamotionvectorpredictorcandidatethatminimizesthecostJmotion.3.1.1.CodingModeIntheHMencoder,aninterCUcanbesegmentedintomultipleinterPUs.AninterCUcanbeencodedwithoneofthefollowingcodingmodes:MODE_SKIP,MODE_DIRECT,MODE_INTER,andMODE_MERGE.ForMODE_SKIP,MODE_DIRECTandMODE_MERGEcases,anysub-partitioningisnotallowed.Onthecontrary,fourtypesoffurtherpartitioningintheCUcanbeallowedforMODE_INTERcase.

TheHMencoderfirstevaluatesCUcodingcostswithskipanddirectmode.

Forthesecases,motionvectorpredictorforthecurrentCUisdirectlyusedascurrentmotionvector.ThemotionvectortobeusedforMODE_SKIPorMODE_DIRECTaredeterminedbycheckingallpossiblemotionvectorcandidatesandselectingthebestonethatminimizesthecostJmode.TheHMencodercomparesthecodingcostofMODE_SKIPandMODE_DIRECTfirst,takesthebettermode,andthenproceedtocheckingotherremainingvariations.3.1.2.Skip/DirectmodeIfthecodingcostJmodeissmallerthantheoneobtainedinSkip/Directmode,pred_modeissetto“MODE_MERGE”.Inthiscase,motionvectoroftheselectedleftorupperpartitionaredirectlyusedascurrentmotionparameters.Aftertheprocessspecifiedintheprevioussection,theHMencoderevaluateswhethermotionmergingshouldbeenabledornotforthecurrentCU.DecisionontheuseofmotionmergingisperformedonlyatCUlevelwhosePUsizeisSIZE_2Nx2N,whichisidenticaltoCUsize.Thedecisionontheuseofmotionmergingstartswithcheckingavailabilitiesofleftandupperneighboringpartitions.Ifthereisnoavailablepartitionformotionmerging,theHMencoderdoesnotchooseMODE_MERGEforthecurrentCU.Otherwise,thecodingcostJmodeiscomputedforavailablepartitions.3.1.3.MergemodeIftheprocesssuggestsfurtherevaluationofinterpredictionmodes,theHMencodercomputescodingcostsofCUpartitioningmodesSIZE_2Nx2NandSIZE_NxNofMODE_INTERcase.Then,theHMencoderfurtherevaluatesSIZE_2NxNandSIZE_Nx2Npartitions.Inthesecases,motionvectorsisderivedbymotionestimationprocessspecifiedinsection3.1.IfthecostJmodeofthebestPUpartitioningamongallevaluatedmodesissmallerthantheoneobtainedinMergemode,pred_modeissetto“MODE_INTER”.3.1.4.IntermodeLumainterpolationfilter1/4-sample,12-tapDCT-basedinterpolationfilter(DCT-IF)forhighefficiencyconfiguration(HE)1/4-sample,6-tapdirectionalinterpolationfilter(DIF)forlowcomplexityconfiguration(LC)Chromainterpolationfilter1/8-sample,bi-linearinterpolationfilterforbothHEandLC3.2.InterpolationfilterMotioncompensationisthekeyfactorforhighefficientvideocompressing,wherefractionalpelaccuracyrequiresinterpolationofthereferenceframe.InH.264/AVC,a6-tapfixedWienerfilterisfirstusedforhalfpelaccuracyinterpolationandthenabilinearcombinationofintegerandhalfpelvaluesisusedtoprovide¼pelaccuracyinterpolation.Insteadofadaptiveinterpolationfilter.HMstilladoptsthefixed12-tapDCT-basedinterpolationfiltertoprovidefractionalpelaccuracyinterpolationbyreplacingthecombinationofWienerandbilinearfilterswithasetofinterpolationfiltersatthedesiredfractionalaccuracy.Morespecifically,onlyonefilteringprocedureisneededtoprovidetheinterpolationpixeltoanypixelaccuracy,insteadofacombinationof6-tapandbilinearfilteringproceduresinH.264/AVC.Thus,themotioncompensationprocesscanbesimplifiedintheimplementationpointofviewandthecomplexitycanalsobereducedforquarter-pelaccuracy.TheDCT-basedinterpolationfilteringprocessusesthehorizontalneighboringintegerpixelsI(x,0),wherex=-5…6,tointerpolatethehorizontalfractionalpixels,a,bandc,byusingthefollowingequations,a=\sum_{x=-5}^{6}I(x,y)*f(x,1/4)(1)b=\sum_{x=-5}^{6}I(x,y)*f(x,2/4)(2)c=\sum_{x=-5}^{6}I(x,y)*f(x,3/4)(3)

wherey=0.Intheaboveequantions,f(x,1/4),f(x,2/4)andf(x,3/4)denotethe12-tapDCT-basedinterpolationfiltercoefficientsinfractionalpositions1/4,2/4,and3/4,respectively,aslistedinTable

AsetofinterpolationfilterforlowcomplexityconfigurationisreferredtoastheDirectionalInterpolationFilter(DIF).Foreachofthethreehorizontalfractionalpositions,a,bandc,andthethreeverticalfractionalpositions,d,handn,whicharealignedwithfullpixelpositions,asingle6-tapfilterisused.ThefiltercoefficientsofDIFare{3,-15,111,37,-10,2}/128for¼position(andmirroredfor¾position),{3,-17,78,78,-17,3}/128for½position.

a=(3H-15I+111J+37K-10L+2M+64)>>7b=(3H-17I+78J+78K-17L+3M+64)>>7c=(2H-10I+37J+111K-15L+3M+64)>>7

d=(3B-15E+111J+37O-10S+2W+64)>>7h=(3B-17E+78J+78O-17S+3W+64)>>7

n=(2B-10E+37J+111O-15S+3W+64)>>7在高效预测模式下，HEVC仍然采用H.264中的等级B预测方式同时还增加了广义B（GeneralizedPandBpicture,GPB）取代低时延场景下中的P的预测方式G。GPB预测结构是指对传统P帧采用类似于B帧的双向预测方式。在这种预测模式下，前向和后向参考列表中的参考图像都必须为当前图像之前的图像。对P帧采用B帧的预测方式增加了运动估计得准确度，提高编码效率，同时也有利于编码流程统一。 4、TransformandQuantization4.1.TransformTransformsofsizesfrom4x4to32x32aresupported.Fortransformsofsize4x4and8x8,thesamemethodasthatspecifiedintheH.264/AVC[6]standardisused.Fortransformsof16x16and32x32,fastintegertransformsusingabutterflystructureareused.4.2.ScalingandQuantizationThesamequantizationmethodasinH.264/AVCisused..经过不同的预测之后，残差存在不同的方向性，经过量化变换后，不同预测方向产生的差异性仍然存在，因此如果使用一致的zig-zag扫描，并不是最佳的扫描方法，为了克服这个缺点，HEVC采用水平，垂直和zig-zag三种扫描方式。为了获得最佳效果，HEVC对不同大小的变换单元和帧内预测模式采用查表的方式获得其扫描方式，并且因为亮度像素和色度像素的模式有很大的不同，所以对亮度像素和色度像素，采用不同的表格

TwoalternativeentropycodingschemesaresupportedintheHM:ContextAdaptiveBinaryArithmeticCoding(CABAC)andLowComplexityEntropyCoding(LCEC).Contextadaptivebinaryarithmeticcoding(CABAC)forhighefficiencyconfigurationLowcomplexityentropycoding(LCEC)phase2forlowcomplexityconfiguration5、EntropyCoding6、LoopFilteringThedeblockingmethodisbasedonthatusedinH.264/AVCRecallthatforblocksizesgreaterthan16×16,only16×16transformisused.Thus,for32×32and64×64blocks,de-blockingfilterisappliedonlyalongthe16×16blockedges.Thisreducesthecomputationalcomplexityforthede-blockingoperationifthebiggerblocksarechosenfrequently.4x4blockboundariesarenotdeblockedinordertoreducethecomplexity.6.1.Deblockingfilter自适应样点补偿技术是一个自适应选择过程，在去块滤波后进行。目前自适应样点补偿方式分为带状补偿（BandOffset,BO）和边缘补偿（EdgeOffset,EO）两大类带状补偿将像素值强度等级分为若干个条带，每个条带内的像素拥有相同的补偿值。进行补偿时，根据重构像素点所处的带选择相应的带状补偿值进行补偿。现在的HM模型将像素值强度从0到最大值划分为32个等级。同时这32个等级条带还分为两类，第一类是位于中间的16个等级条带，剩余的为第二类条带。编码时只将一类条带的补偿信息写入片头；另一类条带信息则不传送。这样方式