分析成果介绍

ANew DenoisingMethodusingTextureMetrican ptiveStructureVarianceYiweiZhang1,GeLi∗2,XiaoqiangGuo3,WenminWang4,RonggangWangSchoolofElectronicandComputerEngineering,PekingUniversityShenzhenGraduateSchoolLishuiRoad2199,NanshanDistrict,Shenzhen,GuangdongProv 1 4 5AcademyofBroadcastingScience,FuxingmenOuterStreet2,XichengDistrict, ,3—Inthispaper,aninnovatedmethodisproposedfordenoising.ThemethodconsistsoftwomajorFirst,anewadaptivesuperpixel texturemetricisproposed. texturemetriciscalculated,whichrelatestodifferentpartsofa stream.Thenanewadaptivestructurevarianceisestimatedbyadoptingfineandcoarsestructures.Finally,anoisefilterbasedontheestimatedweightsofdifferentstructuresina streamisapplied.Bycomparison,theproposedmethodoutperformstraditionalstate-of-artmethods,especiallyinblockartifactsreduction.IndexTerms— denoising,preprocessing,texturemetric,structurevariance,superpixelI.Denoisingtechniqueisawellexploredtopicinthefieldofimageand pre-processing.Itistheactualfoundationforaplentyofapplications,suchasobjectdetection,behavior codecandcomputervision,etc.Overthepastfewdecades,theperformanceofimagedenoisinghasbeenefficientlyimprovedbyemployingamuchmoreelaboratemodelofnaturalimages.Bynow,manygooddenoisingalgo-rithms[1],[2],[3],[4],[5],[6]havebeenproposed,and[7]presentsacomprehensivecomparisonofthem.Eventhough,theoriginalpurposeofdenoisingistoremoveunexpectednoisefromacorruptedimageand .Howevertheartifactscausedbythesealgorithmshavepostedasevereeffectonthequalityofimageand .Thusagooddenoisingmethodshouldintroduceasfewartifactsaspossible.SofarBM3D[4]andBM4D[5]aretwoofthestate-of-artmethodsforimageand denoising.Grousimilar2-Dblockstoa3-DgroupandperformingtransformationarethemajorcontributioninBM3D.Themethodadoptsacollaborativefiltertoperformimagereconstruction,whichretainsbetterdetails.DerivingfromBM3D,BM4Dgroupssimilar3-Dspatiotemporalvolumestoa4-Dgroup.ThosetwomethodsThisprojectwassupportedbyShenzhenPeacockn ),ScienceandTechnologynningProjectofGuangdongProve,(No.2014B )and863projectunderGrant(No.2015AA015905).

performswellinthenoisereduction.However,blockartifactsintroducedbyBM3DandBM4Darestillneedtobeimproved,asshowninFig.1(b).Inthispaper,aninnovatedmethodisproposedwhichcanreducenoisewell,andintheme showslessblockingartifacts.Fig.1.(a)thefirstframeofCrowded3.(b)denoisingresultsofBM4D.(c)denoisingresultsofourmethod.Toachievebetterdenoisingperformance,theapproachiscomposedoftwostages:obtainmentofadaptivesuperpixeltexturemetric;combinationoffineandcoarsestructure.First,weutilizesuperpixelandSingleValue position(SVD)toobtainatexturemetricρforeachpathofeachframe.Bygrouallofpathmetricρ,weobtainabrandnewtexturemetricP.Andalso,Pisutilizedtoestimateproportionoffinestructureandcoarsestructureinthestream.Second,anewadaptivestructurevarianceisobtainedbyadoptingfineandcoarsestructurebasedonP,whichcanbefitwellfordifferentscenarios.Finally,thestreamisobtainedbyapplyinganoisefilterbasedontheestimatedweightsofdifferentstructures.Contribution:twomajorproceduresareproposedinthispaper.First,aninnovateptive texturemetricbasedonsuperpixelisintroduced,whichisrelatedtodifferentpartsofa stream.Second,anewadaptivestructurevarianceisobtainedbyutilizingfineandcoarsestructurestoperformfiltering.Bycomparison,ourproposedmethodoutperformstraditionalstate-of-artmethods,especiallyinblockartifactreduction,asshowninFig.1(c).Theremainderofthispaperisorganizedasfollows.givesthedetailsofourmethod.SectionIgivestheexperimentalresultsofourapproachandcomparisonswithtwostate-of-artalgorithms.Finally,thepaperisconcludedinsectionIV.978-1-5090-5316-2/16/$31.00c2016 VCIP2016,Nov.27–30,2016,.TheProposedTheaimoftheproposedmethodinthispaperistoremovenoisefromcontaminated ,retainmuchmoretexturedetailsandproducefewerblockingartifacts.Inmanyliter-atures,aplentyofnoisemodelsareappliedin orimageprocessing.Inthisstudy,weadoptanadditivewhiteGaussiannoise(AWGN)model.Consideranobserved streamasanoisedimagesequencez:X×Tdefinedasz(x,t)=y(x,t)+η(x,t),x∈X,t∈T wherey(·,·)isamatrixrepresentingtheoriginal(unknown)

AndtheflagindexofeachsuperpixelpathisdefinedF,k=k,k∈Li where,krepresentsthek-thsuperpixelpathof.LabelofSuperpixelGradientPath:HereweGi,k,handGi,k,vasthehorizontalandverticalgradients,krespectively,showninequationGi,k,h,Gi,k,v=Gi,h(x,y),Gi,v(x,y),(x,y)∈XwhereX,kdenotesthesetofcoordinatesof,k.ThenthegradientGi,kof,kisobtainedbyjointingGi,k,hand,η(·,·)∼N(0,σ2) .d.AWGN.And(x,t) the3-Dspatiotemporalcoordinateingrayspaceorthe4-D Gi,k= Gi,k,h(m)Gi,k,v(m) ,m∈{1,...,ωi,k} spatiotemporalcoordinateolorspace.Forsimplicity,onlyinthegrayspaceisstudied,thusX⊂Z2,T⊂Z. obtainy(·,·),weproposeamethoddesigned whereωi,krepresentsthescalesizeof,k.Gi,kisaωi,k-2matrix.Gi,k,h(m),Gi,k,v(m)denotesthegradientsofy(x,t)=P.∗yf(x,t)+(1−P).∗yc(x, wherePisatexturemetricmatrixcontainin erallcharac-tersofsuperpixelsfor stream,derivedinsection-

atthecoordinate(xm,AdaptiveSuperpixelSVD:Wedefineacorrelativeco-variancematrixCasyf(·,·)andyc(·,)·areoutputsofnoisefilterswithsucturevarianceσ2andσ2respectively,asshowninsection-B.

CC=

Forsimplicity,weuseGtoreceGi,k.AfterSVDofG, TextureMetricBasedon denoisingfiltering,sometexturedetailsmaybesmooth-filteredcausingunwantedartifacts.To thisdefect,wefirstadoptthemetricPtoestimatetexture.Inspiredby[8],Piscalculatedby

isrepresented

G=U 0

CalculationofOrientationGradient:Whengradientsofneighboursareisotropic,therelativetextureissmooth;when

U ,V= gradientsareanisotropic,thetextureispoignant.Sowe ωi,kgradientoperatorSober(3)and(4)asthebasicoperatorsfortextureestimation.00010000001000000100021

whereUandVarebothorthonormalmatrix,definedinequation(12).s1ands2arethepr ipaleigenvalueofmatrixG.Combining(11)and(12),weobtaDh=

,Dv=

C v2s2+ v11v21(s2− 111

However,Ginpracticeisinterferedbynoise,thusweDh=8−202,Dv=

ˆtorepresenttherealgradientofsuperpixelpathandCˆ−10

representthecorrelativecovariance.Asaresult,equationwhereDhandDvarehorizontalandverticalSoberfilterrespectively.Forthei-thframe ina ,weobtainits

and(13)aremodifiedas(14)andˆ=ˆsˆ10Vˆ gradientGi,h,Gi,v= , ,1≤i≤

2sˆ2+

C

whereGi,handGi,varehorizontalandverticalgradientsofrespectively,Nfisthenumberof SuperpixelSegmentation:Thereexistsmanyalgorithmstosegmentanimage.Inthispaper,weuse[9]tosplittheframesintomanypaths.FortheframeI,

21 11Herewesetthegradientofsuperpixel-noisepathˆ=G+ Cˆ=GTG+GTGη+GTG+GT divideitintoNipaths.Thenweobtainsuperpixellabelof,definedas

eCisunknown,soweuseE(ˆ)toestimateE(C)Li={1,..., (ˆ)AssumingnoiseisAWGN,GandGηsatisfytheformulaE(GTGη)=E(GTG)

ηE(GT

2

whereξ=1/2whenusingoperator(3),ξ=3/16whenusingoperator(4).Thus,therelationshipbetweenoriginalandrealisticsinglevalueisobtainedas(

√√

α=v11v21,β

v2+

Fig.2.Expectedvalueof 21,γ=

vˆ2+vˆ Inspiredby[10],[11],we

+

Thusweobtaintheexpectedvalueofρi,k describedinequation(27).Fig.2givesthemoreintuitionaldescription

ρ= −sˆ12+tocharacterizethenoise-textureofthesuperpixelpath. E(ρi,k)

isan

α,β,γ,ξ,ωi,kareconstantsinafixedpathrespectively,the (ωi,k−1)!!·π

isanonlychangeswiths1,s2andσ2.Now,supposeσ2isconstant,ρwillbelargerwhenthereisaprominenttexture(s1 inthepath.Asaresult,wedefines2−

ThenwesetathresholdτequaltoE(ρi,k),andmark,kasfinestructurepathifρi,k>τandcoarsestructurepathifρi,k<τ.Finally,variancesofall,karecalculated, ρ=s12+ arelabeledas correspondingtofinestructureand correspondingtocoarseasatexturemetricforthesuperpixeObtainmentOfTextureMetric:Considering

2)FilteringwithWeights:First,finestructurevariancefcandcoarsestructurevarianceσ2ofstreamarefcvaryingwithsuperpixelpaths,weuseρi,ktodenotethetexturemetricof,k.Formoreexplicitly,weintroducesuperpixel

respectively

σ2=wf∗ texturemetricpathˆ,kcorrespondingto,k,defined

i∈Nf cσ2=w∗ c

ˆ,k={(x,y)|V(x,y)=ρi,k,(x,y)∈

whereV(x,y)denotesthevalueof atcoordinate(x, wherewfandwcareutilizedtoadjustσ2andσ2,of ThenPisobtainedbyaggregatingallˆ,k,described thegeneralformulawisdescribed1P={

,k∈

w1+

Combinationof

∆=min{mean(σ2−

Inordertoreconstructthestreamwithgood i∈Nf performanceandlessblockartifacts,weadoptanewadaptivestructurevariance.Thisvarianceisestimatedbyadoptingfineandcoarsestructures.Thenanoisefilterbasedontheestimatedweightsofdifferentstructureisapplied.Thedetailsaredescribedby

a,bandcareconstants.Thenthefinestructurestreamyf(·,·)andthecoarsestructurestreamyc(·,·)areobtainedrespectivelyas(yf(x,t)=Filter{z(x,t),ObtainingStructureVariance:First,Weusethecalcu-latedρi,kasateststatistictodecidewhetherasuperpixel

cyc(x,t)=Filter{z(x,t),c

pathhasastrongertexture.Byintroducingeigenvaluesandconditionnumbersofrandommetrics[12],weobtained

whereFilter{z(x,t),σ2}andFilter{z(x,t),σ2} theprocessoffilteringwithσ2andσ2forz(·,· possibilitydensityfp(ρi,k)ofρi,k,described weobserveρi,kcanestimatetheprobabilityofstructures.ThusweutilizePasaweightforfiltering.fp(ρi,k)=(ωi,k−1)ρi,k(1−ρ2 thereconstructedstreamisobtainedbyequationTABLEIComparisonOfPSNRσFig.3.Denoisingresults

streamnamedStandard2.(a)isthefirst

IV.Inthispaper,wepresentaninnovativedenoisingmethodcombininganewadaptivetexturemetricbasedonsuperpixelandanewstructurevariance.Byutilizingthe metrictoweightthefineandcoarse stream,majorartifactsproducedbytraditionalmethodsareeliminateddra-matically.Experimentresultsshowthattheproposedmethodcanachievterperformancecomparedwithtwostate-of-artframeofthe.(b)to(d)respectivelydenotesthefilteringresultsofdifferentalgorithmsasBM3D,BM4D,andours.I.ExperimentSimulationresultsaregivenincomparisonofBM3DandBM4D.Allthemethodsaretestedandcomparedwith40 streamundervariousnoiselevelsand13 stream1.SectionI-AandsectionI-Baresubjectiveandobjectiveassessmentrespectively.ThemajorparametersoftheproposedmethodaresetasTABLE.MajorParametersOfTheProposedδcababSubjectiveAsshowninFig.3(b-c),ourmethodcanadaptivelyadjusttheproportionbetweenfinestructureandcoarsestructuretoretaindetailedtextureandreduceblockartifactsontheprocess denoising.AsshowninFig.3(d),ourmethodretainsasmuchdetailedtextureasBM4DandproducesfewerblockartifactsthanBM4D.ObjectiveTheperformancecomparisonisshowninTABLE1

H.TalebiandP.Milanfar,“Globalimagedenoising,”ImageProcessing,IEEETransactionson,vol.23,no.2,pp.755–768,2014.W.Dong,G.Li,G.Shi,X.Li,andY.Ma,“Low-ranktensorapprox-imationwithlacianscalemixturemodelingformultiframeimagedenoising,”inProceedingsoftheIEEEInternationalConferenceonComputerVision,2015,pp.442–449.H.Yue,X.Sun,J.Yang,andF.Wu,“Imagedenoisingbyexploringexternalandinternalcorrelations,”ImageProcessing,IEEETransactionson,vol.24,no.6,pp.1967–1982,2015.K.Dabov,A.Foi,andK.Egiazarian,“denoisingbysparse3dtransform-collaborativefiltering,”inSignalProcessingConfer-ence,200715thEuropean.IEEE,2007,pp.145–149.M.Maggioni,G.Boracchi,A.Foi,andK.Egiazarian,“denoising,deblocking,andenhancementthroughseparable4-dnonlocalspatiotem-poraltransforms,”ImageProcessing,IEEETransactionson,vol.21,no.9,pp.3952–3966,2012.Z.Liu,L.Yuan,X.Tang,M.Uyttendaele,andJ.Sun,“Fastburstimagesdenoising,”ACMTransactionsonGraphics(TOG),vol.33,no.6,p.232,L.Shao,R.Yan,X.Li,andY.Liu,“Fromheuristicoptimizationtodictionarylearning:areviewandcomprehensivecomparisonofimagedenoisingalgorithms,”Cybeics,IEEETransactionson,vol.44,no.7,pp.1001–1013,2014.L.Ding,G.Li,R.Wang,andW.Wang,“pre-processingwithjnd-basedgaussianfilteringofsuperpixels,”inIS&T/SPIEElectronicImaging.InternationalSocietyforOpticsandPhotonics,2015,pp.941004–941004.R.Achanta,A.Shaji,K.Smith,A.Lucchi,P.Fua,andS.Susstrunk,“Slicsuperpixelscomparedtostate-of-the-artsuperpixelmethods,”Pat-ternysisandMachineInligence,IEEETransactionson,vol.34,no.11,pp.2274–2282,2012.L.Li,R.Wang,W.Wang,andW.Gao,“Alow-lightimageenhancementmethodforbothdenoisingandcontrastenlarging,”inImageProcessing(ICIP),2015IEEEInternationalConferenceon.IEEE,2015,pp.3730–X.ZhuandP.Milanfar,“Automaticparameterselectionfordenoisingalgorithmsusingano-referencemeasureofimagecontent,”ImageProcessing,IEEETransactionson,vol.19,no.12,pp.3116–3132,2010.A.Edelman,“Eigenvaluesandconditionnumbersofrandommatrices,”SIAMJournalonMatrixysisandApplications,vol.9,no.4,pp.543–560,1988.一种基于纹理度量和自适应结构方差的视频去噪方法1,.2,郭3,,1. 4 中国广播电视研究院，中国北京市西城区复兴门外大街2号中国，1008663guoxi 入尽可能少的伪影。到目前为止BM3D4]BM4D5]2-D3-DBM3D，将类似的3-D时空体积分组为4-D组。这两种方法该项目得到了深圳市孔雀计划 1.(a)Crowded3bBM4D(c)构的结合。首先，我们利用超像素和单值分解（SVD）ρ。通过对1（c）所示。及与两种最先进算法的比较。最后，第四节对本文进行了总结。978-1-5090-5316-2/16/$31.00c2016IEEEVCIP2016，2016112730声模型被应用于视频或图像处理。在本研究中，我们采用加性白噪声（AWGN）模型。zXTz(x,ty(x,t)+η(x,t),xX,∈Ty(·,·)（未知）视频的矩阵，η(·,·～N(0,σ2)AWGN间中的视频，因此XZ2，TZ.为了获得y(·,·)，我们提出了一种设计为y(x,t)=P的方法。yf(x,t)+(1.P)。。yc(x,t)(2)其中P是包含视频流超像素整体特征的纹理度量矩阵，在第-A节中导出；yf(·,·)和yc(·,·)分别是结构方差为σ2和σ2的噪声滤波器的输出，如第-B节所示。而导致不需要的伪影。为了克服这个缺陷，我们首先采用度量P来估计视频纹理。受[8]的启发 的计算公式如下：令人心酸的。因此我们采用梯度算子Sober(3)和(4)作为纹理估计的基本算子。01.Dh=.1200000011.,Dv=.0200.10100.011.Dh.28.10001.112.，Dv0811.202.10(4)其中Dh和Dv分别是水平和垂直Sober滤波器。对于视频中的第i帧，我们获得其梯度Gi,h,Gi,v=Dh,Dv,1≤i≤Nf其中Gi,h和Gi,v分别是的水平和垂直梯度，Nf是视频帧数9]将帧分割成许多路径。对于帧，我们将其分为Ni条路径。然后我们得到的超像素标Li1,...,Ni}F,k其中,k表示的第k个超像素路径Gi,k,h,Gi,k,v=Gi,h(x,y),Gi,v(x,y),(x,y)∈XGi,kGi,k,h(m)Gi,k,v(m,m1,...,ωi,k}其中ωi,k表示,k的尺度大小。Gi,k是ωi,k4)SVDCCGi,kTGi,kG=UVT(11)0秒2U=un,1un,2,V=v21UV(12)s1s2G(112222v1v111s21s2v11v21(s2)C=(13)22v11v21(s1s2)v1v21秒11秒2s.1V.TG=U02222v.11s.1+v.21s.2v.11v.21(.s1.s.2)C.=(15)22v.11v.21(.s1.s.2)v.21s.1+v.11G.=G+Gη(16)C.=GTG+GTGη+GηTG+GTηGηCE(C.)E(C)：E(C)E(C.)E(GTGGTGηGTη+GηTGη)E(GTxiωi,kσ20其中，当使用算子(3)时，Ψ1/2；当使用算子(4)时，Ψ3/16。因此，原始单值与实际单值22s.1.s.2=α(s1.s2)22s.1s.2β(s1s2)+Σγωi,kσ2v11v21v11+v212α=,β=,γ=(21)2222v.11v.21v.11+v.21v.11+ρ=(22)第12条+第22条α、β、γ、xi、ωi、k，ρs1、s2σ2σ2(s1s2)ρ22s1sρ=s1+量。更明确地说，我们引入超像素纹理度量路径,k。对应于,k，定义为,k.={(x,y)|V(x,y)=ρi,k,(x,y)∈Xi,k}(24)其中V(x,y)表示,k的值。在坐标(x,y)处。然后通过聚合所有,k获得P。，描述为={,k。|iÎNf,kÎLi}强的纹理。通过引入随机度量的特征值和条件数[12]，我们得到了ρi,k的可能性密度fp(ρi,k)，描述为fp(ρi,k)=(ωi,k.1)ρi,k(1.ρi,k2)(ωi,k.2)/3(26)2ρi,k（27）ρi,k2E(ρi,k)ωi,k,ωi,kE(ρi,k)=ωi,k1）·,ωi,kωi，k！！2σ2和σ2 对应于粗结构。2）用权重过滤：首