Tikhonov吉洪诺夫正则化

1、TikhonovregularizationFromWikipedia,thefreeencyclopediaTikhonovregularization isthemostcommonlyusedmethodof of named for.In,themethodisalsoknownasridgeregression .Itisrelatedtothe forproblems.Thestandardapproachtosolveanof givenasAx - b,isknownas andseekstominimizetheAx 一 b 2where isthe.However,them

2、atrix maybe or yieldinganon-uniquesolution.Inordertogivepreferencetoaparticularsolutionwithdesirableproperties,theregularizationtermisincludedinthisminimization:Ax - b 2 + lirxll2forsomesuitablychosen Tikhonovmatrix, r .Inmanycases,thismatrixis chosenasther = givingpreferencetosolutionswithsmallerno

3、rms.Inothercases, operators.,a oraweighted)maybeusedtoenforcesmoothnessifthe underlyingvectorisbelievedtobemostlycontinuous.Thisregularization improvestheconditioningoftheproblem,thusenablinganumericalsolution.An explicitsolution,denotedby r,isgivenby:x AtA + r T)1 ATbTheeffectofregularizationmaybev

4、ariedviathescaleofmatrix .For aI,when a=Othisreducestotheunregularizedleastsquaressolutionprovided that(A TA)-iexists.ContentsBayesianinterpretationAlthoughatfirstthechoiceofthesolutiontothisregularizedproblemmaylook artificial,andindeedthematrix r seemsratherarbitrary,theprocesscanbe justifiedfroma

5、.Notethatforanill-posedproblemonemustnecessarily introducesomeadditionalassumptionsinordertogetastablesolution.Statisticallywemightassumethat weknowthat isarandomvariablewitha.Forsimplicitywetakethemeantobezeroandassumethateachcomponentis independentwithx.Ourdataisalsosubjecttoerrors,andwetaketheerr

6、orsin 力tobealsowithzeromeanandstandarddeviation.UndertheseassumptionstheTikhonov-regularizedsolutionisthe solutiongiventhedataandtheaprioridistributionof ,accordingto.TheTikhonovmatrixisthenr = a/forTikhonovfactor a= b / xIftheassumptionof isreplacedbyassumptionsof anduncorrelatednessof, andstillass

7、umezeromean,thenthe entailsthatthesolutionisminimal.GeneralizedTikhonovregularizationForgeneralmultivariatenormaldistributionsfor xandthedataerror,onecan applyatransformationofthevariablestoreducetothecaseabove.Equivalently, onecanseekan xtominimizeAx - b2 +X 一 Xp02wherewehaveused|x|2 tostandforthew

8、eightednormBayesianinterpretation Pistheinverseof b, XistheXTFx(cf.the).Inthe ofx,and Qistheinversecovariancematrixof X.TheTikhonovmatrixisthengivenasa factorizationofthematrix Q= r t.the),andisconsidereda.AtPA +Q)1 AtP(b - Ax0)ThisgeneralizedproblemcanbesolvedexplicitlyusingtheformulaQRegularizatio

9、ninHilbertspaceTypicallydiscretelinearill-conditionedproblemsresultasdiscretizationof,and onecanformulateTikhonovregularizationintheoriginalinfinitedimensional context.Intheabovewecaninterpret &sa on,and xand baselementsinthedomainandrangeof .Theoperator A*A + rtr isthenabounded invertibleoperator.R

10、elationtosingularvaluedecompositionandWienerfilterWith r =a thisleastsquaressolutioncanbeanalyzedinaspecialwayvia the.GiventhesingularvaluedecompositionofAA = UYVtwithsingularvalues theTikhonovregularizedsolutioncanbeexpressedas = VDUrbwhere hasdiagonalvaluesandiszeroelsewhere.Thisdemonstratestheeff

11、ectoftheTikhonovparameter onthe oftheregularizedproblem.Forthegeneralizedcaseasimilar representationcanbederivedusinga.Finally,itisrelatedtothe:wheretheWienerweightsaref = !and Qistheof A.iDetermination of the Tikhonov factorTheoptimalregularizationparameter aisusuallyunknownandofteninpracticalprobl

12、emsisdeterminedbyanadhocmethod.Apossibleapproach reliesontheBayesianinterpretationdescribedabove.Otherapproachesinclude the, and. provedthattheoptimalparameter,inthesenseof minimizes: where H S Sisthe and Tistheeffectivenumber.八 RSSG =T 2XfcXTTXTUsingthepreviousSVDdecomposition,wecansimplifytheabove

13、expression:andRSS =i=1RSS=RSS0i=1b 2 + a 2 i ii=1it= m-E, E a 2 .b2 +a2 = m q + ZjbTZOIi=1 ii=1 iRelationtoprobabilisticformulationTheprobabilisticformulationofan introduces(whenalluncertaintiesareGaussian)acovariancematrix representingtheaprioriuncertaintiesonthemodelparameters,andacovariancematrix

14、 Drepresentingtheuncertaintieson theobservedparameters(see,forinstance,Tarantola,2004).Inthespecialcase whenthesetwomatricesarediagonalandisotropic, .：/and ：，and,inthiscase,theequationsofinversetheoryreducetothe equationsabove,with a= D / M.HistoryTikhonovregularizationhasbeeninventedindependentlyin

15、manydifferent contexts.Itbecamewidelyknownfromitsapplicationtointegralequationsfrom theworkof andD.L.Phillips.Someauthorsusetheterm Tikhonov-Phillips regularization.ThefinitedimensionalcasewasexpoundedbyA.E.Hoerl,who tookastatisticalapproach,andbyM.Foster,whointerpretedthismethodasa- filter.Followin

16、gHoerl,itisknowninthestatisticalliteratureas ridge regression.QReferences(1943).O6ycTO访quBOCTuo6paTHbix3agaqOnthestabilityofinverse problems. 39(5):195 -198.Tychonoff,A.N.(1963). OpemeHuuHeKoppeKTHonocraB刀eHHbix3agaq uMeTogepery刀月pu3auuuSolutionofincorrectlyformulatedproblems andtheregularizationmet

17、hod. DokladyAkademiiNaukSSSR 151: 501-504.Translatedin SovietMathematics 4:1035 -1038.Tychonoff,A.N.;V.Y.Arsenin(1977). Solutionoflll-posedProblems . Washington:Winston&Sons. .Hansen,.,1998, Rank-deficientandDiscreteill-posedproblems ,SIAMHoerlAE,1962, Applicationofridgeanalysistoregressionproblems , ChemicalEngineeringProgress,58,54-59.FosterM,1961, AnapplicationoftheWiener-Kolmogorovsmoothing theorytomatrixinversion J.SIAM,9,387-392PhillipsDL,1962