非参数分类方法-分析归纳很到位

非参数分类方法-分析归纳很到位.pptPatternRecognitionLab501TongjiUniversity2NonparametricTechniqueOutlineDensityEstimationParzenWindow

PatternRecognitionLab501TongjiUniversity3DensityEstimation–Overview(1)Inprobabilityandstatistics,densityestimationistheconstructionofanestimate,basedonobserveddata,ofanunobservableunderlyingprobabilitydensityfunction.Therearetwobasicapproachestoperformdensityestimation:Parametric:thedensityfunctionisassumed(i.e.,Gaussian)andtheparametersofthefunction(i.e.,meanandvariance)arethenoptimizedbyfittingthemodeltothedatasetNon-parametric:nofunctionalformforthedensityfunctionisassumed,andthedensityestimatesisdrivenentirelybythedataPatternRecognitionLab501TongjiUniversity4DensityEstimation–Overview(2)BinomialdistributionPatternRecognitionLab501TongjiUniversity5DensityEstimation–Overview(3)Whenn→∞,thevarianceis0,wecanusek/ntoestimateP.PatternRecognitionLab501TongjiUniversity6DensityEstimation–Overview(4)ThisestimatebecomesmoreaccurateasweincreasethenumberofsamplepointsNandshrinkthevolumeVImportantconditionPatternRecognitionLab501TongjiUniversity7DensityEstimation–Overview(5)InpracticethevalueofN(thetotalnumberofexamples)isfixedInordertoimprovetheaccuracyoftheestimatep(x)wecouldletVapproachzero,butthentheregionwouldthenbecomesosmallthatitwouldenclosenoexampleswewillhavetofindacompromisevalueforthevolumeVLargeenoughtoincludeenoughexampleswithinSmallenoughtosupporttheassumptionthatp(x)isconstantwithinInconclusion,thegeneralexpressionfornon-parametricdensityestimationbecomesPatternRecognitionLab501TongjiUniversity8DensityEstimation–Overview(6)Whenapplyingthisresulttopracticaldensityestimationproblems,twobasicapproachescanbeadopted:WecanchooseafixedvalueofthevolumeVanddeterminekfromthedata.ThisleadstomethodscommonlyreferredtoasKernelDensityEstimation(KDE),i.e.ParzenWindowsWecanchooseafixedvalueofkanddeterminethecorrespondingvolumeVfromthedata.ThisgivesrisetothekNearestNeighbor(kNN)

approach.PatternRecognitionLab501TongjiUniversity9ParzenWindows(1)PatternRecognitionLab501TongjiUniversity10ParzenWindows(2)PatternRecognitionLab501TongjiUniversity11ParzenWindows(3)NumericexerciseGiventhedatasetbelow,useParzenwindowstoestimatethedensityp(x)aty=3,10,15.Useabandwidthofh=4X={4,5,5,6,12,14,15,15,16,17}PatternRecognitionLab501TongjiUniversity12PatternRecognitionLab501TongjiUniversity13Ofcourse,thewindowfunctioncanbeotherfunction,forexampleGaussianfunction(for1-D).ThisissimplytheaverageofnGaussianfunctionswitheachdatapointasacenter.needstobepredetermined.PatternRecognitionLab501TongjiUniversity14PatternRecognitionLab501TongjiUniversity15ParzenWindows(4)Illustrations(P169)PatternRecognitionLab501TongjiUniversity16PatternRecognitionLab501TongjiUniversity17PatternRecognitionLab501TongjiUniversity18PatternRecognitionLab501TongjiUniversity19ParzenwindowbasedclassificationPatternRecognitionLab501TongjiUniversity20PatternRecognitionLab501TongjiUniversity21h=1h=0.5h=0.1PatternRecognitionLab501TongjiUniversity22kNNDensityEstimation(1)Oneofthedifficultieswiththekernelapproachtodensityestimationisthattheparameterh

governingthekernelwidthisfixedforallkernels.Inregionsofhighdatadensity,alargevalueofhmayleadtoover-smoothingandawashingoutofstructurethatmightotherwisebeextractedfromthedata.ReducinghmayleadtonoisyestimatesPatternRecognitionLab501TongjiUniversity23kNNDensityEstimation(2)Thegeneralexpressionfornonparametricdensityestimationis:Atthattime,wementionedthatthisestimatecouldbecomputedbyFixingthevolumeVanddeterminingthenumberkofdatapointsinsideV(KernelDensity)FixingthevalueofkanddeterminingtheminimumvolumeVthatencompasseskpointsinthedataset(kNN)P