data mining bayesian classifiers管理学教授财经信息管理与工程学院

贝叶斯分类器（Bayesian托马斯贝叶斯（Thomas，约1702年－1761年4月17PagePagePage联合概率P(X=x,条件概率P(Y=y|P(X,Y)=P(Y|X)×P(X)=P(X|Y)×ConditionalBayes

P(C|A)P(A,C)P(P(A|C)P(A,C)P(C)P(C|A)P(A|C)P(C)=0.75=P(X=蓝|Y=蓝P(Y=蓝P(X=蓝|Y=红P(Y=红=0.75×0.35+0.3PagePageConsidereachattributeandclasslabelasrandomGivenarecordwithattributes(A1,GoalistopredictclassSpecifically,wewanttofindthevalueofCthatmaximizesP(C|A1,A2,…,An)CanweestimateP(C|A1,A2,…,An)directlyfromcomputetheposteriorprobability（后验概率P(C|A1,A2,…,An)forallvaluesofCusingtheBayesP(C|AAA)P(A1A2

|C)P(C)1A P(1A

2AnChoosevalueofCthatP(C|A1,A2,…,EquivalenttochoosingvalueofCthatmaximizesP(A1,A2,…,An|C)P(C)HowtoestimateP(A1,A2,…,An|CPagePage朴素贝叶斯分类器NveBayesAssumeindependenceamongattributesAiwhenclassisP(A1,A2,…,An|C)=P(A1|Cj)P(A2|Cj)…P(An|CanestimateP(Ai|Cj)forallAiandNewpointisclassifiedtoCjifP(Cj)P(Ai|Cj)isPagePage 1125KNo 1125KNo2No100KNo3No70KNo4120KNo5No95K6No60KNo7220KNo8No85K9No75KNo10No90K P(No)=P(Yes)=Fordiscrete

|

)=

|/where|Aik|isnumberofinstanceshavingattributeAiandbelongstoclassCkForcontinuousDiscretizetherangeintooneordinalattributeperviolatesindependenceTwo-waysplit:(A<v)or(A>chooseonlyoneofthetwosplitsasnewProbabilitydensityAssumeattributefollowsanormalUsedatatoestimateparametersofdistribution(e.g.,meanandstandarddeviation)Onceprobabilitydistributionisknown,canuseittoestimatetheconditionalprobabilityP(Ai|c)1125KNo2No100KNo3NoNo4120KNo5No6NoNo7220KNo8No9NoNoNo P(A|c)

(A Oneforeach(Ai,ci)For(Income,Ifsamplemean=samplevariance=P(Income120|No)

(120110e2(2975 PageExampleofNaeBayesGivenaTestnaiveBayesP(Refund=No|No)=4/7P(Refund=Yes|Yes)=0P(Refund=No|No)=4/7P(Refund=Yes|Yes)=0P(MaritalStatus=Single|No)=2/7P(MaritalStatus=Divorced|No)=1/7P(MaritalStatus=Married|No)=4/7P(MaritalStatus=Single|Yes)=2/7P(MaritalStatus=Married|Yes)=0FortaxableIf Ifclass=Yes: samplemean=90P(Married|=4/74/70.0072=P(Married|=101.210-9=P(No)= SinceP(X|No)P(No)>P(X|Yes)P(Yes) P(No|X)>P(Yes|X)=>Class=NoPagePagePageNaeBayesIfoneoftheconditionalprobabilityiszero,thentheentireexpressionbecomeszeroProbability

|C)

c:numberof

N|C)N

Nic

p:priorprobabilitym:parameterNcii

|C)

mm称为等价样本大小的参数，决定先验概率p和观测概率nc/n当训练样例较少时，mPagePageExampleofNaeBayesA:leopardP(A|M)

P(A|N)

1103

7P(A|M)P(M)0.06

P(A|N)P(N)0.004 ?=>PagePageNaeBayesRobusttoisolatednoiseHandlemissingvaluesbyignoringtheinstanceduringprobabilityestimatecalcul