人工智能不确定性

手写数字识别文本分类图像分割第八章Uncertainty

不确定性对应教材第13章本章大纲Uncertainty不确定性Probability概率SyntaxandSemantics语法与语义Inference推理IndependenceandBayes‘Rule

—独立性及贝叶斯法则

不确定性智能体几乎从来无法了解关于其环境的全部事实。因此其必须在不确定的环境下行动。概率推理

得到了某一证据，那么有多大的几率结论为真？

例如：我颈部痛;我得脑膜炎的可能有多大?不确定性假如有如下规则:

iftoothache（牙疼）then原因是cavity（牙齿有洞）但并不是所有牙疼的病人都是因为牙齿有洞，所以我们可以建立如下规则:

iftoothacheand¬gum-disease（牙龈疾病）and

¬filling（补牙）and...thenproblem=cavity以上规则是复杂的;更好的方法:

iftoothachethenproblemiscavitywith0.8probability

orP(cavity|toothache)=0.8

theprobabilityofcavityis0.8giventoothacheisobserved不确定性 LetactionAt=离起飞时间提前t分钟动身去机场

At会使我准时到达机场吗?

Problems:

1.partialobservability/部分可观察性(roadstate,otherdrivers‘plans)

2.noisysensors(trafficreports)

3.行动结果的不确定性(flattire,etc.)

4.immensecomplexityofmodelingandpredictingtraffic

因此一个纯粹的逻辑描述方法：

1.risksfalsehood（错误风险）:“A25

willgetmethereontime”,or

2.leadstoconclusionsthataretooweakfordecisionmaking:

“A25

willgetmethereontimeifthere’snoaccidentonthebridgeanditdoesn‘trainandmytiresremainintactetcetc.”

(A1440

mightreasonablybesaidtogetmethereontimebutI’dhavetostayovernightintheairport…)世界与模型中的不确定性Trueuncertainty:rulesareprobabilisticinnature

掷骰子，抛硬币惰性:把所有意外的规则都列举出来是很困难的

花费太多时间来确定所有的相关因素

这些规则过于繁杂而难以使用理论的无知:某些领域中还没有完整的理论

(e.g.,medicaldiagnosis)实践的无知:掌握了所有规则但是

并不是所有的相关信息都能被收集到处理不确确定性的的方法概率理论论作为一一种正式式的方法法for：不确定知知识的表表示和推推理命题中的的模型信信度(event,conclusion,diagnosis,etc.)给定可获获得的证证据,A25willgetmethereontimewithprobability0.04概率是不不确定性性的语言言现代AI的中心支支柱Probability概率概率理论论提供了了一种方方法以概概括来自自我们的的惰性和和无知的的不确定定性。ProbabilisticassertionssummarizeeffectsofLaziness（惰性））:failuretoenumerateexceptions（例外））,qualifications（条件））,etc.Ignorance（理论的的无知））:lackofrelevantfacts,initialconditions,etc.Subjectiveprobability（主观概率率）:

Probabilitiesrelatepropositions（命题））toagent'sownstateofknowledgee.g.,P(A25|noreportedaccidents)=0.06Thesearenotassertions（断言））abouttheworld命题的概概率随着着新证据据的发现现而改变变:

e.g.,P(A25|noreportedaccidents,5a.m.)=0.15不确定条条件下的的决策假设下述述概率是是真的:P(A25getsmethereontime|……)=0.04P(A90getsmethereontime|……)=0.70P(A120getsmethereontime|……)=0.95P(A1440getsmethereontime|……)=0.9999Whichactiontochoose?Dependsonmypreferences（偏好））formissingflightvs.timespentwaiting,etc.Utilitytheory（效用理理论）用来对偏偏好进行行表示和和推理Decisiontheory=probabilitytheory+utilitytheory决策理论论=概率理论论+效用理论论Syntax语法基本元素素:randomvariable（随机变变量）Arandomvariableissomeaspectoftheworldaboutwhichwe(may)haveuncertainty通常大写写e.g.,Cavity,Weather,Temperature类似于命命题逻辑辑:未知世界界被随机机变量的的赋值所所定义Booleanrandomvariables（布尔随随机变量量）e.g.,Cavity（牙洞））(doIhaveacavity?)Discreterandomvariables（离散随随机变量量）e.g.,Weatherisoneof<sunny,rainy,cloudy,snow>定义域mustbeexhaustive（穷尽的的）andmutuallyexclusive（互斥的的）Continuousrandomvariables（连续随随机变量量）e.g.,Temp=21.6;alsoallow,e.g.,Temp<22.0SyntaxElementaryproposition（命题））constructedbyassignmentofavaluetoarandomvariable:e.g.,Weather=sunny,Cavity=false(简写为¬cavity)Complexpropositionsformedfromelementarypropositionsandstandardlogicalconnectivese.g.,Weather=sunny∨Cavity=falseSyntaxAtomicevent:Acompletespecificationofthestateoftheworldaboutwhichtheagentisuncertain原子事件件：对智能能体无法法确定的的世界状状态的一一个完

整的详细描描述。E.g.,iftheworldconsistsofonlytwoBooleanvariablesCavityandToothache,thenthereare4distinctatomicevents:Cavity=false∧Toothache=falseCavity=false∧Toothache=trueCavity=true∧Toothache=falseCavity=true∧Toothache=trueAtomiceventsaremutuallyexclusiveandexhaustive穷尽和互互斥概率公理理对任意命命题A,B0≤P(A)≤1P(true)=1andP(false)=0P(A∨B)=P(A)+P(B)-P(A∧B)Priorprobability（先验概概率）Priororunconditionalprobabilities（无条件件概率））ofpropositions在没有任任何其它它信息存存在的情情况下关关于命题题的信度度e.g.,P(Cavity=true)=0.1andP(Weather=sunny)=0.72correspondtobeliefpriortoarrivalofany(new)evidenceProbabilitydistributiongivesvaluesforallpossibleassignments:概率分分布给出一一个随随机变变量所所有可可能取取值的的概率率P(Weather)=<0.72,0.1,0.08,0.1>(normalized（归一一化的的）,i.e.,sumsto1)Jointprobabilitydistributionforasetofrandomvariablesgivestheprobabilityofeveryatomiceventonthoserandomvariables(i.e.,everysamplepoint)联合概概率分分布给给出一一个随随机变变量集集的值值的全全部组组合的的概率率P(Weather,Cavity)=a4××2matrixofvalues:Everyquestionaboutadomaincanbeansweredbythejointdistributionbecauseeveryeventisasumofsamplepoints连续变变量的的概率率Expressdistributionasaparameterized（参数数化的的）functionofvalue:P(X=x)=U[18,26](x)=uniform（均匀匀分布布）densitybetween18and26连续变变量的的概率率MarginalDistributions（边缘缘概率率分布布）Marginaldistributionsaresub-tableswhicheliminatevariablesMarginalization(summingout):CombinecollapsedrowsbyaddingConditionalprobability（条件件概率率）Conditionalorposteriorprobabilities（后验验概率率）P(a|b)证据累累积过过程的的形式式化和和发现现新证证据后后的概概率更更新当一个个命题题为真真的条条件下下，指指定命命题的的概率率e.g.,P(cavity|toothache)=0.8i.e.,鉴于牙牙疼是是已知知证据据(Notationforconditionaldistributions（条件件概率率分布布）:P(cavity|toothache)=asinglenumberP(Cavity,Toothache)=2x2tablesummingto1P(Cavity|Toothache)=2-elementvectorof2-elementvectorsIfweknowmore,e.g.,cavityisalsogiven,thenwehaveP(cavity|toothache,cavity)=1新证据据可能能是不不相关关的，，可以以简化化,e.g.,P(cavity|toothache,sunny)=P(cavity|toothache)=0.8条件概概率定义条件概概率为为:P(a|b)=P(a∧∧b)/P(b)ifP(b)>0Productrule（乘法法规则则）givesanalternativeformulation:P(a∧∧b)=P(a|b)P(b)=P(b|a)P(a)Ageneralversionholdsforwholedistributions,e.g.,P(Weather,Cavity)=P(Weather|Cavity)P(Cavity)(Viewasasetof4××2equations,notmatrixmultiplication)Chainrule（链式式法则则）isderivedbysuccessiveapplicationofproductrule:条件概概率条件概概率跟跟标准准概率率一样样,forexample:0<=P(a|e)<=1conditionalprobabilitiesarebetween0and1inclusiveP(a1|e)+P(a2|e)+...+P(ak|e)=1conditionalprobabilitiessumto1wherea1,……,akareallvaluesinthedomainofrandomvariableAP(¬¬a|e)=1-P(a|e)negationforconditionalprobabilities通过枚枚举的的推理理Startwiththejointprobabilitydistribution（全联联合概概率分分布））:Foranypropositionφ,sumtheatomiceventswhereitistrue:一个命题的概概率等于所有有当它为真时时的原子事件件的概率和通过枚举的推推理Startwiththejointprobabilitydistribution（全联合概率率分布）:Foranypropositionφ,sumtheatomiceventswhereitistrue:一个命题题的概率率等于所所有当它它为真时时的原子子事件的的概率和和通过枚举举的推理理Startwiththejointprobabilitydistribution（全联合合概率分分布）:Foranypropositionφ,sumtheatomiceventswhereitistrue:一个命题题的概率率等于所所有当它它为真时时的原子子事件的的概率和和通过枚举举的推理理Startwiththejointprobabilitydistribution（全联合合概率分分布）:Normalization（归一化化）Denominator（分母））canbeviewedasanormalizationconstantαP(Cavity|toothache)=αP(Cavity,toothache)

=α[P(Cavity,toothache,catch)+P(Cavity,toothache,¬catch)]=α[<0.108,0.016>+<0.012,0.064>]=αα<0.12,0.08>=<0.6,0.4>Generalidea:computedistributiononqueryvariablebyfixingevidencevariables（证据变变量）andsummingoverhiddenvariables（未观测测变量））通过枚举举的推理理Typically,weareinterestedin

theposteriorjointdistributionofthequeryvariables（查询变变量）Ygivenspecificvaluesefortheevidencevariables（证据变变量）ELetthehiddenvariables（未观测测变量））beH=X-Y–EThentherequiredsummationofjointentriesisdonebysummingoutthehiddenvariables:P(Y|E=e)=αP(Y,E=e)=αΣhP(Y,E=e,H=h)ThetermsinthesummationarejointentriesbecauseY,EandHtogetherexhaustthesetofrandomvariables（Y,E,H构成了域域中所有有变量的的完整集集合）Obviousproblems:1.Worst-casetimecomplexityO(dn)wheredisthelargestarity2.SpacecomplexityO(dn)tostorethejointdistribution

3.HowtofindthenumbersforO(dn)entries?Independence（独立性性）AandBareindependentiffP(A|B)=P(A)orP(B|A)=P(B)orP(A,B)=P(A)P(B)E.g:rollof2die:P({1},{3})=1/6*1/6=1/36P(Toothache,Catch,Cavity,Weather)=P(Toothache,Catch,Cavity)P(Weather)32entriesreducedto12;fornindependentbiasedcoins,O(2n)→O(n)Absoluteindependencepowerfulbutrare绝对独立立强大但但罕见Dentistry（牙科领领域）isalargefieldwithhundredsofvariables,noneofwhichareindependent.Whattodo?独立的滥滥用天真的数数学笑话话：一个著名名统计学学家永远远不会坐坐飞机旅旅行,因为他研研究了航航空旅行行和估计计,任何给定定的航班班上有炸炸弹的可可能性是是一百万万分之一一,他不准备备接受这这些可能能性。有一天，，一位同同时在远远离家乡乡的会议议上遇到到他。““你怎么么到这里里的？坐坐火车吗吗？”“不，我飞飞过来的的”“Whataboutthepossibilityofabomb?”“Well,Ibeganthinkingthatiftheoddsofonebombare1:million,thentheoddsoftwobombsare(1/1,000,000)x(1/1,000,000).Thisisavery,verysmallprobability,whichIcanaccept.SonowIbringmyownbombalong!”Conditionalindependence条件独立立性Randomvariablescanbedependent,butconditionallyindependentExample:YourhousehasanalarmNeighborJohnwillcallwhenhehearsthealarmNeighborMarywillcallwhenshehearsthealarmAssumeJohnandMarydon’ttalktoeachotherIsJohnCallindependentofMaryCall?No–IfJohncalled,itislikelythealarmwentoff,whichincreasestheprobabilityofMarycallingP(MaryCall|JohnCall)≠P(MaryCall)条件独立性But,ifweknowthestatusofthealarm,JohnCallwillnotaffectwhetherornotMarycallsP(MaryCall|Alarm,JohnCall)=P(MaryCall|Alarm)WesayJohnCallandMaryCallareconditionally

independentgivenAlarmIngeneral,“AandBareconditionallyindependentgivenC””means:P(A|B,C)=P(A|C)P(B|A,C)=P(B|C)P(A,B|C)=P(A|C)P(B|C)条件独立性P(Toothache,Cavity,Catch)has23-1=7independententries专业领域知识识:Cavitydirectlycausestoothacheandprobe-catches.IfIhave

acavity,theprobabilitythattheprobecatchesinitdoesn‘tdependonwhether

Ihaveatoothache:

(1)P(catch|toothache,cavity)=P(catch|cavity)ThesameindependenceholdsifIhaven’tgotacavity:(2)P(catch|toothache,¬cavity)=P(catch|¬cavity)CatchisconditionallyindependentofToothachegivenCavity:P(Catch|Toothache,Cavity)=P(Catch|Cavity)Equivalentstatements:P(Toothache|Catch,Cavity)=P(Toothache|Cavity)P(Toothache,Catch|Cavity)=P(Toothache|Cavity)P(Catch|Cavity)条件独立性Writeoutfulljointdistributionusingchainrule:P(Toothache,Catch,Cavity)

=P(Toothache|Catch,Cavity)P(Catch,Cavity)

=P(Toothache|Catch,Cavity)P(Catch|Cavity)P(Cavity)

=P(Toothache|Cavity)P(Catch|Cavity)P(Cavity)

I.e.,2+2+1=5independentnumbersInmostcases,theuseofconditionalindependencereducesthesizeoftherepresentationofthejointdistributionfromexponentialinntolinearinn.在大多数情况况下，使用条条件独立性能能将全联合概概率的表示由由n的指数关系减减为n的线性关系。。Conditionalindependenceisourmostbasicandrobustformofknowledgeaboutuncertainenvironments.Bayes’’Rule（贝叶斯法则则）Bayes’’Rule（贝叶斯法则则）乘法原则⇒Bayes‘rule:orindistributionform为什么该法则则非常有用?将条件倒转通常一个条件件是复杂的，，一个是简单单的许多系统的基基础(e.g.语音识别)现代AI基础！Bayes’’Rule（贝叶斯法则则）Usefulforassessingdiagnosticprobability（诊断概率）fromcausalprobability（因果概率）:E.g.,letMbemeningitis（脑膜炎）,Sbestiffneck（脖子僵硬））:Note:脑膜炎的后验验概率依然非非常小!Note:依然要先检测测脖子僵硬!Why?Bayes’’RuleinPractice使用贝叶斯法法则:IH=“havingaheadache“头痛F=“comingdownwithFlu””流感P(H)=1/10P(F)=1/40P(H|F)=1/2有一天你早上上醒来发现头头很痛，于是是得到以下结结论：“因为为得了流感以以后50%的几率会引起起头痛，所以以我有50%的几率得了流流感”Isthisreasoningcorrect?使用贝叶斯法法则:IH="havingaheadache“F="comingdownwithFlu"P(H)=1/10P(F)=1/40P(H|F)=1/2TheProblem:

P(F|H)=?使用贝叶斯法法则:IH="havingaheadache“F="comingdownwithFlu"P(H)=1/10P(F)=1/40P(H|F)=1/2TheProblem:

P(F|H)=P(H|F)P(F)/P(H)=1/8≠P(H|F)使用贝叶斯法法则:II在一个包裹里里有2个信封一个信封里有有一个红球(worth$100)和一个黑球另一个信封里里有2个黑球.黑球一文不值值然后你随机拿拿出一个信封封，并随机拿拿出一个球–it’sblack此时此刻给你你个机会换一一个信封.是换呢还是换换呢还是换呢呢?使用贝叶斯法法则:IIE:envelope,1=(R,B),2=(B,B)B:theeventofdrawingablackballP(E|B)=P(B|E)*P(E)/P(B)WewanttocompareP(E=1|B)vs.P(E=2|B)P(B|E=1)=0.5,P(B|E=2)=1P(E=1)=P(E=2)=0.5P(B)=P(B|E=1)P(E=1)+P(B|E=2)P(E=2)=(.5)(.5)+(1)(.5)=.75P(E=1|B)=P(B|E=1)P(E=1)/P(B)=(.5)(.5)/(.75)=1/3P(E=2|B)=P(B|E=2)P(E=2)/P(B)=(1)(.5)/(.75)=2/3因此在已发现现一个黑球后后,该信封是1的后验概率(thusworth$100)比信封是2的后验概率低低所以还是换吧吧课堂测验一名医生做了了一个具有99%可靠性的测试试：也就是说说,99%的病人其检测测呈阳性,99%的健康人士检检测呈阴性.该医生估计1%的人类病了。。。。Question:一个患者检测测呈阳性.该患者得病的的几率是多少少?0-25%,25-75%,75-95%,or95-100%?课堂测验Adoctorperformsatestthathas99%reliability,i.e.,99%of

peoplewhoaresicktestpositive,and99%ofpeoplewhoarehealthytestnegative.Thedoctorestimatesthat1%ofthepopulationissick.Question:Apatienttestspositive.Whatisthechancethatthepatientissick?0-25%,25-75%,75-95%,or95-100%?Intuitiveanswer:99%;Correctanswer:50%Bayes’’rulewith多重证据和条条件独立性P(Cavity|toothache∧catch)

=αP(toothache∧catch|Cavity)P(Cavity)

=αP(toothache|Cavity)P(catch|Cavity)P(Cavity)ThisisanexampleofanaïveBayesmodel（朴素贝叶斯斯模型）:Totalnumberofparameters（参数）islinearinn链式法则全联合分布using链式法则:

P(Toothache,Catch,Cavity)=P(Toothache|Catch,Cavity)P(Catch,Cavity)=P(T