人工智能不确定性

手写数字识别文本分类图像分割第八章Uncertainty

不确定性对应教材第13章本章大纲Uncertainty不确定性Probability概率SyntaxandSemantics语法与语义Inference推理IndependenceandBayes‘Rule

—独立性及贝叶斯法则

不确定性智能体几乎从来无法了解关于其环境的全部事实。因此其必须在不确定的环境下行动。概率推理

得到了某一证据，那么有多大的几率结论为真？

例如：我颈部痛;我得脑膜炎的可能有多大?不确定性假如有如下规则:

iftoothache（牙疼）then原因是cavity（牙齿有洞）但并不是所有牙疼的病人都是因为牙齿有洞，所以我们可以建立如下规则:

iftoothacheand¬gum-disease（牙龈疾病）and

¬filling（补牙）and...thenproblem=cavity以上规则是复杂的;更好的方法:

iftoothachethenproblemiscavitywith0.8probability

orP(cavity|toothache)=0.8

theprobabilityofcavityis0.8giventoothacheisobserved不确定性 LetactionAt=离起飞时间提前t分钟动身去机场

At会使我准时到达机场吗?

Problems:

1.partialobservability/部分可观察性(roadstate,otherdrivers‘plans)

2.noisysensors(trafficreports)

3.行动结果的不确定性(flattire,etc.)

4.immensecomplexityofmodelingandpredictingtraffic

因此一个纯粹的逻辑描述方法：

1.risksfalsehood（错误风险）:“A25

willgetmethereontime”,or

2.leadstoconclusionsthataretooweakfordecisionmaking:

“A25

willgetmethereontimeifthere’snoaccidentonthebridgeanditdoesn‘trainandmytiresremainintactetcetc.”

(A1440

mightreasonablybesaidtogetmethereontimebutI’dhavetostayovernightintheairport…)世界与模型中的不确定性Trueuncertainty:rulesareprobabilisticinnature

掷骰子，抛硬币惰性:把所有意外的规则都列举出来是很困难的

花费太多时间来确定所有的相关因素

这些规则过于繁杂而难以使用理论的无知:某些领域中还没有完整的理论

(e.g.,medicaldiagnosis)实践的无知:掌握了所有规则但是

并不是所有的相关信息都能被收集到处理不确定性的方法概率理论作为一种正式的方法for：

不确定知识的表示和推理

命题中的模型信度(event,conclusion,diagnosis,etc.)

给定可获得的证据,

A25

willgetmethereontimewithprobability0.04概率是不确定性的语言

现代AI的中心支柱Probability概率概率理论提提供了一种种方法以概概括来自我我们的惰性性和无知的的不确定性性。ProbabilisticassertionssummarizeeffectsofLaziness（惰性）:failuretoenumerateexceptions（例外）,qualifications（条件）,etc.Ignorance（理论的无无知）:lackofrelevantfacts,initialconditions,etc.Subjectiveprobability（主观概率）:

Probabilitiesrelatepropositions（命题）toagent'sownstateofknowledge

e.g.,P(A25|noreportedaccidents)=0.06Thesearenotassertions（断言）abouttheworld命题的概率率随着新证证据的发现现而改变:

e.g.,P(A25|noreportedaccidents,5a.m.)=0.15不确定条件件下的决策策假设下述概概率是真的的:P(A25getsmethereontime|……)=0.04

P(A90getsmethereontime|……)=0.70

P(A120getsmethereontime|……)=0.95

P(A1440getsmethereontime|……)=0.9999Whichactiontochoose?Dependsonmypreferences（偏好）formissingflightvs.timespentwaiting,etc.Utilitytheory（效用理论论）用来对偏好好进行表示示和推理Decisiontheory=probabilitytheory+utilitytheory决策理论=概率理论+效用理论Syntax语法基本元素:randomvariable（随机变量量）Arandomvariableissomeaspectoftheworldaboutwhichwe(may)haveuncertainty通常大写e.g.,Cavity,Weather,Temperature类似于命题题逻辑:未知世界被被随机变量量的赋值所所定义Booleanrandomvariables（布尔随机机变量）e.g.,Cavity（牙洞）(doIhaveacavity?)Discreterandomvariables（离散随机机变量）e.g.,Weatherisoneof<sunny,rainy,cloudy,snow>定义域mustbeexhaustive（穷尽的））andmutuallyexclusive（互斥的））Continuousrandomvariables（连续随机机变量）e.g.,Temp=21.6;alsoallow,e.g.,Temp<22.0SyntaxElementaryproposition（命题）constructedbyassignmentofavaluetoarandomvariable:e.g.,Weather=sunny,Cavity=false(简写为¬cavity)Complexpropositionsformedfromelementarypropositionsandstandardlogicalconnectivese.g.,Weather=sunny∨Cavity=falseSyntaxAtomicevent:Acompletespecificationofthestateof

theworldaboutwhichtheagentisuncertain原子事件：对智能体体无法确定定的世界状状态的一个个完

整的详细描述述。E.g.,iftheworldconsistsofonlytwoBooleanvariablesCavityandToothache,thenthereare4distinctatomicevents:Cavity=false∧Toothache=falseCavity=false∧Toothache=trueCavity=true∧Toothache=falseCavity=true∧Toothache=trueAtomiceventsaremutuallyexclusiveandexhaustive穷尽和互斥斥概率公理对任意命题题A,B0≤P(A)≤1

P(true)=1andP(false)=0

P(A∨B)=P(A)+P(B)-P(A∧B)Priorprobability（先验概率率）Priororunconditionalprobabilities（无条件概概率）ofpropositions在没有任何何其它信息息存在的情情况下关于于命题的信信度e.g.,P(Cavity=true)=0.1andP(Weather=sunny)=0.72correspondtobeliefpriortoarrivalofany(new)evidenceProbabilitydistributiongivesvaluesforallpossibleassignments:概率分布给出一个随随机变量所所有可能取取值的概率率P(Weather)=<0.72,0.1,0.08,0.1>(normalized（归一化的的）,i.e.,sumsto1)Jointprobabilitydistributionforasetofrandomvariablesgivestheprobabilityofeveryatomiceventonthoserandomvariables(i.e.,everysamplepoint)联合概率分分布给出一一个随机变变量集的值值的全部组组合的概率率P(Weather,Cavity)=a4××2matrixofvalues:Everyquestionaboutadomaincanbeansweredbythejointdistributionbecauseeveryeventisasumofsamplepoints连续变量的的概率Expressdistributionasaparameterized（参数化的的）functionofvalue:P(X=x)=U[18,26](x)=uniform（均匀分布布）densitybetween18and26连续变量的的概率MarginalDistributions（边缘概率率分布）Marginaldistributionsaresub-tableswhicheliminatevariablesMarginalization(summingout):CombinecollapsedrowsbyaddingConditionalprobability（条件概率率）Conditionalorposteriorprobabilities（后验概率率）P(a|b)证据累积过过程的形式式化和发现现新证据后后的概率更更新当一个命题题为真的条条件下，指指定命题的的概率e.g.,P(cavity|toothache)=0.8i.e.,鉴于牙疼是是已知证据据(Notationforconditionaldistributions（条件概率率分布）:P(cavity|toothache)=asinglenumberP(Cavity,Toothache)=2x2tablesummingto1P(Cavity|Toothache)=2-elementvectorof2-elementvectorsIfweknowmore,e.g.,cavityisalsogiven,thenwehaveP(cavity|toothache,cavity)=1新证据可能能是不相关关的，可以以简化,e.g.,P(cavity|toothache,sunny)=P(cavity|toothache)=0.8条件概率定义条件概率为为:

P(a|b)=P(a∧∧b)/P(b)ifP(b)>0Productrule（乘法规则则）givesanalternativeformulation:

P(a∧b)=P(a|b)P(b)=P(b|a)P(a)Ageneralversionholdsforwholedistributions,e.g.,P(Weather,Cavity)=P(Weather|Cavity)P(Cavity)(Viewasasetof4××2equations,notmatrixmultiplication)Chainrule（链式法则则）isderivedbysuccessiveapplicationofproductrule:条件概率条件概率跟跟标准概率率一样,forexample:0<=P(a|e)<=1conditionalprobabilitiesarebetween0and1inclusiveP(a1|e)+P(a2|e)+...+P(ak|e)=1conditionalprobabilitiessumto1wherea1,…,akareallvaluesinthedomainofrandomvariableAP(¬a|e)=1-P(a|e)negationforconditionalprobabilities通过枚举的的推理Startwiththejointprobabilitydistribution（全联合概概率分布））:Foranypropositionφ,sumtheatomiceventswhereitistrue:一个命题的的概率等于于所有当它它为真时的的原子事件件的概率和和通过枚举的的推理Startwiththejointprobabilitydistribution（全联合概概率分布））:Foranypropositionφ,sumtheatomiceventswhereitistrue:一个命题的的概率等于于所有当它它为真时的的原子事件件的概率和和通过枚举的的推理Startwiththejointprobabilitydistribution（全联合概概率分布））:Foranypropositionφ,sumtheatomiceventswhereitistrue:一个命题的的概率等于于所有当它它为真时的的原子事件件的概率和和通过枚举的的推理Startwiththejointprobabilitydistribution（全联合概概率分布））:Normalization（归一化））Denominator（分母）canbeviewedasanormalizationconstantαP(Cavity|toothache)=αP(Cavity,toothache)

=α[P(Cavity,toothache,catch)+P(Cavity,toothache,¬catch)]

=α[<0.108,0.016>+<0.012,0.064>]

=αα<0.12,0.08>=<0.6,0.4>Generalidea:computedistributiononqueryvariablebyfixingevidencevariables（证据变量量）andsummingoverhidden

variables（未观测变变量）通过枚举的的推理Typically,weareinterestedintheposteriorjointdistributionofthequeryvariables（查询变量量）Ygivenspecificvaluesefortheevidencevariables（证据变量量）ELetthehiddenvariables（未观测变变量）beH=X-Y–EThentherequiredsummationofjointentriesisdonebysummingoutthehiddenvariables:P(Y|E=e)=αP(Y,E=e)=αΣhP(Y,E=e,H=h)ThetermsinthesummationarejointentriesbecauseY,EandHtogetherexhaustthesetofrandomvariables（Y,E,H构成了域中中所有变量量的完整集集合）Obviousproblems:

1.Worst-casetimecomplexityO(dn)wheredisthelargestarity2.SpacecomplexityO(dn)tostorethejointdistribution3.HowtofindthenumbersforO(dn)entries?Independence（独立性））AandBareindependentiffP(A|B)=P(A)orP(B|A)=P(B)orP(A,B)=P(A)P(B)E.g:rollof2die:P({1},{3})=1/6*1/6=1/36P(Toothache,Catch,Cavity,Weather)=P(Toothache,Catch,Cavity)P(Weather)32entriesreducedto12;fornindependentbiasedcoins,O(2n)→O(n)Absoluteindependencepowerfulbutrare绝对独立强强大但罕见见Dentistry（牙科领域域）isalargefieldwithhundredsofvariables,noneofwhichareindependent.Whattodo?独立的滥用用天真的数学学笑话：一个著名统统计学家永永远不会坐坐飞机旅行行,因为他研究究了航空旅旅行和估计计,任何给定的的航班上有有炸弹的可可能性是一一百万分之之一,他不准备接接受这些可可能性。有一天，一一位同时在在远离家乡乡的会议上上遇到他。。“你怎么么到这里的的？坐火车车吗？”“不，我飞过过来的”“Whataboutthepossibilityofabomb?”“Well,Ibeganthinkingthatiftheoddsofonebombare1:million,thentheoddsoftwobombsare(1/1,000,000)x(1/1,000,000).Thisisavery,verysmallprobability,whichIcanaccept.SonowIbringmyownbombalong!”Conditionalindependence条件独立性性Randomvariablescanbedependent,butconditionallyindependentExample:YourhousehasanalarmNeighborJohnwillcallwhenhehearsthealarmNeighborMarywillcallwhenshehearsthealarmAssumeJohnandMarydon’’ttalktoeachotherIsJohnCallindependentofMaryCall?No–IfJohncalled,itislikelythealarmwentoff,whichincreasestheprobabilityofMarycallingP(MaryCall|JohnCall)≠≠P(MaryCall)条件件独独立立性性But,ifweknowthestatusofthealarm,JohnCallwillnotaffectwhetherornotMarycallsP(MaryCall|Alarm,JohnCall)=P(MaryCall|Alarm)WesayJohnCallandMaryCallareconditionallyindependentgivenAlarmIngeneral,““AandBareconditionallyindependentgivenC””means:P(A|B,C)=P(A|C)P(B|A,C)=P(B|C)P(A,B|C)=P(A|C)P(B|C)条件件独独立立性性P(Toothache,Cavity,Catch)has23-1=7independententries专业业领领域域知知识识:Cavitydirectlycausestoothacheandprobe-catches.IfIhaveacavity,theprobabilitythattheprobecatchesinitdoesn‘‘tdependonwhetherIhaveatoothache:(1)P(catch|toothache,cavity)=P(catch|cavity)ThesameindependenceholdsifIhaven’’tgotacavity:(2)P(catch|toothache,¬cavity)=P(catch|¬¬cavity)CatchisconditionallyindependentofToothachegivenCavity:P(Catch|Toothache,Cavity)=P(Catch|Cavity)Equivalentstatements:P(Toothache|Catch,Cavity)=P(Toothache|Cavity)P(Toothache,Catch|Cavity)=P(Toothache|Cavity)P(Catch|Cavity)条件件独独立立性性Writeoutfulljointdistributionusingchainrule:P(Toothache,Catch,Cavity)=P(Toothache|Catch,Cavity)P(Catch,Cavity)=P(Toothache|Catch,Cavity)P(Catch|Cavity)P(Cavity)=P(Toothache|Cavity)P(Catch|Cavity)P(Cavity)I.e.,2+2+1=5independentnumbersInmostcases,theuseofconditionalindependencereducesthesizeoftherepresentationofthejointdistributionfromexponentialinntolinearinn.在大多多数情情况下下，使使用条条件独独立性性能将将全联联合概概率的的表示示由n的指数数关系系减为为n的线性性关系系。Conditionalindependenceisourmostbasicandrobustformofknowledgeaboutuncertainenvironments.Bayes’Rule（贝叶叶斯法法则））Bayes’Rule（贝叶叶斯法法则））乘法原原则⇒Bayes‘rule:orindistributionform为什么么该法法则非非常有有用?将条件件倒转转通常一一个条条件是是复杂杂的，，一个个是简简单的的许多系系统的的基础础(e.g.语音识识别)现代AI基础！！Bayes’Rule（贝叶叶斯法法则））Usefulforassessingdiagnosticprobability（诊断概率））fromcausalprobability（因果概率））:E.g.,letMbemeningitis（脑膜膜炎））,Sbestiffneck（脖子子僵硬硬）:Note:脑膜炎炎的后后验概概率依依然非非常小小!Note:依然要要先检检测脖脖子僵僵硬!Why?Bayes’RuleinPractice使用贝贝叶斯斯法则则:IH=““havingaheadache““头痛F=““comingdownwithFlu”流感P(H)=1/10P(F)=1/40P(H|F)=1/2有一天天你早早上醒醒来发发现头头很痛痛，于于是得得到以以下结结论：：“因因为得得了流流感以以后50%的几率率会引引起头头痛，，所以以我有有50%的几率率得了了流感感”Isthisreasoningcorrect?使用贝贝叶斯斯法则则:IH="havingaheadache““F="comingdownwithFlu"P(H)=1/10P(F)=1/40P(H|F)=1/2TheProblem:P(F|H)=?使用贝贝叶斯斯法则则:IH="havingaheadache““F="comingdownwithFlu"P(H)=1/10P(F)=1/40P(H|F)=1/2TheProblem:P(F|H)=P(H|F)P(F)/P(H)=1/8≠≠P(H|F)使用贝贝叶斯斯法则则:II在一个个包裹裹里有有2个信封封一个信信封里里有一一个红红球(worth$100)和一个个黑球球另一个个信封封里有有2个黑球球.黑球一一文不不值然后你你随机机拿出出一个个信封封，并并随机机拿出出一个个球–it’’sblack此时此此刻给给你个个机会会换一一个信信封.是换呢呢还是是换呢呢还是是换呢呢?使用贝贝叶斯斯法则则:IIE:envelope,1=(R,B),2=(B,B)B:theeventofdrawingablackballP(E|B)=P(B|E)*P(E)/P(B)WewanttocompareP(E=1|B)vs.P(E=2|B)P(B|E=1)=0.5,P(B|E=2)=1P(E=1)=P(E=2)=0.5P(B)=P(B|E=1)P(E=1)+P(B|E=2)P(E=2)=(.5)(.5)+(1)(.5)=.75P(E=1|B)=P(B|E=1)P(E=1)/P(B)=(.5)(.5)/(.75)=1/3P(E=2|B)=P(B|E=2)P(E=2)/P(B)=(1)(.5)/(.75)=2/3因此在在已发发现一一个黑黑球后后,该信封封是1的后验验概率率(thusworth$100)比信封封是2的后验验概率率低所以还还是换换吧课堂测测验一名医医生做做了一一个具具有99%可靠性性的测测试：：也就就是说说,99%的病人人其检检测呈呈阳性性,99%的健康康人士士检测测呈阴阴性.该医生生估计计1%的人类类病了了。。。。Question:一个患患者检检测呈呈阳性性.该患者者得病病的几几率是是多少少?0-25%,25-75%,75-95%,or95-100%?课堂测测验Adoctorperformsatestthathas99%reliability,i.e.,99%ofpeoplewhoaresicktestpositive,and99%ofpeoplewhoarehealthytestnegative.Thedoctorestimatesthat1%ofthepopulationissick.Question:Apatienttestspositive.Whatisthechancethatthepatientissick?0-25%,25-75%,75-95%,or95-100%?Intuitiveanswer:99%;Correctanswer:50%Bayes’rulewith多重证证据和和条件件独立立性P(Cavity|toothache∧catch)=αP(toothache∧catch|Cavity)P(Cavity)=αP(toothache|Cavity)P(catch|Cavity)P(Cavity)ThisisanexampleofanaïïveBayesmodel（朴素素贝叶叶斯模模型））:Totalnumberofparameters（参数数）islinearinn链式法法则全联合合分布布using链式法法则:P(Toothache,Catch,Cavity)=P(Toothache|Catch,Cavity)P(Catch,Cavity)=P(Toothache|Catch,Cavity)P(Catch|Cavity)P(Cavity)=P(Toothache|Cavity)P(