李开复-对话式界面

ConversationalComputers:

Always10YearsAway?Kai-FuLeeCorporateVicePresidentMicrosoftCorporationWhyConversationalInterface?Speech:“invented”forinteraction“[Speech&languageare]abiologicaladaptationtocommunicateinformation…Oneofnature’sengineeringmarvels”–StevenPinker“Visionevolvedfromtheneedtosurvive;speechevolvedfromtheneedtocommunicate”–MichaelDertouzos.Benefitsof“ConversationalInterface”“Tome,speechrecognitionwillbeatransformingcapability…whenyoucanspeaktoyourcomputeranditwillunderstandwhatyou'resayingincontext.”–GordonMoore“Speechandnaturallanguageunderstandingarethekeytechnologiesthatwillhavethemostimpactinthenext15years.”–BillGatesFutureUIvisionassumeconversationalUIApple’s“KnowledgeNavigator”.Microsoft’s“informationatyourfingertips”.SciencefictionmoviesassumeconversationalUIBut“Always”10YearsAway1950JeromeWeisnerpredictedby1960machinetranslationmaybepossible1957HerbertSimonpredictedby1967machinewillmatchhumanperformanceinmanyareas1969USExpertPanelpredicted“voiceI/Owillbeincommonuseby1978”1993Ipredictedby2003everyPCwillshipwithspeechrecognition1998GartnerGrouppredictedPCUIwillassumevoiceinputby2003DecomposingthePredictionSpeechrecognitionTexttospeechNaturallanguageunderstandingWhyhavewebeenaconstant10yearsaway?My3-year&10-yearpredictionsNaturalLanguageUnderstandingSpeechRecognitionTexttoSpeechTalkOutlineTalkOutlineNaturalLanguageUnderstandingSpeechRecognitionTexttoSpeechSpeechrecognitionTexttospeechNaturallanguageunderstandingWhyhavewebeenaconstant10yearsaway?My3-year&10-yearpredictionsFundamentalEquationofSpeechRecognitionXistheacousticwaveformWisthewordstringAspeechrecognizerfindsWsuchthat

W=argmaxp(W|X)=argmaxp(X|W)p(W)p(X|W)istheacousticmodelp(W)isthelanguagemodelStatisticalModelingImprovingtheacousticmodel–p(X|W)StatisticalApproachBuildadetailedstatisticalmodelforeachword.Detailcouldbebasedonphonetics,speaker,dialect,gender,ordata-drivendetailsetc.Collectalotmoresamplesforeachword.Thereisnodatalikemoredata.Gotostepone.Improvingthelanguagemodel–p(W)StatisticalApproach–Trigrams.Thereisnodatalikemoredata.Thishelpsrecognition,notunderstanding.DoesMoore’sLawHelpSpeech?Moore’slawisnecessarybutnotsufficientJustfasterchipsmeansrecognitionerrorsappearfaster.Super-Moore’slawforspeech:Fasterprocessors/memory/disk+Gettingmorerealdata&feedbackloop+ImprovedstatisticalmodelsResult:Moore’slawdoublesperformancein18monthsSuper-Moore’slawhalveserrorsin60monthsSpeechRecognition:

ApproachingHumanErrorRateMicrosoftlicensedCMUSphinx-IIWhisperinMSRSpeechinOfficeXPSpeechinTablet/Office11SpeechinLonghornHumanErrorRateTalkOutlineNaturalLanguageUnderstandingSpeechRecognitionTexttoSpeechSpeechrecognitionTexttospeechNaturallanguageunderstandingWhyhavewebeenaconstant10yearsaway?My3-year&10-yearpredictionsFundamentalApproachforTTSConcatenativeSynthesisConcatenationofpre-recordedspeechunitsFront-endNaturallanguageprocessing(wordbreaking,POS…)Determineemphasistodrivespeed,pitch,loudness.Back-endCollectalotofdataCarefullysegment&storeinadatabaseSelectthebestunitsfromthedatabaseFindstatisticalmetricsthatmatch“naturalness”,e.g.,smoothnessratherthanspecificdurationtargetsUsethesemetricstoselectunitsTexttoSpeech

ApproachingHumanNaturalnessNaturalnessHumanNaturalnessASR&TTS:Optimization&Engineering

ByleveragingMoore’slawExponentialimprovementsfrom…FasterCPU+biggerdatabase+betteralgorithmApproachinghumanabilities,butnotAI,but…Optimization,or“speechengineering”Stillfallsshortofhumanson:Learning,adaptation.Robustnesstoenvironment.ButmanyapplicationsjustfromASR&TTS:ASR:Dictation,speechsearch,speakerverification,languagelearning…TTS:Telephonyinfoaccess,voicefonts,voiceconversion…TalkOutlineNaturalLanguageUnderstandingSpeechRecognitionTexttoSpeechSpeechrecognitionTexttospeechNaturallanguageunderstandingWhyhavewebeenaconstant10yearsaway?My3-year&10-yearpredictionsSyntax(rulesofthehuman’slanguage)Nouns,verbs,etc.andhowtheycombine“BookaboutatriptoChicago”vs.“BookatriptoChicago”Normalizelinguisticvariations.SemanticsMeaningofthewordsBookmeansreserveaticket;requiresfrom-city,to-city,etc.Context(additionalhints)Domainknowledge:NotrainfromHawaiitoChicagoStatistics:Bookasanoun>Bookasaverb“BookChicago”PersonalPreferences:Whereyoulive,yourcalendar,howyoupay…Modeloftime,urgency,

presenceDialog(resolvingambiguity&determineintent)“Buyabookorbooktravel?”“Whatdatewouldyouliketotravel?”NaturalLanguageUnderstandingCombines:ApplyingStatisticstoUnderstandingEngineeringapproach:Focusononedomain,engineeralltheknowledge.Collectdata&createfeedbacklooptoimprove.ApplyingBayesRuletounderstandingWisthewordstring

MisthemeaningAspeechrecognizerfindsMsuchthat

M=argmaxp(M|W)=argmaxp(W|M)p(W)p(W|M)modelsallthewaystoexpressa“meaning”p(M)isthesemanticmodelWhatis“unsolved”byStatistics?FusionofmanysourcesofknowledgeDomain-freeunderstandingInstantcontextswitchingGeneralknowledgeHistory,sports,etc.Commonsensereasoning“Leastcommonofallsenses”Ambiguity“Mr.WrightshouldwritetoMrs.Wright

rightaway”Emotion,humor,etc.Manyofthechallengesare“AI-complete”MilestonesinSpeechTechnologyResearch

1962 1967 1972 1977 1982 1987 1992 1997 2002IsolatedWordsFilter-bankanalysis;Time-normalization;DynamicprogrammingIsolatedWords;ConnectedDigits;ContinuousSpeechPatternrecognition;LPCanalysis;Clusteringalgorithms;ContinuousSpeech;SpeechUnderstandingStochasticlanguageunderstanding;Finite-statemachines;Statisticallearning;SmallVocabulary,AcousticPhonetics-basedMediumVocabular,Template-basedLargeVocabulary;Syntax,Semantics,ConnectedWords;ContinuousSpeechLargeVocabulary,Statistical-basedHiddenMarkovmodels;StochasticLanguagemodeling;Spokendialog;MultiplemodalitiesVeryLargeVocabulary;Semantics,MultimodalDialog,TTSConcatenativesynthesis;Machinelearning;Mixed-initiativedialog;FueledbyMoore’sLaw+Data+ResearchTalkOutlineSpeechrecognitionTexttospeechNaturallanguageunderstandingWhyhavewebeenaconstant10yearsaway?My3-year&10-yearpredictionsNaturalLanguageUnderstandingSpeechRecognitionTexttoSpeechWhyConstant10YearsAway?ImmaturetechnologyImprovingbutonlyrecentlybecomingusefulOver-soldexpectationsSciencefictionmoviesEffective(butnotrealproduct)demosUnder-estimatedrisksUserhabitsarehardtochangeCostofdevelopingspeechapplicationishighThingsaredifferentnow!TechnologyisreadyAndwehavelearnedourlessons.WhatHaveWeLearned?Don’tmakepredictions.…basedonextrapolatingfromonedatapoint!Thereisnodatalikemoredata.Realdata&feedback>Moore’sLaw.Changetheworld,onedomainatatime.Breakthroughfromdata+rigorisjustfine.Startwithuser’scomfortzone.Startwiththegreatestcustomerneed&businessopportunity.TalkOutlineSpeechrecognitionTexttospeechNaturallanguageunderstandingWhyhavewebeenaconstant10yearsaway?My3-year&10-yearpredictionsNaturalLanguageUnderstandingSpeechRecognitionTexttoSpeech3-YearSpeechPrediction:

MostRealisticNear-TermSpeechApplicationMeeting/VoicemailTranscriptionMarketOpportunityMobileDevices/CarsTelephony/CallCenterAccessibilityDesktopDictationWindowsCommands&Applications/APITechnologyReadinessCustomerNeedPoorAlternative10-YearSpeechPredictionsTelephonyDevicesDesktopDictation&NewapplicationsAllphoneshavespeech;Mainstreamapp2005Accessibility&AsianDictationMobility&Automotiv