FAFU机器学习03-1-Introduction of Machine Learning课件

FoundationsofMachineLearningIntroductionofMachineLearningContents1ClassicalMachineLearning234Whatismachinelearning?EnsembleMethodsReinforcementLearning5DeepLearningAboutmachinelearningFromLearningtoMachineLearningLearning:AcquiringskillWithexperienceaccumulatedfromobservationsFromLearningtoMachineLearningLearning:AcquiringskillWithexperienceaccumulatedfromobservationsMachineLearning:AcquiringskillWithexperienceaccumulated/computedfromdataWhatisskill?AMoreConcreteDefinitionskill⇔

improve

some

performance

measure

(e.g.

prediction

accuracy)MachineLearning:improvingperformance

measure

withexperiencecomputedfromdataAMoreConcreteDefinitionAprogramcanbesaidtolearnfromexperienceEwithrespecttosomeclassoftasksTandperformancemeasureP,ifitsperformanceattasksinT,asmeasuredbyP,improvedwithexperienceE.ImproveonTaskwithrespecttoPerformancemetricbasedonExperienceT:PlayingcheckersP:Percentageofgameswonagainstanarbitraryopponent

E:PlayingpracticegamesagainstitselfWhyusemachinelearningML:analternativeroutetobuildcomplicatedsystemLearnfromthispictureandrecognize:3-year-oldcandoDefineflowersandhand-program:difficultML-basedflowersrecognitionsystemcanbeeasiertobuildthanhand-programmedsystemMLRouteML:analternativeroutetobuildcomplicatedsystemSomeScenariostouseMLwhenhumancannotprogramthesystemmanuallyNavigatingonMarswhenhumancannotdefinethesolutioneasilySpeechrecognitionWhenneedingrapiddecisionsthathumancannotdoHigh-frequencytradingWhenneedingtobeuser-orientedinamassivescaleConsumer-targetedmarketingKeyessenceofMLKeyessence:helpdecidewhethertouseMLMachineLearning:improvingperformance

measure

withexperiencecomputedfromdataExistssomeunderlyingpatterntobelearnedSoperformancemeasurecanbeimprovedButnoprogrammabledefinitionSoMLisneededSomehowthereisdataaboutthepatternSoMLhassomeinputstolearnfromThreecomponentsofmachinelearningDataWanttodetectspam?Getsamplesofspammessages.Wanttoforecaststocks?Findthepricehistory.Wanttofindoutuserpreferences?ParsetheiractivitiesonWebChat.Therearetwomainwaystogetthedata—manualandautomatic.Manuallycollecteddatacontainsfarfewererrorsbuttakesmoretimetocollect.Automaticapproachischeaperbutwithmoreerrors.SomesmartasseslikeGoogleusetheirowncustomerstolabeldataforthemforfree.RememberReCaptchawhichforcesyouto"Selectallstreetsigns"?That'sexactlywhatthey'redoing.Freelabour!Nice.ThreecomponentsofmachinelearningDataFeaturesAlsoknownasparametersorvariables.Thosecouldbecarmileage,user'sgender,stockprice,wordfrequencyinthetext.Inotherwords,thesearethefactorsforamachinetolookat.Whendatastoredintablesit'ssimple—featuresarecolumnnames.Butwhataretheyifyouhave100Gbofcatpics?Wecannotconsidereachpixelasafeature.That'swhyselectingtherightfeaturesusuallytakeswaylongerthanalltheotherMLparts.That'salsothemainsourceoferrors.ThreecomponentsofmachinelearningDataFeaturesAlgorithmsMostobviouspart.Anyproblemcanbesolveddifferently.Themethodyouchooseaffectstheprecision,performance,andsizeofthefinalmodel.Thereisoneimportantnuancethough:ifthedataiscrappy,eventhebestalgorithmwon'thelp.Sometimesit'sreferredas"garbagein–garbageout".Sodon'tpaytoomuchattentiontothepercentageofaccuracy,trytoacquiremoredatafirst.LearningvsIntelligenceArtificialintelligenceisthenameofawholeknowledgefield,similartobiologyorchemistry.MachineLearningisapartofartificialintelligence.Animportantpart,butnottheonlyone.NeuralNetworksareoneofmachinelearningtypes.Apopularone,butthereareothergoodguysintheclass.DeepLearningisamodernmethodofbuilding,training,andusingneuralnetworks.Basically,it'sanewarchitecture.Nowadaysinpractice,nooneseparatesdeeplearningfromthe"ordinarynetworks".Weevenusethesamelibrariesforthem.LearningvsIntelligence深度学习都是神经网络吗？机器学习下面应该是表示学习，包括所以使用机器学习挖掘表示本身的方法。

ThemapofmachinelearningworldThemapofmachinelearningworldLet'sstartwithabasicoverview.Nowadaystherearefourmaindirectionsinmachinelearning.Contents1ClassicalMachineLearning234Whatismachinelearning?EnsembleMethodsReinforcementLearning5DeepLearningClassicalMachineLearningClassicalmachinelearningisoftendividedintotwocategories–SupervisedandUnsupervisedLearning.SupervisedLearningTherearetwotypesofSupervisedLearning:classification–anobject'scategoryprediction,andregression–predictionofaspecificpointonanumericaxis.Classification"Splitsobjectsbasedatoneoftheattributesknownbeforehand.Separatesocksbybasedoncolor,documentsbasedonlanguage,musicbygenre".Todayusedfor:Spamfiltering,Languagedetection,Asearchofsimilardocuments,Sentimentanalysis,Recognitionofhandwrittencharactersandnumbers,Frauddetection,etc.Popularalgorithms:NaiveBayes,DecisionTree,LogisticRegression,K-NearestNeighbours,SupportVectorMachineClassificationInspamfilteringtheNaiveBayesalgorithmwaswidelyused.Themachinecountsthenumberof"viagra"mentionsinspamandnormalmail,thenitmultipliesbothprobabilitiesusingtheBayesequation,sumstheresultsandyay,wehaveMachineLearning.ClassificationHere'sanotherpracticalexampleofclassification.Let'ssayyouneedsomemoneyoncredit.Howwillthebankknowifyou'llpayitbackornot?Usingthisdata,wecanteachthemachinetofindthepatternsandgettheanswer.There'snoissuewithgettingananswer.Theissueisthatthebankcan'tblindlytrustthemachineanswer.Todealwithit,wehaveDecisionTrees.Allthedataautomaticallydividedtoyes/noquestions.Theycouldsoundabitweirdfromahumanperspective,e.g.,whetherthecreditorearnsmorethan$128.12?Though,themachinecomesupwithsuchquestionstosplitthedatabestateachstep.ClassificationSupportVectorMachines(SVM)isrightfullythemostpopularmethodofclassicalclassification.Itwasusedtoclassifyeverythinginexistence:plantsbyappearanceinphotos,documentsbycategories,etc.TheideabehindSVMissimple–it'stryingtodrawtwolinesbetweenyourdatapointswiththelargestmarginbetweenthem.Lookatthepicture:Regression"Drawalinethroughthesedots.Yep,that'sthemachinelearning“Todaythisisusedfor:StockpriceforecastsDemandandsalesvolumeanalysisMedicaldiagnosisAnynumber-timecorrelationsPopularalgorithmsareLinearandPolynomialregressions.RegressionRegressionisbasicallyclassificationwhereweforecastanumberinsteadofcategory.Examplesarecarpricebyitsmileage,trafficbytimeoftheday,demandvolumebygrowthofthecompanyetc.Regressionisperfectwhensomethingdependsontime.UnsupervisedlearningUnsupervisedwasinventedabitlater,inthe'90s.Itisusedlessoften,butsometimeswesimplyhavenochoice.Labeleddataisluxury.ButwhatifIwanttocreate,let'ssay,abusclassifier?ShouldImanuallytakephotosofmillionfuckingbusesonthestreetsandlabeleachofthem?There'salittlehopeforcapitalisminthiscase.Thankstosocialstratification,wehavemillionsofcheapworkersandserviceslikeMechanicalTurkwhoarereadytocompleteyourtaskfor$0.05.Andthat'showthingsusuallygetdonehere.Clustering"Dividesobjectsbasedonunknownfeatures.Machinechoosesthebestway“Nowadaysused:Formarketsegmentation(typesofcustomers,loyalty)TomergeclosepointsonamapForimagecompressionToanalyzeandlabelnewdataTodetectabnormalbehaviorPopularalgorithms:K-means_clustering,Mean-Shift,DBSCANDimensionalityReduction"Assemblesspecificfeaturesintomorehigh-levelones“Nowadaysisusedfor:Recommendersystems(★)BeautifulvisualizationsTopicmodelingandsimilardocumentsearchFakeimageanalysisRiskmanagementPopularalgorithms:PrincipalComponentAnalysis(PCA),SingularValueDecomposition(SVD),LatentDirichletallocation(LDA),LatentSemanticAnalysis(LSA,pLSA,GLSA),t-SNE(forvisualization)Associationrulelearning"Lookforpatternsintheorders'stream"Nowadaysisused:ToforecastsalesanddiscountsToanalyzegoodsboughttogetherToplacetheproductsontheshelvesToanalyzewebsurfingpatternsPopularalgorithms:Apriori,Euclat,FP-growthContents1ClassicalMachineLearning234Whatismachinelearning?EnsembleMethodsReinforcementLearning5DeepLearningEnsembleMethods"Bunchofstupidtreeslearningtocorrecterrorsofeachother"Nowadaysisusedfor:Everythingthatfitsclassicalalgorithmapproaches(butworksbetter)Searchsystems(★)ComputervisionObjectdetectionPopularalgorithms:RandomForest,GradientBoostingStackingOutputofseveralparallelmodelsispassedasinputtothelastonewhichmakesafinaldecision.RegressionRegressionisbasicallyclassificationwhereweforecastanumberinsteadofcategory.Examplesarecarpricebyitsmileage,trafficbytimeoftheday,demandvolumebygrowthofthecompanyetc.Regressionisperfectwhensomethingdependsontime.UnsupervisedlearningUnsupervisedwasinventedabitlater,inthe'90s.Itisusedlessoften,butsometimeswesimplyhavenochoice.Labeleddataisluxury.ButwhatifIwanttocreate,let'ssay,abusclassifier?ShouldImanuallytakephotosofmillionfuckingbusesonthestreetsandlabeleachofthem?There'salittlehopeforcapitalisminthiscase.Thankstosocialstratification,wehavemillionsofcheapworkersandserviceslikeMechanicalTurkwhoarereadytocompleteyourtaskfor$0.05.Andthat'showthingsusuallygetdonehere.Clustering"Dividesobjectsbasedonunknownfeatures.Machinechoosesthebestway“Nowadaysused:Formarketsegmentation(typesofcustomers,loyalty)TomergeclosepointsonamapForimagecompressionToanalyzeandlabelnewdataTodetectabnormalbehaviorPopularalgorithms:K-means_clustering,Mean-Shift,DBSCANDimensionalityReduction"Assemblesspecificfeaturesintomorehigh-levelones“Nowadaysisusedfor:Recommendersystems(★)BeautifulvisualizationsTopicmodelingandsimilardocumentsearchFakeimageanalysisRiskmanagementPopularalgorithms:PrincipalComponentAnalysis(PCA),SingularValueDecomposition(SVD),LatentDirichletallocation(LDA),LatentSemanticAnalysis(LSA,pLSA,GLSA),t-SNE(forvisualization)Associationrulelearning"Lookforpatternsintheorders'stream"Nowadaysisused:ToforecastsalesanddiscountsToanalyzegoodsboughttogetherToplacetheproductsontheshelvesToanalyzewebsurfingpatternsPopularalgorithms:Apriori,Euclat,FP-growthContents1ClassicalMachineLearning234Whatismachinelearning?EnsembleMethodsReinforcementLearning5DeepLearningEnsembleMethods"Bunchofstupidtreeslearningtocorrecterrorsofeachother"Nowadaysisusedfor:Everythingthatfitsclassicalalgorithmapproaches(butworksbetter)Searchsystems(★)ComputervisionObjectdetectionPopularalgorithms:RandomForest,GradientBoostingStackingOutputofseveralparallelmodelsispassedasinputtothelastonewhichmakesafinaldecision.BaggingUsethesamealgorithmbuttrainitondifferentsubsetsoforiginaldata.Intheend—justaverageanswers.BaggingUsethesamealgorithmbuttrainitondifferentsubsetsoforiginaldata.Intheend—justaverageanswers.ThemostfamousexampleofbaggingistheRandomForestalgorithm,whichissimplybaggingonthedecisiontrees(whichwereillustratedabove).Whenyouopenyourphone'scameraappandseeitdrawingboxesaroundpeople'sfaces—it'sprobablytheresultsofRandomForestwork.BoostingAlgorithmsaretrainedonebyonesequentially.Eachsubsequentonepayingmostofitsattentiontodatapointsthatweremispredictedbythepreviousone.Repeatuntilyouarehappy.Sameasinbagging,weusesubsetsofourdatabutthistimetheyarenotrandomlygenerated.Now,ineachsubsamplewetakeapartofthedatathepreviousalgorithmfailedtoprocess.Thus,wemakeanewalgorithmlearntofixtheerrorsofthepreviousone.Nowadaystherearethreepopulartoolsforboosting,youcanreadacomparativereportinCatBoostvs.LightGBMvs.XGBoostContents1ClassicalMachineLearning234Whatismachinelearning?EnsembleMethodsReinforcementLearning5DeepLearningReinforcementLearning"Throwarobotintoamazeandletitfindanexit"Nowadaysusedfor:Self-drivingcarsRobotvacuumsGamesAutomatingtradingEnterpriseresourcemanagementPopularalgorithms:Q-Learning,SARSA,DQN,A3C,GeneticalgorithmReinforcementLearningReinforcementlearningisusedincaseswhenyourproblemisnotrelatedtodataatall,butyouhaveanenvironmenttolivein.Likeavideogameworldoracityforself-drivingcar.Survivinginanenvironmentisacoreideaofreinforcementlearning.Throwpoorlittlerobotintoreallife,punishitforerrorsandrewarditforrightdeeds.Samewayweteachourkids,right?Contents1ClassicalMachineLearning234Whatismachinelearning?EnsembleMethodsReinforcementLearning5DeepLearningNeuralNetworksandDeepLeaning"Wehaveathousand-layernetwork,dozensofvideocards,butstillnoideawheretouseit.Let'sgeneratecatpics!"Usedtodayfor:ReplacementofallalgorithmsaboveObjectidentificationonphotosandvideosSpeechrecognitionandsynthesisImageprocessing,styletransferMachinetranslationPopulararchitectures:Perceptron,ConvolutionalNetwork(CNN),RecurrentNetworks(RNN),AutoencodersNeuralNetworksandDeepLeaningAnyneuralnetworkisbasicallyacollectionofneuronsandconnectionsbetweenthem.Neuron

isafunctionwithabunchofinputsandoneoutput.Itstaskistotakeallnumbersfromitsinput,performafunctiononthemandsendtheresulttotheoutput.NeuralNetworksandDeepLeaningneuronsConnections

arelikechannelsbetweenneurons.Theyconnectoutputsofoneneuronwiththeinputsofanothersotheycansenddigitstoeachother.Eachconnectionhason