问题为什么现在的计算机处理智能信息效率很低ppt课件

1、Prof. Liqing ZhangDept. Computer Science & Engineering, Shanghai Jiaotong UniversityStatistical Learning & InferenceBooks and ReferencesTrevor Hastie Robert Tibshirani Jerome Friedman , The Elements of statistical Learning: Data Mining, Inference, and Prediction, 2001, Springer-VerlagVladimir N. Vap

2、nik, The Nature of Statistical Learning Theory, 2nd ed., Springer, 2000S. Mendelson, A few notes on Statistical Learning Theory in Advanced Lectures in Machine Learning: Machine Learning Summer School 2002, S. Mendelson and A. J. Smola (eds), Lecture Notes in Computer Science, 2600, Springer, 2003M.

3、 Vidyasagar, Learning and generalization: with applications to neural networks, 2nd ed., Springer, 2003Overview of the CourseIntroductionOverview of Supervised LearningLinear Method for Regression and ClassificationBasis Expansions and RegularizationKernel MethodsModel Selections and InferenceSuppor

4、t Vector MachineBayesian InferenceUnsupervised LearningWhy Statistical Learning?我门被信息淹没，但却缺乏知识。- R. Roger恬静的统计学家改动了我们的世界；不是经过发现新的现实或者开发新技术，而是经过改动我们的推理、实验和观念的构成方式。- I. Hacking问题：为什么如今的计算机处置智能信息效率很低？图像、视频、音频认知言语ML: SARS Risk PredictionSARS RiskAgeGenderBlood PressureChest X-RayPre-Hospital AttributesA

5、lbuminBlood pO2White CountRBC CountIn-Hospital AttributesML: Auto Vehicle NavigationSteering DirectionProtein FoldingThe Scale of Biomedical Data计算科学与脑科学计算机信息处置基于逻辑的计算和数据分别数据处置与存储简单智能信息处置复杂、慢认知才干弱信息处置方式：逻辑概念统计信息大脑信息处置基于统计信息的计算计算和数据集成一体数据处置与存储未知智能信息处置简单、快速认知才干强信息处置方式：统计信息概念逻辑Function Estimation Model

6、The Function Estimation Model of learning examples:Generator (G) generates observations x (typically in Rn), independently drawn from some fixed distribution F(x)Supervisor (S) labels each input x with an output value y according to some fixed distribution F(y|x)Learning Machine (LM) “learns from an

7、 i.i.d. l-sample of (x,y)-pairs output from G and S, by choosing a function that best approximates S from a parameterised function class f(x,), where is in the parameter setFunction Estimation ModelKey concepts: F(x,y), an i.i.d. k-sample on F, functions f(x,) and the equivalent representation of ea

8、ch f using its index xGSLMyyThe loss functional (L, Q)the error of a given function on a given exampleThe risk functional (R)the expected loss of a given function on an example drawn from F(x,y) the (usual concept of) generalisation error of a given function The Problem of Risk Minimization The Prob

9、lem of Risk MinimizationThree Main Learning ProblemsPattern Recognition:Regression Estimation:Density Estimation:General FormulationThe Goal of LearningGiven an i.i.d. k-sample z1, zk drawn from a fixed distribution F(z)For a function class loss functionals Q (z ,), with in We wish to minimise the r

10、isk, finding a function *General FormulationThe Empirical Risk Minimization (ERM) Inductive PrincipleDefine the empirical risk (sample/training error):Define the empirical risk minimiser:ERM approximates Q (z ,*) with Q (z ,k) the Remp minimiserthat is ERM approximates * with kLeast-squares and Maxi

11、mum-likelihood are realisations of ERM4 Issues of Learning TheoryTheory of consistency of learning processesWhat are (necessary and sufficient) conditions for consistency (convergence of Remp to R) of a learning process based on the ERM Principle?Non-asymptotic theory of the rate of convergence of l

12、earning processesHow fast is the rate of convergence of a learning process?Generalization ability of learning processesHow can one control the rate of convergence (the generalization ability) of a learning process?Constructing learning algorithms (i.e. the SVM)How can one construct algorithms that c

13、an control the generalization ability?Change in Scientific MethodologyTRADITIONALFormulate hypothesisDesign experimentCollect dataAnalyze resultsReview hypothesisRepeat/PublishNEWDesign large experimentsCollect large dataPut data in large databaseFormulate hypothesisEvaluate hypothesis on databaseRu

14、n limited experiments Review hypothesisRepeat/PublishLearning & AdaptationIn the broadest sense, any method that incorporates information from training samples in the design of a classifier employs learning.Due to complexity of classification problems, we cannot guess the best classification decisio

15、n ahead of time, we need to learn it.Creating classifiers then involves positing some general form of model, or form of the classifier, and using examples to learn the complete classifier.Supervised learningIn supervised learning, a teacher provides a category label for each pattern in a training se

16、t. These are then used to train a classifier which can thereafter solve similar classification problems by itself.Unsupervised learningIn unsupervised learning, or clustering, there is no explicit teacher or training data. The system forms natural clusters of input patterns and classifiers them base

17、d on clusters they belong to .Reinforcement learningIn reinforcement learning, a teacher only says to classifier whether it is right when suggesting a category for a pattern. The teacher does not tell what the correct category is.ClassificationThe task of the classifier component is to use the featu

18、re vector provided by the feature extractor to assign the object to a category.Classification is the main topic of this course.The abstraction provided by the feature vector representation of the input data enables the development of a largely domain-independent theory of classification.Essentially the classifier divides the feature space into regions corresponding to different categories.ClassificationThe degree of difficulty of the classification pr