模式识别与机器学习综述(共6页)

1、Machine Learning and Pattern Recognition1. Basic Introduction 1. Machine Learning Learning is a very important feature of intelligent behavior. But the machine learning . H.A.simon believes that learning is adaptive changes made to the system, making the system more effective when completed the same

2、 or similar tasks next time. R.s.Michaiski said that learning is to construct or modify things for experienced representation .People engaged in the development if Expert-Systerm think that learning is the acquisition of knowledge. These views have different emphases. The first emphasizes the effect

3、 of the external behavior of learning, the second emphasizes the internal processes of learning and the third is mainly from the practical point of departure of knowledge engineering. Machine learning has a very important position in the study of artificial intelligence. An intelligent system that d

4、oes not have the ability to learn is difficult to be called a real intelligent systems. But in the past generally intelligent systems lack the capacity to learn. For example, they can not be self-correcting when errors are encountered; do not improve their performance through experience; does not au

5、tomatically obtain and discovery the required knowledge . They are limited to deductive reasoning and lack of induction, so it can only prove the fact and theorem witch have existed, and do not discovery a new theorems, laws and rules. With the development of artificial intelligence, these limitatio

6、ns behave even more prominent. It is in this case, machine learning has gradually become one of the core of artificial intelligence research. Its application has been throughout all branches of artificial intelligence, such as expert systems, automated reasoning Natural language understanding, patte

7、rn recognition, computer vision, intelligent robotics and other fields. In particular, the typical expert system knowledge acquisition bottleneck problem, people have been trying to try to use machine learning methods to overcome them. The research of machine learning is based on the understanding o

8、f physiology and cognitive science ,build model or models of human understanding of the learning process, develop Various learning theories and learning methods; Learning algorithm research and general theoretical analysis, establish learning system with application-specific task-oriented. The goal

9、of this study affect each other and promote each other. 2. Pattern Recognition Pattern recognition is a fundamental human intelligence , in everyday life , people often conducting pattern recognition . With the rise of the computer artificial intelligence occurred in the 1940s and 1950s , Of course,

10、 one also hopes to use the computer to replace or extend the mental part ofhumanity . Pattern Recognition rapid and become a new discipline in the early 1960s . Pattern recognition means ( text and logical relationships between the values of ) the characterization of the various forms of objects or

11、phenomena information processing and analysis to describe phenomena or things , identification , classification and interpretation process , information science and an important part of artificial intelligence . Pattern recognition research focuses on two aspects, One study of objects ( including pe

12、ople ) is how we perceive objects belonging to the scope of scientific knowledge , the second is given the task of how to use a computer to implement the theories and methods of pattern recognition . The former is a research physiologist , psychologists , biologists and neurophysiologists , the latt

13、er through the mathematicians , informatics experts and computer scientists ,in recent decades efforts have been made in this research system. Using computer to identify or classify a process. Events or processes may be identified by text, sound, images, and other specific objects, these objects are

14、 different from the information in digitalform is called pattern information. The number of classes of pattern recognition classification is determined by the specific identification problems. Sometimes, you can not know the actual number of classes at the start, you need to identify the system afte

15、r repeated observations to determine the object to be identified.Pattern recognitions have a relationship with statistical pattern recognition, psychology, linguistics, computer science, biology, cybernetics, etc. It also has relations with artificial intelligence research, image processing. Such as

16、 adaptive or self-organizing pattern recognition system contains artificial intelligence learning mechanisms; scene understanding of artificial intelligence research, natural language understanding also includes pattern recognition problems. Another example is pattern recognition preprocessing and f

17、eature extraction, image processing applicationstechnology sectors; image processing pattern recognition, image analysis techniques are applied.2. The relationship Pattern recognition is derived from engineering, it is a kind of problem (problem); Machine learning derived from mathematics, is a kind

18、 of method (methodology). For a specific pattern recognition problems, you can use handcrafted rule - -based approach to solving, but more complicated PR problems often adopt the method of machine learning. 1. The classification of machine learning According to the study of different pattern, the ma

19、chine learning in general can be divided into four categories: Supervised Learning Training set with all the input of the target value is called supervised learning. This study aims to find the relationship between the input variables and target. According to the target value, supervised learning ca

20、n be divided into two types of problems: if the target value is a discrete variable, called classification; If it is a continuous variable, called regression.Unsupervised LearningAll input variables without a target value are called unsupervised learning. This study aims to find the internal links b

21、etween the input variables. According to the specific internal contact type, unsupervised learning also can be divided into a variety of problems, such as clustering, density estimation, the visualization.Semi - supervised LearningInput variables some have the target value and some have no is called

22、 semi -supervised learning. In fact, the book did not refer directly to semi - supervised learning.Reinforcement learningThis type of learning is on the basis of supervised learning, it allows the machine to choose training data. At the same time, training in access to information at the same time a

23、lso can bring the cost or loss, triggering a tradeoff.2. The basic process of machine learningThe most basic of the machine learning process is: 1. Determine the type of model 2. To determine the model complexity (i.e., the number of free parameters) 3. Make sure all the parameters of each model 4.

24、Finally, the comparison between model one of the best choice, also known asmodel selectionTrainning model generally refers to one or a set of analytical expressions, through which can use the analytic method to express knowledge or direct optimization decision. According to the generalization abilit

25、y is different, trainning model can only face the step 3, also can at the same time to cover all 4 steps. Trainning cannot express for the part of the model, either through conputational algorithm to enumeration or more, or to specific application model assumptions.Over fitting and Model SelectionTh

26、e specific meaning of over fitting description is not clear in the book, generally refers to such a phenomenon: sometimes the model error on the training set is very small, but a large error on the training set. If algorithm over fitting phenomena, traditionally to choosing a subset of the training

27、data (called the validation set), and based on the validation set to do the model selection. The difference between the validation data and test data is that the former can be in another part of the run of the training data, different run for the same set of data to take a different training - to di

28、vide the validation; While the latter is not used in the process of training, is usually used in the experiment.The validation of the faults are two:1. The validation takes up extra training data, the data in the application are particularly affected. Cross - validation technology is used to allevia

29、te the defects. It in turn to select a small number of data from the training set to do the validation set, and finally to combine multiple results. But cross - validation to introduce several rounds of the validation, increased the amount of calculation. 2. As a result of the existence of the valid

30、ation, training model can not according to the training data parsing model comparison. When need to enumerate compare model complexity parameters change, the validation of miscellaneous complexity index rose.3. Pattern recognition methodDecision theory method Also known as statistical method, and it

31、 is a way that the develop early. Identify the object first and make digital transformation to fit the computer . A model is often represented with a large amount of information. Many pattern recognition systems in the digital link after preprocessing, also used to remove the interference with infor

32、mation and reduce some deformation and distortion. After, that is followed for feature extraction from digital or after preprocessing of input patterns extracted a set of features. So-called feature is the measure of a selected it for normal deformation and distortion remains the same or almost the

33、same, and only contain redundant information of as little as possible. Feature extraction process map the input mode from the object space to feature space. At this time, the model can be used a point in the feature space or a feature vector representation. This mapping not only compress the amount

34、of information, and easy to classification. In decision theory method, the feature extraction is very important, but there is no general theoretical guidance, only by analyzing specific recognition object the feature selection. Feature extraction can be carried out after the classification, that is,

35、 from the feature space and then mapped to the decision space. For introducing the identification function, calculated from characteristic vector corresponding to various other identification function value, the categorized by identifying function values.Syntactic methods Also called structure metho

36、d or linguistics. Its basic idea is to put a model described as the combination of simpler subpattern, sub model and can be described as the combination of simpler subpattern, end up with a tree structure to describe, at the bottom of the simplest primitive subpattern called mode. In syntactic appro

37、ach selected primitive problem is equivalent to select characteristic problems in decision theory method. Usually requires the selected primitives reflects on model provides a compact description of its structural relationship, and easy to use the syntax method to extract them. Obviously, the primit

38、ive itself should not contain important structural information. Model to a set of motifs and to describe their combination relationship, referred to as the model to describe the statement, the same as in the language sentence combination of words and phrases, words with the same character combinatio

39、ns. Primitive rules combined into patterns, by so-called syntax to specify. Once primitives have been identified, the identification process can be through the syntactic analysis, the analysis of the given pattern statement is in line with the specified syntax, meet is of a kind of grammar points in

40、 the class. Choice depends on the nature of the problem of pattern recognition method. If the identified object is very complex, and contains rich structural information, general syntax methods; By the object recognition is not very complex or do not contain obvious structural information, generally

41、 USES decision theory method. These two methods cannot be separated, in syntactic approach, primitive itself is made of decision theory method to extract. In application, combine the two methods were applied to different levels, often can get good effect.Statistical pattern recognitionStatistical pattern recognition (statistic pattern recognition), the basic principle is: the similarities of the samples in the pattern space close to each other, and form a group, namely birds of a feather