斯坦福大学机器学习课程讲义第二讲――单变量的线性回归模型表

1、 Linear r egressionwith o ne v ariable Modelrepresenta6on Machine L earning 100 200 300400 500 050010001500200025003000Housing P rices (Portland, O RPrice (in 1000s of d ollars Size (feet 2Supervised L earningGiven t he “right a nswer” f or each e xample i n t he d ata.Regression P roblemPredict r e

2、al-­valued o utputNota6on:m = N umber o f t raining e xamples x s = “input” v ariable / f eaturesy s = “output” v ariable / “target” v ariableSize i n f eet 2 (x Price ($ i n 1000's (y 2104 460 1416 232 1534 315 852 178 Training s et o f housing p rices (Portland, O R Training S etLearning

3、A lgorithmhSize o f houseEs6mated priceHow d o w e r epresent h ?Linear r egression w ith o ne v ariable. Univariate l inear r egression. Cost f unc6onMachine L earningLinear r egression with o ne v ariable How t o c hoose s ?Training S etHypothesis:s: P arameters Size i n f eet 2 (x Price ($ i n 10

4、00's (y 2104 460 1416 232 1534 315 852 178 0 1 2 3 01230 1 2 3 01230 1 2 3 0123 yxIdea: C hoose s o t hati s c lose t o f or o urtraining e xamples Cost f unc6on intui6on IMachine L earningLinear r egression with o ne v ariable Hypothesis: Parameters: Cost F unc6on: Goal: Simplied 1230 1 2 3yx(f

5、or xed , t his i s a f unc6on o f x (func6on o f t he p arameter 123-­0.5 0 0.5 1 1.5 2 2.5 1230 1 2 3yx(for xed , t his i s a f unc6on o f x (func6on o f t he p arameter 123-­0.5 0 0.5 1 1.5 2 2.5 12 3 -­0.5 0 0.5 1 1.5 2 2.5yx(for xed , t his i s a f unc6on o f x (func6on o f t he p

6、 arameter 0 1 2 312 3 Cost f unc6on intui6on I IMachine L earningLinear r egression with o ne v ariableHypothesis: Parameters: Cost F unc6on: Goal: (for xed , t his i s a f unc6on o f x (func6on o f t he p arameters 01002003004005000 1000 2000 3000Price ($ in1000s Size i n f eet 2 (x (for xed , t hi

7、s i s a f unc6on o f x (func6on o f t he p arameters (for xed , t his i s a f unc6on o f x (func6on o f t he p arameters (for xed , t his i s a f unc6on o f x (func6on o f t he p arameters (for xed , t his i s a f unc6on o f x (func6on o f t he p arameters Gradient descentMachine L earning Linear r

8、egression with o ne v ariableHave s ome f unc6on Want Outline: Start w ith s omeKeep c hanging t o r educeun6l w e h opefully e nd u p a t a m inimum1 0J(0,101 J(0,1Gradient d escent a lgorithm Correct: S imultaneous u pdate Incorrect: Gradient d escentintui6on Machine L earning Linear r egression w

9、ith o ne v ariableGradient d escent a lgorithm If i s t oo s mall, g radient d escent can b e s low.If i s t oo l arge, g radient d escent can o vershoot t he m inimum. I t m ay fail t o c onverge, o r e ven d iverge. at l ocal o p6ma Current v alue o f Gradient d escent c an c onverge t o a l ocal

10、minimum, e ven w ith t he l earning r ate xed.As w e a pproach a l ocalminimum, g radientdescent w ill a utoma6callytake s maller s teps. S o, n oneed t o d ecrease o ver6me. Gradient d escent f orlinear r egression Machine L earning Linear r egression with o ne v ariableGradient d escent a lgorithm

11、 Linear R egression M odel Gradient d escent a lgorithm updateandsimultaneously10J(0,101J(0,1 (for xed , t his i s a f unc6on o f x (func6on o f t he p arameters (for xed , t his i s a f unc6on o f x (func6on o f t he p arameters (for xed , t his i s a f unc6on o f x (func6on o f t he p arameters (f

12、or xed , t his i s a f unc6on o f x (func6on o f t he p arameters (for xed , t his i s a f unc6on o f x (func6on o f t he p arameters (for xed , t his i s a f unc6on o f x (func6on o f t he p arameters (for  xed                      ,  

13、;this  is  a  func6on  of  x   (func6on  of  the  parameters                           Andrew  Ng   (for  xed                   &#

14、160;  ,  this  is  a  func6on  of  x   (func6on  of  the  parameters                           Andrew  Ng   (for  xed                      ,  this  is  a  func6on  of  x   (func6on  of  the  parameters