斯坦福大学机器学习第十四讲Lecture14

1、 Reduc1onMo1va1on I:Data C ompressionMachine L earning(inche s (cm Reduce d ataf rom2D t o 1DReduce d ata f rom2D t o 1D (inches (cmData C ompressionReduce d ata f rom 3D t o 2D Reduc1onMo1va1on I I:Data V isualiza1onMachine L earningData V isualiza2onCountryGDP(trillions o fUS$Per c apitaGDP(thousa

2、ndsof i ntl. $HumanDevelop-­ment I ndexLifeexpectancyPovertyIndex(Gini a spercentageMeanhouseholdincome(thousandsof U S$ Canada 1.577 39.17 0.908 80.7 32.6 67.293 China 5.878 7.54 0.687 73 46.9 10.22 India 1.632 3.41 0.547 64.7 36.8 0.735 Russia 1.48 19.84 0.755 65.5 39.9 0.72 Singapore 0.223 5

3、6.69 0.866 80 42.5 67.1 USA 14.527 46.86 0.91 78.3 40.8 84.3 Data V isualiza2onCountry Data V isualiza2on Reduc1onPrincipal C omponentAnalysis p roblemformula1onMachine L earning Principal C omponent A nalysis (PCA p roblem f ormula2onPrincipal C omponent A nalysis (PCA p roblem f ormula2onReduce f

4、rom 2-­dimension t o 1-­dimension: F ind a d irec1on (a v ector onto w hich t o p roject t he d ata s o a s t o m inimize t he p rojec1on e rror. Reduce f rom n-­dimension t o k-­dimension: F ind v ectorsonto w hich t o p roject t he d ata, s o a s t o m inimize t he p rojec1on e

5、 rror. Reduc1onPrincipal C omponentAnalysis a lgorithmMachine L earningData p reprocessing Training s et:Preprocessing (feature s caling/mean n ormaliza1on:Replace e ach w ith .If d ierent f eatures o n d ierent s cales (e.g., s ize o f h ouse, n umber o f b edrooms, s cale f eatures t o h ave c omp

6、arable range o f v alues.Reduce d ata f rom 2D t o 1D Reduce d ata f rom 3D t o 2DReduce d ata f rom -­dimensions t o -­dimensions Compute “covariance m atrix”:Compute “eigenvectors” o f m atrix :U,S,V = svd(Sigma;From , w e g et: U,S,V = svd(Sigma Principal C omponent A nalysis (PCA a lgo

7、rithm s ummary Ader m ean n ormaliza1on (ensure e very f eature h as zero m ean a nd o p1onally f eature s caling: Sigma =U,S,V = svd(Sigma; Ureduce = U(:,1:k; z = Ureduce*x; Reduc1onReconstruc1on f romcompressedrepresenta1onMachine L earningReconstruc2on f rom c ompressed r epresenta2on Reduc1onCho

8、osing t he n umber o fprincipal c omponentsMachine L earningAverage s quared p rojec1on e rror:Total v aria1on i n t he d ata:Typically, c hoose t o b e s mallest v alue s o t hat“99% o f v ariance i s r etained” (1% Algorithm: Try P CA w ith Compute Check i fU,S,V = svd(SigmaU,S,V = svd(Sigma Pick

9、s mallest v alue o f f or w hich (99% o f v ariance r etained Reduc1onAdvice f orapplying P CAMachine L earningSupervised l earning s peedup Extract i nputs: U nlabeled d ataset: New t raining s et: Note: M apping s hould b e d ened b y r unning P CA only o n t he t raining s et. T his m apping c an

10、 b e a pplied a s w ell t o the e xamples a nd i n t he c ross v alida1on a nd t estApplica2on o f P CA-­Compression-­Reduce m emory/disk n eeded t o s tore d ata -­Speed u p l earning a lgorithm-­Visualiza1on Bad u se o f P CA: T o p revent o verEngUse i nstead o f t o r educe t

11、 he n umber o f features t o Thus, f ewer f eatures, l ess l ikely t o o vert. This m ight w ork O K, b ut i snt a g ood w ay t o a ddress overlng. U se r egulariza1on i nstead. PCA i s s ome2mes u sed w here i t s houldnt b eDesign o f M L s ystem:-­Get t raining s et-­Run P CA t o r educe i n d imension t o g et -­Train l ogis1c r egression o n -­Test o n t est s et: M ap t o . R un o n How a bout d oing t he w hole t hing w ithout u sing P CA?Before i mpl