机器学习之线性回归二-随堂课件算法

法 法EDUCATIONTOCREATEABRIGHT 者在本课程使用，不得在课程范围外向任何第散播。任何其他人或者机构不得盗版、、仿造其中的创意和 课 咨 回归算

EDUCATIONTOCREATEABRIGHT

EDUCATIONTOCREATEABRIGHT认真听，善摘录，勤思多温故，乐实践，再发 不请 ，不拖延作“四不原

EDUCATIONTOCREATEABRIGHT EDUCATIONTOCREATEABRIGHT

EDUCATIONTOCREATEABRIGHT

EDUCATIONTOCREATEABRIGHT 关系，在算法的学习过程中，试图寻找一个函数h:RdR使得参

EDUCATIONTOCREATEABRIGHT值年工作年8最近三个月消费当前额可提高额

EDUCATIONTOCREATEABRIGHT房屋面积租赁价格123

(x)0

1x1

2x2

EDUCATIONTOCREATEABRIGHT111232011x1nxn0x01x1nn

EDUCATIONTOCREATEABRIGHT xT i0最终要求是计算出的值，并选择最优的yiTxi

EDUCATIONTOCREATEABRIGHT 原因：中心极限定

i i

EDUCATIONTOCREATEABRIGHTyiTxi pi

yi

pyi|x

;

m im |

exp

2

mm

EDUCATIONTOCREATEABRIGHTlogL

loss(y,)J1

i

i1

xilog

exp11

2 m

1

θ的求解过程J

EDUCATIONTOCREATEABRIGHTmJ1m

hxiyi

1XYTXY

21m221m2minJ2i1 1J1

XY

XY

11

Y

1TXTXTXTYYTXYTY 22XTXXXTX1XT

EDUCATIONTOCREATEABRIGHTXTX1XTXTX是可逆的；为了防止不可逆或者过拟合的问题XTXI1XT

EDUCATIONTOCREATEABRIGHT数据来IndividualhouseholdelectricpowerconsumptionData建议：使用python的sklearn库的linear_model中LinearRegression来获取算

EDUCATIONTOCREATEABRIGHT

EDUCATIONTOCREATEABRIGHT

J1,YJJ0,YJ1,YfXt0,YfXt

EDUCATIONTOCREATEABRIGHTJh

iJ h

yiJ logh

xi

EDUCATIONTOCREATEABRIGHTm目标函数：J1m

hxiyi

EDUCATIONTOCREATEABRIGHT22i jJ1h j

jjj

L2-

EDUCATIONTOCREATEABRIGHT11J

n2n1

jj

J

hx 2

j

使用L2正则的线性1归模型就称为Ridge

EDUCATIONTOCREATEABRIGHTJ 2

EDUCATIONTOCREATEABRIGHT使用L1正则的线性回归模型就称为LASSO回归(LeastAbsoluteShrinkageSelectionn nm2

EDUCATIONTOCREATEABRIGHT

EDUCATIONTOCREATEABRIGHTElasitc

EDUCATIONTOCREATEABRIGHT1

2

J

pj

p[0,12

EDUCATIONTOCREATEABRIGHT

EDUCATIONTOCREATEABRIGHT

2

EDUCATIONTOCREATEABRIGHTMSE

yimm

yi RMSE

MSE

yim

yi

yi R2

1

y yi

i

EDUCATIONTOCREATEABRIGHTTSS：总平方和TSS(TotalSumofSquares)，表示样本之间的差异情况，是伪方RSS：残差平方和RSS（ResidualSumofSquares），表示预测值和样本值之

EDUCATIONTOCREATEABRIGHT 训练数 测试数目标函数θ求解J1h

EDUCATIONTOCREATEABRIGHTm22i

i i-Jα：学习率、步

J1

EDUCATIONTOCREATEABRIGHT 1

i

hxi

yijj

hx 2

y

Jhxy

EDUCATIONTOCREATEABRIGHT

m

xh my my

jj

y

x h x

j Jhxy

EDUCATIONTOCREATEABRIGHTfori=1to yi

hxi

i }

EDUCATIONTOCREATEABRIGHT

EDUCATIONTOCREATEABRIGHT这正是小批量梯度下降法（Mini-batchGradientDescent，简称MBGD）的初jfori=1toj }

ykk

hk

k

EDUCATIONTOCREATEABRIGHT 对同样的模拟数据进行两种算法的比较(pythonsklearnLinearRegression和

EDUCATIONTOCREATEABRIGHTy0

y

xx2

y

x

x4 34

EDUCATIONTOCREATEABRIGHT i

i

minJm损失函数/目标函数：Jm

i

EDUCATIONTOCREATEABRIGHT

EDUCATIONTOCREATEABRIGHTJi局 回归损失函数

hxi

yixJ h xi

i

i

yi

EDUCATIONTOCREATEABRIGHT

xi

w exp 2k

EDUCATIONTOCREATEABRIGHT数

EDUCATIONTOCREATEABRIGHT

EDUCATIONTOCREATEABRIGHT

EDUCATIONTOCREATEABRIGHT数

EDUCATIONTOCREATEABRIGHT

EDUCATIONTOCREATEABRIGHT

phx

x

1eT

y

y 1ez

01gz

e1ez

1 1 1 gz1gz

1ez假设：Py1|x;hxPy0|x;1hx

θEDUCATIONTOCREATEθPy|x;

x

hx1y似然函数

mLpy|X;pyi|xi;m

mh

i

yi

h

xi 1yii im

mEDUCATIONTOCREATEABRIGHTm

m yi

1yi

hxi

y

h

1hxi i1mm

yi

1yi

gTxii1

1

gm 1yi g

xi

xi i1 j j T

yimm

nn

EDUCATIONTOCREATEABRIGHT

mi1m

j yij

hxi

iiii

EDUCATIONTOCREATEABRIGHTmLm

|xi;

mm

pyi1

pi

1eTxii

1emimlnL

lnp

yi

1 1

i

ln

1

1e

1efi i

i

1

i

ln1e ,yi

,

EDUCATIONTOCREATEABRIGHT 数据来

EDUCATIONTOCREATEABRIGHT

EDUCATIONTOCREATEABRIGHT向量θk，组成的二维矩阵为 ee kKpyk|x; ,k1,2,kKlll

eT

EDUCATIONTOCREATEABRIGHTyiy

Txi

L

pyk|xi;

yi

e

xii k

i k i i

elK

Txi对数似然lnLy lne yKe i k1 lyKe

k

kTT

xlnl

Tx

yx

eT

kx k

eT

kxk

pyk|x;kk

lklkKeKl

T

K lK

Tx ll

EDUCATIONTOCREATEABRIGHT数据来

EDUCATI