数据分析入门-以鸢尾花分类为例

1、以鸢尾花分类为例本以鸢尾花数据为例，总结数据分析般过程，python数据分析库的部分法，并完成鸢尾花分类模型构建数据获取以及导import requestsimport numpy as np#request.get(URL)可以读取站信息，返回respose对象,将其存变量r中。r =requests.get(/ml/machine-learning-databases/iris/iris.data )#这返回r，可以看出rResponse对象r#os.getcwd()可以返回当前编辑录path =os.getcwd()pathC:Users44587Python#pythonwith op

2、en法以write模式在当前path下创建iris.data并将存储于r中的数据写#response.text表获取response中的本信息with open(path+iris.data,w) as f:f.write(r.text)#数据写后使pandasread_csv法读取件，names参数可赋值个list以更改列名df =pd.read_csv(path +iris.data,names =sepal length,sepal width,petal length,petal width,Class)探索性数据分析这部分的的是对数据有个总体的认知，并发现些明显的信息，并且对数据进清

3、洗()RangeIndex: 150 entries, 0 to 149Data columns (total 5 columns):sepal length 150 non-null float64sepal width 150 non-null float64petal length 150 non-null float64petal width 150 non-null float64Class150 non-null objectdtypes: float64(4), object(1)memory usage: 5.9+ KB可以看出，数据是分完整整齐的，没有缺失值。#查看数据表的统

4、计信息df.describe()05.8433330.8280664.3000005.1000005.8000006.4000007.90000003.0540000.4335942.0000002.8000003.0000003.3000004.400000.03.7586671.7644201.0000001.6000004.3500005.1000006.90000001.1986670.7631610.1000000.3000001.3000001.8000002.500000min25%50%75%maxdf.head()Class01234.65.03.53.00.2Iris-se

5、tosaIris-setosaIris-setosaIris-setosaIris-setosadf.iloc:3,:4sepal lengthsepal widthpetal lengthpetal width03.53.00.2df.loc:3,sepal lengthName: sepal length, dtype: float64#查看类别,.unique()可以返回列中所有不同数据，类似SQL中的uniquedf.Class.unique()array(Iris-setosa, Iris-versicolor, Iris-virginica, dtype=object)#查看详细分

6、组信息,可以看到类别前50为Setosa类，中间50为Versicolor，后50为Virginicadf.groupby(Class).groupsIris-virginica: Int64Index(100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112,113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125,126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138,139, 1

7、40, 141, 142, 143, 144, 145, 146, 147, 148, 149,dtype=int64)df.count()dtype: int64#取出特征组成列表labels =list(df.columns:4)labelssepal length, sepal width, petal length, petal width#取出类别为Virginica的数据并重置indexdf1 =dfdf.Class =Iris-virginicadf1.reset_index()index1001011021031041051061071081091101111121131141

8、15116Class01Iris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginica1.927.13.046.53.05.82.257.63.01.81.82.510111213141

9、52.02.55.02.03.05.51.8Class171819202122232425262728293031323334353637383940414243444546474849Iris-virginica1181191201211221231241251261271281291301311321331341351361371381391401411421431441451461471481497.76.06.02.83.03.02.53.03.43.06.95.02.02.02.02.31.8Iris-virginicaIris-virginicaIris-virginicaIris

10、-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virgi

11、nicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginica#df.corr()dfdf.iloc:,:5df.corr()sepal length1.000000-0.1093690.8717540.817954sepal width-0.1093691.000000-0.420516-0.356544petal length0.871754-0.4205161.0000000.962757petal width0.817954-0.3565440.

12、9627571.000000sepal lengthsepal widthpetal lengthpetal width#Seaborn库是基于matplotlib的阶绘图库，可以简洁优美的绘制图形sns.pairplot(df,hue =Class)从图中可以观察到，petallength以及petal width两特征可以较好的对鸢尾花进分类，后的randomforest也证实了这点，两特征分类贡献近乎各占四成。观察图1,0可以发现，setosa的sepallength 与sepalwidth有定的线性关系，后对此进线性回归分析。#绘制提琴图，展各个特征分类数据分布fig,ax =plt.

13、subplots(2,2,figsize (8,8)sns.violinplot(x =dfClass,ydfsepal length,ax ax0,0)sns.violinplot(x =dfClass,ydfsepal width,ax ax0,1)sns.violinplot(x =dfClass,ydfpetal length,ax ax1,0)sns.violinplot(x =dfClass,ydfpetal width,ax ax1,1)plt.tight_layout#绘制直图查看sepal width的分布plt.style.use(ggplot)fig,ax =plt.su

14、bplots(1,1,figsize(4,4)ax.hist(dfsepal width,color =black)ax.set_xlabel(sepal width)plt.tight_layout()Setosa 的Sepal Width 与Sepal Length线性相关性分析#绘制两特征散点图fig,axes =plt.subplots(figsize =(7,7)axes.set_ylabel(Sepal length)axes.set_title(Setosa Sepal Width vs. Sepal Length,y =1.02)Text(0.5, 1.02, Setosa S

15、epal Width vs. Sepal Length)result =sm.OLS(y,).fit()print(result.summary()D:Anacondalibsite-packagesnumpycorefromnumeric.py:2389: FutureWarning: Method .ptp is deprecated and will be removed in a future version. Usenumpy.ptp instead.return ptp(axis=axis, out=out, *kwargs)OLS Regression Results=Metho

16、d:Date:Time:1Covariance Type:nonrobusttOmnibus:Prob(Omnibus):Skew:Kurtosis:2.599 Cond. No.1 Standard Errors assume that the covariance matrix of the errors is correctly specified.可以看出，回归程：sepal length = 0.6909*sepal width +2.6447回归系数t检验p-val较，结果显著，由于元线性回归的局限性，R以及Adjusted R并不太，模型拟合程度较低，但是模型F检验显著。#在散点

17、图中绘制回归线plt.legend(loc =Best)D:Anacondalibsite-packagesipykernel_launcher.py:6: MatplotlibDeprecationWarning: Unrecognized location Best. Falling back on best; valid locationsThis will raise an exception in 3.3.使随机森林构建分类模型from sklearn.ensemble import RandomForestClassifierfrom sklearn.model_selection import train_test_split#构建并训练分类器X=df.iloc:,:4y =df.iloc:,4X_train,X_test,y_train,y_test =train_test_spli