北大暑期课程回归分析LinearRegressionAnalysis讲义pku4

1、Class 4: Inference in multiple regression.I. The Logic of Statistical InferenceThe logic of statistical inference is simple: we would like to make inferences about a population from what we observe from the sample that has been drawn randomly from the population. The samples' characteristics are

2、 called "point estimates." It is almost certain that the sample's characteristics are somewhat different from the population's characteristics. But because the sample was drawn randomly from the population, the sample's characteristics cannot be "very different" from

3、the population's characteristics. What do I mean by "very different"? To answer this question, we need a distance measure (or dispersion measure), called the standard deviation of the statistic. To summarize, statistical inferences consist of two steps:(1) Point estimates (sample stati

4、stics)(2) Standard deviations of the point estimates (dispersion of sample statistics in a hypothetical sampling distribution). II. Review: For a sample of fixed size is the dependent variable; contains independent variables. We can write the model in the following way: Under certain assumptions, th

5、e LS estimator As certain desirable properties:A1 => unbiasedness and consistencyA1 + A2 => BLUE, with A1 + A2 + A3 =>BUE (even for small samples)III. The Central Limit TheoremStatement: The mean of iid random variables (with mean of , and variance of ) approaches a normal distribution as t

6、he number of random variables increases. The property belongs to the statistic - sample mean in the sampling distribution of all sample means, even though the random variables themselves are not normally distributed. You can never check this normality because you can only have one sample statistic.

7、In regression analysis, we do not assume that e is normally distributed if we have a large sample, because all estimated parameters approach to normal distributions.Why: all LS estimates are linear function of e (proved last time). Recall a theorem: a linear transformation of a variable distributed

8、as normal is also distributed as normal.IV. Inferences about Regression CoefficientsA. Presentation of Regression ResultsCommon practice: give a star beside the parameter estimate for significance level of 0.05, two stars for 0.01, and three stars for 0.001. For example:Dependent Variable: EarningsI

9、ndependent Variable: Father's education 0.900*Mother's education0.501*Shoe size -2.16What is the problem with this practice? First, we want to have a quantitative evaluation of the significance level. We should not blindly rely on statistical tests. For example, Father's education0.900*

10、(0.450)Mother's education0.501* (0.001)Shoe size -2.16 (1.10)In this case, is father's education much more significant than shoe size? Not really. They are very similar. By contrast, mother's is far more significant than the other two. A second practice is to report the t or z values:Coe

11、ffi.t.Father's education0.900 2.0Mother's education0.501 500.Shoe size-2.16 -1.96This solution is much better. However, very often, our hypothesis is not about deviation from zero, but from other hypothetical values. For example, we are interested in the hypothesis whether a one-year increas

12、e in father's education will increase son's education by one year. The hypothesis here is 1 instead of 0. The preferred way of presentation is:Coeff.(S.E.)Father's education0.900(0.450)Mother's education0.501(0.001)Shoe size-2.16(1.10)B. Difference between Statistical Significance an

13、d the Size of an EffectStatistical significance always refers to a stated hypothesis. You will see a lot of misuses in the literature, sometimes by well-known sociologists. They would say that this variable is highly significant. That one is not significant. This is not correct. I am not responsible

14、 for their mistakes, but I want to warn you not to commit the same mistakes again. In our example, you could say Mother's education is highly significant from zero. But it is not significant from 0.5. Had your hypothesis been that the parameter for Mother's education is 0.5, the result would

15、 be consistent with the hypothesis. That is, statistical significance should always be made with reference to a hypothesis. FollowAnother common mistake is to equate statistical significance with the size of an effect. A variable can be statistically significant from zero. But the estimated coeffici

16、ent is small. The contribution of father's education to the dependent variable is larger than that of mother's education even though mother's education is more statistically significant from zero than father's education.Important: you should look at both coefficients and their standa

17、rd errors.C. Confidence Intervals for Single ParametersD. Hypothesis Testing for Single ParametersCompare ,if z is outside the range of -1.96 and 1.96, the hypothesis is rejected. Otherwise, we fail to reject the hypothesis. V. Inferences about Linear Combinations of Two ParametersExample 1: (equali

18、ty hypothesis), =>Example 2: (proportionality hypothesis), =>Example 3: (surplus hypothesis), =>In general form, we may have Hypothesis testing: Confidence interval: lies between (low limit, upper limit)Procedure: A. Point estimate: Compute B. Degree of Imprecisionand then take square root

19、of . We would need to obtain the variance-covariance matrix of the parameter vector in order to carry out the calculation. are on the diagonal, is off-diagonal.Let us look at the first half the example. Compute the confidence interval for the hypothesis that b1=b2. Step 1. b1 - b2 = - = 0.2586.Step

20、2: V(b1 - b2) = + -2* () = 0.1406.SD(b1 - b2) = 0.14061/2 = 0.3750.Step 3:Compute t2 = (0.2586 -0)/ 0.3750 = 0.6897, insignificant (unsurprising result). Note DF = 5-3 = 2. I use two parameters as an example.Examples of Hypothesis Testing: Example 1: (equality hypothesis), =>Example 2: (proportio

21、nality hypothesis), =>Example 3: (surplus hypothesis), =>In general form, We may have Hypothesis testing: Confidence interval: lies between (low limit, upper limit)Procedure: A. Find point estimate: Compute + B. Find degree of Imprecision and then take square root of .We would need to obtain the