maximum-likelihood estimation requires more stringent statistical assumptions than the least-squares estimators. The least-squares estimators require only second-moment assumptions (assumptions about the expected value, the variances, and the covariances among the random errors). The maximum-likelihood estimators require a full distributional assumption, in this case that the random errors follow a normal distribution with the same second moments as required for the least- squares estimates. For more information on maximum-likelihood estimation in regression models, see Graybill [1961, 1976], Myers [1990], Searle [1971], and Seber [1977].
2.13 CASE WHERE THE REGRESSOR x IS RANDOM
The linear regression model that we have presented in this chapter assumes that the values of the regressor variable x are known constants. This assumption makes the confidence coefficients and type I (or type II) errors refer to repeated sampling on y at the same x levels. There are many situations in which assuming that the x’s are fixed constants is inappropriate. For example, consider the soft drink delivery time data from Chapter 1 (Figure 1.1). Since the outlets visited by the delivery person are selected at random, it is unrealistic to believe that we can control the delivery volume x. It is more reasonable to assume that both y and x are random variables.
Fortunately, under certain circumstances, all of our earlier results on parameter estimation, testing, and prediction are valid. We now discuss these situations.
2.13.1 x and y Jointly Distributed
Suppose that x and y are jointly distributed random variables but the form of this joint distribution is unknown. It can be shown that all of our previous regression results hold if the following conditions are satisfied:
1 The conditional distribution of y given x is normal with conditional mean β0 + β1x and conditional variance σ2.
2 The x’s are independent random variables whose probability distribution does not involve β0, β1, and σ2.
While all of the regression procedures are unchanged when these conditions hold, the confidence coefficients and statistical errors have a different interpretation. When the regressor is a random variable, these quantities apply to repeated sampling of (xi, yi) values and not to repeated sampling of yi at fixed levels of xi.
2.13.2 x and y Jointly Normally Distributed: Correlation Model
Now suppose that y and x are jointly distributed according to the bivariate normal distribution. That is,
(2.60)
where μ1 and
is the correlation coefficient between y and x. The term σ12 is the covariance of y and x.
The conditional distribution of y for a given value of x is
(2.61)
where
(2.62a)
(2.62b)
and
(2.62c)
That is, the conditional distribution of y given x is normal with conditional mean
(2.63)
and conditional variance
The method of maximum likelihood may be used to estimate the parameters β0 and β1. It may be shown that the maximum-likelihood estimators of these parameters are
and
(2.64b)
The estimators of the intercept and slope in Eq. (2.64) are identical to those given by the method of least squares in the case where x was assumed to be a controllable variable. In general, the regression model with y and x jointly normally distributed may be analyzed by the methods presented previously for the model where x is a controllable variable. This follows because the random variable y given x is independently and normally distributed with mean β0 + β1x and constant variance
It is possible to draw inferences about the correlation coefficient ρ in this model. The estimator of ρ is the sample correlation coefficient
(2.65)
Note that
so that the slope