Note that equation (4) is a statement that gives us the probability that the scalar random variable, bi, lies in some interval on the real line. So, if we are interested in two elements of b at once, consider a probability statement of the following type:
Pr.[(bi , bj)’ is in R] = (1 – α) . (6)
This is just a statement that gives us the probability that a random vector lies in some two-dimensional region, say R. Just as equation (4) can be manipulated to give us the confidence interval in equation (5), the statement in equation (6) can be manipulated to give us a confidence region that has the corresponding interpretation. That is, it will be region whose boundaries are random, and the interpretation will be that if we repeatedly construct such a region many, many times, then such regions will cover the true value of the vector, (βi , βj)’ , 100(1 – α)% of the time.
Now, the question is, “what does such a region look like?”. Look back at the sampling distribution for the full b vector in equation (2), and comments (i) and (ii) that follow it. In addition, recall that if the full b vector follows this multivariate normal distribution, then all of the marginal distributions associated with its elements will also be normal. In particular, the sub-vector associated with the pair of elements that we are interested in will have the following bivariate normal sampling distribution:
(bi , bj)’ ~ N[(βi , βj)’ , V*] , (7)
where the elements in the (estimated) covariance matrix, V*, come from the appropriate (2 × 2) sub-matrix of σ2(X’X)-1, in (2). The leading diagonal; elements of this sub-matrix will be var.(bi) and var.(bj), and the off-diagonal; elements will each be cov.(bi , bj ). Given points (i) and (ii) above, in general var.(bi) ≠ var.(bj) and cov.(bi , bj ) ≠ 0.
Just as the univariate (scalar) normal random variable bi becomes a univariate Student-t random variable when we standardize it, and replace the unobserved s.d.(bi) with the observable s.e.(bi), the bivariate normal random variable becomes a bivariate Student-t random variable when we essentially standardize each element and replace the unobserved cov.(bi , bj), s.d.(bi) and s.d.(bj) with the observable côv(bi , bj), s.e.(bi) and s.e.(bj). (Of course, these quantities are obtained from the appropriate sub-matrix of .) Strictly speaking, what we do is to work with the (bi , bj)’ vector in its entirety, and construct a new vector,
(bi* , bj*)’ = V*-1/2(bi , bj)’, (8)
where V*is the estimated covariance matrix for (bi , bj)’, and V*-1/2 satisfies the relationship V*-1/2V*-1/2 = V*-1.
As you would anticipate from the earlier blog post, if the two elements of the random vector have different variances and/or there is a non-zero covariance between them, the plots change. Specifically, the circular contours will become elliptical. The following two graphs relate to the case where the correlation is changed from 0.0 to -0.7, and you know already (from the earlier post) what would happen if the correlation were (say) 0.4:
When we relate this to the construction of a confidence region for (β1 , β2)’ in our regression model, we can see that once we choose the confidence level (say, 95%) we are concentrating on just one of the contours, and that the region will be “centered” at the point determined by (b1 , b2)’ . This is what we see when we estimate a regression model with EViews, and then select View, Coefficient Diagnostics, Confidence Ellipse. We then have the opportunity to specify the confidence level, which coefficients are to be considered in a pair-wise manner, and how we want to display the individual confidence intervals for each individual coefficient:
In this last plot, we see that the confidence ellipse for a 95% confidence level is “centered” at the point (1.42, -0.007), which corresponds to the OLS estimates for the intercept and slope coefficients in the regression output above. If we repeated this exercise many, many times then 95% of the regions created would cover the true values of the intercept and slope coefficients in this model. Of course, we will never know if this particular regions does. Notice that the dotted straight vertical lines in the confidence ellipses plot give us the limits for a 95% confidence interval for just the intercept coefficient by itself. In this case this interval has a lower limit of just under 1.2, and an upper limit of just under 1.7. Using the OLS regression output above, you should be able to quickly determine the exact values for the limits of this interval. In the same manner, the two horizontal straight dotted lines give us the lower and upper limits for a 95% confidence interval for just the slope coefficient by itself. Again, you can use the OLS regression output to convince yourself that these limits are correct.
Note that as you increase the confidence level, the area of the confidence ellipse will increase, in the same way that a confidence interval becomes wider as you increase the confidence level, ceteris paribus. Finally, the direction in which the confidence ellipse is “sloped” in this example indicates that b1 and b2 must have a negative covariance. This is readily verified by selecting View, Covariance matrix and observing that the covariance is -0.000251:
Postscript ~ What would the confidence region look like if we were dealing with 3 coefficients?