Why Estimate the Intercept and Slope Together?
In a simple linear regression model, the intercept and slope are both unknown population parameters. Their estimates are usually related: changing the slope can be partly offset by changing the intercept, particularly when the predictor values are not centered near zero. A pair of separate confidence intervals does not display this relationship. A joint confidence region does: it gives a set of intercept-slope pairs that are collectively compatible with the data at a stated confidence level.
The region is an ellipse in the parameter plane. Its center is the pair of least-squares estimates, and its orientation and shape depend on the predictor values and the covariance between the estimates. This tutorial constructs the region using an F distribution and explains why its coverage differs from that of two separate marginal intervals.
Write \(\hat\beta_0\) and \(\hat\beta_1\) for the least-squares estimates, and define \(S_{xx}=\sum_{i=1}^n(x_i-\bar{x})^2\). Let \(\mathrm{SSE}=\sum_{i=1}^n(Y_i-\hat\beta_0-\hat\beta_1x_i)^2\) and \(s^2=\mathrm{SSE}/(n-2)\). The model assumptions above are those for the exact finite-sample F construction; without normal errors, this particular exact coverage result need not hold.
These are regression-inference quantities, not interest or life-contingency calculations: no interest rate or mortality assumption applies. Each worked example states the hypothetical regression distribution and the values used.
The F-Based Joint Confidence Region
Let \(F_{2,n-2;1-\alpha}\) denote the \(1-\alpha\) quantile of an F distribution with 2 and \(n-2\) degrees of freedom. For a confidence level \(1-\alpha\), the joint confidence region consists of all proposed population pairs \((\beta_0,\beta_1)\) satisfying the following inequality.
Equivalently, multiply by \(2s^2\) and use the resulting inequality to test a candidate pair. The numerator is a quadratic form in the two coefficient differences, so its boundary is an ellipse. The estimates \((\hat\beta_0,\hat\beta_1)\) are at its center.
Why the F Formula Has This Form
The joint region comes from inverting a test of a proposed pair. To see the source of the F statistic, write the regression in matrix form as \(Y=X\beta+\varepsilon\), where the two columns of \(X\) are a column of ones and the observed predictor values. Since at least two predictor values differ, \(X\) has rank 2. Under the model, the least-squares estimation error has a two-dimensional normal distribution, and its standardized squared distance from the proposed true coefficient pair has a chi-square distribution with 2 degrees of freedom.
More specifically, under a proposed pair that is the true pair, \((\hat\beta-\beta)'X'X(\hat\beta-\beta)/\sigma^2\) has a \(\chi^2_2\) distribution. The residual sum of squares satisfies \(\mathrm{SSE}/\sigma^2\sim\chi^2_{n-2}\), independently of the coefficient estimates. Dividing each chi-square variable by its degrees of freedom gives an F statistic. In the coefficient notation above, this is exactly the quadratic expression divided by \(2s^2\).
For the true coefficient pair, the probability that this statistic is at most \(F_{2,n-2;1-\alpha}\) is \(1-\alpha\). Replacing the unknown true pair by all pairs that satisfy the same inequality therefore gives a region with joint coverage \(1-\alpha\). Expanding the quadratic form using \(\sum x_i=n\bar{x}\) and \(\sum x_i^2=S_{xx}+n\bar{x}^2\) yields the displayed formula. The denominator uses \(s^2=\mathrm{SSE}/(n-2)\), so \(n>2\) is essential: if \(n=2\), the residual variance estimate and the F statistic are undefined.
Worked Examples
Worked Example: Constructing and Checking a Joint Region
A hypothetical regression of annual maintenance cost \(Y\), in thousands of dollars, on a device-age index \(x\) has \(n=8\), \(\bar{x}=2\), \(S_{xx}=12\), \(\hat\beta_0=10\), \(\hat\beta_1=3\), and \(\mathrm{SSE}=18\). Assume independent errors with distribution \(N(0,\sigma^2)\), where \(\sigma^2\) is unknown. There is no interest or mortality assumption. Construct the 95% joint region and determine whether \((\beta_0,\beta_1)=(8,4)\) is included.
Method. Calculate the residual degrees of freedom and residual variance estimate, then substitute the summaries into the F-region inequality.
The residual degrees of freedom are \(n-2=8-2=6\), and \(s^2=18/6=3\). The 95% critical value is \(F_{2,6;0.95}\approx5.143\). Therefore the right side after multiplying the inequality by \(2s^2\) is \(2(3)(5.143)=30.858\). The joint region is
For the proposed pair \((8,4)\), the estimate-minus-candidate differences are \(10-8=2\) and \(3-4=-1\). Substitution gives \(8[2+2(-1)]^2+12(-1)^2=8(0)^2+12=12\). Since \(12\leq30.858\), the pair is inside the region.
Check. At the estimated pair \((10,3)\), both differences are zero and the left side is zero, as required for the center of the region. The method avoids treating the two coefficients as unrelated when assessing a pair.
Worked Example: Separate Marginal Intervals Can Include a Pair Outside the Region
Use the hypothetical regression summaries from the preceding example: \(n=8\), \(\bar{x}=2\), \(S_{xx}=12\), estimates \(10\) and \(3\), and \(s^2=3\). Errors are independent \(N(0,\sigma^2)\) with unknown \(\sigma^2\). There is no interest rate or mortality assumption. Calculate separate 95% t intervals for the intercept and slope, then compare them with the 95% joint region.
Method. The standard errors are \(\operatorname{SE}(\hat\beta_1)=\sqrt{s^2/S_{xx}}\) and \(\operatorname{SE}(\hat\beta_0)=\sqrt{s^2(1/n+\bar{x}^2/S_{xx})}\). With 6 residual degrees of freedom, \(t_{6;0.975}\approx2.447\).
For the slope, the standard error is \(\sqrt{3/12}=\sqrt{0.25}=0.500\). Its interval is \(3\pm2.447(0.500)=3\pm1.2235\), or approximately \([1.776,4.224]\).
For the intercept, \(1/n+\bar{x}^2/S_{xx}=1/8+4/12=3/24+8/24=11/24\). Thus its standard error is \(\sqrt{3(11/24)}=\sqrt{1.375}\approx1.173\). Its interval is \(10\pm2.447(1.173)\approx10\pm2.869\), or approximately \([7.131,12.869]\). Endpoints are rounded to three decimals.
The pair \((10,4)\) lies in both marginal intervals: the intercept is at its estimate, and 4 is within the slope interval. Yet its joint-region expression is \(8[(10-10)+2(3-4)]^2+12(3-4)^2=8( -2)^2+12=44\), which exceeds \(30.858\). Thus this pair is outside the 95% joint region despite lying in both separate 95% intervals.
Check. The intervals concern each coefficient individually. They do not claim that both intervals cover their respective parameters simultaneously with probability 0.95. The common mistake is to interpret two 95% marginal intervals as a 95% joint statement.
Worked Example: Evaluating a Proposed Pair in a Different Regression
A hypothetical study relates a portfolio’s annualized service cost \(Y\), in thousands of dollars, to a centered workload measure \(x\). The summaries are \(n=10\), \(\bar{x}=0\), \(S_{xx}=30\), \(\hat\beta_0=5\), \(\hat\beta_1=-0.4\), and \(\mathrm{SSE}=16\). Assume independent errors distributed as \(N(0,\sigma^2)\), with unknown variance. No interest or mortality model is involved. At 95% confidence, determine whether \((4,-0.1)\) lies in the joint region.
Method. First find \(s^2=16/(10-2)=2\). The F critical value is \(F_{2,8;0.95}\approx4.459\), so the right side of the unscaled quadratic inequality is \(2(2)(4.459)=17.836\). Since \(\bar{x}=0\), the region is
For \((4,-0.1)\), the coefficient differences are \(5-4=1\) and \(-0.4-(-0.1)=-0.3\). The left side is \(10(1)^2+30(-0.3)^2=10+30(0.09)=12.7\). Since \(12.7\leq17.836\), the proposed pair is inside the region.
Check. At the fitted pair \((5,-0.4)\), the left side is zero. For this centered predictor, the cross-term vanishes in the displayed expression; that simplification follows from \(\bar{x}=0\), not from a general rule that intercept and slope estimates are independent.
What the Region Says—and What It Does Not Say
The joint region is one confidence statement about the vector \((\beta_0,\beta_1)\). Under the model, the random region constructed by this procedure contains the true pair with probability \(1-\alpha\) over repeated samples. After the data are observed, the region is a fixed set; the frequentist coverage statement does not assign a probability to the fixed parameter pair.
The boundary accounts for the covariance between the two estimates. When \(\bar{x}\neq0\), the term \((\hat\beta_0-\beta_0)+\bar{x}(\hat\beta_1-\beta_1)\) shows how intercept and slope changes can partly compensate. The ellipse is generally tilted in the coefficient plane. When the predictor is centered so that \(\bar{x}=0\), this particular quadratic expression has no cross-term, and its axes align with the intercept and slope directions.
Separate marginal t intervals answer two individual questions, one about \(\beta_0\) and one about \(\beta_1\). Their rectangular overlap is not the F-based joint region, and its coverage is not generally the stated joint confidence level. Conversely, the joint region is not simply the two marginal intervals drawn together: the F critical value and the covariance structure determine a different boundary.
Common Mistakes and a Reliable Procedure
- Using the wrong degrees of freedom. The F critical value has 2 numerator degrees of freedom and \(n-2\) denominator degrees of freedom. The residual degrees of freedom must be positive, so require \(n>2\).
- Forgetting the factor \(2s^2\). The quadratic numerator is compared with \(2s^2F_{2,n-2;1-\alpha}\) when written without division by \(2s^2\).
- Using separate intervals as a joint region. Two individual intervals do not automatically provide the same simultaneous coverage as the F-based region.
- Ignoring predictor variation. At least two distinct predictor values are needed to estimate both coefficients; if \(S_{xx}=0\), the slope and the stated region are not defined.
- Reversing the coefficient differences inconsistently. Both differences can be written estimate minus candidate or candidate minus estimate, because they are squared as a quadratic form. Keep the choice consistent in every term.
A dependable construction is to verify \(n>2\) and \(S_{xx}>0\), calculate \(s^2=\mathrm{SSE}/(n-2)\), select \(F_{2,n-2;1-\alpha}\), and substitute each proposed pair into the quadratic inequality. This gives a direct inclusion check and keeps the joint interpretation explicit.
Check Your Understanding
For each problem, state whether the candidate pair is inside the stated joint region when requested. The errors are independent normal with common unknown variance in each setting; no interest or mortality assumption applies.
- A regression has \(n=7\), \(\bar{x}=1\), \(S_{xx}=14\), estimates \((2,1)\), and \(\mathrm{SSE}=10\). Write the 90% joint confidence region. (Answer: \(s^2=10/5=2\), \(F_{2,5;0.90}\approx3.780\), so \(7[(2-\beta_0)+(1-\beta_1)]^2+14(1-\beta_1)^2\leq15.120\).)
- A regression has \(n=9\), \(\bar{x}=0\), \(S_{xx}=18\), estimates \((4,-0.5)\), and \(\mathrm{SSE}=14\). Give the 95% joint region in terms of the F quantile. (Answer: \(s^2=14/7=2\), so \(9(4-\beta_0)^2+18(-0.5-\beta_1)^2\leq4F_{2,7;0.95}\), approximately \(18.95\) on the right.)
- A regression has \(n=6\), \(\bar{x}=2\), \(S_{xx}=10\), estimates \((1,2)\), and \(\mathrm{SSE}=8\). At 95%, is \((0,1)\) inside the region? (Answer: \(s^2=8/4=2\); the left side is \(6[(1-0)+2(2-1)]^2+10(2-1)^2=64\), while \(2s^2F_{2,4;0.95}\approx27.776\), so it is outside.)
- For \(n=8\), \(\bar{x}=2\), \(S_{xx}=12\), \(s^2=3\), and a 95% joint region, what is the largest possible absolute slope difference \(|\hat\beta_1-\beta_1|\) among points in the region? Use \(F_{2,6;0.95}=5.143\). (Answer: \(\sqrt{2(3)(5.143)/12}=\sqrt{2.5715}\approx1.604\).)
- For the same summaries in question 4, what is the largest possible absolute intercept difference \(|\hat\beta_0-\beta_0|\) in the joint region? (Answer: \(\sqrt{2(3)(5.143)(1/8+2^2/12)}=\sqrt{14.143}\approx3.761\).)