F Tests For Joint Restrictions On Intercept And Slope

Why Test Two Regression Restrictions Together?

A regression question may concern a particular combination of the intercept and slope, rather than either coefficient on its own. For example, a model might be required to pass through a specified point and have a specified slope. Testing these conditions separately does not, in general, answer whether they hold together. A joint F test assesses the pair of restrictions in one calculation and accounts for the covariance between the estimated coefficients.

This tutorial considers the simple linear regression model with an intercept and one slope. The focus is a test of two independent linear restrictions on those two coefficients. The method is aligned with the SOA VEE Mathematical Statistics syllabus.

Model and assumptions: For fixed predictor values $x_1,\ldots,x_n$, suppose $Y_i=\beta_0+\beta_1x_i+\varepsilon_i$, where the errors are independent and normally distributed with mean 0 and common variance $\sigma^2$. The design contains at least two distinct predictor values, so the intercept and slope are estimable. Under this model, the usual exact finite-sample F test applies. If the errors are not normal, that exact distributional result need not hold.

Writing Two Restrictions as a Matrix Hypothesis

Let $\boldsymbol{\beta}=(\beta_0,\beta_1)'$, and let $\hat{\boldsymbol{\beta}}=(\hat{\beta}_0,\hat{\beta}_1)'$ be the least-squares estimate. Two linear restrictions can be written $H_0:R\boldsymbol{\beta}=\boldsymbol{r}$, where $R$ is a $2\times2$ matrix and $\boldsymbol{r}$ is a two-entry vector. The restrictions are independent when $R$ has rank 2. In this tutorial, the number of restrictions is therefore $q=2$.

Definition: The joint F test compares the discrepancy $\boldsymbol{d}=R\hat{\boldsymbol{\beta}}-\boldsymbol{r}$ with the estimated covariance matrix of that discrepancy. If $\widehat{\operatorname{Cov}}(\hat{\boldsymbol{\beta}})=\boldsymbol{V}$, then $$ F=\frac{\boldsymbol{d}'(R\boldsymbol{V}R')^{-1}\boldsymbol{d}}{q}. $$ Under $H_0$ and the stated model, this statistic has an $F$ distribution with $q=2$ numerator degrees of freedom and $n-2$ denominator degrees of freedom.

The estimated coefficient covariance matrix $\boldsymbol{V}$ in this formula already includes the residual mean square. In matrix notation, $\boldsymbol{V}=s^2(X'X)^{-1}$, where $X$ is the regression design matrix and $s^2=\operatorname{SSE}/(n-2)$. Thus, if $\boldsymbol{V}$ is supplied, do not multiply or divide the quadratic form by $s^2$ again. An equivalent form uses $R(X'X)^{-1}R'$ and explicitly divides the quadratic form by $s^2$.

Why the Statistic Has an F Distribution

The result combines the sampling distribution of the estimated restrictions with that of the residual sum of squares. Under the null hypothesis, the discrepancy has mean zero. Its covariance matrix is determined by the coefficient covariance matrix and the restriction matrix.

Standardize the restriction discrepancy.
Under $H_0$, $\boldsymbol{d}$ is normally distributed with covariance $\sigma^2R(X'X)^{-1}R'$. Because $R$ has rank 2, the quadratic form using the inverse of this covariance has a chi-square distribution with 2 degrees of freedom.

Use the residual variation.
Under the normal-error model, $\operatorname{SSE}/\sigma^2$ has a chi-square distribution with $n-2$ degrees of freedom and is independent of the least-squares coefficient estimates.

Form the ratio.
Dividing each independent chi-square quantity by its degrees of freedom gives the F statistic. Replacing $\sigma^2$ in the restriction covariance by $s^2$ yields $F=\boldsymbol{d}'(R\boldsymbol{V}R')^{-1}\boldsymbol{d}/2$.

More explicitly, when the true error variance is used, the chi-square statement is

\boldsymbol{d}'\left[\sigma^2R(X'X)^{-1}R'\right]^{-1}\boldsymbol{d} \sim \chi^2_2.

Equivalently, the same expression is $\boldsymbol{d}'[R(X'X)^{-1}R']^{-1}\boldsymbol{d}/\sigma^2$. These are two ways to write the same standardization: the first inverse already contains $\sigma^2$, while the second form divides by it explicitly.

Worked Calculations

Worked Example: Testing a Target Intercept and Slope

A hypothetical study fits a simple linear regression to $n=8$ observations. The predictor values are centered at zero and have $S_{xx}=8$. The residual mean square is $s^2=8$, and the estimated coefficients are $\hat{\beta}_0=5$ and $\hat{\beta}_1=2$. Test $H_0:\beta_0=4$ and $\beta_1=1$, jointly, using the supplied model summaries. Use a 5% significance level and the critical value $F_{2,6;0.95}=5.143$.

Method. Set $R=I_2$, $\boldsymbol{r}=(4,1)'$, and calculate the covariance-standardized discrepancy. Because the predictor values are centered, the intercept and slope estimates have zero covariance. Their estimated variances are $s^2/n=8/8=1$ and $s^2/S_{xx}=8/8=1$, respectively. Hence $\boldsymbol{V}=I_2$.

The discrepancy and its quadratic form are

\boldsymbol{d}= \begin{pmatrix}5\\2\end{pmatrix} - \begin{pmatrix}4\\1\end{pmatrix} = \begin{pmatrix}1\\1\end{pmatrix}, \qquad \boldsymbol{d}'\boldsymbol{V}^{-1}\boldsymbol{d} = \begin{pmatrix}1&1\end{pmatrix} \begin{pmatrix}1&0\\0&1\end{pmatrix} \begin{pmatrix}1\\1\end{pmatrix} =2.

There are $q=2$ restrictions, so $F=2/2=1.0000$. The degrees of freedom are $(2,n-2)=(2,6)$. Since $1.0000<5.143$, do not reject the joint null hypothesis at the 5% level. The data do not provide sufficient evidence against the specified intercept-and-slope pair.

Check. With centered predictor values and the stated sums, the coefficient covariance matrix is the identity, so the quadratic form is the sum $1^2+1^2=2$. A common mistake is to divide by $s^2=8$ again even though $\boldsymbol{V}$ already includes it; that would incorrectly reduce the statistic.

Worked Example: Testing Two Combinations of Coefficients

A hypothetical regression has $n=10$, centered predictor values with $S_{xx}=10$, and residual mean square $s^2=10$. The coefficient estimates are $(\hat{\beta}_0,\hat{\beta}_1)=(3,2)$, and their estimated covariance matrix is $I_2$. Test the two restrictions $\beta_0+\beta_1=4$ and $\beta_0-\beta_1=0$, at the 5% level. Use $F_{2,8;0.95}=4.459$.

Method. The restrictions are represented by $R=\begin{pmatrix}1&1\\1&-1\end{pmatrix}$ and $\boldsymbol{r}=(4,0)'$. The determinant of $R$ is $-2$, so $R$ has rank 2 and the restrictions are independent.

First calculate the estimated restricted quantities and their discrepancy:

R\hat{\boldsymbol{\beta}} = \begin{pmatrix}1&1\\1&-1\end{pmatrix} \begin{pmatrix}3\\2\end{pmatrix} = \begin{pmatrix}5\\1\end{pmatrix}, \qquad \boldsymbol{d} = \begin{pmatrix}5\\1\end{pmatrix} - \begin{pmatrix}4\\0\end{pmatrix} = \begin{pmatrix}1\\1\end{pmatrix}.

The covariance matrix for the two estimated combinations is

R\boldsymbol{V}R' = RR' = \begin{pmatrix}2&0\\0&2\end{pmatrix}, \qquad (R\boldsymbol{V}R')^{-1} = \begin{pmatrix}1/2&0\\0&1/2\end{pmatrix}.

Thus the quadratic form is $1(1/2)+1(1/2)=1$, giving $F=1/2=0.5000$. The degrees of freedom are $(2,10-2)=(2,8)$. Since $0.5000<4.459$, do not reject the two restrictions jointly at the 5% level.

Check. The off-diagonal elements of $RR'$ are zero, so the two estimated combinations have zero covariance in this example. A common mistake is to treat the restrictions as tests of $\beta_0$ and $\beta_1$ separately; they concern the sum and difference, so the rows of $R$ must be applied to both the estimate and its covariance matrix.

Worked Example: Using a Correlated Coefficient Covariance Matrix

A hypothetical regression has $n=9$, predictor mean $\bar{x}=-1$, $S_{xx}=4$, and residual mean square $s^2=4$. The estimates are $(\hat{\beta}_0,\hat{\beta}_1)=(6,-1)$. Test $H_0:\beta_0=5,\ \beta_1=-1$ jointly. Use $F_{2,7;0.95}=4.737$.

Method. The estimated covariance matrix can be calculated from the regression summaries. The slope variance is $s^2/S_{xx}=4/4=1$. The intercept-slope covariance is $-\bar{x}s^2/S_{xx}=-(-1)(4)/4=1$. The intercept variance is $s^2(1/n+\bar{x}^2/S_{xx})=4(1/9+1/4)=13/9$. Therefore,

\boldsymbol{V} = \begin{pmatrix}13/9&1\\1&1\end{pmatrix}.

Here $R=I_2$ and $\boldsymbol{d}=(1,0)'$. The determinant of $\boldsymbol{V}$ is $13/9-1=4/9$, so its inverse is

\boldsymbol{V}^{-1} = \frac{9}{4} \begin{pmatrix}1&-1\\-1&13/9\end{pmatrix} = \begin{pmatrix}9/4&-9/4\\-9/4&13/4\end{pmatrix}.

Multiplication gives $\boldsymbol{V}^{-1}\boldsymbol{d}=(9/4,-9/4)'$, and then $\boldsymbol{d}'\boldsymbol{V}^{-1}\boldsymbol{d}=9/4$. Consequently, $F=(9/4)/2=9/8=1.1250$, with degrees of freedom $(2,7)$. Since $1.1250<4.737$, do not reject the joint null hypothesis at the 5% level.

Check. The determinant and inverse above give a positive quadratic form, as required for a positive-definite covariance matrix. The off-diagonal covariance affects the inverse even though the slope discrepancy is zero; ignoring it would give the wrong joint statistic.

Interpreting the Test and Avoiding Traps

The test is an omnibus test of the restrictions as a set. Rejecting $H_0$ means that the observed coefficient estimates are sufficiently inconsistent with at least one of the restrictions, after accounting for their joint sampling variability. It does not, by itself, identify which restriction is responsible. Not rejecting does not prove that both restrictions are true; it means the data do not provide enough evidence against them at the chosen significance level.

The statistic is nonnegative because it is a quadratic form based on a positive-definite covariance matrix. Its numerator degrees of freedom equal the number of independent restrictions, not the number of observations. For this simple regression, the residual degrees of freedom are $n-2$, since two coefficients are estimated.

Do not double-count residual mean square. If the supplied covariance is $\widehat{\operatorname{Cov}}(\hat{\boldsymbol{\beta}})$, use it directly in the F formula. Use an extra division by $s^2$ only when the matrix being inverted is $R(X'X)^{-1}R'$, which omits that scale.
Check rank. Two rows that are multiples of one another do not impose two independent restrictions. The numerator degrees of freedom are the rank of the restriction matrix, not simply the number of written equations.
Keep the covariance terms. If the coefficient estimates are correlated, the off-diagonal entries can affect the statistic. Replacing the covariance matrix by its diagonal is generally incorrect.
Use the correct residual degrees of freedom. With an intercept and one slope, the denominator degrees of freedom are $n-2$, not $n-1$.
Distinguish a joint test from the overall regression test. The overall regression significance test concerns a particular null about the slope. A general pair of restrictions on intercept and slope is tested using its own $R$, $\boldsymbol{r}$, and covariance transformation.

Key takeaway: For two independent restrictions, calculate $\boldsymbol{d}=R\hat{\boldsymbol{\beta}}-\boldsymbol{r}$, transform the estimated coefficient covariance to $R\boldsymbol{V}R'$, and form $F=\boldsymbol{d}'(R\boldsymbol{V}R')^{-1}\boldsymbol{d}/2$. Under the stated normal-error model, compare it with an $F_{2,n-2}$ distribution.

Check Your Understanding

For each problem, use the stated significance level and any supplied critical value. Treat a supplied coefficient covariance matrix as already including the residual mean square.

A regression with $n=14$ has estimated coefficient covariance $\boldsymbol{V}=\operatorname{diag}(0.75,3)$. The estimate differs from the two hypothesized coefficient values by $\boldsymbol{d}=(1,2)'$. Calculate the joint F statistic and its degrees of freedom. (Answer: $F=1.3333$, degrees of freedom $(2,12)$.)
For $n=11$, let $\boldsymbol{V}=\begin{pmatrix}2&1\\1&2\end{pmatrix}$ and $\boldsymbol{d}=(2,0)'$. Calculate the joint F statistic and degrees of freedom. (Answer: $F=1.3333$, degrees of freedom $(2,9)$.)
A test has $R=\begin{pmatrix}1&1\\1&-1\end{pmatrix}$, $\boldsymbol{V}=I_2$, and $\boldsymbol{d}=(2,-2)'$. Calculate $R\boldsymbol{V}R'$, the joint F statistic, and the numerator degrees of freedom. (Answer: $R\boldsymbol{V}R'=\begin{pmatrix}2&0\\0&2\end{pmatrix}$, $F=2.0000$, numerator degrees of freedom $2$.)
For a regression with $n=12$, $\boldsymbol{V}=I_2$, and $\boldsymbol{d}=(3,1)'$, test the restrictions at the 5% level using $F_{2,10;0.95}=4.103$. State the statistic and decision. (Answer: $F=5.0000$; reject the joint null hypothesis at the 5% level.)
A proposed restriction matrix has rows $(1,2)$ and $(2,4)$. How many independent restrictions does it impose, and what is the numerator degrees of freedom for the corresponding nonredundant test? (Answer: One independent restriction; numerator degrees of freedom $1$.)

What You'll Learn