Why One-Restriction F Tests Equal Squared t Tests

Why Two Tests Can Be the Same Test

A general linear hypothesis F test compares an unrestricted least-squares fit with a fit constrained to satisfy specified linear restrictions. For a single independent restriction, this comparison has one numerator degree of freedom. A t test, by contrast, works directly with the estimated contrast and its standard error. These descriptions look different, but for the same nonzero estimable restriction they produce statistics related by an exact identity: $F=t^2$.

The identity is algebraic; it does not depend on an approximation. Under the normal linear model, the usual exact t and F distributions then imply that the tests give the same p-value and decision when the t test is two-sided. We derive the identity, check it in concrete models, and clarify why the zero contrast must be excluded.

Model and the One-Restriction Setup

Let $Y\in\mathbb{R}^n$ follow the normal linear model $Y=X\beta+\varepsilon$, where $X$ is a fixed $n\times p$ design matrix of rank $r$, $\beta\in\mathbb{R}^p$, and $\varepsilon\sim N(0,\sigma^2 I_n)$ with $\sigma^2>0$. The residual degrees of freedom are $\nu=n-r>0$. Write $\hat\beta$ for any least-squares solution and $\mathrm{SSE}_U$ for the residual sum of squares from the unrestricted fit. Define $\mathrm{MSE}=\mathrm{SSE}_U/\nu$.

Definition: A vector $c\in\mathbb{R}^p$ is an estimable contrast if $c'\beta$ is uniquely determined by the mean vector $X\beta$; equivalently, $c$ belongs to the row space of $X$. A single restriction $c'\beta=\delta$ is one independent estimable restriction when $c$ is estimable and $c\ne 0$. For such $c$, the t statistic is $$ t=\frac{c'\hat\beta-\delta}{\sqrt{\mathrm{MSE}\,c'(X'X)^+c}}, $$ where $(X'X)^+$ is the Moore–Penrose inverse. The corresponding one-restriction F statistic is $$ F=\frac{\mathrm{SSE}_R-\mathrm{SSE}_U}{\mathrm{MSE}}, $$ where $\mathrm{SSE}_R$ is the residual sum of squares under $c'\beta=\delta$.

The nonzero condition matters. If $c=0$, then $c'\beta=\delta$ is either always true (when $\delta=0$) or impossible (when $\delta\ne0$); it is not one independent estimable restriction. In particular, $c'(X'X)^+c=0$, so the displayed t statistic is undefined. All results below assume $c\ne0$.

The Restricted Fit Adds a Squared Contrast Error

The key calculation is the increase in residual sum of squares caused by imposing the restriction. Put $G=X'X$ and $A=G^+$. Because $c$ is estimable, $c$ lies in the range of $G$. The matrix $A$ is symmetric and positive semidefinite, and $GAG=G$. In particular, $c'Ac>0$: if $c=Gz$, then $c'Ac=z'GAGz=z'Gz$, which is positive for nonzero $c$ in the range of $G$.

Derived result: For a nonzero estimable $c$, the increase in residual sum of squares from imposing $c'\beta=\delta$ is $$ \mathrm{SSE}_R-\mathrm{SSE}_U =\frac{(c'\hat\beta-\delta)^2}{c'(X'X)^+c}. $$

Proof. Let $\hat\beta$ be an unrestricted least-squares solution and let $e=Y-X\hat\beta$. The least-squares normal equations give $X'e=0$. Any other parameter vector can be written $\hat\beta+h$, and its residual sum of squares is $$ \|e-Xh\|^2=\|e\|^2+h'Gh, $$ because the cross term is $-2e'Xh=0$. Thus the restricted fit minimizes $h'Gh$ subject to $c'h=-d$, where $d=c'\hat\beta-\delta$.

Since $c$ lies in the range of $G$, $GA c=c$. For any $h$, the spectral decomposition of $G$ gives $$ c'h=(A^{1/2}c)'(G^{1/2}h). $$ Indeed, $A^{1/2}G^{1/2}c=c$ on the range of $G$. By the Cauchy–Schwarz inequality, $$ (c'h)^2\le (c'Ac)(h'Gh). $$ Every feasible $h$ therefore has $h'Gh\ge d^2/(c'Ac)$. This bound is attained by $h_*=-dAc/(c'Ac)$: it satisfies $c'h_*=-d$, and, using $AGA=A$, $$ h_*'Gh_*=\frac{d^2\,c'AGA c}{(c'Ac)^2} =\frac{d^2}{c'Ac}. $$ Consequently the minimum added residual sum of squares is $d^2/(c'Ac)$, as claimed. $\square$

The Identity and Its Test Interpretation

The preceding result shows directly that the F statistic has numerator equal to the squared contrast error divided by its unscaled variance factor. Substituting the definitions of $F$ and $t$ gives the promised identity.

Theorem: Under the model setup above, for testing the single nonzero estimable restriction $H_0:c'\beta=\delta$, $$ F=\frac{\mathrm{SSE}_R-\mathrm{SSE}_U}{\mathrm{MSE}} =\frac{(c'\hat\beta-\delta)^2}{\mathrm{MSE}\,c'(X'X)^+c} =t^2. $$

Proof. Divide the restricted-sum-of-squares formula by $\mathrm{MSE}$. The result is exactly the square of the t statistic defined for this same contrast and null value. Hence $F=t^2$. $\square$

Under $H_0$, with the stated Gaussian spherical-error model and $\nu>0$, the exact contrast t statistic has a $t_\nu$ distribution and the general linear hypothesis F statistic has an $F_{1,\nu}$ distribution. Their distributional relationship is also direct: if $T=Z/\sqrt{U/\nu}$, where $Z\sim N(0,1)$, $U\sim\chi^2_\nu$, and $Z$ and $U$ are independent, then $T\sim t_\nu$ and $$ T^2=\frac{Z^2}{U/\nu}\sim F_{1,\nu}. $$ Because $F=t^2$ for each observed data set, the F upper-tail p-value equals the two-sided t p-value: $$ P(F_{1,\nu}\ge t_{\mathrm{obs}}^2) =P(|T_\nu|\ge |t_{\mathrm{obs}}|). $$ The equality does not make a one-sided t test equivalent to an F test: squaring removes the sign of the estimated contrast.

Worked Calculations

Worked Example: A Coordinate Restriction in an Orthogonal Design

Suppose $X'X=\begin{pmatrix}4&0\\0&2\end{pmatrix}$, $\hat\beta=(5,3)'$, and the restriction is $\beta_2=1$. Let $\nu=10$ and $\mathrm{SSE}_U=40$. Check the two statistics.

Here $c=(0,1)'$, $\delta=1$, and $(X'X)^+=\operatorname{diag}(1/4,1/2)$. Thus $c'(X'X)^+c=1/2$, while $c'\hat\beta-\delta=3-1=2$. The restriction adds $$ \mathrm{SSE}_R-\mathrm{SSE}_U=\frac{2^2}{1/2}=8. $$ Hence $\mathrm{SSE}_R=48$ and $\mathrm{MSE}=40/10=4$. The t statistic and its square are $$ t=\frac{2}{\sqrt{4(1/2)}}=\frac{2}{\sqrt{2}},\qquad t^2=2. $$ The F statistic is $F=8/4=2$. The two calculations agree.

Worked Example: Testing a Slope in a Simple Regression

In a simple regression with an intercept and centered predictor satisfying $\sum_i x_i^2=10$, the slope estimate is $0.6$. Test the restriction that the slope is $0.1$, given $\mathrm{MSE}=2$ and $\nu=8$.

For the slope contrast, $c=(0,1)'$. Centering makes the intercept and slope columns orthogonal, so the slope entry of $(X'X)^{-1}$ is $1/10$. The contrast error is $0.6-0.1=0.5$, and the restricted fit adds $$ \frac{(0.5)^2}{1/10}=\frac{0.25}{0.1}=2.5 $$ to the unrestricted SSE. The t statistic is $$ t=\frac{0.5}{\sqrt{2(1/10)}}=\frac{0.5}{\sqrt{0.2}}, \qquad t^2=\frac{0.25}{0.2}=1.25. $$ The F statistic is $2.5/2=1.25$. This F test corresponds to the two-sided test of the slope value $0.1$; it does not encode whether the estimate is above or below $0.1$.

Worked Example: A Non-Coordinate Contrast

Suppose $X'X=\operatorname{diag}(2,4)$, $\hat\beta=(2,1)'$, and the restriction is $\beta_1+\beta_2=2$. Take $\nu=6$ and $\mathrm{MSE}=2$. Compute both statistics.

The contrast vector is $c=(1,1)'$, so $c'(X'X)^{-1}c=1/2+1/4=3/4$. The estimated contrast is $c'\hat\beta=3$, giving discrepancy $3-2=1$. The increase in SSE is $$ \mathrm{SSE}_R-\mathrm{SSE}_U=\frac{1^2}{3/4}=\frac{4}{3}. $$ The t statistic satisfies $$ t=\frac{1}{\sqrt{2(3/4)}}=\frac{1}{\sqrt{3/2}}, \qquad t^2=\frac{2}{3}. $$ Meanwhile $F=(4/3)/2=2/3$. The identity holds even though the restriction involves both coefficients rather than just one coordinate.

Common Mistakes and Why the Identity Matters

Forgetting the nonzero condition. An estimable vector can be zero, but $c=0$ does not define one independent restriction. Its contrast variance factor is zero, so the t statistic and the formula dividing by that factor are undefined. Require $c\ne0$.
Using a different variance estimate in the two tests. The identity uses the same unrestricted $\mathrm{MSE}$ in the t standard error and in the F denominator. Substituting a different variance estimate breaks this equality.
Claiming that F supplies a direction. Since $F=t^2$, positive and negative t values with the same magnitude give the same F statistic. A one-sided alternative requires the signed t statistic.
Comparing different null hypotheses. The contrast, null value, design, and error estimate must match. A t test of one coefficient is not automatically the same as an F test for a different restriction.

The practical value of the identity is both conceptual and diagnostic. It shows that a one-restriction general linear hypothesis test is not a separate inferential procedure from a two-sided contrast t test. It also provides a useful computation check: if the restricted-fit calculation and $t^2$ disagree, inspect the restriction, contrast variance factor, residual degrees of freedom, and MSE.

Check Your Understanding

Use the definitions and derivation above to answer these questions.

Why must a one-restriction test require both that $c$ is estimable and that $c\ne0$?
If $c'\hat\beta-\delta=-3$ and $\mathrm{MSE}\,c'(X'X)^+c=2$, what are $t^2$ and $F$?
Explain why the F test for one restriction agrees with a two-sided t test but does not distinguish the two directions of a one-sided alternative.
In the restricted-sum-of-squares derivation, why does the cross term between the unrestricted residual $e$ and $Xh$ vanish?
If the restriction is $0'\beta=0$, what fails in the t-statistic formula, and why is this not a one independent restriction?