Noncentral F Distributions for General Linear Hypotheses

From a Linear Hypothesis to an Exact F Distribution

A multi-parameter hypothesis asks whether several restrictions on a regression coefficient vector hold together. In a normal linear model, the usual F statistic has an exact distribution under the null. Away from the null, it has a noncentral F distribution. Its noncentrality parameter records how far the true coefficient vector departs from the hypothesis, on a scale determined by the design and the error variance.

We derive that distribution for a general rank-$q$ hypothesis, then show precisely how its noncentrality parameter is the squared distance to the null set under the design metric. The result is exact for finite samples under the stated normal-model assumptions.

Model and hypothesis: Let $Y=X\beta+\varepsilon$, where $Y$ is an $n$-vector, $X$ is a fixed $n\times p$ matrix of full column rank, and $\varepsilon\sim N_n(0,\sigma^2 I_n)$ with $\sigma^2>0$. Assume $n>p$. The parameter is $\beta\in\mathbb{R}^p$, and $\sigma^2$ is unknown. Consider $H_0:A\beta=b$, where $A$ is a $q\times p$ matrix of rank $q$, $1\le q\le p$, and $b\in\mathbb{R}^q$.

Full column rank ensures that all components of $\beta$ are identifiable in this model. The rank condition on $A$ means that the $q$ restrictions are linearly independent. Write $\hat\beta=(X'X)^{-1}X'Y$ for the least-squares estimator, and let $\operatorname{SSE}=(Y-X\hat\beta)'(Y-X\hat\beta)$. The F statistic compares the estimated departure from the restrictions with the residual mean square.

Definition: Put $C=A(X'X)^{-1}A'$. Under the stated rank conditions, $C$ is positive definite. The statistic for testing $H_0:A\beta=b$ is $$ F=\frac{(A\hat\beta-b)'C^{-1}(A\hat\beta-b)/q}{\operatorname{SSE}/(n-p)}. $$ The numerator is the quadratic discrepancy from the hypothesized restrictions, scaled by its covariance; the denominator estimates $\sigma^2$.

The Exact Distribution

A noncentral chi-square variable with $q$ degrees of freedom and noncentrality parameter $\lambda\ge 0$ is the distribution of $\sum_{j=1}^q(Z_j+m_j)^2$, where the $Z_j$ are independent $N(0,1)$ variables and $\sum_{j=1}^q m_j^2=\lambda$. A noncentral F variable with degrees of freedom $q,\nu$ and noncentrality $\lambda$ is the distribution of \[ \frac{U/q}{V/\nu}, \] where $U$ has this noncentral chi-square distribution, $V\sim\chi^2_\nu$, and $U,V$ are independent. When $\lambda=0$, these become the usual central chi-square and F distributions.

Theorem: Under the model and rank conditions above, for any true $\beta$, $$ F\sim F_{q,n-p}(\lambda),\qquad \lambda=\frac{(A\beta-b)'[A(X'X)^{-1}A']^{-1}(A\beta-b)}{\sigma^2}. $$ In particular, if $H_0$ is true, then $\lambda=0$ and $F$ has the central $F_{q,n-p}$ distribution.

Proof. Let $\nu=n-p$ and $M_X=I-X(X'X)^{-1}X'$, the residual projection matrix. From the model, $$ \hat\beta=\beta+(X'X)^{-1}X'\varepsilon,\qquad A\hat\beta-b\sim N_q(A\beta-b,\sigma^2 C). $$ Since $C$ is positive definite, choose a symmetric square root $C^{-1/2}$. The vector $C^{-1/2}(A\hat\beta-b)/\sigma$ has covariance $I_q$ and mean $C^{-1/2}(A\beta-b)/\sigma$. Its squared length therefore has a noncentral chi-square distribution with $q$ degrees of freedom and $$ \lambda=\frac{(A\beta-b)'C^{-1}(A\beta-b)}{\sigma^2}. $$ Consequently, the numerator quadratic form divided by $\sigma^2$ has that distribution.

The residual sum of squares is $\operatorname{SSE}=\varepsilon'M_X\varepsilon$. The matrix $M_X$ is symmetric and idempotent, with rank $n-p=\nu$. Orthogonally diagonalizing this projection shows that $\operatorname{SSE}/\sigma^2\sim\chi^2_\nu$. To verify independence, the numerator error term depends linearly on $X'\varepsilon$, while the residual vector is $M_X\varepsilon$. Their cross-covariance is zero because $X'M_X=0$. These are jointly normal, so zero cross-covariance implies independence; hence the numerator quadratic form and $\operatorname{SSE}$ are independent. Dividing their scaled versions by $q$ and $\nu$, respectively, gives the asserted noncentral F distribution. If $A\beta=b$, the mean of the whitened numerator vector is zero, so $\lambda=0$. $\square$

Noncentrality as Distance to the Null Set

The expression for $\lambda$ can be read as a distance, but the relevant metric is determined by the design. Define $G=X'X$, which is positive definite. For any coefficient vectors $\beta$ and $\gamma$, the squared design-metric distance between them is $(\beta-\gamma)'G(\beta-\gamma)$. The null set is the affine set of vectors $\gamma$ satisfying $A\gamma=b$.

Result: Let $\delta=A\beta-b$. The squared design-metric distance from $\beta$ to the null set is $$ \min_{\gamma:\,A\gamma=b}(\beta-\gamma)'X'X(\beta-\gamma) =\delta'[A(X'X)^{-1}A']^{-1}\delta. $$ Thus the noncentrality parameter is this squared distance divided by $\sigma^2$.

Proof. Set $h=\beta-\gamma$, so the constraint becomes $Ah=\delta$. With $G=X'X$ and $C=AG^{-1}A'$, define $h_*=G^{-1}A'C^{-1}\delta$. Then $Ah_*=CC^{-1}\delta=\delta$, so $h_*$ satisfies the constraint. For any other feasible $h$, write $h=h_*+v$, where $Av=0$. The cross-term in the squared length vanishes: $$ h_*'Gv=\delta'C^{-1}Av=0. $$ Therefore $h'Gh=h_*'Gh_*+v'Gv$, and $v'Gv\ge 0$ because $G$ is positive definite. The minimum is attained at $h_*$. Its value is $$ h_*'Gh_*=\delta'C^{-1}AG^{-1}A'C^{-1}\delta =\delta'C^{-1}\delta, $$ as claimed. Dividing by $\sigma^2$ gives the theorem's $\lambda$. $\square$

This distance depends on $X'X$: coefficient directions estimated precisely count differently from directions estimated imprecisely. It is not generally the ordinary Euclidean distance between coefficient vectors. The variance scale matters too: for a fixed departure from the null set, larger $\sigma^2$ means smaller noncentrality.

Worked Examples

Worked Example: A One-Restriction Test with Correlated Coefficient Estimates

Suppose $p=2$, $n=4$, and the columns of $X$ are $(1,1,1,1)'$ and $(1,1,1,-1)'$. Then $$ X'X=\begin{pmatrix}4&2\\2&4\end{pmatrix}, \qquad (X'X)^{-1}=\begin{pmatrix}1/3&-1/6\\-1/6&1/3\end{pmatrix}. $$ Test $H_0:\beta_1+\beta_2=0$, so $A=(1,1)$, $b=0$, and $q=1$. Take the true parameter to be $\beta=(2,1)'$ and $\sigma^2=4$.

The covariance matrix of $\hat\beta$ is $4(X'X)^{-1}$. Its diagonal entries are $4/3$, and its off-diagonal entries are $-2/3$, so the coefficient estimates are correlated, with correlation $(-2/3)/(4/3)=-1/2$. For the tested linear combination, $$ C=A(X'X)^{-1}A'=1/3-1/6-1/6+1/3=1/3. $$ The departure is $A\beta-b=2+1=3$, giving $$ \lambda=\frac{3^2}{(1/3)4}=\frac{27}{4}=6.75. $$ Here $\nu=n-p=2$, so the statistic has distribution $F_{1,2}(6.75)$ at these parameter values. Under the null, the same statistic instead has the central $F_{1,2}$ distribution.

Worked Example: A Two-Restriction Hypothesis

Let $n=6$, $p=3$, and suppose the design has $X'X=4I_3$. Test $H_0:\beta_1=0,\ \beta_2=0$, represented by $$ A=\begin{pmatrix}1&0&0\\0&1&0\end{pmatrix},\qquad b=\begin{pmatrix}0\\0\end{pmatrix}. $$ Take $\beta=(1,2,-1)'$ and $\sigma^2=2$. The hypothesis matrix has rank $q=2$, and the residual degrees of freedom are $\nu=6-3=3$.

Since $(X'X)^{-1}=I_3/4$, we have $C=I_2/4$ and $C^{-1}=4I_2$. The departure vector is $\delta=(1,2)'$, so $$ \delta'C^{-1}\delta =(1,2)\begin{pmatrix}4&0\\0&4\end{pmatrix}\begin{pmatrix}1\\2\end{pmatrix} =4+16=20. $$ Thus $\lambda=20/2=10$, and $F\sim F_{2,3}(10)$. The third coefficient does not enter $\delta$, but the residual degrees of freedom still depend on the full model dimension $p$.

Worked Example: Checking the Distance by Constrained Minimization

Let $X'X=G=\begin{pmatrix}5&1\\1&2\end{pmatrix}$, a positive-definite matrix since its leading diagonal entry is positive and its determinant is $9$. Test $H_0:\beta_1-\beta_2=0$, with $A=(1,-1)$, $b=0$, and let $\beta=(3,1)'$. Then $\delta=A\beta=2$. The inverse design matrix and contrast variance factor are $$ G^{-1}=\frac{1}{9}\begin{pmatrix}2&-1\\-1&5\end{pmatrix}, \qquad C=AG^{-1}A'=1. $$ Therefore the squared design-metric distance is $\delta^2/C=4$, and $\lambda=4/\sigma^2$.

To verify the minimum directly, the closest null vector in the $G$-metric is $\gamma=\beta-h_*$, where $$ h_*=G^{-1}A'C^{-1}\delta =\frac{1}{9}\begin{pmatrix}2&-1\\-1&5\end{pmatrix} \begin{pmatrix}1\\-1\end{pmatrix}2 =\begin{pmatrix}2/3\\-4/3\end{pmatrix}. $$ Hence $\gamma=(7/3,7/3)'$, which satisfies the null restriction. Also $Gh_*=(2,-2)'$, so $$ h_*'Gh_*=(2/3)(2)+(-4/3)(-2)=4. $$ This agrees with the contrast formula. For example, if $\sigma^2=2$, then $\lambda=2$; with $n=5,p=2$, the statistic has distribution $F_{1,3}(2)$.

Interpretation and Common Pitfalls

The noncentral F distribution is useful because it describes the statistic not only when the restrictions are true, but also under specified alternatives. The parameter $\lambda$ combines the departure $A\beta-b$, the precision of the tested estimates through $A(X'X)^{-1}A'$, and the error variance. This exact distribution is a foundation for evaluating how an F test behaves under alternatives.

Using $q$ for the residual degrees of freedom. The numerator degrees of freedom are the rank $q$ of $A$; the denominator degrees of freedom are $n-p$, the residual degrees of freedom in the full model.
Dropping covariance terms. The restrictions may involve correlated coefficient estimates. The quadratic form must use the full matrix $A(X'X)^{-1}A'$, not separate marginal variances.
Calling the numerator central under every parameter value. It is central only when $A\beta=b$. Under an alternative, its noncentrality is determined by the true departure.
Using ordinary coefficient distance without checking the metric. The distance relevant here is measured using $X'X$, not necessarily by summing squared coefficient differences.

Key takeaway: For a rank-$q$ hypothesis in the full-rank normal linear model, the exact statistic has a noncentral $F_{q,n-p}(\lambda)$ distribution. Its noncentrality is the squared design-metric distance from the true coefficient vector to the null set, divided by $\sigma^2$.

Check Your Understanding

Use the definitions and results above to answer these questions.

In the stated model, what assumptions ensure that $A(X'X)^{-1}A'$ is invertible?
For a rank-$q$ hypothesis, why are the numerator and denominator degrees of freedom $q$ and $n-p$, respectively?
What does the noncentrality parameter become when $A\beta=b$, and what is the resulting F distribution?
Why does the design metric, rather than ordinary Euclidean distance, appear in the distance-to-null-set result?
If the error variance doubles while the true coefficient departure and design remain fixed, how does $\lambda$ change?