Noncentrality and the Geometry of Hypothesis Spaces

Why Geometry Matters for Power

A multi-parameter test can reject only when the data provide evidence in directions represented by its null hypothesis. An alternative may be large in some sense yet have little effect on the test if most of that effect lies outside those directions. The noncentrality parameter makes this precise: it measures the alternative's squared distance from the null hypothesis space after scaling for the covariance of the estimator.

We work with the normal linear model $Y\sim N(X\beta,\sigma^2 I_n)$, where $Y$ is an $n$-vector, $X$ is a fixed $n\times p$ matrix of rank $p$, $\beta$ is a $p$-vector, and $\sigma^2>0$. The hypothesis is $H_0:C\beta=d$, where $C$ is a $q\times p$ matrix of rank $q\le p$, and $d$ is a specified $q$-vector. Assume $n>p$ when using an estimated residual variance.

Definition: In the normal linear model, the noncentrality parameter for testing $H_0:C\beta=d$ is $$ \lambda=\frac{(C\beta-d)'[C(X'X)^{-1}C']^{-1}(C\beta-d)}{\sigma^2}. $$ It is the squared distance of the true mean from the null hypothesis space, measured in units determined by the model covariance.

The matrix $C(X'X)^{-1}C'$ is the covariance matrix of $C\hat{\beta}$, apart from the factor $\sigma^2$. Thus $\lambda$ does not simply measure the raw size of $C\beta-d$. It measures that departure relative to its sampling variability. The geometry behind this scaling will also show why an effect in an untested direction contributes nothing.

Standardized Coordinates and the Tested Directions

The coordinates $\beta$ are not generally on an equal geometric scale: the design matrix determines how changes in different coefficients affect the fitted mean. To account for this, define $\gamma=(X'X)^{1/2}\beta$, where $(X'X)^{1/2}$ is the positive-definite square root, and define $A=C(X'X)^{-1/2}$. Then $C\beta=A\gamma$, while the least-squares estimator $\hat{\gamma}=(X'X)^{1/2}\hat{\beta}$ has covariance $\sigma^2 I_p$.

The null hypothesis becomes $A\gamma=d$. The row space of $A$ is the subspace of standardized parameter coordinates that the hypothesis tests. A change in $\gamma$ orthogonal to that row space cannot change $A\gamma$, and so cannot move the alternative farther from or closer to the null.

Let $P=A'(AA')^{-1}A$. Since $A$ has rank $q$, $P$ is the orthogonal projection onto the row space of $A$. The vector $A^+d=A'(AA')^{-1}d$ is in that row space and satisfies $A(A^+d)=d$. It is the shortest vector satisfying the null constraint in standardized coordinates.

Geometric identity: For every $\beta$, $$ \lambda =\frac{\|P\gamma-A^+d\|^2}{\sigma^2} =\frac{\displaystyle\min_{\beta_0:\,C\beta_0=d}\|X(\beta-\beta_0)\|^2}{\sigma^2}. $$ Thus the test's noncentrality is the squared distance from the true mean $X\beta$ to the set of null means, divided by $\sigma^2$.

Deriving the Geometric Identity and the Test Distribution

The first equality follows directly from the definition of $P$. Because $A^+d$ lies in the row space of $A$, and $A\gamma-d=A(P\gamma-A^+d)$, we have

\begin{aligned} \|P\gamma-A^+d\|^2 &=(A\gamma-d)'(AA')^{-1}A\gamma-d\\ &=(A\gamma-d)'(AA')^{-1}(A\gamma-d). \end{aligned}

In the first line, the middle expression is understood as $(A\gamma-d)'(AA')^{-1}(A\gamma-d)$: applying $A^+$ to $A\gamma-d$ gives $A'(AA')^{-1}(A\gamma-d)$, whose squared norm is exactly this quadratic form. Since $AA'=C(X'X)^{-1}C'$ and $A\gamma=C\beta$, division by $\sigma^2$ gives $\lambda$.

For the second equality, every null parameter $\beta_0$ corresponds to $\gamma_0=(X'X)^{1/2}\beta_0$ satisfying $A\gamma_0=d$. The closest such $\gamma_0$ to $\gamma$ is $P\gamma+A^+d-P\gamma$ more simply written as $\gamma_0=(I_p-P)\gamma+A^+d$. Indeed, $A(I_p-P)=0$, so $A\gamma_0=d$, and $\gamma-\gamma_0=P\gamma-A^+d$. Any other solution differs from this one by a vector in the null space of $A$, which is orthogonal to the row space; the squared distance therefore cannot be smaller. Finally, $\|X(\beta-\beta_0)\|^2=(\beta-\beta_0)'X'X(\beta-\beta_0)=\|\gamma-\gamma_0\|^2$. This proves the identity. $\square$

The same projection representation gives the exact sampling distribution. Under the model, $\hat{\gamma}\sim N(\gamma,\sigma^2 I_p)$. The test's quadratic form is $Q=(C\hat{\beta}-d)'[C(X'X)^{-1}C']^{-1}(C\hat{\beta}-d)$. As above, $Q=\|P\hat{\gamma}-A^+d\|^2$. The projection $P$ has rank $q$, so $P\hat{\gamma}$, expressed in any orthonormal basis of its range, has $q$ independent normal coordinates with variance $\sigma^2$. Their squared sum divided by $\sigma^2$ is a noncentral chi-square variable.

Theorem: Under the stated normal linear model and rank conditions, $$ \frac{Q}{\sigma^2}\sim\chi^2_q(\lambda), \qquad \lambda=\frac{\|P\gamma-A^+d\|^2}{\sigma^2}. $$ If $s^2=\|Y-X\hat{\beta}\|^2/(n-p)$, then $$ \frac{Q/q}{s^2}\sim F_{q,n-p}(\lambda). $$

To justify the estimated-variance statement, write $W=X(X'X)^{-1/2}$, so $W'W=I_p$, and $X\hat{\beta}=WW'Y$. The numerator depends on $W'Y$; the residual vector is $(I_n-WW')Y=(I_n-WW')\varepsilon$. These are orthogonal projections of a multivariate normal error vector, so they are independent. The residual sum of squares divided by $\sigma^2$ is therefore an independent central $\chi^2_{n-p}$ variable. Taking the ratio of the noncentral chi-square numerator per $q$ to this independent residual mean square gives the stated noncentral $F$ distribution. $\square$

Worked Examples

Worked Example: An Effect Outside the Tested Coordinates

Suppose $X=I_4$, $\sigma^2=4$, and the mean vector is $\beta=(2,-1,5,0)'$. Test $H_0:\beta_1=\beta_2=0$, so $C$ selects the first two coordinates and $d=(0,0)'$.

Here $X'X=I_4$, and $C(X'X)^{-1}C'=I_2$. Therefore $$ \lambda=\frac{(2,-1) \begin{pmatrix}1&0\\0&1\end{pmatrix} \binom{2}{-1}}{4} =\frac{2^2+(-1)^2}{4} =\frac54. $$

The components $\beta_3=5$ and $\beta_4=0$ do not enter this calculation: they lie outside the tested coordinates. Changing $\beta_3$, even by a large amount, leaves the noncentrality unchanged. The test is informative about the first two coordinates, not about every possible departure of the mean from zero.

Worked Example: Covariance Scaling with Nonorthogonal Design Columns

Let $X=\begin{pmatrix}1&1\\0&1\\0&0\end{pmatrix}$, so its columns are $x_1=(1,0,0)'$ and $x_2=(1,1,0)'$. Let $\sigma^2=4$, take the true coefficient vector $\beta=(3,5)'$, and test $H_0:\beta_1=0$.

The columns are not orthogonal. Direct calculation gives $$ X'X=\begin{pmatrix}1&1\\1&2\end{pmatrix}, \qquad (X'X)^{-1}=\begin{pmatrix}2&-1\\-1&1\end{pmatrix}. $$ With $C=(1,0)$ and $d=0$, we obtain $$ C(X'X)^{-1}C'=2, \qquad C\beta-d=3, \qquad \lambda=\frac{3^2}{2\cdot4}=\frac98. $$

This value can also be checked as a distance in observation space. The true mean is $X\beta=3x_1+5x_2$. Under the null, a fitted mean has the form $b x_2$, because $\beta_1=0$. The squared distance to the closest such mean is $$ \min_b\|3x_1+(5-b)x_2\|^2 =\min_t\|3x_1+t x_2\|^2. $$ Since $\|x_1\|^2=1$, $x_1'x_2=1$, and $\|x_2\|^2=2$, the expression to minimize is $9+6t+2t^2$. Its minimum occurs at $t=-3/2$ and equals $9-9+9/2=9/2$. Dividing by $\sigma^2=4$ gives $\lambda=9/8$, as before. The coefficient departure of 3 is not itself the geometric distance: the design makes part of the $x_1$ effect align with the nuisance direction $x_2$.

Worked Example: A Nonzero Null Space

Let $X=I_3$, $\sigma^2=4$, and $\beta=(5,1,7)'$. Test $H_0:\beta_1=2,\ \beta_2=-1$, leaving $\beta_3$ unrestricted. Here $C=\begin{pmatrix}1&0&0\\0&1&0\end{pmatrix}$ and $d=(2,-1)'$.

Since $C(X'X)^{-1}C'=I_2$, the departure from the restrictions is $C\beta-d=(5-2,\ 1-(-1))'=(3,2)'$. Thus $$ \lambda=\frac{(3,2)(3,2)'}{4} =\frac{9+4}{4} =\frac{13}{4}. $$

The nearest null mean is $(2,-1,7)'$: the first two coordinates are moved to their required values, while the third is left unchanged because the null places no restriction on it. The squared distance is $3^2+2^2=13$, confirming the result after division by 4. This example also shows why nonzero $d$ causes no difficulty: the relevant displacement is from the null value, not from the origin.

What the Geometry Tells Us

The noncentrality parameter is zero exactly when $C\beta=d$, that is, when the true parameter satisfies the null. Otherwise it is positive. For a fixed model and variance, alternatives farther from the null space have larger $\lambda$, and the noncentral $F$ distribution shifts toward larger values. This is the geometric basis for how the noncentrality parameter enters exact power calculations.

The relevant distance is not an unscaled Euclidean distance between coefficient vectors. The design matrix sets the metric: a coefficient change that barely changes the fitted mean may be less detectable than an equally large raw change in a direction that substantially changes it. Equivalently, the covariance matrix of the tested estimator determines how departures are scaled.

Common Mistake

A frequent error is to use $(C\beta-d)'(C\beta-d)/\sigma^2$ as the noncentrality parameter. That ignores the covariance of $C\hat{\beta}$, and is correct only in special cases where $C(X'X)^{-1}C'=I_q$. The correction is to use its inverse in the quadratic form. For example, in the nonorthogonal-design example, the unscaled calculation would give $9/4$, but the correct value is $9/8$. The difference reflects that the tested coefficient estimate has variance $2\sigma^2$, not $\sigma^2$.

Key takeaway: Noncentrality is the squared distance from the true mean to the null hypothesis space, divided by the error variance. Only the projection onto the tested directions contributes; effects in unrestricted directions do not.

Check Your Understanding

Use the geometry and the covariance-scaled formula to answer these questions.

In standardized coordinates, what subspace does $P=A'(AA')^{-1}A$ project onto?
Why can changing a parameter component orthogonal to the row space of $A$ leave $\lambda$ unchanged?
For $X=I_2$, $\sigma^2=9$, and the test $H_0:\beta_1=0$, what is $\lambda$ when $\beta=(6,100)'$?
Why is the inverse of $C(X'X)^{-1}C'$ needed in the noncentrality formula?
In the nonorthogonal-design example, what is the squared distance from the true mean to the null mean space before division by $\sigma^2$?