Noncentrality and Covariance-Standardized Mean Effects

Why Standardize a Mean Difference?

A mean difference of two units can be substantial in a variable with little variability and modest in a variable with large variability. In several dimensions, the same issue is more complicated: the variables may have different scales and may be correlated. A useful measure of separation must account for both features. The Mahalanobis distance does this by measuring a mean difference in units determined by the covariance matrix.

For the one-sample Gaussian mean problem with known covariance, this distance also has a direct inferential role. Under a fixed alternative, the known-covariance quadratic statistic has a noncentral chi-square distribution. Its noncentrality parameter is the sample size multiplied by the squared Mahalanobis distance between the alternative mean and the null mean.

Definition: Let $\Sigma$ be a positive-definite covariance matrix, and let $\mu_0$ and $\mu$ be two $p$-dimensional mean vectors. The squared Mahalanobis distance from $\mu_0$ to $\mu$, measured using $\Sigma$, is $$ \Delta^2=(\mu-\mu_0)^\mathsf{T}\Sigma^{-1}(\mu-\mu_0). $$ The Mahalanobis distance is $\Delta=\sqrt{\Delta^2}$. For a sample of size $n$, the corresponding noncentrality parameter is $\lambda=n\Delta^2$.

The covariance matrix is part of the definition: the same pair of mean vectors can have different Mahalanobis distances under different covariance matrices. The squared distance is dimensionless. It measures the size of the mean displacement after accounting for the covariance structure, rather than measuring displacement in the original units.

Deriving the Noncentral Chi-Square Law

Assume $X_1,\ldots,X_n$ are independent and identically distributed $p$-dimensional normal random vectors with mean $\mu$ and known positive-definite covariance matrix $\Sigma$. Consider testing $H_0:\mu=\mu_0$ with the known-covariance statistic $$ Q=n(\bar X-\mu_0)^\mathsf{T}\Sigma^{-1}(\bar X-\mu_0), $$ where $\bar X=n^{-1}\sum_{i=1}^n X_i$. The sampling law of the sample mean gives $\bar X\sim N_p(\mu,\Sigma/n)$. The following derivation identifies the distribution of $Q$ when the true mean is $\mu$, including when $\mu\ne\mu_0$.

Theorem: Under the assumptions above, $Q$ has a noncentral chi-square distribution with $p$ degrees of freedom and noncentrality parameter $$ \lambda=n(\mu-\mu_0)^\mathsf{T}\Sigma^{-1}(\mu-\mu_0)=n\Delta^2. $$ When $\mu=\mu_0$, $\lambda=0$, and this reduces to the central chi-square distribution with $p$ degrees of freedom.

Standardize the sample mean.
Because $\Sigma$ is positive definite, it has a symmetric inverse square root $\Sigma^{-1/2}$. Define $Z=\sqrt n\,\Sigma^{-1/2}(\bar X-\mu_0)$. Its covariance is $I_p$, and its mean is $a=\sqrt n\,\Sigma^{-1/2}(\mu-\mu_0)$.

Express the statistic as a squared length.
Using $\Sigma^{-1/2}\Sigma^{-1/2}=\Sigma^{-1}$, we obtain $Q=Z^\mathsf{T}Z$.

Identify the noncentrality.
The vector $Z$ is normal with identity covariance, so $Q$ is the squared length of a standard-covariance normal vector with mean $a$. Its noncentrality is $a^\mathsf{T}a=n(\mu-\mu_0)^\mathsf{T}\Sigma^{-1}(\mu-\mu_0)$.

For completeness, a noncentral chi-square variable with $p$ degrees of freedom and parameter $\lambda$ is the sum of the squares of $p$ independent normal variables with variance one whose means have squared sum $\lambda$. To verify that the squared length above has this form, take an orthogonal matrix $U$ that rotates $a$ to $(\sqrt{a^\mathsf{T}a},0,\ldots,0)^\mathsf{T}$. The vector $UZ$ is still normal with identity covariance, since $U I_p U^\mathsf{T}=I_p$, and its squared length equals $Z^\mathsf{T}Z$, since $U^\mathsf{T}U=I_p$. Thus the rotated coordinates are independent unit-variance normal variables, and their means have squared sum $a^\mathsf{T}a=\lambda$. This proves the theorem. $\square$

This derivation also shows why the parameter contains the sample size. Averaging $n$ independent observations reduces the covariance of the sample mean from $\Sigma$ to $\Sigma/n$. Multiplying the standardized displacement by $\sqrt n$ therefore multiplies its squared length by $n$. The mean difference stays fixed under a fixed alternative, but its separation relative to the sampling variability grows with the sample size.

What the Noncentrality Measures

The value $\Delta$ describes the alternative mean’s distance from the null in covariance-standardized units. The value $\lambda=n\Delta^2$ describes that squared separation on the scale of the statistic’s sampling distribution. These are related but not interchangeable quantities: $\Delta$ is a distance, $\Delta^2$ is a squared distance, and $\lambda$ also incorporates sample size.

A useful check follows directly from the theorem. Under the alternative, write $Z_j=a_j+\varepsilon_j$, where the $\varepsilon_j$ are independent $N(0,1)$ variables. Since $E[Z_j^2]=1+a_j^2$, $$ E[Q]=\sum_{j=1}^p(1+a_j^2)=p+\lambda. $$ Thus increasing $\lambda$ shifts the mean of the statistic upward. More precisely, $\operatorname{Var}(Z_j^2)=2+4a_j^2$, so independence gives $\operatorname{Var}(Q)=2p+4\lambda$. These moments describe the distribution; they do not by themselves give a rejection probability, which also depends on the test’s critical region.

Worked Example: A Correlated Two-Variable Mean Difference

Suppose the null mean is $\mu_0=(0,0)^\mathsf{T}$, the alternative mean is $\mu=(2,3)^\mathsf{T}$, and the known covariance is $$ \Sigma=\begin{pmatrix}4&2\\2&9\end{pmatrix}. $$ Find the squared Mahalanobis distance and the noncentrality parameter for $n=12$.

The determinant of $\Sigma$ is $4(9)-2(2)=32$, which is positive; its diagonal entries are positive, so this symmetric matrix is positive definite. Its inverse is $$ \Sigma^{-1}=\frac{1}{32}\begin{pmatrix}9&-2\\-2&4\end{pmatrix}. $$ The mean displacement is $d=\mu-\mu_0=(2,3)^\mathsf{T}$. First multiply: $$ \Sigma^{-1}d=\frac{1}{32}\begin{pmatrix}9(2)-2(3)\\-2(2)+4(3)\end{pmatrix} =\begin{pmatrix}3/8\\1/4\end{pmatrix}. $$ Then $$ \Delta^2=d^\mathsf{T}\Sigma^{-1}d =2(3/8)+3(1/4)=3/4+3/4=3/2. $$ Therefore $\Delta=\sqrt{3/2}$, and for $n=12$, $$ \lambda=12(3/2)=18. $$ The test statistic under this alternative has a noncentral chi-square distribution with two degrees of freedom and noncentrality 18.

Worked Example: Equal Standardized Effects, Different Raw Differences

Let $\Sigma=\operatorname{diag}(1,9)$ and $\mu_0=(0,0)^\mathsf{T}$. Compare the alternatives $\mu_A=(2,0)^\mathsf{T}$ and $\mu_B=(0,6)^\mathsf{T}$. The raw Euclidean lengths of their displacements are 2 and 6, respectively. Their squared Mahalanobis distances are $$ \Delta_A^2=2^2/1+0^2/9=4,\qquad \Delta_B^2=0^2/1+6^2/9=4. $$ Thus both alternatives are equally far from the null in covariance-standardized units, even though the raw displacements differ. For $n=25$, each has noncentrality $$ \lambda=25(4)=100. $$ This equality occurs because the second coordinate has variance nine times the first coordinate’s variance: a displacement of six in that coordinate is scaled by its larger variability.

Worked Example: Correlation Changes the Effect of Direction

Take $$ \Sigma=\begin{pmatrix}1&0.8\\0.8&1\end{pmatrix}. $$ Its eigenvectors point along $(1,1)^\mathsf{T}$ and $(1,-1)^\mathsf{T}$, with corresponding eigenvalues $1.8$ and $0.2$. A displacement along an eigenvector with eigenvalue $v$ has squared Mahalanobis distance equal to its squared Euclidean length divided by $v$.

For $d_+=(1,1)^\mathsf{T}$, the squared Euclidean length is 2 and the eigenvalue is 1.8, so $$ \Delta_+^2=2/1.8=10/9. $$ For $d_-=(1,-1)^\mathsf{T}$, the squared Euclidean length is also 2, but its eigenvalue is 0.2, so $$ \Delta_-^2=2/0.2=10. $$ At $n=9$, these alternatives have noncentralities $\lambda_+=9(10/9)=10$ and $\lambda_-=9(10)=90$. The equal-length displacements are not equally distinguishable on the covariance-standardized scale: the second points along a direction with much less variability.

Scaling and Interpretation

For a fixed covariance matrix and a fixed alternative mean, the noncentrality is linear in sample size. If the sample size is multiplied by a positive factor $c$, then $\lambda$ is also multiplied by $c$, while $\Delta$ and $\Delta^2$ remain unchanged. Conversely, at fixed sample size, changing the alternative’s direction or the covariance structure can change the standardized effect even if the raw displacement length stays the same.

The Mahalanobis metric is also unchanged by an invertible linear change of coordinates applied consistently to the mean vectors and covariance matrix. If $Y=AX$ for an invertible matrix $A$, then the displacement becomes $Ad$ and the covariance becomes $A\Sigma A^\mathsf{T}$. Since $$ (A\Sigma A^\mathsf{T})^{-1}=(A^\mathsf{T})^{-1}\Sigma^{-1}A^{-1}, $$ we have $$ (Ad)^\mathsf{T}(A\Sigma A^\mathsf{T})^{-1}(Ad) =d^\mathsf{T}\Sigma^{-1}d. $$ Thus changing units or applying an invertible re-expression of the variables does not change the standardized effect, provided the covariance is transformed along with the mean.

Common Mistakes

Using raw distance in place of standardized distance. The reasoning “this alternative is twice as far because its Euclidean length is twice as large” ignores scale and correlation. The correction is to calculate $d^\mathsf{T}\Sigma^{-1}d$, using the covariance for the variables being analyzed.
Confusing $\Delta$, $\Delta^2$, and $\lambda$. They are respectively the distance, its square, and the square multiplied by sample size. The noncentral chi-square parameter is $\lambda=n\Delta^2$, not $n\Delta$.
Dropping the sample size from the statistic’s distribution. The alternative mean itself does not depend on $n$, but the sample mean’s covariance is $\Sigma/n$. Consequently, the noncentrality grows linearly with $n$ for a fixed alternative.
Applying the result when its assumptions do not hold. The exact derivation here uses independent Gaussian observations and a known positive-definite covariance matrix. Replacing $\Sigma$ by an estimate changes the statistic’s finite-sample distribution.

Key takeaway: In the known-covariance Gaussian mean problem, the alternative’s squared covariance-standardized effect is $\Delta^2=(\mu-\mu_0)^\mathsf{T}\Sigma^{-1}(\mu-\mu_0)$, and the test statistic’s noncentrality is exactly $n\Delta^2$. Covariance determines how direction and scale matter; sample size multiplies the resulting squared separation.

Check Your Understanding

Use the definitions and derivation above to answer the following questions.

What is the difference between the Mahalanobis distance and the noncentrality parameter?
For fixed $\mu$, $\mu_0$, and $\Sigma$, how does the noncentrality change if the sample size is tripled?
Why can two mean displacements with equal Euclidean length have different squared Mahalanobis distances?
In the two-variable example with covariance $\operatorname{diag}(1,9)$, why do the displacements $(2,0)^\mathsf{T}$ and $(0,6)^\mathsf{T}$ have equal squared Mahalanobis distance?
Name the distribution of the known-covariance statistic under a fixed alternative, and give its degrees of freedom and noncentrality parameter.