Mahalanobis Tests with Known Covariance

Why a Mean Test Needs a Covariance-Adjusted Distance

In a one-sample multivariate problem, the estimate of a mean is a vector. A difference of $(1,1)$ from a specified mean does not have one universal interpretation: its components may have different variabilities, and the components may be correlated. A useful test statistic must account for both features. The known-covariance quadratic statistic does this by measuring the discrepancy in units determined by the covariance of the sample mean.

The statistic is a measure of discrepancy, not by itself a complete testing procedure. To carry out a test at a specified significance level, one also needs a critical value with the appropriate null calibration. Here the focus is how to construct the statistic and what its value means.

Model, Hypothesis, and Statistic

Definition: Let $X_1,\ldots,X_n$ be independent random vectors with $X_i\sim N_p(\mu,\Sigma)$, where the $p\times p$ covariance matrix $\Sigma$ is known and positive definite. Write $\bar X=n^{-1}\sum_{i=1}^n X_i$. For testing $H_0:\mu=\mu_0$, where $\mu_0$ is specified, the known-covariance quadratic statistic is $$ Q=n(\bar X-\mu_0)^\mathsf{T}\Sigma^{-1}(\bar X-\mu_0). $$

Positive definiteness ensures that $\Sigma^{-1}$ exists and that the quadratic form is nonnegative. The statistic is large when the observed sample mean is far from $\mu_0$ relative to the variability described by $\Sigma$. A test with a chosen critical value $c$ rejects $H_0$ when $Q>c$. The value of $c$ must come from the test’s null calibration; the formula for $Q$ alone does not supply it.

The factor $n$ is essential. Under the model, the sample mean has covariance matrix $\Sigma/n$. Thus the same discrepancy in the sample mean is more striking when it is based on more observations, because the mean is then less variable. Equivalently, the statistic uses the inverse covariance of the sample mean:

Q=(\bar X-\mu_0)^\mathsf{T}(\Sigma/n)^{-1}(\bar X-\mu_0).

This equality follows from $(\Sigma/n)^{-1}=n\Sigma^{-1}$. It also helps keep the roles of the two covariance matrices clear: $\Sigma$ describes a single observation, while $\Sigma/n$ describes the sample mean.

Worked Example: Different Variances in Two Coordinates

Suppose $n=25$, $\mu_0=(0,0)^\mathsf{T}$, the known covariance is $\Sigma=\begin{pmatrix}4&0\\0&1\end{pmatrix}$, and the observed sample mean is $\bar x=(1,0.4)^\mathsf{T}$. Calculate $Q$ and interpret the result.

The mean discrepancy and inverse covariance are $$ d=\bar x-\mu_0=\begin{pmatrix}1\\0.4\end{pmatrix}, \qquad \Sigma^{-1}=\begin{pmatrix}1/4&0\\0&1\end{pmatrix}. $$

Therefore, $$ d^\mathsf{T}\Sigma^{-1}d = \begin{pmatrix}1&0.4\end{pmatrix} \begin{pmatrix}1/4&0\\0&1\end{pmatrix} \begin{pmatrix}1\\0.4\end{pmatrix} =\frac14+0.16=0.41, $$ and $$ Q=25(0.41)=10.25. $$ The first coordinate has variance $4$, so a discrepancy of $1$ is less than one standard deviation for a single observation. The second has variance $1$, so a discrepancy of $0.4$ is smaller still. The sample-size-scaled, covariance-standardized displacement has length $\sqrt{10.25}$, approximately $3.20$. Whether that value crosses a test threshold depends on the null calibration and chosen level.

Deriving the Standardized-Displacement Interpretation

A covariance matrix defines both the scales of individual coordinates and how their fluctuations move together. To see how the statistic uses this information, standardize the discrepancy in all directions at once. Since $\Sigma$ is positive definite, it has a symmetric positive-definite square root $\Sigma^{1/2}$ and inverse square root $\Sigma^{-1/2}$.

Result: Under $H_0$, define the standardized sample-mean discrepancy $$ Z=\sqrt n\,\Sigma^{-1/2}(\bar X-\mu_0). $$ Then $Q=Z^\mathsf{T}Z=\lVert Z\rVert^2$. Thus $Q$ is the squared Euclidean length of the discrepancy after covariance standardization and scaling for the sample size.

To verify the result, the model gives $E[\bar X]=\mu$ and $\operatorname{Cov}(\bar X)=\Sigma/n$. Under $H_0$, $E[\bar X-\mu_0]=0$. Consequently, $$ \operatorname{Cov}(Z) =n\Sigma^{-1/2}\frac{\Sigma}{n}\Sigma^{-1/2}=I_p. $$ For the quadratic form itself, use the symmetry of $\Sigma^{-1/2}$: $$ Z^\mathsf{T}Z =n(\bar X-\mu_0)^\mathsf{T} \Sigma^{-1/2}\Sigma^{-1/2}(\bar X-\mu_0) =n(\bar X-\mu_0)^\mathsf{T}\Sigma^{-1}(\bar X-\mu_0)=Q. $$ So the statistic adds the squared coordinates of a jointly whitened discrepancy; it does not simply add squared raw differences. $\square$

Under the Gaussian model, $Z$ is a standard multivariate normal vector under the null. That fact is useful for calibrating a test, but obtaining the reference distribution and critical value is a separate step. The geometric interpretation above follows directly from standardization and does not depend on choosing a test level.

Worked Example: Correlation Changes the Measured Distance

Let $n=16$, $\mu_0=(0,0)^\mathsf{T}$, and $$ \Sigma=\begin{pmatrix}4&2\\2&2\end{pmatrix}, \qquad \bar x=\begin{pmatrix}2\\1\end{pmatrix}. $$ Calculate $Q$, and compare its distance interpretation with the raw Euclidean distance.

First, $\det(\Sigma)=4(2)-2(2)=4>0$, and the diagonal entries are positive, so this symmetric matrix is positive definite. Its inverse is $$ \Sigma^{-1} =\frac14\begin{pmatrix}2&-2\\-2&4\end{pmatrix} =\begin{pmatrix}1/2&-1/2\\-1/2&1\end{pmatrix}. $$ With $d=(2,1)^\mathsf{T}$, $$ \Sigma^{-1}d =\begin{pmatrix}1/2&-1/2\\-1/2&1\end{pmatrix} \begin{pmatrix}2\\1\end{pmatrix} =\begin{pmatrix}1/2\\0\end{pmatrix}, \qquad d^\mathsf{T}\Sigma^{-1}d =\begin{pmatrix}2&1\end{pmatrix}\begin{pmatrix}1/2\\0\end{pmatrix}=1. $$ Thus $Q=16(1)=16$, and the standardized sample-mean displacement has length $4$.

The raw Euclidean length of $d$ is $\sqrt{2^2+1^2}=\sqrt5$, but that length ignores covariance. Here the discrepancy points along $(2,1)^\mathsf{T}$, a direction in which the covariance has relatively large variation. The quadratic statistic measures that observed displacement against the covariance structure rather than treating all directions as equally variable.

Eigen-Directions Explain the Weighting

The geometry can be made more explicit by expressing the discrepancy along the principal directions of $\Sigma$. Let $u_1,\ldots,u_p$ be an orthonormal eigenbasis, with positive eigenvalues $\lambda_1,\ldots,\lambda_p$. Write $d=\bar X-\mu_0=\sum_{j=1}^p a_j u_j$, where $a_j=u_j^\mathsf{T}d$ is the discrepancy along direction $u_j$.

Result: The quadratic statistic decomposes as $$ Q=n\sum_{j=1}^p\frac{a_j^2}{\lambda_j}. $$ A given squared displacement contributes less in a direction with larger variance and more in a direction with smaller variance.

For a complete derivation, the spectral decomposition of the symmetric positive-definite matrix is $\Sigma=\sum_{j=1}^p\lambda_j u_j u_j^\mathsf{T}$, so $\Sigma^{-1}=\sum_{j=1}^p\lambda_j^{-1}u_j u_j^\mathsf{T}$. Substituting $d=\sum_k a_k u_k$ into the quadratic form gives $$ d^\mathsf{T}\Sigma^{-1}d =\sum_{j=1}^p\sum_{k=1}^p\sum_{\ell=1}^p \frac{a_k a_\ell}{\lambda_j} (u_k^\mathsf{T}u_j)(u_j^\mathsf{T}u_\ell) =\sum_{j=1}^p\frac{a_j^2}{\lambda_j}, $$ because orthonormality makes each inner product zero unless its indices agree. Multiplication by $n$ yields the stated expression. $\square$

This decomposition also shows why standardizing each original coordinate separately can be wrong when covariance is not diagonal. Correlated coordinates do not generally align with the eigen-directions. The inverse covariance accounts for their joint structure, whereas a calculation that only divides each coordinate by its own standard deviation discards the off-diagonal covariances.

Worked Example: The Sample-Size Factor

Suppose the known covariance is $I_2$, the specified mean is $(0,0)^\mathsf{T}$, and an observed sample mean is $(0.5,0.5)^\mathsf{T}$. Compare the statistic for $n=4$ and $n=16$, assuming the same observed discrepancy in each case.

Since $\Sigma^{-1}=I_2$, the unscaled quadratic discrepancy is $$ d^\mathsf{T}\Sigma^{-1}d =(0.5)^2+(0.5)^2=0.25+0.25=0.5. $$ For $n=4$, $Q=4(0.5)=2$, and the standardized length is $\sqrt2$. For $n=16$, $Q=16(0.5)=8$, and the standardized length is $\sqrt8$, approximately $2.83$. The raw difference has not changed; the larger sample makes that same difference larger relative to the sampling variability of the mean.

This comparison explains why omitting $n$ changes the inferential meaning of the statistic. The unscaled quadratic form describes a distance relative to the covariance of one observation. The test statistic describes the corresponding distance for the sample mean.

From Statistic to Test, and Common Pitfalls

For a specified mean hypothesis, the calculation proceeds in a fixed order: form the observed mean discrepancy, weight it using the inverse of the known covariance, and scale for the sample size. The result is one nonnegative number summarizing the discrepancy across all $p$ coordinates. A test then compares that number with a critical value chosen for the desired null error rate. The statistic alone does not determine whether a particular value is significant.

Form the discrepancy.
Compute $d=\bar x-\mu_0$, keeping the ordering of the coordinates consistent with $\Sigma$.

Apply covariance weighting.
Compute $d^\mathsf{T}\Sigma^{-1}d$, using the known covariance matrix, including its off-diagonal entries.

Scale for the sample mean.
Multiply by $n$ to obtain $Q$, then compare with a valid critical value if conducting a calibrated test.

A frequent mistake is to use $d^\mathsf{T}\Sigma d$ instead of $d^\mathsf{T}\Sigma^{-1}d$. The inverse is needed because larger variance should reduce the weight assigned to a fixed displacement; using $\Sigma$ would do the opposite in a high-variance direction. Another mistake is to use the known covariance of an observation as if it were the covariance of the sample mean, thereby leaving out the factor $n$. Finally, replacing the known covariance with a sample covariance changes the procedure and its calibration; it is not a harmless computational substitution.

Key takeaway: $Q=n(\bar X-\mu_0)^\mathsf{T}\Sigma^{-1}(\bar X-\mu_0)$ is the squared length of the sample-mean discrepancy after covariance standardization. It adjusts for scale, correlation, and sample size. Its numerical value measures displacement; a calibrated threshold is still needed to make a level-controlled decision.

Check Your Understanding

Use the definitions and derivations above to answer these questions.

Why does the known-covariance statistic use $\Sigma^{-1}$ rather than $\Sigma$?
If the covariance of each observation is $\Sigma$, what is the covariance of the sample mean under independence?
What does $\sqrt Q$ represent in terms of a standardized sample-mean discrepancy?
How does a displacement’s contribution change when it lies along a covariance eigen-direction with a larger eigenvalue?
Why does computing $Q$ not, by itself, establish a significance level for a test?