Likelihood-Ratio Testing with Known Covariance

From a Gaussian Likelihood to a Mean Test

A multivariate mean can differ from a specified value in many directions. Those directions need not be equally variable: a displacement that is substantial relative to a low-variance direction may be unremarkable relative to a high-variance one. With the covariance matrix known, the likelihood-ratio test accounts for this by measuring the sample mean’s distance from the null mean using the inverse covariance matrix.

The purpose here is to derive that test from the likelihood, rather than begin with a distance statistic. The derivation shows why the sample size multiplies the squared distance, why the inverse covariance appears, and how the likelihood-ratio rejection rule becomes a chi-square-calibrated threshold.

Model and hypotheses: Let \(X_1,\ldots,X_n\) be independent random vectors in \(\mathbb{R}^p\), each with distribution \(N_p(\mu,\Sigma)\), where the covariance matrix \(\Sigma\) is known and positive definite. Write \(\bar X=n^{-1}\sum_{i=1}^n X_i\). We test the simple null hypothesis \(H_0:\mu=\mu_0\) against the unrestricted alternative \(H_1:\mu\in\mathbb{R}^p,\ \mu\ne\mu_0\).

The alternative allows any mean vector, so its likelihood is maximized over the full parameter space \(\mathbb{R}^p\). The likelihood ratio convention used here is the maximized likelihood under the null divided by the maximized likelihood under the full model. It therefore lies between zero and one, and small values count against the null.

Maximizing the Likelihood

For observed vectors \(x_1,\ldots,x_n\), the Gaussian likelihood at a candidate mean \(\mu\) is

L(\mu) =(2\pi)^{-np/2}|\Sigma|^{-n/2} \exp\left\{-\frac12\sum_{i=1}^n(x_i-\mu)^\mathsf{T}\Sigma^{-1}(x_i-\mu)\right\}.

Because \(\Sigma\) is fixed, maximizing this likelihood is equivalent to minimizing the quadratic expression in its exponent. The following decomposition identifies its minimizer and isolates the part that depends on the hypothesized mean.

Result: For every \(\mu\in\mathbb{R}^p\),

\sum_{i=1}^n(x_i-\mu)^\mathsf{T}\Sigma^{-1}(x_i-\mu) = \sum_{i=1}^n(x_i-\bar x)^\mathsf{T}\Sigma^{-1}(x_i-\bar x) +n(\bar x-\mu)^\mathsf{T}\Sigma^{-1}(\bar x-\mu).

Consequently, the unrestricted maximum-likelihood estimate is \(\hat\mu=\bar x\).

Proof. For each observation, write \(x_i-\mu=(x_i-\bar x)+(\bar x-\mu)\) and expand the quadratic form. Summing the two cross-product terms gives zero because \(\sum_{i=1}^n(x_i-\bar x)=0\). The remaining terms are the within-sample sum of quadratic forms and \(n\) copies of \((\bar x-\mu)^\mathsf{T}\Sigma^{-1}(\bar x-\mu)\), which proves the identity. Since \(\Sigma\) is positive definite, \(\Sigma^{-1}\) is positive definite too. Thus the final term is nonnegative and is zero exactly when \(\mu=\bar x\). This proves the stated maximum-likelihood estimate. \(\square\)

Under the null, the only allowed mean is \(\mu_0\). Under the unrestricted model, the maximizing value is \(\bar x\). In the likelihood ratio, the common normalizing constant and the within-sample term cancel, leaving only the difference between these two fits.

Theorem (likelihood-ratio statistic): For the hypotheses above, define

\Lambda(x)=\frac{L(\mu_0)}{\sup_{\mu\in\mathbb{R}^p}L(\mu)}. $$ Then $$ \Lambda(x)= \exp\left\{-\frac n2(\bar x-\mu_0)^\mathsf{T} \Sigma^{-1}(\bar x-\mu_0)\right\}, \qquad -2\log\Lambda(x)= n(\bar x-\mu_0)^\mathsf{T}\Sigma^{-1}(\bar x-\mu_0).

Derivation. The quadratic term in the log-likelihood at \(\mu_0\) exceeds the term at \(\bar x\) by \(n(\bar x-\mu_0)^\mathsf{T}\Sigma^{-1}(\bar x-\mu_0)\), by the decomposition just proved. The log-likelihood has coefficient \(-1/2\) on that quadratic term. Therefore \(\ell(\mu_0)-\ell(\bar x)\) equals \(-n/2\) times the displayed quadratic form. Exponentiating gives \(\Lambda(x)\); multiplying the log-ratio by \(-2\) gives the second formula. \(\square\)

The quadratic form \((\bar x-\mu_0)^\mathsf{T}\Sigma^{-1}(\bar x-\mu_0)\) is the squared Mahalanobis distance from \(\bar x\) to \(\mu_0\), using \(\Sigma\) as the covariance matrix. Thus the likelihood-ratio statistic is \(n\) times that squared distance. The factor \(n\) matters: the sample mean becomes more precise as the sample size increases, so the same displacement from \(\mu_0\) is stronger evidence when based on more observations.

Rejection Region and Calibration

The likelihood-ratio test rejects for small \(\Lambda(x)\). Since the exponential function is increasing, the formula for \(\Lambda(x)\) shows that this is equivalent to rejecting for large values of

T=n(\bar X-\mu_0)^\mathsf{T}\Sigma^{-1}(\bar X-\mu_0).

Under \(H_0\), \(\bar X\sim N_p(\mu_0,\Sigma/n)\). Equivalently, \(\sqrt n\,\Sigma^{-1/2}(\bar X-\mu_0)\) has a \(p\)-dimensional standard normal distribution, and its squared Euclidean length is \(T\). Hence \(T\sim\chi^2_p\) under the null. This is the exact known-covariance calibration.

Level-\(\alpha\) likelihood-ratio test: For \(0<\alpha<1\), let \(\chi^2_{p,1-\alpha}\) denote the \(1-\alpha\) quantile of the chi-square distribution with \(p\) degrees of freedom. Reject \(H_0\) when

T>\chi^2_{p,1-\alpha}.

Equivalently, reject when \(\Lambda(x)<\exp\{-\chi^2_{p,1-\alpha}/2\}\). Under \(H_0\), this test has rejection probability \(\alpha\).

The equivalence follows directly by taking logarithms: \(\Lambda(x)<\exp\{-c/2\}\) holds exactly when \(-2\log\Lambda(x)>c\), which is \(T>c\). The null distribution of \(T\) then gives the stated rejection probability. A corresponding p-value is the upper-tail probability \(P(\chi^2_p\ge T_{\mathrm{obs}})\).

Worked Examples

Worked Example: A Scalar Mean from the Observed Data

Suppose \(n=3\) independent observations come from \(N(\mu,1)\), and the observed values are \(1,2,3\). Test \(H_0:\mu=1\) at level \(0.05\).

The sample mean is \(\bar x=(1+2+3)/3=2\). To check the likelihood comparison directly, the sum of squared deviations at the unrestricted estimate is \( (1-2)^2+(2-2)^2+(3-2)^2=2\). Under the null it is \( (1-1)^2+(2-1)^2+(3-1)^2=5\). The null log-likelihood is therefore lower than the unrestricted log-likelihood by \(-\tfrac12(5-2)=-1.5\), so \(\Lambda=\exp(-1.5)\).

The statistic is \(T=n(\bar x-\mu_0)^2/\sigma^2=3(2-1)^2/1=3\). The \(0.95\) quantile of \(\chi^2_1\) is approximately \(3.841\). Since \(3<3.841\), do not reject at level \(0.05\). The calculation from the likelihood agrees: \(-2\log\Lambda=3\).

Worked Example: A Correlated Two-Variable Mean

Let \(n=10\), \(\mu_0=(0,0)^\mathsf{T}\), and \(\Sigma=\begin{pmatrix}4&1\\1&2\end{pmatrix}\). Suppose the observed sample mean is \(\bar x=(1,2)^\mathsf{T}\). Test at level \(0.05\).

The determinant of \(\Sigma\) is \(4\cdot2-1\cdot1=7\), so \(\Sigma^{-1}=\tfrac17\begin{pmatrix}2&-1\\-1&4\end{pmatrix}\). With \(d=\bar x-\mu_0=(1,2)^\mathsf{T}\),

d^\mathsf{T}\Sigma^{-1}d =(1,2)\frac17 \begin{pmatrix}2&-1\\-1&4\end{pmatrix} \begin{pmatrix}1\\2\end{pmatrix} =(1,2)\begin{pmatrix}0\\1\end{pmatrix} =2.

Thus \(T=10\cdot2=20\), and \(\Lambda=\exp(-20/2)=\exp(-10)\). The \(0.95\) quantile of \(\chi^2_2\) is approximately \(5.991\). Since \(20>5.991\), reject \(H_0\). The off-diagonal covariance matters: the calculation uses the full inverse matrix, not separate univariate comparisons.

Worked Example: The Same Raw Displacement Can Be Weak Evidence

Suppose \(n=4\), \(\mu_0=(0,0)^\mathsf{T}\), \(\Sigma=\begin{pmatrix}1&0\\0&9\end{pmatrix}\), and \(\bar x=(1,1)^\mathsf{T}\). At level \(0.05\), determine whether to reject.

Here \(\Sigma^{-1}=\begin{pmatrix}1&0\\0&1/9\end{pmatrix}\), so the squared Mahalanobis distance is \((1,1)\begin{pmatrix}1&0\\0&1/9\end{pmatrix}(1,1)^\mathsf{T}=1+1/9=10/9\). Therefore \(T=4(10/9)=40/9\approx4.444\), and \(\Lambda=\exp\{-T/2\}=\exp(-20/9)\).

For two degrees of freedom, the \(0.95\) critical value is approximately \(5.991\). Because \(40/9<5.991\), do not reject. The two coordinates each differ from their null values by one unit, but the second coordinate has variance nine times as large as the first. Its displacement therefore contributes only \(1/9\) to the squared Mahalanobis distance.

Common Mistakes and Why the Form Matters

The likelihood-ratio construction is useful because it translates the comparison of two fitted likelihoods into a single statistic with a known null distribution. It also makes clear what is being held fixed: the covariance matrix is known, positive definite, and common to the null and alternative. Replacing it with an estimated covariance changes the likelihood optimization and the calibration; the result derived here does not apply unchanged.

Using \(\Sigma\) rather than \(\Sigma^{-1}\). The quadratic form must standardize displacements by their covariance. Directions with greater variance should contribute less, not more, to the statistic.
Omitting \(n\). The statistic is \(n\) times the squared distance of the sample mean, not merely the squared distance. The covariance of the sample mean is \(\Sigma/n\).
Reversing the likelihood ratio. Here \(\Lambda\) is null likelihood divided by maximized unrestricted likelihood. Small \(\Lambda\), or equivalently large \(-2\log\Lambda\), is evidence against the null.
Using the wrong degrees of freedom. The chi-square degrees of freedom are \(p\), the number of unrestricted mean coordinates tested, not the sample size \(n\).

A common but incorrect argument is: “The observed mean is one unit from the null in each coordinate, so the evidence is determined by the ordinary squared distance \(1^2+1^2\).” That ignores the scale and dependence encoded in \(\Sigma\). The correction is to calculate \((\bar x-\mu_0)^\mathsf{T}\Sigma^{-1}(\bar x-\mu_0)\), multiply by \(n\), and compare the result with the \(p\)-degree-of-freedom chi-square critical value.

Key takeaway: In the known-covariance Gaussian model, the likelihood-ratio statistic for testing a specified mean is exactly \(n\) times the squared Mahalanobis distance between the sample mean and the null mean. Its null distribution is \(\chi^2_p\), so the level-\(\alpha\) test rejects above the corresponding chi-square quantile.

Check Your Understanding

Use the likelihood-ratio definition and the known-covariance test derived above.

Why is the unrestricted maximum-likelihood estimate of \(\mu\) equal to \(\bar x\) when \(\Sigma\) is positive definite?
For \(p=3\), \(n=5\), and squared Mahalanobis distance \(1.2\), what is the likelihood-ratio statistic?
State the level-\(\alpha\) rejection rule in terms of \(T\), and identify its null degrees of freedom.
Explain why a displacement in a high-variance direction generally contributes less to the test statistic than the same-sized displacement in a low-variance direction.
What error is made by rejecting for large values of \(\Lambda\) under the likelihood-ratio convention used here?