Likelihood-Ratio Test Of A Specified Covariance

Why a Specified-Covariance Test Has Two Matrix Terms

A covariance matrix describes both the scale of variation in each coordinate and the dependence between coordinates. Testing a proposed matrix therefore asks whether the observed multivariate scatter is compatible with all of those features at once. In a normal model, the likelihood-ratio test compares the likelihood at the specified covariance with the largest likelihood available when the covariance is unrestricted.

This tutorial treats the mean vector as known. That condition determines which deviations enter the likelihood: each observation is centered at the known mean, not at a mean estimated from the sample. The test statistic has a determinant term, reflecting overall volume, and a trace term, reflecting the scatter measured relative to the proposed covariance. Its calibration—the choice of a rejection threshold—is a separate question.

Model, Likelihood, and Regularity Conditions

Write the known-mean scatter matrix and its scaled version as

The matrix \(\widehat{\Sigma}\) is the maximum-likelihood estimator under the unrestricted alternative when \(A\) is positive definite. Its denominator is \(n\), not \(n-1\). Here \(\mu\) is known, so no degree of freedom is used to estimate the mean.

The ordinary finite likelihood-ratio calculation requires \(A\) to be positive definite. Under the stated model with nonsingular covariance, this holds with probability one when \(n\ge p\). If the observed \(A\) is singular, the unrestricted normal likelihood is unbounded as a covariance matrix approaches singularity in directions with no observed scatter; there is then no finite unrestricted maximum of the form used below.

For a positive-definite \(\Sigma\), the joint density, viewed as a function of \(\Sigma\), is

This follows by multiplying the \(n\) normal densities and using \(\sum_i (X_i-\mu)^\top\Sigma^{-1}(X_i-\mu) =\operatorname{tr}(\Sigma^{-1}A)\).

The Unrestricted Covariance Maximum

To verify the maximum, compare the log-likelihood at an arbitrary positive-definite \(\Sigma\) with its value at \(\widehat{\Sigma}\). Let \(B=\widehat{\Sigma}^{1/2}\Sigma^{-1}\widehat{\Sigma}^{1/2}\), which is positive definite. Since \(A=n\widehat{\Sigma}\), the trace term at \(\Sigma\) is \(n\operatorname{tr}(B)\). Also \(|\Sigma|=|\widehat{\Sigma}|/|B|\), and at \(\widehat{\Sigma}\) the trace term is \(np\). Therefore

Let \(b_1,\ldots,b_p\) be the positive eigenvalues of \(B\). Then \(\log|B|=\sum_j\log b_j\) and \(\operatorname{tr}(B)=\sum_j b_j\). The scalar inequality \(\log b\le b-1\) for \(b>0\) gives \(\log|B|-\operatorname{tr}(B)+p\le0\), with equality only if every \(b_j=1\). Thus the likelihood is maximized at \(\Sigma=\widehat{\Sigma}\), as claimed. \(\square\)

Deriving the Likelihood-Ratio Statistic

Define the likelihood ratio as the maximum likelihood under the null divided by the maximum likelihood under the unrestricted alternative:

Substituting the likelihood into this ratio cancels the common normalizing constant. At \(\widehat{\Sigma}=A/n\), the trace term is \(\operatorname{tr}(\widehat{\Sigma}^{-1}A)=np\). Hence

Because \(A=n\widehat{\Sigma}\), the same expression can be written in terms of the scaled scatter matrix:

A small \(\Lambda\) means the specified covariance gives a likelihood much smaller than the best unrestricted covariance. It is often convenient to use the deviance, defined as minus twice the log likelihood ratio:

The determinant expression follows from \(\log|\Sigma_0^{-1}\widehat{\Sigma}| =\log|\widehat{\Sigma}|-\log|\Sigma_0|\). Although the product \(\Sigma_0^{-1}\widehat{\Sigma}\) need not be symmetric, it has the same positive eigenvalues as the symmetric positive-definite matrix \(\Sigma_0^{-1/2}\widehat{\Sigma}\Sigma_0^{-1/2}\), so its determinant and trace are well-defined and real.

Why the Deviance Is Nonnegative

The deviance cannot be negative: the null likelihood cannot exceed the maximum likelihood under the unrestricted alternative. The matrix expression makes this fact explicit. Let \(C=\Sigma_0^{-1/2}\widehat{\Sigma}\Sigma_0^{-1/2}\), and let \(c_1,\ldots,c_p\) be its positive eigenvalues. Trace and determinant invariance give

For every \(c>0\), the function \(g(c)=c-\log c-1\) satisfies \(g(1)=0\) and \(g'(c)=1-1/c\). It decreases for \(0<c<1\), increases for \(c>1\), and therefore has minimum zero at \(c=1\). Every summand is nonnegative, so \(D\ge0\). Equality holds exactly when all \(c_j=1\), which for the symmetric matrix \(C\) means \(C=I\), or equivalently \(\widehat{\Sigma}=\Sigma_0\). \(\square\)

This also clarifies what the two terms measure. The determinant compares covariance volumes, while the trace aggregates the observed scatter along directions standardized by the null covariance. Neither term alone gives the full likelihood comparison.

Worked Calculations

Common Mistake and Practical Interpretation

A frequent error is to center the observations at their sample mean even though the mean is specified as known. That replaces \(A=\sum_i(X_i-\mu)(X_i-\mu)^\top\) with a different scatter matrix and changes the likelihood being compared. The correction is to use the stated known mean throughout: calculate every residual as \(X_i-\mu\), form \(A\), and then use \(\widehat{\Sigma}=A/n\). Estimating an unknown mean leads to a different likelihood calculation.

Another tempting shortcut is to compare only determinants. Equal determinants do not imply equal covariance structure: matrices can have the same volume but very different eigenvalues and orientations. The trace term is also necessary. The likelihood-ratio deviance combines both contributions and is nonnegative, with zero only when the unrestricted estimate equals the specified matrix.

The algebra provides the statistic and its direction: reject for sufficiently small \(\Lambda\), equivalently sufficiently large \(D\). It does not by itself specify a numerical cutoff. That requires a reference distribution or another calibration appropriate to the testing procedure.