Likelihood-Ratio Tests Of Nested Covariance Restrictions

Why Compare Nested Covariance Models?

Covariance assumptions often form a sequence of increasingly flexible models. For example, a spherical covariance matrix has one free variance parameter; a diagonal matrix allows a separate variance for each variable; and an unrestricted covariance matrix also allows all covariances. A likelihood-ratio test asks whether the additional flexibility in a larger model improves fit enough to justify its extra parameters.

The comparison is meaningful only when the models are nested: every covariance matrix allowed under the smaller model must also be allowed under the larger one. The test statistic measures the loss in maximized likelihood caused by imposing the smaller model’s restrictions. Its usual large-sample reference distribution is chi-square, with degrees of freedom determined by the difference in model dimensions—provided the models satisfy regularity conditions.

Definition and Exact Likelihood-Ratio Statistic

Write \(\bar X=n^{-1}\sum_{i=1}^n X_i\) and \(S=n^{-1}\sum_{i=1}^n(X_i-\bar X)(X_i-\bar X)^\mathsf T\). The divisor is \(n\), because \(S\) is the unrestricted Gaussian maximum-likelihood estimator of the covariance matrix when the mean is unknown. For a fixed positive-definite \(\Sigma\), maximizing over \(\mu\) gives \(\hat\mu=\bar X\). Substitution into the log-likelihood yields

Let \(\widehat\Sigma_j\) be a covariance maximum-likelihood estimate in model \(\mathcal M_j\). Subtracting the two maximized log-likelihoods gives an exact expression, including when neither model is unrestricted.

To verify it, substitute each estimate into the displayed log-likelihood. The constant \(-np\log(2\pi)/2\) appears in both models and cancels. The difference in log-determinant terms is \(n\log(|\widehat\Sigma_0|/|\widehat\Sigma_1|)/2\), and the difference in trace terms is \(n\{\operatorname{tr}(\widehat\Sigma_0^{-1}S)-\operatorname{tr}(\widehat\Sigma_1^{-1}S)\}/2\). Multiplying the log-likelihood difference by two proves the formula. Since the models are nested, \(L_0\leq L_1\), so \(T\geq0\).

When \(\mathcal M_1\) is the unrestricted covariance model, \(\widehat\Sigma_1=S\), assuming \(S\) is positive definite. Then \(\operatorname{tr}(S^{-1}S)=p\), and the expression reduces to

In many models the restricted estimate also satisfies \(\operatorname{tr}(\widehat\Sigma_0^{-1}S)=p\). This occurs, for example, when the model has a freely estimated overall scale and the likelihood score for that scale is zero at the maximum. In that case the statistic simplifies further to \(n\log(|\widehat\Sigma_0|/|S|)\). Do not assume this simplification without checking the model or the likelihood equations.

Degrees of Freedom and Wilks’ Theorem

The dimension of a covariance model is the number of locally free parameters needed to specify its covariance matrices. An unrestricted symmetric \(p\times p\) covariance matrix has \(p(p+1)/2\) distinct entries. For regular nested models with dimensions \(d_0\) and \(d_1\), the number of independent restrictions is \(q=d_1-d_0\).

The dimension difference follows directly from local parameter counting. In regular local coordinates, the larger model has \(d_1\) free covariance coordinates, while the null model restricts those coordinates to a smooth subset with \(d_0\) free coordinates. Its local codimension is therefore \(d_1-d_0\), the number of independent restrictions. Wilks’ theorem calibrates the likelihood loss from those restrictions by a chi-square distribution with that codimension as its degrees of freedom. This is an asymptotic result, not an exact finite-sample distribution in general.

For example, an unrestricted covariance matrix in three dimensions has six parameters. A diagonal covariance matrix has three, so testing diagonality against the unrestricted model imposes three restrictions and has three asymptotic degrees of freedom. A parameter count is valid only when it describes the local dimension: redundant parameterizations, boundary points, or singular information can make the simple subtraction misleading.

Worked Comparisons

Conditions, Interpretation, and a Common Pitfall

The usual chi-square calibration requires more than a nested pair of formulas. The covariance matrices must be positive definite at the null, the model dimensions must be well defined locally, and the true null parameter must not lie on a boundary where the model changes shape. The information must be nonsingular, and the likelihood must have the usual smooth local behavior. The independent Gaussian sampling model and fixed dimension are part of the stated calibration. If any of these conditions fail, the degrees-of-freedom subtraction may not give the null distribution; a boundary can, for example, lead to a nonstandard mixture rather than an ordinary chi-square law.

Common mistake: “The null has four fewer named parameters, so the test statistic must have a chi-square distribution with four degrees of freedom.” The correction is that parameter counting supplies the degrees of freedom only for a regular nested comparison. One must first identify the effective local dimensions and check the regularity assumptions. A parameter on the edge of its allowed range, a non-identifiable parameter, or a singular model can invalidate the ordinary Wilks calibration even when the models appear nested.

The likelihood-ratio framework is useful because it compares models on a common scale and applies to many covariance structures, including diagonal, block-diagonal, equal-variance, and compound-symmetry restrictions. It also separates two tasks: maximize the likelihood under each model, then calibrate the resulting loss using the dimension difference and the regularity conditions. Those tasks should not be conflated: a correct likelihood calculation does not by itself guarantee a reliable chi-square approximation.