Selecting a Structured Covariance Model

Why Covariance-Model Selection Needs Care

A structured covariance model replaces an unrestricted covariance matrix with a more economical description: for example, a block, Toeplitz, Kronecker, or variance-component structure. This can reduce the number of estimated parameters and stabilize estimation, but it also creates a selection problem. A simpler structure may fit less well, while a more flexible one can improve the likelihood simply by having more freedom.

The comparison method depends on the relationship between the candidates. A likelihood-ratio test is designed for nested models and has a standard chi-square calibration only under regularity conditions. For nonnested models, information criteria such as AIC and BIC provide penalized comparisons. Neither route makes parameter counting or boundary constraints optional.

Assume first that the observed independent units are Gaussian vectors \(Y_1,\ldots,Y_n\), with a specified mean model and covariance \(\Sigma(\theta)\). Each candidate must be fitted to the same observed data under a comparable likelihood. Write \(\ell(\hat\theta_j)\) for the maximized log-likelihood under model \(M_j\), and \(k_j\) for the number of identifiable parameters estimated in that likelihood. If the same mean parameters appear in both models, they still belong in each \(k_j\), although their common contribution cancels in a parameter-count difference.

Likelihood-Ratio Tests for Nested Models

For nested models \(M_0\subset M_1\), the likelihood-ratio statistic is

The maximized likelihood under the larger model cannot be smaller, so \(\Lambda\geq 0\). Under the null hypothesis that \(M_0\) is correct, Wilks's theorem gives an asymptotic \(\chi^2_{k_1-k_0}\) distribution when the models are identifiable and regular, the null value is an interior point of the larger model, the relevant information matrix is nonsingular, and the number of independent units grows with fixed model dimension. The models must also describe the same data and mean structure. These conditions are substantive, not just formalities.

The following calculation derives a useful likelihood-ratio statistic for a simple structured comparison. It also shows precisely where the parameter difference and chi-square reference come from.

A common exception occurs when the null sets a variance component to zero. For example, a random-intercept covariance may be written within each cluster as \(\sigma^2 I+\tau^2\mathbf{1}\mathbf{1}^{\mathsf T}\), with \(\tau^2\geq0\). The null \(\tau^2=0\) is on the boundary, not in the interior. Ordinary Wilks calibration does not follow. With one such scalar component, a positive interior residual variance, and standard regularity conditions, the common limiting reference is the mixture \(\frac12\chi^2_0+\frac12\chi^2_1\), where \(\chi^2_0\) is a point mass at zero. More complicated boundary constraints can give other mixtures; one must not assume the one-component formula automatically applies.

Information Criteria When a Likelihood-Ratio Test Is Not Suitable

For candidates that are nonnested, or for nested candidates where a standard likelihood-ratio calibration is not justified, AIC and BIC compare fit against a complexity penalty. Their conventional forms are

Here \(k\) counts all identifiable parameters estimated in the likelihood, including mean and nuisance parameters, and \(n\) is the number of independent units contributing likelihood information. Lower values are preferred within the set being compared. AIC’s fixed penalty is motivated by estimating predictive fit; BIC’s penalty increases with sample size and has a large-sample marginal-likelihood motivation. Neither criterion is a test of a null hypothesis, and their numerical values are meaningful for comparison only when based on a consistent data and likelihood basis.

The BIC penalty can be derived under regular large-sample conditions. Let a model have fixed dimension \(k\), an interior identifiable maximum-likelihood estimate \(\hat\theta\), and a prior density \(\pi(\theta)\) that is positive and continuous near that estimate. Suppose the observed information \(J_n=-\nabla^2\ell(\hat\theta)\) is of order \(n\) and \(J_n/n\) converges to a positive-definite matrix. A second-order Taylor expansion near \(\hat\theta\) gives \(\ell(\theta)\approx\ell(\hat\theta)-\frac12(\theta-\hat\theta)^{\mathsf T}J_n(\theta-\hat\theta)\). Integrating this local Gaussian approximation yields the marginal likelihood \(m(Y)\approx e^{\ell(\hat\theta)}\pi(\hat\theta)(2\pi)^{k/2}|J_n|^{-1/2}\).

Since \(J_n=n\{J_n/n\}\), its determinant satisfies \(\log|J_n|=k\log n+\log|J_n/n|\). Taking minus twice the logarithm of the approximation gives \(-2\log m(Y)=-2\ell(\hat\theta)+k\log n+O_p(1)\). The terms collected in \(O_p(1)\) include the prior density and the limiting information determinant. BIC keeps the leading fit and dimension terms. This derivation requires regularity; boundary or nonidentifiable models may not have this simple approximation.

Parameter Counts, REML, and Comparability

Parameter counting is part of the statistical model, not an administrative detail. Count free, identifiable parameters in the likelihood used for comparison. Do not count a redundant scale parameter in a nonidentifiable Kronecker representation as though the data estimated it independently. Conversely, do count estimated mean parameters when computing AIC or BIC from the full likelihood, even if they cancel in a likelihood-ratio difference between models with the same mean specification.

Restricted maximum likelihood (REML) requires particular care. Restricted likelihoods are constructed after accounting for fixed effects, so REML likelihoods generally cannot be compared across different fixed-effect design matrices. For covariance comparisons with the same fixed-effect design, restricted likelihood comparisons are commonly used; the design and likelihood basis must remain the same. If fixed effects differ, fit using maximum likelihood for a comparable full likelihood rather than comparing REML values.

The same principle applies to AIC and BIC: do not compare values from different response observations, different likelihood contributions, or incomparable likelihood definitions. For a Gaussian model, candidates fitted to the same complete observations and mean structure offer a straightforward comparison. If the set of observed data differs across fits, a better criterion value may reflect the data change rather than a better covariance description.

Practical Selection and a Common Pitfall

A defensible selection starts by stating the comparison and its assumptions. If one structure is a regular nested restriction of another, use a likelihood-ratio test with the parameter difference as degrees of freedom and verify interiority and identifiability. If the comparison is nonnested or the null lies on a boundary, use an appropriate information criterion or a specifically justified nonstandard test. Check whether the scientific goal is inference about a restriction, predictive performance, or a parsimonious representation: these are not identical goals.

• Wrong reasoning: “The larger covariance model has a higher maximized likelihood, so it is better.” The likelihood cannot decrease when a model is expanded, even if the extra parameters are fitting chance variation.
• Correction: account for complexity. Use a calibrated likelihood-ratio test only when its assumptions hold, or compare penalized criteria on the same likelihood basis.
• Wrong reasoning: “There are two models, so the likelihood-ratio statistic must have a chi-square distribution.”
• Correction: verify nesting, regularity, and whether the null is interior. A variance component equal to zero is a common boundary case.