Constrained Maximum Likelihood for Structured Covariance

Why Constrained Covariance Likelihood Requires Care

A covariance model can be statistically appealing but mathematically infeasible: its fitted matrix may fail to be positive definite, or an unconstrained optimizer may violate restrictions such as nonnegative variance components. Maximum likelihood for structured covariance therefore has two linked tasks: optimize the likelihood and ensure every candidate covariance belongs to the specified model and is a valid covariance matrix.

The central calculation is a covariance score: the derivative of the Gaussian log-likelihood in a permitted covariance direction. It shows which structural parameters can be adjusted to improve fit, and it distinguishes an interior solution, where the score vanishes, from a boundary solution, where an active constraint can prevent further improvement.

Likelihood and Feasible Covariances

Suppose \(X_1,\ldots,X_n\) are independent \(p\)-dimensional observations from \(N_p(\mu,\Sigma(\theta))\). Treat the mean \(\mu\) as fixed while differentiating with respect to the covariance parameters \(\theta\). Define the residual second-moment matrix \(S=n^{-1}\sum_{i=1}^n (X_i-\mu)(X_i-\mu)'\). Up to a constant not depending on \(\theta\), the log-likelihood is

This expression applies when \(\Sigma(\theta)\) is positive definite. If the mean is also an unknown parameter, the joint likelihood uses residuals from the fitted mean; covariance derivatives are still taken holding the mean fixed. At a joint maximum, the covariance score equations are evaluated at the jointly fitted parameters.

For example, a linear covariance-component model has \(\Sigma(\theta)=\sum_{j=1}^q \theta_j A_j\), with known symmetric component matrices \(A_j\). The coefficients may also be required to satisfy inequalities, such as \(\theta_j\geq0\) for variance components. These inequalities alone do not always ensure positive definiteness, so both kinds of restrictions must be checked.

Deriving the Covariance Score

The following result gives the score for any differentiable covariance parameterization. It is useful whether the model is linear in its parameters or is defined through a nonlinear map.

Proof. For a differentiable positive-definite matrix, the differential identities are \(d\log|\Sigma|=\operatorname{tr}(\Sigma^{-1}d\Sigma)\) and \(d\Sigma^{-1}=-\Sigma^{-1}(d\Sigma)\Sigma^{-1}\). The first follows by differentiating the determinant, and the second follows by differentiating \(\Sigma\Sigma^{-1}=I\). Applying them to the log-likelihood gives

In the last equality, cyclic invariance of the trace puts \(d\Sigma\) at the end of the product. Setting \(d\Sigma=\Sigma_j\,d\theta_j\) and dividing by \(d\theta_j\) proves the formula. \(\square\)

For a linear covariance-component model, \(\Sigma_j=A_j\), so the score for component \(j\) is the trace in the theorem with \(A_j\) in place of \(\Sigma_j\). At a feasible interior maximum, all such scores must be zero. This condition is necessary, not sufficient: it does not by itself establish that the point is a maximum or that it is globally optimal.

Boundary Constraints and Inequality Restrictions

When a parameter is constrained, an optimum need not have zero score in every coordinate. For maximizing \(\ell(\theta)\) subject to smooth inequalities \(g_k(\theta)\geq0\), define \(q(\theta)=-\ell(\theta)\) and minimize \(q\). At a regular local solution, the Karush–Kuhn–Tucker necessary conditions require multipliers \(\lambda_k\geq0\) for active inequalities such that

These are necessary conditions when the objective and constraints are differentiable and an appropriate constraint qualification holds (for example, the linear independence constraint qualification). They say that an active constraint can balance a nonzero likelihood score. Positive definiteness is normally enforced by restricting the parameterization to positive-definite matrices; if an optimum is sought on a boundary where the covariance becomes singular, the ordinary nondegenerate Gaussian likelihood no longer applies.

Direct and Factor-Based Parameterizations

A direct parameterization works with covariance entries or structural coefficients and imposes positive definiteness as a constraint. It makes the structural meaning of parameters explicit and often gives simple score directions. Its weakness is that positive definiteness can be awkward to maintain during numerical optimization.

A factor-based parameterization writes \(\Sigma=LL'\), where \(L\) is nonsingular, often lower triangular with positive diagonal entries. This guarantees positive definiteness: for every nonzero vector \(u\), \(u'\Sigma u=(L'u)'(L'u)>0\). However, a generic factor \(L\) does not automatically preserve a desired covariance structure. The factor must be restricted or constructed so that its product \(LL'\) lies in the target family.

The score transforms by the chain rule. Let \(G=\frac{n}{2}(\Sigma^{-1}S\Sigma^{-1}-\Sigma^{-1})\), the symmetric matrix in the covariance differential. For a differentiable factor \(L(\phi)\),

Thus each factor-coordinate score is obtained by substituting \(dL=(\partial L/\partial\phi_j)d\phi_j\). If all entries of \(L\) were free, the matrix gradient would be \(2GL\); for a lower-triangular factor, only the allowed entries contribute. This chain rule gives a score in feasible coordinates without treating the covariance entries as independent.

Common Pitfall and Practical Checks

A common mistake is to solve the unconstrained score equations and accept the resulting matrix without checking the model restrictions. A stationary point can have a negative variance component, violate a structural equality, or produce a covariance matrix that is not positive definite. The correction is to define the feasible set first, optimize only within it, and check boundary candidates as well as interior score roots. A second mistake is to use a generic Cholesky factor and assume it preserves every structure: it guarantees positive definiteness, but only a suitably restricted factorization guarantees the required structure.

In practice, evaluate the likelihood using a factorization of the candidate covariance rather than explicitly forming its inverse when possible. This improves numerical stability, but it does not change the mathematical requirements: the candidate must satisfy the structural model, remain positive definite, and meet the appropriate interior or boundary optimality conditions.