Design-Based Identifiability in Clustered Data

Why the Design Matters Before Fitting

An optimizer can search for random-effect variances and covariances only insofar as the design gives those parameters distinct implications for the data. If two different covariance matrices produce exactly the same random-effect contribution for every cluster in the design, the parameters are not distinguishable from that design. Changing optimizers or tightening tolerances cannot supply the missing information.

This is a question of design-based identifiability: whether the cluster structure and predictor values permit specified random-effect covariance parameters to have distinct implications. It is separate from whether a particular finite data set estimates those parameters precisely, or whether an optimization algorithm has converged.

Model and Identifiability Criterion

For cluster \(i\), let \(Y_{ij}\) be the response for observation \(j\), and let \(x_{ij}\) denote its predictors. A GLMM specifies a response distribution conditional on random effects, with conditional mean linked to a linear predictor such as \(\eta_{ij}=o_{ij}+x_{ij}^{T}\beta+z_{ij}^{T}b_i\). Here \(o_{ij}\) is a known offset, \(\beta\) is a vector of fixed effects, \(z_{ij}\) is the row of the random-effects design matrix \(Z_i\), and \(b_i\) is a vector of cluster-specific random effects.

For the design checks below, assume clusters are independent, \(b_i\) has mean zero and a common covariance matrix \(G\), and the conditional response model and link are specified. In the usual Gaussian random-effects formulation, \(b_i\sim N(0,G)\), with \(G\) positive semidefinite. Observations are conditionally independent given the random effects and fixed predictors. These assumptions define the model being checked; they do not guarantee that the design identifies its parameters.

The entries of \(Z_iGZ_i^{T}\) describe the covariance pattern contributed by the random effects to the linear predictor within cluster \(i\). In a Gaussian model with an identity link, these contributions form part of the response covariance structure. In a non-Gaussian GLMM, the marginal response covariance is generally not simply \(Z_iGZ_i^{T}\), because integrating over random effects also involves the response distribution and link. The matrices remain a useful design check, but full likelihood identifiability can depend on the whole model.

A Rank Check for Covariance Parameters

Suppose \(G\) is an unrestricted symmetric \(q\times q\) covariance matrix. It has \(q(q+1)/2\) distinct entries. Each entry of \(Z_iGZ_i^{T}\) is a linear combination of those entries. Collect the coefficients for all clusters and all pairs of observations within each cluster into a matrix \(A\). Then a sufficient and, for this linear covariance mapping, necessary condition for the unrestricted entries of \(G\) to be distinguishable is that \(A\) have full column rank \(q(q+1)/2\).

The notation \(\operatorname{vec}_{\mathrm{sym}}\) means that only distinct entries of a symmetric matrix are listed. The rank criterion follows directly: if \(A\) has a nonzero null vector \(v\), then a change \(v\) in the listed covariance entries leaves every mapped matrix unchanged, since \(Av=0\). Conversely, if \(A\) has full column rank, no nonzero change in those entries leaves all mapped matrices unchanged. This is a linear-algebra result about the mapping, not a guarantee of precise estimation from a particular sample. Positive-semidefinite restrictions can also affect identifiability at a boundary, so the unrestricted rank check is best understood as a check for whether the design can distinguish the full covariance parameterization.

How Predictor Placement Changes What Can Be Distinguished

Within-cluster variation often makes random slopes distinguishable from random intercepts, but the actual criterion is the rank of the covariance mapping. A predictor that varies within clusters supplies more than one random-effects design row in a cluster. Predictor values that are all identical can collapse distinct random effects into the same contribution. The following example shows that having two values is not, by itself, enough to separate all three covariance entries.

Practical Design Checks and Failure Modes

Before interpreting a fitted covariance matrix, check the design that generates its random-effects columns. For each proposed random effect, ask which values its design column takes within each cluster and which combinations of columns occur together. Then assess whether the resulting covariance patterns can distinguish the parameters of interest. For a general structure, constructing the coefficient matrix \(A\) and checking its rank makes this question explicit.

A rank deficiency is structural for the chosen covariance mapping: some parameter changes leave the design-implied matrices unchanged. By contrast, a full-rank matrix can still be poorly conditioned, and a finite data set may provide weak information. Those are practical estimation difficulties rather than exact aliasing. Earlier material on convergence and singular fits helps diagnose numerical and boundary behavior, but such diagnostics do not replace this design check.

• Random intercept and slope: inspect the range and pattern of the slope predictor within clusters, not only its overall range.
• Random slopes for factors: check whether the relevant factor levels occur in the clusters whose random effects are being modeled. Missing combinations can remove covariance contrasts.
• Multiple random effects: check whether their design columns vary together so that different covariance entries imply distinguishable patterns.
• Centering and coding: a change of predictor origin or factor coding changes the parameter representation. It may make estimates easier to interpret or computation more stable, but it does not create information absent from the design.

Design Changes, Constraints, and Limits

When a design cannot distinguish specified covariance parameters, the most direct remedy for future data is often to collect observations at predictor values or combinations that add new covariance contrasts within clusters. In existing data, a simpler covariance structure may be appropriate if it matches the scientific question. For example, setting an intercept–slope covariance to zero removes that parameter from the model, but it is a substantive restriction, not a discovery that the design estimated the covariance to be zero.

Reparameterizing a covariance matrix can clarify which combinations are estimable, but cannot increase the rank of the information supplied by the design. Nor does a design-level rank check prove that the full GLMM likelihood uniquely identifies every model parameter: the response family, link, fixed-effects structure, and random-effects distribution also matter. It answers a narrower, useful question: whether the specified cluster and predictor layout separates the random-effect covariance patterns being requested.