Identifiability of Nested Variance Components

Why Covariance Patterns Determine Identifiability

A nested model may name a separate variance for each level, but naming the components does not guarantee that the data can distinguish them. The relevant question is whether different choices of those variances imply different probability distributions for the observations. In a Gaussian model, this can be examined through the marginal covariance matrix: each random-effect level contributes a characteristic pattern of covariances among observations.

The central design features are replication within a lower-level unit and multiple units within a higher-level group. These create different kinds of pairs of observations. Repeated measurements on the same unit share every random effect along their path through the hierarchy; measurements on different units in the same group share only higher-level effects. If the design lacks one of these pair types, some variance components may contribute only through an inseparable sum.

A Three-Level Nested Random-Intercept Model

Consider observations \(Y_{gir}\), where \(g\) indexes a top-level group, \(i\) indexes a unit nested within group \(g\), and \(r\) indexes a replicate within that unit. For example, the levels could be clinics, patients within clinics, and repeated measurements on each patient. Use the model

Here \(x_{gir}^{\mathsf T}\beta\) is the fixed-effects mean. The group effect, unit effect, and residual satisfy

Assume all these random quantities are mutually independent, with the same variance at a given level, and that the design and nesting assignments are known. The variance components are nonnegative. This is a Gaussian response model; in a non-Gaussian GLMM, the same nested random effects determine covariance on the latent linear-predictor scale, but observed-response covariance also depends on the response distribution and link.

For two observations, their covariance is the sum of the variances of the random effects they share. Thus, observations in different groups have covariance zero; observations in different units of the same group have covariance \(\tau_G^2\); and distinct replicates of the same unit have covariance \(\tau_G^2+\tau_U^2\). The marginal variance of one observation is \(\tau_G^2+\tau_U^2+\sigma^2\).

Deriving the Covariance Contrasts

Let \(A_G\) be the matrix with entry 1 when two observations share a top-level group and 0 otherwise. Let \(A_U\) have entry 1 when they share the same nested unit and 0 otherwise, and let \(I\) be the identity matrix. The marginal covariance matrix is

The diagonal entries of \(A_G\) and \(A_U\) are 1 because an observation shares its group and unit with itself. To see when the components are identifiable, consider entries of \(V\) corresponding to different pair types. Each equation isolates a different contrast:

Consequently, if all three pair patterns are represented, the mapping from components to covariance entries is one-to-one:

This is a structural argument: it shows that no two different component triples produce the same covariance matrix for that design. A particular finite data set may not yield these exact population covariances, and estimates obtained by subtracting sample covariance summaries may even be negative. Those are finite-sample or estimation issues, not a failure of structural identifiability.

What Missing Replication Does

If a pair type is absent, the covariance matrix may no longer contain enough distinct patterns to separate the components. In particular, without repeated observations within a unit, the unit variance and residual variance can enter only as a sum. Without multiple units in a group, the group and unit effects can have the same pattern across the observed units.

The other aliasing pattern occurs when there is only one unit in a group, even if that unit is measured repeatedly. Every pair of distinct observations then shares both the group and unit effects. For example, with two replicates, the covariance matrix is

The repeated-measurement covariance distinguishes the combined random-intercept variance from the residual variance, but it does not divide that combined variance into group and unit components. Having multiple units within a group supplies the contrasting pair that shares the group effect but not the unit effect.

Checking Identifiability and Recognizing Failure

A design-based check can be made before fitting a model. List the kinds of observation pairs that actually occur, then write the covariance each kind implies. If the resulting equations distinguish each component, the Gaussian covariance mapping is structurally one-to-one. Equivalently, inspect whether the component pattern matrices \(A_G\), \(A_U\), and \(I\), restricted to the observed design, are linearly independent. If one is a linear combination of the others, some parameter combinations are aliased.

After fitting, warning signs can include a nearly flat likelihood over a range of component values, estimates at zero, very large standard errors, or strong sensitivity to small changes in the model or data. These signals indicate weak information or boundary behavior; they do not by themselves prove structural nonidentifiability. Conversely, an optimizer returning three numbers does not establish that the design separates the components.

The pairwise covariance argument assumes the stated random effects are independent and that the covariance structure is correctly specified. If random effects are correlated, residuals are serially correlated, or the nesting assignments are wrong, observed covariance patterns can differ from the model's predictions. For non-Gaussian outcomes, the displayed covariance equations describe the latent random-effects contribution, not generally the covariance of the observed response. Identifiability then needs to be considered under the specified likelihood and scale rather than inferred solely from a Gaussian response covariance formula.

Identifiability, Precision, and Alternatives

Structural identifiability and precision answer different questions. Identifiability asks whether the model distributions differ for different component values. Precision asks how much information the realized sample provides to distinguish those values. Even when one top-level group is enough for covariance-matrix identifiability in the example above, many independent groups will typically be important for estimating between-group variation reliably. Increasing replication at the unit level primarily helps distinguish unit and residual variation; increasing the number of units within groups helps distinguish group and unit variation.

A design with missing pair types cannot recover the separate components from the specified covariance structure merely by using a more sophisticated optimizer. The analysis must instead acknowledge the limitation: collect data with the missing replication, impose a scientifically justified restriction that combines or fixes components, or report only the identifiable sum. Treating an aliased component as a separate estimand without additional assumptions gives results that depend on arbitrary modeling or numerical choices.

A common mistake is to say that every variance component requires many independent top-level groups to be structurally identifiable. That confuses the need for information and precision with the definition of identifiability. Under the Gaussian covariance definition used here, one group with repeated observations within at least two units has distinct entries for the group, unit, and residual contrasts. More groups can greatly improve estimation of group-level variation, but are not a necessary condition for the covariance mapping to be one-to-one.