Replication Needed to Separate Variance Components

Why the Location of Replication Matters

A large data set does not automatically contain enough information to estimate every variance component. What matters is not only how many observations there are, but which observations share which group memberships. Two observations that share a group but differ in a lower-level unit inform a different covariance pattern from two observations that share the same lower-level unit. Repeating an observation under exactly the same factor combination can distinguish still another source of variation.

This tutorial treats replication as a design question: where must repeated observations occur so that the variation associated with different levels of a nested or crossed structure can be separated? The central tool is the covariance between pairs of observations with specified membership relationships.

Nested Structures: Replicate at the Lowest Level You Need to Separate

Consider a Gaussian random-intercept model with groups \(g\), subgroups \(h\) nested within groups, and observations \(r\) within subgroups:

The random effects and errors are assumed mutually independent across their indices, with mean zero and the stated variances. The mean structure is assumed correctly specified; if covariates are included, the covariance reasoning applies after accounting for their mean contributions. Conditional on the random effects, the errors are independent and have common variance \(\sigma_e^2\). The covariance results below follow from these assumptions; normality is useful for Gaussian likelihood inference but is not required for the covariance identities themselves.

Two observations in different subgroups of the same group share only \(u_g\), so their covariance is \(\sigma_g^2\). Two distinct observations in the same subgroup share both \(u_g\) and \(v_{gh}\), so their covariance is \(\sigma_g^2+\sigma_h^2\). An observation’s variance adds the residual component as well.

The three patterns make the separation possible: observe pairs sharing a group but not a subgroup, pairs sharing a subgroup, and individual observations whose total variance can be compared with the shared-subgroup covariance. If there is only one observation per subgroup, there are no distinct within-subgroup pairs. When the model includes both a subgroup effect and residual error, their contributions cannot be distinguished from that pairwise pattern. If there is only one subgroup per group, group and subgroup membership are perfectly aligned, so the two corresponding covariance components cannot be separated by these patterns.

Crossed Structures: Replicate Within Cells to Separate Interaction and Error

Now suppose each level \(i\) of factor \(A\) occurs with each level \(j\) of factor \(B\), and there are repeated observations \(k\) for a given \(A\)-by-\(B\) combination. A model that allows a random interaction is

where the independent, mean-zero random terms have variances \(\sigma_A^2\), \(\sigma_B^2\), \(\sigma_{AB}^2\), and \(\sigma_e^2\), respectively. Assume there are multiple levels of each factor and, for the full set of covariance patterns, multiple observations within each factor-level combination. As in the nested model, these formulas describe a correctly specified mean and independent, homoscedastic errors.

Two observations with the same \(A\) level and different \(B\) levels share only \(a_i\); two with the same \(B\) level and different \(A\) levels share only \(b_j\). Distinct observations in the same cell share \(a_i\), \(b_j\), and \(c_{ij}\), but not their individual residual errors. The variance of a single observation includes all four terms.

Thus, repeated observations within the same cell are what distinguish interaction variation from residual variation in this model. With only one observation per cell, there is no distinct within-cell pair. More fundamentally, the interaction and residual terms then both attach to the sole observation in each cell: their separate variances cannot be recovered from the cell-specific variation alone.

A Short Identifiability Derivation

The confounding under one observation per crossed cell can be seen directly. If \(k=1\), the model contains one cell-specific value \(c_{ij}+\varepsilon_{ij1}\) for each cell. Under the independent Gaussian assumptions, this sum has mean zero and variance \(\sigma_{AB}^2+\sigma_e^2\). Across distinct cells, the interaction and residual sums are independent. Consequently, the likelihood contribution from these cell-specific terms depends on the two variances only through their sum: any nonnegative pair with the same sum gives the same distribution for those terms. For example, a combined variance of \(3.6\) is consistent with \((\sigma_{AB}^2,\sigma_e^2)=(1.0,2.6)\) or \((2.0,1.6)\). Replicating observations within cells changes the covariance between distinct observations in a cell: that covariance includes \(\sigma_{AB}^2\), whereas the additional variance of each individual observation includes \(\sigma_e^2\). This supplies the missing distinction.

Replication in a Generalized Linear Mixed Model

For a non-Gaussian response, a generalized linear mixed model (GLMM) specifies a conditional response distribution given random effects, along with a link connecting its conditional mean to fixed and random effects. For example, a binary response may be modeled as conditionally Bernoulli with a logit link. Conditional independence of responses does not mean their marginal outcomes are independent: observations sharing random effects can be associated after those effects are integrated out.

The design logic still applies. Repeating observations within the same membership combination supplies information about the dependence induced by the random effects at that combination. However, a Bernoulli GLMM does not usually include a separately estimated Gaussian observation-level residual variance in the way the Gaussian model above does; the conditional response variance follows from the response distribution and its mean. Therefore, do not transfer the Gaussian variance decomposition mechanically to a GLMM. Which components can be distinguished depends on the response family, link, random-effects structure, covariate patterns, and amount of replication. A structurally informative design can still yield imprecise or boundary estimates.

Design Checks, Alternatives, and Limitations

Before fitting a model, list the memberships of each observation and identify the pair types that the proposed variance components predict will have different covariances. Then check whether the design actually contains observations for each required pair type. The number of total records alone cannot answer this question.

These are structural checks, not guarantees of precise estimation. With only a few groups, subgroups, or factor levels, variance estimates can be highly uncertain even when the design contains all the relevant patterns. Components may be estimated at zero, a boundary that can reflect weak information rather than proof that the source is absent. An optimizer warning, a singular fit, large uncertainty, or strong sensitivity to model specification deserves investigation. Also check whether residuals plausibly meet the assumed conditional independence and variance structure: repeated measurements over time may have additional serial dependence, and unequal residual variances can invalidate simple covariance interpretations.

There are alternatives when replication is unavailable, but each changes what the model can claim. One can omit a component, thereby imposing a substantive assumption; combine confounded sources into a single variance term; or redesign data collection to obtain the missing membership combinations. In a crossed model with one observation per cell, for example, omitting the interaction can allow a simpler additive model, but it assumes that cell-specific interaction variation is absent or is absorbed into the error term. It does not empirically separate interaction from residual variation. More sophisticated estimation cannot recover information that the design does not contain.

Common Pitfall

Wrong reasoning: “There are hundreds of observations, so all the variance components must be estimable.” This overlooks whether the observations repeat at the memberships needed to produce distinct covariance patterns. Hundreds of unique crossed cells with one observation each still provide no within-cell pair for separating a random interaction from residual error.

Correction: Identify the component-specific membership pattern first, then verify that the data contain it. Replication must occur at the level whose variation is to be separated; additional observations elsewhere may improve precision for other components but cannot replace the missing pattern.