Identifiability in Crossed Designs

Why overlap determines what can be learned

In a crossed design, each observation belongs to a level of each of two factors. For example, a response might be recorded for a particular operator using a particular device. The factors are crossed when their levels can occur in multiple combinations, rather than one factor being contained within levels of the other. The earlier material on crossed classifications and covariance derivations establishes how shared factor levels induce dependence among observations. Here the key question is whether the observed pattern of sharing lets us distinguish the variance components that produce that dependence.

A model can be written down even when the data cannot distinguish its variance components. This is a design property, not merely a question of whether an estimation algorithm converges. Exact confounding means different parameter values imply the same probability distribution for the data. Weak identification means the distributions differ, but only slightly for the design and sample size at hand, so estimates can be unstable.

Definition: A parameter is structurally identifiable when distinct parameter values imply distinct distributions for the observations under the stated model. A variance component is not structurally identifiable separately from other components if changing it can be offset by changing another component without changing that distribution. Practical identifiability concerns whether the available design and data provide enough information for precise estimation, even when structural identifiability holds.

A covariance criterion for a crossed random-intercept model

Consider \(n\) observations with a fixed-effects design matrix \(X\), and two crossed random intercepts. Let \(Z_A\) and \(Z_B\) be the incidence matrices: each row marks the level of factor A and factor B, respectively, for one observation. Use the Gaussian model

y=X\beta+Z_Aa+Z_Bb+\varepsilon, \qquad a\sim N(0,\tau_A^2 I),\quad b\sim N(0,\tau_B^2 I),\quad \varepsilon\sim N(0,\sigma^2 I).

Assume the random effects and residuals are mutually independent, and that observations have the specified memberships and common component variances. Marginally, \(y\) is multivariate normal with mean \(X\beta\) and covariance

V=\tau_A^2M_A+\tau_B^2M_B+\sigma^2I, \qquad M_A=Z_AZ_A',\quad M_B=Z_BZ_B'.

The entry in \(M_A\) for observations \(r\) and \(s\) is 1 if they share an A level and 0 otherwise; \(M_B\) works the same way for B. Thus the covariance between two distinct observations is \(\tau_A^2\) if they share only A, \(\tau_B^2\) if they share only B, both components added if they share both, and 0 if they share neither. The diagonal includes both random-intercept variances and residual variance.

Theorem: For this Gaussian model with an unrestricted fixed mean, the three covariance components are structurally identifiable from the marginal distribution if and only if \(M_A\), \(M_B\), and \(I\) are linearly independent. For a restricted-likelihood analysis, apply the same criterion to the projected matrices \(K'M_AK\), \(K'M_BK\), and \(K'K\), where the columns of \(K\) span the residual-contrast space satisfying \(K'X=0\). Dependence among the relevant matrices means some component combinations cannot be separated by that likelihood.

The criterion follows directly from the covariance formula. If the matrices are linearly independent, equality of two covariance matrices implies

(\tau_A^2-\widetilde{\tau}_A^2)M_A+ (\tau_B^2-\widetilde{\tau}_B^2)M_B+ (\sigma^2-\widetilde{\sigma}^2)I=0.

Linear independence forces each coefficient to be zero, so the parameter values are equal. Conversely, if the matrices are dependent, there is a nonzero coefficient vector whose weighted sum is zero. Starting from strictly positive variance components, a sufficiently small change in that direction keeps all components nonnegative while leaving \(V\) unchanged. The distribution therefore cannot distinguish those nearby parameter values. For restricted likelihood, the same argument applies to covariance after projecting out the fixed-effects space.

This is an exact, model-based criterion. It does not say that a variance can be estimated precisely from a small sample. Also, the projected criterion matters: a covariance pattern visible in the full covariance matrix may provide no residual-contrast information after the fixed effects are removed.

Worked Example: A Complete Two-by-Two Crossing

Suppose a hypothetical study observes one response for each of four A-by-B combinations: \((A_1,B_1)\), \((A_1,B_2)\), \((A_2,B_1)\), and \((A_2,B_2)\). Assume the random-intercept model above, with hypothetical component values \(\tau_A^2=4\), \(\tau_B^2=9\), and \(\sigma^2=1\). Order observations as listed.

Each observation has marginal variance \(4+9+1=14\). The first two observations share A only, so their covariance is 4; the first and third share B only, so their covariance is 9. The first and fourth share neither, so their covariance is 0. Applying the same rule to all pairs gives

V= \begin{pmatrix} 14&4&9&0\\ 4&14&0&9\\ 9&0&14&4\\ 0&9&4&14 \end{pmatrix}.

The off-diagonal entries for sharing A reveal \(\tau_A^2\), and those for sharing B reveal \(\tau_B^2\). Once these are distinguished, the diagonal remainder is \(14-4-9=1\), which is \(\sigma^2\). In matrix terms, the three covariance patterns are linearly independent. This design is structurally able to separate the components, although four responses would generally give very imprecise estimates.

Sparse overlap and exact confounding

A useful way to inspect a crossed design is as a bipartite graph. One set of nodes represents A levels and another represents B levels; an observed A-by-B combination is an edge. Repeated observations on the same combination can be represented by multiple observations on that edge. The graph shows which pairs of observations share a factor level, but the covariance-matrix criterion remains the precise test.

The simplest confounding occurs when every observed A level appears in exactly one observation. Then \(Z_AZ_A'=I\), so the A random-intercept contribution has exactly the same covariance pattern as the residual contribution. No response values can separate \(\tau_A^2\) from \(\sigma^2\) under this model. This is different from a component that is merely estimated imprecisely: the likelihood depends on their sum, not on their separate values.

Worked Example: A Factor Observed Once Per Level

Consider three hypothetical observations at \((A_1,B_1)\), \((A_2,B_1)\), and \((A_3,B_2)\). Every A level occurs once, while the first two observations share B. Suppose, for illustration, that \(\tau_A^2=3\), \(\tau_B^2=2\), and \(\sigma^2=1\).

The first and second observations have covariance 2 because they share B. Other off-diagonal covariances are 0. Each diagonal entry is \(3+2+1=6\), so

V= \begin{pmatrix} 6&2&0\\ 2&6&0\\ 0&0&6 \end{pmatrix} = 3I+2 \begin{pmatrix} 1&1&0\\ 1&1&0\\ 0&0&1 \end{pmatrix} +1I.

Here \(M_A=I\). Therefore the A and residual terms combine as \((\tau_A^2+\sigma^2)I\); the displayed diagonal value identifies their sum \(3+1=4\), not the two parts. For example, \(\tau_A^2=2\) and \(\sigma^2=2\) would produce the same covariance matrix if \(\tau_B^2\) remained 2. B is distinguishable because its shared-level pattern creates the off-diagonal covariance 2. The numerical component values illustrate the covariance pattern; they are not separately recoverable from this design.

Disconnected designs are not automatically nonidentifiable

A design is disconnected when its bipartite graph has more than one component: no chain of observed combinations links one component to another. Disconnection can make some fixed-effect contrasts unavailable and can reduce information. But it does not, by itself, make the random-effect variance components nonidentifiable. If the components share the same variance parameters and the model assumes independent random effects for distinct levels, within-component covariance patterns can still distinguish those parameters.

Worked Example: Two Disconnected Complete Blocks

Suppose a hypothetical design contains two separate complete two-by-two blocks. Block 1 uses levels \(A_1,A_2\) and \(B_1,B_2\); block 2 uses \(A_3,A_4\) and \(B_3,B_4\). There are no observed combinations linking the blocks. Assume shared component variances \(\tau_A^2=2\), \(\tau_B^2=5\), and \(\sigma^2=3\). In either block, a diagonal variance is \(2+5+3=10\), a same-A covariance is 2, and a same-B covariance is 5.

For instance, order the first block as \((A_1,B_1),(A_1,B_2),(A_2,B_1),(A_2,B_2)\), and use the corresponding ordering for the second block. The full covariance has block form

V= \begin{pmatrix} C&0\\ 0&C \end{pmatrix}, \qquad C= \begin{pmatrix} 10&2&5&0\\ 2&10&0&5\\ 5&0&10&2\\ 0&5&2&10 \end{pmatrix}.

Within each block, same-A pairs expose 2, same-B pairs expose 5, and the diagonal remainder is \(10-2-5=3\). Thus the covariance patterns remain distinct despite the zero cross-block covariances. Disconnection does not cause structural confounding here. However, the two blocks do not provide cross-block comparisons, and a model that assigns separate parameters to blocks would be a different model with different identifiability requirements.

Diagnostics, alternatives, and limitations

Before interpreting an estimated variance component, inspect the membership pattern and the implied covariance matrices. A component whose matrix duplicates another component's matrix is exactly confounded. Near-duplication after projection can instead produce weak identification. In practice, warning signs include a nearly singular information matrix, a very flat likelihood profile over a range of variance values, large changes under small model or data perturbations, and estimates repeatedly reaching zero or another boundary. A boundary estimate alone does not prove nonidentifiability; it can also occur when a component is identifiable but the data support little variation.

There are two useful ways to check the design. The covariance-matrix approach directly examines linear independence of the component matrices, or their restricted versions. An incidence-graph inspection is a quick structural aid: it identifies isolated levels, repeated combinations, and which pairs share one or both factors. The graph is not a replacement for the matrix check, especially with repeated observations or additional random effects. Traditional expected-mean-square arguments can also reveal which sources of variation enter distinct observable contrasts in balanced designs, but they are less general for sparse or unbalanced patterns.

These conclusions rely on the specified model: common variance components across levels, independent random effects for distinct levels, and the stated residual structure. A different residual model, random-effect correlation structure, or fixed-effects design changes the relevant covariance patterns. In a generalized linear mixed model, the incidence structure still determines which observations share latent random effects, but the Gaussian marginal covariance formula and its linear-independence test are not, by themselves, a complete identifiability proof for a nonlinear response likelihood.

Common pitfall: “The factor graph is disconnected, so its variance component cannot be estimated.” That conclusion does not follow. Disconnection means there are no links between graph components; it does not erase distinct within-component covariance patterns. Conversely, a connected graph does not guarantee identifiability: if one factor's levels each occur once, its covariance matrix can still equal the residual identity matrix. Check the component covariance patterns, including any fixed-effect projection, rather than using connectedness alone as the test.

Key takeaway: Crossed-design variance components are separated by distinct covariance patterns created by overlap. Exact identifiability is a matrix-independence question; sparse overlap may either create exact confounding or leave parameters technically identifiable but weakly estimated.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

In a crossed random-intercept model, what does an off-diagonal entry of \(M_A\) record?
Suppose every A level appears exactly once and \(M_A=I\). Which two components are confounded?
Does a disconnected incidence graph necessarily imply that the variance components are not identifiable? State why or why not.
For a design with diagonal covariance entries 12, same-A off-diagonal entries 3, and same-B off-diagonal entries 7, what residual variance is implied if those patterns identify the components?
Why can a likelihood profile be nearly flat even when the covariance component matrices are linearly independent?

Solutions. (1) It is 1 when the two observations share an A level and 0 otherwise. (2) The A random-intercept variance and residual variance, since both multiply \(I\) in \(V\). (3) No. Disconnected components can still contain distinct within-component sharing patterns that identify common variance parameters. (4) The diagonal is the sum of the three variances, so \(\sigma^2=12-3-7=2\) in squared response units. (5) Linear independence establishes structural identifiability, not precision; a small or sparse data set may contain little information to distinguish the resulting distributions.