Integrating Crossed Random Effects without Nesting Assumptions

Why the Integration Structure Matters

In a generalized linear mixed model, observations may share more than one grouping factor. A response can depend on a subject effect and an item effect, for example, with the same subjects responding to multiple items and the same items appearing for multiple subjects. Conditional on all random effects, the responses may be independent. But marginalizing those effects does not generally preserve a product of independent subject-specific or item-specific likelihoods.

That distinction matters when constructing the marginal likelihood for estimation. Treating a crossed design as though items were nested within subjects, or vice versa, changes which observations share random effects and therefore changes the model being integrated. The earlier material on blockwise integration for nested random-effects models concerns a structure in which lower-level groups belong to particular higher-level groups. In a crossed design, the same item effect can be shared across subjects, so the same block separation is not generally available.

Model and Marginal Likelihood

Definition: Let \(\mathcal O\) be the set of observed subject-item pairs, with pair \((i,j)\) denoting an observation from subject \(i\) on item \(j\). A crossed random-intercept GLMM has linear predictor \(\eta_{ij}=x_{ij}^{T}\beta+b_i+u_j\), where \(b_i\) is the random intercept for subject \(i\), and \(u_j\) is the random intercept for item \(j\). A standard model assumes \(b_i\overset{\text{ind}}{\sim}N(0,\sigma_s^2)\), \(u_j\overset{\text{ind}}{\sim}N(0,\sigma_t^2)\), and independence between the subject and item effects. Conditional on all these effects and the covariates, responses are independent and have the specified GLM response distribution and link.

For example, with binary responses and the logit link, \(P(Y_{ij}=1\mid b_i,u_j)=\operatorname{logit}^{-1}(\eta_{ij})\), where \(\operatorname{logit}^{-1}(a)=1/(1+\exp(-a))\). More generally, write \(f_{ij}(y_{ij}\mid b_i,u_j;\beta)\) for the conditional response probability or density. With independent Gaussian random intercepts, the marginal likelihood is

L(\beta,\sigma_s^2,\sigma_t^2) = \int_{\mathbb R^{I}}\int_{\mathbb R^{J}} \left\{\prod_{(i,j)\in\mathcal O} f_{ij}(y_{ij}\mid b_i,u_j;\beta)\right\} \left\{\prod_{i=1}^{I}\phi(b_i;0,\sigma_s^2)\right\} \left\{\prod_{j=1}^{J}\phi(u_j;0,\sigma_t^2)\right\} \,d\mathbf u\,d\mathbf b .

Here \(I\) and \(J\) are the numbers of represented subjects and items, \(\mathbf b=(b_1,\ldots,b_I)^T\), \(\mathbf u=(u_1,\ldots,u_J)^T\), and \(\phi(\,\cdot\,;0,\sigma^2)\) is the normal density with mean zero and variance \(\sigma^2\). The integral is over every distinct random effect in the model, not one effect per observation. If the model has random-effect vectors rather than random intercepts, the corresponding multivariate Gaussian densities replace the scalar densities, and each observation's predictor includes the appropriate design-matrix products.

Worked Example: Writing the Likelihood for an Incomplete Crossed Design

Suppose three hypothetical subjects, A, B, and C, respond to items 1 and 2, but subject C has no recorded response to item 2. There are five binary responses. Let \(y_{ij}\) denote the recorded response, \(b_A,b_B,b_C\) the subject intercepts, and \(u_1,u_2\) the item intercepts. Take \(\eta_{ij}=\beta_0+b_i+u_j\).

For a binary response, the conditional probability contribution is \(p_{ij}^{y_{ij}}(1-p_{ij})^{1-y_{ij}}\), where \(p_{ij}=\operatorname{logit}^{-1}(\beta_0+b_i+u_j)\). The conditional likelihood includes exactly the observed pairs:

L(\beta_0,\sigma_s^2,\sigma_t^2) = \int_{\mathbb R^3}\int_{\mathbb R^2} \prod_{(i,j)\in\{(A,1),(A,2),(B,1),(B,2),(C,1)\}} p_{ij}^{y_{ij}}(1-p_{ij})^{1-y_{ij}} \prod_{i\in\{A,B,C\}}\phi(b_i;0,\sigma_s^2) \prod_{j\in\{1,2\}}\phi(u_j;0,\sigma_t^2) \,d\mathbf u\,d\mathbf b.

There are three subject integration dimensions and two item dimensions, for five dimensions total. The absent C-item 2 response contributes no response factor, but the subject C and item 2 effects remain in the integral because they occur in other observed pairs. In particular, item 1 is shared by A, B, and C, while each of those subjects is also observed on item 2 except C. This is not a design in which each item's random effect belongs to just one subject.

Why Crossed Effects Do Not Generally Separate

The integrand contains one response factor for each observed pair. A factor for pair \((i,j)\) depends on both \(b_i\) and \(u_j\). When an item appears for multiple subjects, its single effect \(u_j\) is shared across their response factors. When a subject responds to multiple items, its single effect \(b_i\) is shared across those factors. Integrating one effect therefore leaves the other effects coupled through the observations they jointly influence.

A small complete \(2\times2\) design makes the dependency visible. Its conditional response likelihood contains four terms, for pairs \((1,1),(1,2),(2,1),(2,2)\). The factors for the first subject involve both item effects, and the factors for each item involve both subject effects. Thus the likelihood is not a product of one term involving only subject 1 and item effects assigned uniquely to it, followed by an independent term for subject 2. The same item effects occur in both subjects' terms.

This can also be expressed through the subject-item incidence graph: subjects and items are vertices, and an observed pair is an edge connecting its subject to its item. A shared item connects the subjects observed on that item; a shared subject connects the items observed for that subject. Within a connected component containing shared effects, the joint likelihood generally remains coupled. Separate connected components do factor, because they involve disjoint random effects and response factors. Such component factorization is not the same as treating each subject or item as an independent nested block.

Key result: Conditional independence of responses given all random effects permits the conditional likelihood to be written as a product over observed pairs. It does not imply that the marginal likelihood factors by subject or by item after integration. Such a further factorization requires that the relevant response factors and random effects separate; shared crossed effects generally violate that condition.

Worked Example: A Coarse Quadrature Illustration of Shared-Effect Dependence

Consider two hypothetical binary responses from one subject on two distinct items. Set the fixed intercept to zero, let the subject and item random intercepts each have standard normal distributions, and assume conditional independence given all three effects. The exact probability that both responses equal 1 is

P(Y_1=1,Y_2=1) = \int \phi(b;0,1) \left\{\int \phi(u_1;0,1)\operatorname{logit}^{-1}(b+u_1)\,du_1\right\} \left\{\int \phi(u_2;0,1)\operatorname{logit}^{-1}(b+u_2)\,du_2\right\} \,db.

Use a deliberately coarse symmetric two-point rule, placing equal weight on each of \(-1\) and \(1\) for each standard normal integral. This is an illustrative approximation, not an exact evaluation of the Gaussian integral. For a fixed subject-effect node \(b=c\), the approximate single-response probability after averaging over an item effect is \(q(c)=\{\operatorname{logit}^{-1}(c-1)+\operatorname{logit}^{-1}(c+1)\}/2\). At \(c=1\),

q(1)\approx\frac{\operatorname{logit}^{-1}(0)+\operatorname{logit}^{-1}(2)}{2} =\frac{0.5000+0.8808}{2} =0.6904.

By symmetry, \(q(-1)\approx 1-q(1)=0.3096\). Averaging over the two subject-effect nodes gives an approximate marginal success probability \(\{0.6904+0.3096\}/2=0.5000\) for either response. But the two responses share the same subject effect. Their joint probability under this rule is

P(Y_1=1,Y_2=1)\approx\frac{0.6904^2+0.3096^2}{2} =\frac{0.4767+0.0959}{2} \approx 0.2863.

Multiplying the two marginal probabilities as if the responses were marginally independent would give \(0.5000^2=0.2500\), about \(0.0363\) lower than the joint approximation. The difference illustrates the dependence induced by their shared subject effect; it is not an estimate of the exact difference for Gaussian random effects, since the quadrature rule is intentionally coarse. The calculation also shows why multiplying integrated per-observation probabilities discards the common effect.

A Short Justification of the Factorization Condition

The factorization issue follows directly from the structure of the integrand. Let \(g(\mathbf b,\mathbf u)\) be the product of all conditional response factors and all random-effect densities. If the observed pairs split into disjoint groups such that each group uses a disjoint set of random effects, then the integrand can be written \(g_1(\mathbf b_1,\mathbf u_1)g_2(\mathbf b_2,\mathbf u_2)\), with disjoint integration variables. Repeated integration then gives

\int g_1(\mathbf b_1,\mathbf u_1)g_2(\mathbf b_2,\mathbf u_2) \,d\mathbf b_1\,d\mathbf u_1\,d\mathbf b_2\,d\mathbf u_2 = \left(\int g_1\,d\mathbf b_1\,d\mathbf u_1\right) \left(\int g_2\,d\mathbf b_2\,d\mathbf u_2\right).

The disjoint-variable condition is essential. In a crossed component, a response factor can involve a subject effect that also occurs in another item's factors, or an item effect that also occurs in another subject's factors. The integrand therefore cannot generally be assigned to independent blocks with disjoint variables. Integrating a shared variable does not remove the dependence it creates among the factors that contained it.

Worked Example: When the Design Really Is Nested

Suppose each of four hypothetical subjects responds to items used only by that subject: subjects A and B each receive items 1 and 2, while subjects C and D each receive items 3 and 4. If item effects are indexed separately for each subject-item combination, no item effect is shared across subjects. Under independent subject and item effects, the four subject-specific sets of response factors and random effects use disjoint integration variables.

Write \(L_i(b_i,\mathbf u_i)\) for the product of subject \(i\)'s conditional response factors and the densities of the item effects assigned only to that subject, with the subject-effect density included as well. Then the full marginal likelihood factors as

L=\prod_{i\in\{A,B,C,D\}}\int L_i(b_i,\mathbf u_i)\,d\mathbf u_i\,db_i.

Here blockwise integration by subject is justified because each item effect belongs to one subject only. If item 1 were instead the same item observed by both A and B with one shared \(u_1\), that variable would occur in both subjects' response factors. The displayed product of independent subject-specific integrals would then no longer represent the model.

Practical Choices, Diagnostics, and Limitations

Once the integral has been written correctly, an approximation method must respect its joint dimension and dependence structure. Tensor-product quadrature can be applied to all random-effect coordinates, but its node count grows rapidly with dimension. Sparse-grid methods and Laplace approximation are alternatives for some problems; they still require the correct crossed-effects integrand. Replacing the model with a nested one merely to simplify integration is not a numerical approximation to the same likelihood: it changes which observations share effects.

Before computing an approximation, check the design and the integrand:

Check the incidence pattern. Confirm which subject-item pairs are observed and whether a subject or item effect is reused across multiple response factors.
Count unique effects, not rows. The integration dimension is determined by the distinct random effects (or coordinates in the random-effect vectors), not by the number of observations.
Check factorization claims. A proposed block decomposition must assign each random-effect variable and each response factor to a block without sharing variables across blocks.
Check the model against the scientific design. If the same item is presented to different subjects, a model that gives each subject a separate item effect does not represent one shared item effect.

A factorization error can produce plausible-looking numerical output and still target the wrong likelihood. Other limitations remain even with the correct integral: high dimension can make numerical integration costly, and an approximation can be inaccurate if the integrand is sharply concentrated or has important features that the chosen numerical rule does not resolve. Convergence of a numerical computation cannot validate an incorrectly specified nesting structure.

Common Pitfall

Common mistake: “Responses are conditionally independent, so I can integrate each subject's responses separately and multiply the results.”

Correction: Conditional independence gives a product of response factors before integration. Separate subject-wise marginal likelihoods are valid only when the subject-wise blocks use disjoint random effects and response factors. If an item effect is shared by subjects, integrating that effect jointly couples their response factors; multiplying separately integrated subject likelihoods removes the sharing in the model.

Check Your Understanding

Use the definitions and factorization condition above to answer the questions. Brief solutions follow the questions.

A design has one shared item observed by each of six subjects. Does conditional independence imply that the marginal likelihood factors into six independent subject likelihoods? Why or why not?
A crossed model has 12 subjects and 8 items, with random intercepts for each. How many scalar random-effect coordinates are integrated over if all are represented?
When can disconnected parts of a subject-item incidence graph be integrated as separate likelihood contributions?
In the quadrature illustration, why is the approximate probability of two successes not \(0.5000^2\)?
What design change would justify integrating item effects within subject-specific blocks?

Solutions. (1) No. The shared item effect appears in response factors for multiple subjects, so their likelihood contributions remain coupled after that effect is integrated out. (2) There are \(12+8=20\) scalar coordinates: one for each subject and one for each item. (3) Separate integration is valid when the components use disjoint random effects and response factors, with independent random-effect distributions across components as assumed by the model. (4) Both responses share the same subject effect, so their conditional probabilities vary together as that effect changes. The coarse rule gives \(\{0.6904^2+0.3096^2\}/2\approx0.2863\), whereas the product of marginal probabilities is \(0.2500\). (5) Each item effect must belong to only one subject, with no item effect reused across subjects; then subject-specific blocks have disjoint integration variables.