From the Sampling Design to the Model
A complex hierarchy is easiest to model when its structure is described before any formula is written. Consider a hypothetical study in which participants are recruited at sites, sites belong to regions, and each participant answers a subset of a common bank of items. A response can share features with another response because it comes from the same region, site, or participant, or because it concerns the same item. Those are distinct sources of dependence, and the model should represent the distinctions the sampling design supports.
The site and participant memberships are nested: each site belongs to one region, and each participant belongs to one site. Items are crossed with participants when the same item can be answered by participants from multiple groups. The observed data need not contain every possible participant–item pair. An incomplete set of combinations is not, by itself, a reason to treat the item as nested within the participant.
For precision, let \(r\) index regions, \(s\) sites within region \(r\), \(p\) participants within site \(s\), and \(j\) items. Let \(O\) be the set of observed participant–item responses; it can be a proper subset of all possible combinations. Let \(Y_{rspj}\) be a binary response and \(x_{rspj}\) a fixed predictor, such as a treatment indicator. One possible logistic GLMM is:
Here \(b_r\), \(u_{rs}\), \(v_{rsp}\), and \(w_j\) are respectively the region, site-within-region, participant-within-site, and item random intercepts. A standard starting assumption is that these effects are mutually independent and normally distributed with mean zero:
The model also assumes that responses are conditionally independent given the random effects and fixed predictors, and that the conditional Bernoulli distribution and logit mean model are appropriate. The independence assumption applies after conditioning on all included effects; it does not claim that the observed responses are marginally independent. The random effects are assumed to describe variation among levels from populations represented by the sampling design. If the sites or items are deliberately selected and inference is only about those exact levels, a fixed-effect representation may be more suitable.
Justifying Each Term
A random-effects term is not justified merely because a column in the data has a name. The key questions are what was sampled, which records share that sampled unit, and what broader population of levels the analysis is intended to represent.
Record which region contains each site, which site contains each participant, and which participants answered each item. Confirm whether item labels refer to the same item across groups.
Responses from one region may share regional conditions; responses from one site may share site-specific conditions; repeated responses from one participant may be correlated; and a difficult or easy item may affect all participants who answer it.
Use nested terms for site within region and participant within site, and a crossed item term when items recur across participants. Add fixed predictors for the scientific contrasts, not as substitutes for grouping structure.
Inspect replication, overlap, and connectedness. Incomplete combinations can be used, but sparse or disconnected patterns may leave some variance components weakly identified or prevent useful comparisons.
The model's implied covariance gives a direct justification for its terms. On the latent linear-predictor scale, two distinct responses have covariance equal to the sum of random-effect variances for grouping levels they share, under the independence assumptions above. For example, two responses from the same participant and different items share their region, site, and participant, but not their item effect. Responses from different participants in the same site that use the same item share region, site, and item effects. They do not share a participant effect. Thus the random-effects structure encodes which pairs of records are expected to be related and why.
Worked Model-Building Examples
Worked Example: A Region–Site–Participant Hierarchy With Shared Items
Hypothetical design. A team samples 4 regions, 3 sites in each region, and 20 participants in each site. Participants answer a subset of 30 survey items. Each item is used by participants from multiple sites, although not every participant answers every item. The outcome is a Yes/No response, and a treatment indicator varies within each site.
The response records are indexed by region \(r\), site \(s\) within region, participant \(p\) within site, and item \(j\). Region, site, and participant memberships are nested in sequence. The item labels are shared across sites and participants, so item is crossed with participant rather than nested within participant. Because treatment varies within a site, it is a fixed predictor in this model; whether it also needs a participant-specific random slope depends on the treatment assignment and replication.
For this initial random-intercept specification, the model is:
There are \(4\) region levels, \(4\times3=12\) site levels, and \(4\times3\times20=240\) participant levels. There are 30 item levels. These are counts of grouping levels, not counts of independent observations; the number of observed responses is the number of actually completed participant–item combinations. The incomplete combinations remain absent from the response data rather than being coded as No.
The term \(u_{rs}\) is site variation conditional on its region membership: every site has one parent region. The term \(v_{rsp}\) is participant variation conditional on the site containing that participant. The shared \(w_j\) term allows responses to the same item by different participants to be related. A model using a distinct item effect for each participant would instead represent a different design and would discard the intended sharing across participants.
Worked Example: Quantifying Dependence From Shared Levels
Suppose a hypothetical fitted model has latent-scale variance estimates \(0.20\) for regions, \(0.30\) for sites, \(0.50\) for participants, and \(0.40\) for items. Under the independent random-intercept assumptions, calculate the random-effect covariance for two kinds of response pairs.
First, consider two different items answered by the same participant at the same site. They share region, site, and participant, but have different item effects. Therefore:
Second, consider two different participants at the same site answering the same item. They share region, site, and item, but not participant. The corresponding calculation is:
These are covariances contributed by the random intercepts on the linear-predictor scale; they are not response-scale covariances or correlations. In a logistic model, the nonlinear link means that these values cannot be read as differences in response probability.
Worked Example: Keeping an Incomplete Crossed Design Incomplete
In a hypothetical site, participant A answers items 1 and 2, while participant B answers items 2 and 3. The observed outcomes are \(Y_{A1}=1\), \(Y_{A2}=0\), \(Y_{B2}=0\), and \(Y_{B3}=1\). The combinations A–3 and B–1 are unobserved.
Suppose, for illustration, the fitted conditional response probabilities for these four observed records are \(0.70\), \(0.40\), \(0.30\), and \(0.80\), respectively. Conditional on the random effects, the Bernoulli likelihood contribution from the observed records is:
The likelihood contains four response contributions because four combinations were observed. There is no contribution for A–3 or B–1, and neither missing combination is assigned an outcome of zero. In an actual GLMM, the likelihood for the observed data is integrated over the random-effects distribution; the calculation above is the conditional likelihood contribution at specified effects, not the integrated likelihood.
This example establishes how to represent the observed pattern, not why responses are missing. If completion depends on unobserved outcomes or unmodeled participant characteristics, the missingness mechanism may require separate treatment; a crossed random effect alone does not make informative missingness harmless.
Worked Example: Interpreting a Conditional Prediction
Suppose a hypothetical fitted model has \(\hat{\beta}_0=-0.40\), \(\hat{\beta}_1=0.70\), and estimated random-effect values for one record of \(b_r=0.10\), \(u_{rs}=-0.20\), \(v_{rsp}=0.30\), and \(w_j=-0.10\). The record has \(x=1\). Its fitted conditional linear predictor is:
Transforming from log odds to probability gives:
This is a conditional fitted probability for the specified region, site, participant, and item effects. It is not the population-average probability for a new participant or item. The treatment coefficient \(0.70\) is a conditional log-odds contrast for \(x=1\) versus \(x=0\), holding the random effects fixed; its conditional odds ratio is \(\exp(0.70)\approx2.014\).
Diagnostics, Alternatives, and Limitations
Before fitting, check that factor labels reflect actual membership. Participant identifiers reused at different sites must be coded so that each participant is uniquely identified within the appropriate site. Otherwise, the model can accidentally treat distinct participants as one shared level. Likewise, if “item 4” refers to different content at different sites, it is not a single globally crossed item level. If items are shared only within regions, the proposed global item term may not match the design.
After fitting, examine convergence warnings, estimated variances near zero, large uncertainty in variance estimates, and sensitivity to reasonable alternative specifications. These can indicate weak information, sparse replication, or an unsupported random-effects structure; they do not automatically establish that a particular term should be deleted. Check whether residual dependence remains aligned with a grouping factor that the model omitted. Also inspect the observed membership graph: a nominally crossed design with little overlap, or disconnected components, may provide limited information for separating effects. Missing combinations can coexist with a usable model, but they do not guarantee estimability.
The main alternatives answer different questions. Treating items as nested within participants is appropriate only if each item level is genuinely unique to a participant; otherwise, it fails to represent shared item variation. Item fixed effects can control differences among a specified set of items, but do not estimate a population variance for sampled items. Omitting site or participant effects imposes stronger conditional-independence assumptions and may understate uncertainty if those levels induce dependence. A simpler model may be useful when the design or data cannot support the full structure, but the simplification should be explicit and justified by the inferential target.
A random-intercept model also has limits. It assumes a common conditional treatment effect unless a random slope or interaction is added. A participant-level random treatment slope is relevant when treatment varies within participants and participant-specific treatment responses are scientifically plausible and supported by the data. It is not automatically appropriate when treatment is constant within participant. More elaborate random-effects structures require adequate replication and can be unstable in sparse designs. Finally, this tutorial's binary-response example does not settle the choice of outcome distribution for other response types.
Common Mistake
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- Participants belong to one site each, while the same items are answered by participants from several sites. Which factor is nested and which is crossed with participants?
- Two records are from the same participant and same item, but different response occasions. Which of the four random intercepts in the model do they share?
- A participant does not answer item 7. Should that absent combination be recorded as a No response in the Bernoulli outcome?
- Under the model assumptions, calculate the shared random-intercept covariance for two different participants in the same site and region, answering different items, when the region and site variance estimates are \(0.15\) and \(0.25\).
- Why might item fixed effects be preferable to an item random intercept?
Solutions. (1) Participant is nested within site; item is crossed with participants because item levels are shared across participants. (2) They share the region, site, participant, and item effects, assuming the same item level and the specified hierarchy. (3) No. The combination is unobserved and contributes no Bernoulli outcome term; coding it as No would invent an observation. (4) They share region and site, but not participant or item, so the covariance contribution is \(0.15+0.25=0.40\) on the linear-predictor scale. (5) Fixed effects can be preferable when inference concerns only a particular, deliberately selected set of items rather than a population of sampled items, or when estimating item-specific contrasts is the goal.