Why the Unit of Inspection Matters
A pooled table can hide how binary outcomes and predictors are distributed across clusters. If a predictor is associated with the outcome in the pooled observations, that pattern may reflect contrasts within clusters, differences in which clusters contribute each predictor value, or both. The distinction matters when assessing separation: a pooled pattern does not by itself show that the predictor separates outcomes within clusters.
The earlier material on separation established the central idea: separation occurs when a direction in the predictors distinguishes successes from failures, so that the likelihood can improve as a coefficient moves without bound rather than reaching a finite maximum. Here the focus is the level at which the pattern occurs. The same grouped binary data can have a pooled separation pattern, cluster-specific patterns, both, or neither.
Model and Grouped Data
Let \(Y_{ij}\) be the binary outcome for observation \(i\) in cluster \(j\), with \(Y_{ij}=1\) denoting a success and \(Y_{ij}=0\) a failure. A simple random-intercept logistic GLMM is
This specification assumes conditional independence of outcomes given the random effects and covariates, and normally distributed, mean-zero random intercepts with variance \(\sigma_b^2\). The separation checks in this tutorial are data-pattern diagnostics for the fixed predictor and cluster composition; they are not a substitute for assessing the fitted GLMM and its assumptions.
For a binary predictor, grouped data can be summarized by cluster, predictor value, number of trials, and number of successes. A cell with \(y=0\) has only failures; a cell with \(y=n\) has only successes; a cell with \(0<y<n\) contains both outcomes. These counts show where the available within-cluster contrasts are—and where they are absent.
Worked Examples: Comparing the Two Levels
Worked Example: Neither the Pooled Table Nor the Clusters Separate
Suppose a hypothetical study records a binary predictor \(x\) in two clusters. Each table entry gives successes out of the stated number of trials.
| Cluster | \(x\) | Trials | Successes |
|---|---|---|---|
| A | 0 | 10 | 2 |
| A | 1 | 10 | 8 |
| B | 0 | 10 | 3 |
| B | 1 | 10 | 7 |
Pooled across clusters, \(x=0\) has \(2+3=5\) successes among \(10+10=20\) trials, giving \(5/20=0.25\). For \(x=1\), there are \(8+7=15\) successes among 20 trials, giving \(15/20=0.75\). The pooled success proportions differ, but each predictor value has both successes and failures: \(20-5=15\) failures at \(x=0\), and \(20-15=5\) failures at \(x=1\). Thus this binary predictor does not completely separate the pooled outcomes.
The same check within clusters gives A: \(2/10\) at \(x=0\) and \(8/10\) at \(x=1\); B: \(3/10\) and \(7/10\), respectively. Every cluster-by-predictor cell contains both outcomes, and both values of \(x\) occur in each cluster. These tables show a consistent association, not complete separation within either cluster.
Worked Example: Pooled Separation Without Within-Cluster Predictor Contrast
Consider a different hypothetical data set.
| Cluster | \(x\) | Trials | Successes |
|---|---|---|---|
| A | 0 | 12 | 0 |
| B | 1 | 12 | 12 |
Pooled across clusters, all 12 observations with \(x=0\) are failures, and all 12 with \(x=1\) are successes. Thus \(x\) completely separates the pooled outcomes. But \(x\) is constant within each cluster: A contributes only \(x=0\), while B contributes only \(x=1\). There is no within-cluster comparison of \(x=0\) with \(x=1\).
The pooled diagnosis is correct as a description of the pooled table, but it does not show that a change in \(x\) separates outcomes among observations in the same cluster. Predictor value is aligned with cluster membership here, so cluster composition and predictor value cannot be disentangled by within-cluster contrasts in these data. A pooled pattern is not evidence that the predictor has within-cluster support.
Worked Example: Opposing Within-Cluster Patterns Hidden by Pooling
Now suppose the hypothetical counts are as follows.
| Cluster | \(x\) | Trials | Successes |
|---|---|---|---|
| A | 0 | 8 | 0 |
| A | 1 | 8 | 8 |
| B | 0 | 8 | 8 |
| B | 1 | 8 | 0 |
Within A, all outcomes at \(x=0\) are failures and all at \(x=1\) are successes, so the within-cluster pattern separates in the increasing direction. Within B, all outcomes at \(x=0\) are successes and all at \(x=1\) are failures, so the pattern separates in the opposite direction.
After pooling, the \(x=0\) group has \(0+8=8\) successes among 16 trials, and the \(x=1\) group has \(8+0=8\) successes among 16 trials. Each pooled predictor value contains both outcomes, so \(x\) does not separate the pooled data. Pooling has concealed the cluster-specific patterns and their opposing directions. This does not mean the clusters have no separation pattern; it means a single pooled table does not reveal it.
Why Separation Can Prevent a Finite Maximum
A short likelihood argument explains the significance of a separating pattern. In logistic regression, for grouped observations with predictor vector \(x\), linear predictor \(\eta\), \(y\) successes, and \(n-y\) failures, the binomial log-likelihood contribution is
Suppose a coefficient direction makes \(\eta\) increase for cells containing only successes and decrease for cells containing only failures. Along that direction, the success-cell contribution approaches its upper limit of zero as \(\eta\) increases, and the failure-cell contribution approaches zero as \(\eta\) decreases. If no mixed-outcome cells obstruct the direction, the likelihood can approach its supremum as the coefficient grows in magnitude, rather than attain it at a finite value. This is the likelihood basis of the standard separation diagnosis.
When observations are grouped, the same logic applies to the underlying outcome patterns: cells containing only one outcome can support separation, while cells containing both outcomes constrain that direction. In the GLMM, the likelihood also involves the random effects and their distribution. Therefore, the table argument is a diagnostic for the observed predictor–outcome structure, not a proof that the integrated GLMM likelihood has or lacks a finite maximizer.
A Practical Diagnostic Sequence
For a categorical predictor, tabulate trials and successes by predictor value, summing over clusters. Identify empty outcome cells and values with only successes or only failures.
Repeat the tabulation within every cluster. Note whether a predictor value is absent, whether both predictor values occur, and whether each cell contains both outcomes.
Separate an apparent pooled pattern from an actual within-cluster contrast. Record whether cluster-level directions agree, oppose one another, or cannot be assessed because predictor values do not vary within a cluster.
Report which data level shows separation-like patterns and which does not. Do not infer that a diagnostic settles finite estimation for the GLMM; also examine the model fit and its assumptions.
For a continuous predictor or several predictors, the same question is geometric: is there a direction in the predictor values that orders successes and failures? Assess the pattern pooled and within clusters using the relevant separation diagnostic. For a cluster with too few observations or too little predictor variation, a within-cluster check may be uninformative; that is a limitation of the available data, not evidence of overlap.
Alternatives, Failure Modes, and Limits
A pooled logistic regression or pooled contingency table is useful for describing an overall predictor–outcome pattern, but it discards cluster labels for the separation check. Cluster-specific tables expose heterogeneity and missing within-cluster support, although sparse clusters can make their patterns unstable. A model with cluster indicators provides another way to represent cluster differences, but it is a different specification from a random-intercept GLMM. These approaches answer related but nonidentical questions; no one table replaces checking the actual model and its design.
- Pooled-only checking: may attribute a pattern to predictor contrasts even when predictor values are segregated across clusters.
- Cluster-only checking: can miss a pooled pattern or a consistent direction that is visible only when information is combined.
- Ignoring direction: clusters can show clear but opposing patterns, which pooling may cancel.
- Equating a warning with a verdict: all-success or all-failure cells flag possible separation, but grouped tables alone do not determine the finite-estimation properties of every GLMM.
- Treating a failed check as reassurance: lack of within-cluster variation means the within-cluster predictor effect cannot be assessed from that cluster; it does not establish adequate information.
Convergence warnings, extreme coefficient estimates, or sensitivity to small data changes may prompt closer inspection, but they are not interchangeable with a separation diagnosis. Conversely, an apparently successful optimizer does not make a pooled-only assessment sufficient. The key limitation is that separation labels summarize particular predictor–outcome patterns; interpretation must also account for the model, cluster structure, and available support.
Check Your Understanding
For each question, identify the relevant data level and state what can be concluded from the described counts.
- In one cluster, \(x=0\) has 9 trials and 0 successes; \(x=1\) has 9 trials and 9 successes. What pattern is present?
- Two clusters have constant predictor values: Cluster A contributes only \(x=0\) failures, and Cluster B only \(x=1\) successes. Does the pooled table establish a within-cluster predictor contrast?
- Two clusters show complete separation in opposite directions, with equal trial counts at each predictor value. What may pooling do to the predictor’s pattern?
- Every pooled predictor cell contains both successes and failures, but one cluster has only successes at its observed predictor values. Is it correct to conclude that no cluster-level pattern needs inspection?
Solutions. (1) The cluster has a within-cluster separation pattern in the increasing direction: all failures occur at \(x=0\) and all successes at \(x=1\). (2) No. The pooled table separates, but \(x\) does not vary within either cluster, so the table provides no within-cluster comparison of predictor values. (3) Pooling can cancel the opposing patterns, leaving both outcomes at each predictor value and no pooled separation by \(x\). (4) No. The pooled table can conceal a cluster-specific pattern; inspect the cluster tables before drawing a conclusion.