Why Sparse Outcome Patterns Deserve Attention
In a binary-outcome GLMM, a predictor category with very few observations—or no observed events—can make an estimated effect unstable or non-finite. A contingency table is a simple first screen: it displays how many \(Y=1\) and \(Y=0\) observations occur in each predictor category. Its value is not that it proves a model has separation in every setting. Rather, it makes visible data patterns that can lead to separation or leave an effect estimated from very little information.
For a binary response, a common GLMM specifies the event probability conditional on a cluster’s random effects. In the notation below, \(Y_{ij}\) is the binary outcome for observation \(i\) in cluster \(j\), \(x_{ij}\) and \(z_{ij}\) are fixed- and random-effects design vectors, and \(b_j\) is a cluster-specific random effect.
The usual model assumptions include a correctly specified conditional mean structure and the stated random-effects distribution; outcomes are conditionally independent given the random effects and covariates. A simple table pooling observations across clusters is descriptive. It does not account for that dependence, nor does it show every covariate or random-effects term in the model. This distinction matters: the table can identify a pattern worth investigating, but the full model design and cluster structure determine whether the fitted GLMM is separated.
Reading a Predictor-by-Outcome Table
For a categorical predictor, list its levels in rows and the two outcome values in columns. Record the number of observations in each cell, not just row percentages. Row percentages show the observed event proportion in each category, but the cell counts reveal whether that proportion is based on many observations or only one or two.
For a binary predictor \(X\), a two-by-two table can be written with counts \(a\) and \(b\) for \(X=1\), and \(c\) and \(d\) for \(X=0\):
When all four counts are positive, the crude sample odds ratio is \(ad/(bc)\), and its log is finite. If a cell is zero, this calculation may be zero or undefined, and the corresponding empirical log-odds contrast can lie on a boundary. This table calculation is a descriptive screen, not a substitute for checking separation in the GLMM’s actual design.
Worked Example: An Empty Event Cell for a Binary Predictor
Consider hypothetical clustered observations summarized by exposure status and outcome. There are 18 exposed observations with no events, and 30 unexposed observations, of which 8 have an event and 22 do not.
| Exposure | Event, \(Y=1\) | No event, \(Y=0\) | Total |
|---|---|---|---|
| Exposed | 0 | 18 | 18 |
| Unexposed | 8 | 22 | 30 |
The observed event proportions are \(0/18=0\) among exposed observations and \(8/30=0.267\) among unexposed observations. The crude odds ratio, taking exposed versus unexposed, is
Its log is not finite. In the simple logistic model containing only this binary predictor, the exposed group’s observed event proportion is zero, so its group-specific fitted log-odds has no finite value. This is strong evidence of a separation problem for that simple model. For a GLMM with additional covariates and random effects, the table remains an important warning, but it cannot by itself establish separation of the full conditional model. Cluster and covariate patterns must also be considered.
Why a Zero Count Can Push an Estimate to a Boundary
The boundary behavior can be seen directly in a binomial likelihood. Suppose, for the moment, one predictor category has \(m\) observations and event probability \(p\), with \(r\) observed events. Ignoring a constant that does not depend on \(p\), its likelihood is
If \(0<r<m\), differentiating the log-likelihood gives \(r/p-(m-r)/(1-p)=0\), so the maximum occurs at \(\hat p=r/m\), strictly between zero and one. If \(r=0\), then \(L(p)\propto(1-p)^m\), which is maximized at \(p=0\). But \(\operatorname{logit}(p)=\log\{p/(1-p)\}\) tends to \(-\infty\) as \(p\) approaches zero. Thus the category’s empirical log-odds cannot be represented by a finite coefficient in a model that estimates a separate log-odds for that category. If \(r=m\), the analogous boundary is \(p=1\), with logit tending to \(+\infty\).
This argument explains the simplest empty-cell pattern; it does not prove every empty cell causes separation in every fitted model. Model terms constrain which probabilities can vary independently. In a GLMM, the conditional predictor also includes random effects, so the relevant diagnosis depends on the model’s full design, not solely on a marginal table.
Sparse Cells Without Empty Cells
A table can be sparse even when every cell contains at least one observation. In that case, finite empirical proportions may be available, but a small factor level can still give a highly uncertain estimate. A factor level represented by two observations contributes much less information about its event probability than a level represented by hundreds. If the model has several coefficients, interactions, or other predictors, the effective support for a particular contrast can be thinner still.
Worked Example: A Small Factor Level With Nonzero Cells
Suppose a hypothetical three-level factor has the following counts. Every level contains both outcome values, but level C has only two observations.
| Factor level | Event, \(Y=1\) | No event, \(Y=0\) | Total |
|---|---|---|---|
| A | 1 | 9 | 10 |
| B | 5 | 5 | 10 |
| C | 1 | 1 | 2 |
The sample event proportions are \(0.100\), \(0.500\), and \(0.500\) for A, B, and C. In a simple logistic model with B as the reference and a separate coefficient for C, the crude log-odds contrast for C versus B is
A rough standard-error calculation for that log odds ratio is
The corresponding rough Wald interval on the log-odds scale is \(0\pm1.96(1.549)\), or approximately \((-3.036,3.036)\). Exponentiating gives an odds-ratio interval of about \((0.048,20.8)\). These are simple logistic-model approximations, not a GLMM interval, but they illustrate how little precision two observations can provide. No cell is empty, so this table alone does not show the boundary pattern of the previous example. It does show that an apparently ordinary point estimate can be supported by very little data.
For a GLMM, the displayed standard-error calculation should not be treated as the model-based uncertainty calculation: it ignores clustering, random-effects estimation, and other covariates. The diagnostic lesson is about support. A tiny level should prompt scrutiny of the fitted effect, its uncertainty, and whether the model is trying to estimate more detail than the observations can sustain.
Rare Outcomes and Predictor Combinations
A low overall event rate is not itself separation. What matters for a predictor effect is how events and non-events are distributed across the values or combinations represented in the model. The following example shows that a rare outcome can still occur in every cell of a simple table.
Worked Example: Rare Events With No Empty Cells
In hypothetical data, a binary predictor divides 100 observations into two equal groups. Each group has 2 events and 48 non-events.
| Predictor | Event, \(Y=1\) | No event, \(Y=0\) | Total |
|---|---|---|---|
| \(X=0\) | 2 | 48 | 50 |
| \(X=1\) | 2 | 48 | 50 |
The overall event proportion is \((2+2)/100=0.040\), or 4.0%. Yet neither predictor group has an empty outcome cell. The crude odds ratio is
Thus the table shows a rare outcome but no separation pattern for this one-predictor comparison. The estimate may still be imprecise because there are only four events. In a more complicated model, cross-classifying the same observations by additional predictors could reveal empty or tiny cells that are hidden by this two-row summary. The table should therefore match the terms and scientifically relevant combinations in the proposed model as closely as is useful, while avoiding so many subdivisions that the display itself becomes unreadable.
A Practical Screening Workflow
For each categorical predictor of interest, tabulate both outcome values and totals. Keep the raw counts visible alongside any percentages.
Note whether a category has only events or only non-events, and whether a nonzero count is still so small that the estimated proportion is fragile.
If the model contains combinations of factors or interactions, a marginal table may hide sparsity in those combinations. Use a targeted cross-tabulation rather than assuming marginal counts tell the whole story.
Look for extreme or unstable fixed-effect estimates, very large standard errors, and fit warnings. Then assess separation using the actual model design; do not treat a warning sign or an optimizer message as a definitive diagnosis by itself.
This workflow complements, rather than replaces, the formal separation checks covered in the earlier material on linear programming tests for separation. A contingency table is easier to interpret and can help locate a problematic predictor level. A formal design-based check can assess the joint pattern across the model’s predictors, which a collection of one-predictor tables cannot do. Conversely, formal output is easier to investigate when the tables have already shown where data support is thin.
Common Mistakes and Limits
- “The outcome is rare, so the model is separated.” Rarity alone is not the criterion. Check whether outcome values are absent from predictor categories or combinations relevant to the model.
- “Every cell is nonzero, so separation is impossible.” Nonzero marginal cells do not rule out separation involving multiple predictors, and small counts can still produce unstable estimates.
- “An empty cell in any table proves the GLMM is separated.” The table may omit predictors, random effects, or cluster structure. Treat the cell as a warning and check the full model design.
- “A large coefficient proves separation.” An extreme estimate can arise from sparse data, scaling, or other fitting problems. Investigate the data pattern and model diagnostics rather than relying on the coefficient alone.
- “Merge levels until every cell is large.” Combining categories can change the scientific meaning and the model. Any regrouping needs substantive justification; it should not be done solely to make a table look less sparse.
Tables also have limits as a diagnostic. They summarize observed counts, not the information supplied by all model terms jointly, and a marginal table of clustered observations does not represent independent sampling. Their strongest use is transparent triage: identify where the outcome pattern may be forcing an estimate toward a boundary or leaving a coefficient weakly supported, then investigate that exact structure in the GLMM.
Check Your Understanding
Answer each question, then compare your reasoning with the brief solutions.
- A predictor category contains 0 events and 14 non-events. What does this indicate, and what can you conclude for a GLMM from that table alone?
- A factor level contains 1 event and 1 non-event, while all other levels have large counts. Is this an empty-cell pattern? What concern remains?
- In a two-by-two table, \(a=3\), \(b=27\), \(c=6\), and \(d=24\). Calculate the crude odds ratio and its log. Does this table have an empty cell?
- Why can a rare overall event rate coexist with no separation pattern in a particular predictor table?
Solutions. (1) The category has an empty event cell, a warning for a boundary estimate in a simple logistic model. For a GLMM, the table alone does not establish separation in the full conditional model. (2) No outcome cell is empty, but the level is supported by only two observations, so its estimated effect may be highly imprecise. (3) \(\widehat{\mathrm{OR}}=(3)(24)/\{(27)(6)\}=72/162=0.4444\), and \(\log(0.4444)\approx-0.8109\). All four cells are positive. (4) Rarity describes the total number of events; separation depends on how events and non-events align with the modeled predictors. A rare outcome can still appear in every predictor category.