Why Check Empty and Singleton Cells?
In sparse count data, the pattern near zero can be more informative than a global fit summary. A model may reproduce the overall mean count yet generate too few cells with no events, or too few cells with exactly one event. Such mismatches can matter when many cells have low expected counts, as in repeated measurements across subjects, clinics, or time periods. Counting empty and singleton cells gives a focused way to ask whether a fitted generalized linear mixed model (GLMM) reproduces this part of the observed data.
This is a predictive check: compute a summary from the observed outcomes, generate replicated outcomes from the fitted model under the same design, and compare the observed summary with the summaries from those replications. The comparison is evidence about model adequacy, not a stand-alone hypothesis test and not proof that a model is true or false.
Model and Replication Conditions
For a common sparse-count GLMM, assume a conditional Poisson model:
Here \(o_{ij}\) is a known offset, \(x_{ij}\) and \(z_{ij}\) are observed covariate vectors, \(\beta\) is a vector of fixed effects, and \(b_j\) is a cluster-level random effect with covariance matrix \(D\). The Poisson assumption applies conditional on the random effects; cells within a cluster need not be independent after those effects are integrated out. Other count-response GLMMs can be checked with the same two statistics, but the replication must follow the fitted response distribution.
A useful check requires that the cell definitions, covariates, offsets, and cluster structure in each replicate match those in the observed data. The model must specify a plausible conditional count distribution, link, fixed-effects structure, and random-effects distribution. Replicates should be generated using the fitted model, with the treatment of random effects and estimated parameters stated clearly. For example, a plug-in simulation uses fitted parameter estimates; it does not automatically include all uncertainty about those estimates. The interpretation of the comparison depends on the replication scheme.
The observed and replicated statistics must be computed in exactly the same way. Do not, for example, count structural cells with no opportunity for an event as empty in the observed data but omit them in the replicates. If cells have unequal exposure, preserve their offsets; otherwise a model can appear to miss the low-count pattern simply because the comparison has ignored different expected rates.
Why the Two Counts Are Informative
For a Poisson count with mean \(\mu\), the probabilities of zero and one event are \(P(Y=0)=e^{-\mu}\) and \(P(Y=1)=\mu e^{-\mu}\). Thus, for cell-specific Poisson means \(\mu_{ij}\), summing the indicator expectations gives:
The justification is direct: the expectation of an indicator \(I(Y_{ij}=0)\) is the probability that \(Y_{ij}=0\), and expectation is additive across cells. This sum-of-expectations result does not require the indicators to be independent. But dependence does affect the variability and joint distribution of \(T_0\) and \(T_1\). In a GLMM, the random effects induce dependence across cells in a cluster, so simply treating the cell indicators as independent can give an incorrect reference distribution. Replicated data from the fitted model retain the modeled dependence.
The formula is conditional on specified Poisson means. In a GLMM, a marginal probability for a cell generally averages over the random-effects distribution, and may not equal the probability calculated using a single plug-in value of \(b_j\). Simulating from the fitted GLMM is therefore a practical way to compare the observed statistics with the distribution implied by its full specified replication scheme.
Worked Example: Expected Empty and Singleton Counts
Suppose five hypothetical cells have Poisson means \(0.2,\ 0.4,\ 0.8,\ 1.2,\) and \(2.0\). The observed counts are \(0,\ 1,\ 0,\ 2,\) and \(0\), respectively. For this calculation, treat the means as fixed and the cells as independent Poisson variables. The observed summaries are \(T_0=3\) and \(T_1=1\).
The expected number of empty cells is the sum of the five zero probabilities:
The expected number of singleton cells is the sum of \(\mu e^{-\mu}\) for the five means:
The observed three empty cells and one singleton cell are not, by themselves, evidence of a serious discrepancy: expected values alone do not show how much these statistics vary. In a GLMM, the relevant comparison would normally use replicated summaries that also reflect the fitted random effects and cluster dependence.
Comparing Observed Statistics With Replicates
Generate \(B\) replicated data sets from the fitted model, keeping the design and cell definitions fixed. For replicate \(r\), calculate \(T_0^{(r)}\) and \(T_1^{(r)}\). Then inspect the observed values against the simulated distributions: useful summaries include the simulated median, a central simulation interval, a plot of the replicate statistics, and the proportion of replicates at least as extreme in a direction chosen before looking at the result.
For example, if an unusually large number of empty cells is the concern, an upper-tail simulation proportion can be computed as \(\{1+\sum_{r=1}^{B} I(T_0^{(r)}\ge T_{0,\mathrm{obs}})\}/(B+1)\). The added 1 avoids a reported value of zero when no simulated statistic reaches the observed value. This proportion describes how often the simulated statistic reaches or exceeds the observed statistic under the chosen replication scheme. Because parameters were estimated from the data and the check is targeted, it should not be presented as an exact, calibrated frequentist p-value or as the probability that the model is correct.
Worked Example: An Unusually High Number of Empty Cells?
In a hypothetical sparse-count GLMM check, the observed data contain \(T_{0,\mathrm{obs}}=14\) empty cells and \(T_{1,\mathrm{obs}}=7\) singleton cells. In \(B=1000\) replicated data sets, the middle 90% of empty-cell counts runs from 6 to 15, and 37 replicates have at least 14 empty cells. The middle 90% of singleton counts runs from 4 to 13, and 92 replicates have at most 7 singleton cells.
The observed empty-cell count is inside its central simulation interval, though toward its upper end. Its upper-tail simulation proportion is:
The observed singleton count is also inside its central interval. Its lower-tail simulation proportion is:
These summaries give a possible indication that the observed data have more empty cells than many fitted-model replicates, while the singleton count is not comparably unusual in the lower direction. The two statistics are related and should not be treated as independent confirmations. The simulation proportions are diagnostic summaries, not probabilities that the GLMM is true or that a particular alternative is true. The appropriate conclusion is to investigate the pattern and model assumptions, not to reject the model solely because one summary is near a conventional cutoff.
Preserve Cluster Structure
A pooled \(T_0\) or \(T_1\) can conceal where the low counts occur. If empty cells are concentrated in one cluster, that can be diagnostically different from the same total spread evenly across clusters. A useful extension within this check is to retain the cluster labels and compare a cluster-specific summary, such as the maximum number of empty cells in any cluster, with its replicated distribution. The statistic should be chosen to answer a clear question, rather than selected after searching many summaries for an extreme result.
Worked Example: Empty Cells Concentrated in One Cluster
Suppose a hypothetical data set has four clusters with ten count cells each. The observed numbers of empty cells are \(8,\ 2,\ 2,\) and \(2\), so the pooled total is \(T_0=8+2+2+2=14\). Consider the statistic \(M_0\), the largest number of empty cells in any one cluster. Here \(M_{0,\mathrm{obs}}=8\).
Under 1000 replicates from the fitted GLMM, suppose the middle 90% of pooled empty-cell counts is 8 to 18, with a median of 13. The observed pooled total of 14 is ordinary by that summary. Suppose the middle 90% of the replicated maximum cluster count is 4 to 7, however, and only 6 replicates have \(M_0^{(r)}\ge8\). Then:
The fitted model often produces a similar total number of empty cells, but rarely concentrates that many in one cluster. This suggests checking whether the model captures the relevant between-cluster pattern. It does not identify a unique cause: possible issues include an inadequate random-effects structure, an unusual cluster, or other misspecification. The calculation is meaningful only if the simulated replicates retain the observed cluster structure and the chosen summary was not selected solely because it looked extreme.
Diagnostics, Alternatives, and Limitations
A predictive check is most useful when the cell definition and discrepancy are scientifically interpretable. Look at the replicate distributions, not just a single tail proportion. Check that simulated outcomes are generated on the original count scale, that offsets and cluster membership are retained, and that the fitting and replication assumptions match. If the observed value is near or beyond a simulation interval, examine whether the discrepancy is concentrated in particular clusters, covariate ranges, or exposure levels. Residual plots and calibration checks for sparse counts address related but different aspects of fit; no one diagnostic replaces the others.
Empty- and singleton-cell counts have limitations. They compress the data and discard distinctions among counts above one. Different model failures can produce similar values of \(T_0\) and \(T_1\); a close match does not establish that the rest of the outcome distribution or dependence structure is adequate. Conversely, an unusual statistic can arise from an isolated feature, multiple diagnostic comparisons, or parameter estimation uncertainty. Interpret the check alongside the design, model assumptions, and other diagnostics.
A common mistake is to compare the observed number of zeros only with its expected value and conclude that any difference indicates misfit. For example, observing three zeros when \(E(T_0)=2.3749\) does not establish a discrepancy: the relevant question is whether three is unusual under the model's predictive distribution, including the dependence induced by random effects. Another mistake is to call a simulation tail proportion the probability that the null model is true. It is instead a frequency computed from replicated data under a specified fitted model and replication procedure.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- Under a Poisson model with mean \(\mu=0.5\) for each of four cells, what are the expected numbers of empty and singleton cells?
- Why can dependence among cells affect a predictive check even though the expected number of empty cells is a sum of cell-level probabilities?
- In 499 replicates, 9 have an empty-cell count at least as large as observed. Calculate the add-one upper-tail simulation proportion and state what it describes.
- Why might a cluster-specific empty-cell statistic detect a pattern missed by the pooled total?
Solutions. (1) Each cell has zero probability \(e^{-0.5}\approx0.6065\), so \(E(T_0)=4e^{-0.5}\approx2.426\). Each has singleton probability \(0.5e^{-0.5}\approx0.3033\), so \(E(T_1)=4(0.5)e^{-0.5}\approx1.213\). (2) Additivity of expectation does not require independence, but dependence changes the variation and joint distribution of the sums. GLMM replications should retain the modeled cluster dependence. (3) The add-one proportion is \((1+9)/(499+1)=10/500=0.0200\). It is the fraction, with the add-one convention, of replicated statistics at least as large as observed under the specified fitted-model simulation; it is not the probability that the model is true. (4) The same pooled number of empty cells can be evenly distributed or concentrated in one cluster. A cluster-specific statistic distinguishes these arrangements and can reveal a mismatch that the pooled count hides.