When Counts Are Too Concentrated
Count diagnostics often ask whether observed outcomes are more variable than a fitted model predicts. The reverse can matter too: an outcome may be underdispersed relative to the model, meaning that observed counts are less variable, or more concentrated in particular ranges, than replicated counts from that model. A check aimed only at unusually large variability can miss this evidence of mismatch.
The comparison must be made with the fitted model’s predictions, not with a rule such as “the sample variance should equal the sample mean.” In a generalized linear mixed model, different observations can have different fitted means, and shared random effects affect the joint predictive distribution. The earlier material on predictive checks provides the general simulation framework; here the focus is on what to look for in its lower tail and across count frequencies.
For example, let \(Y_{ij}\) be the count for observation \(i\) in cluster \(j\), and let \(b_j\) denote that cluster’s random effect. A conditional Poisson GLMM specifies
Conditional on \(b_j\), the Poisson distribution has variance \(\mu_{ij}\). But this conditional statement does not imply that all observed counts have the same mean or that the marginal variance across clusters equals a single mean. A useful check therefore compares the observed data with replicated data generated under the fitted model, preserving the design and using a clearly stated treatment of the random effects.
What to Compare
A simple discrepancy is the Pearson-type statistic
where \(\widehat{\mu}_{ij}\) is the fitted mean used for the comparison. A small value indicates that the observed counts are close to their fitted means in aggregate. To assess whether it is unusually small, calculate the same statistic for each replicated data set, using the same fitting and calculation procedure. The relevant evidence is in the lower tail of the simulated statistics.
This statistic is only one view of concentration. Frequency checks can also ask whether there are more counts near a central range, and fewer unusually small or large counts, than the model predicts. Those patterns may be informative even when the Pearson statistic is not striking. Conversely, a small Pearson statistic can result from some observations being close to their means while others are not; it does not describe every feature of the count distribution.
Worked Example: Correcting and Interpreting a Pearson Discrepancy
Suppose a hypothetical fitted model gives eight fitted means, in order, of \(0.5, 1, 2, 4, 0.5, 1, 2, 4\). The corresponding observed counts are \(0, 1, 1, 5, 1, 0, 3, 3\). Calculate the Pearson discrepancy, then compare it with 20 simulated discrepancies.
Substitution gives
The observed value is \(3.5\), not \(3.3125\). Suppose, hypothetically, the 20 simulated discrepancies sorted from smallest to largest are \(0.5, 0.75, 1, 1.25, 1.5, 1.75, 2, 2.25, 2.5, 2.5, 2.75, 2.75, 3, 3, 3.25, 3.25, 3.5, 3.5, 3.75,\) and \(4.0\). There are 18 values at or below the correct observed value of \(3.5\).
The empirical lower-tail proportion is \(18/20=0.900\). With the optional add-one estimate, it is \((18+1)/(20+1)=19/21\approx0.905\). Either summary says the observed discrepancy is smaller than most simulated discrepancies in this small set of replications, which is evidence of greater concentration than the model typically produced. It is a simulation-based diagnostic summary, not the probability that the model is true or a formal test with a guaranteed Type I error rate. In particular, a count of simulations below \(3.3125\) would not establish how many lie below \(3.5\); the comparisons must use the corrected observed statistic.
Frequency Patterns Can Reveal More
A variance summary compresses a whole distribution to one number. Underdispersion can instead appear as a systematic excess of counts near the fitted center, accompanied by too few counts in one or both tails. A useful check compares observed frequencies with the same frequencies in each replicated data set. When means differ across observations, define ranges with the fitted model in mind, or compare observed and replicated counts within groups of similar fitted means. Otherwise, variation in the means themselves can obscure the pattern.
Worked Example: Checking for Excess Counts Near the Center
Consider 12 hypothetical independent count observations with a common fitted mean of \(22/12\approx1.833\). The observed counts are \(1,2,2,1,3,2,1,2,3,2,1,2\). Their sum is \(4(1)+6(2)+2(3)=22\). Every observation lies between 1 and 3, inclusive, so the observed number in that central range is \(C_{\mathrm{obs}}=12\).
Suppose 500 parametric-bootstrap replications are generated under the fitted model, with the model refitted in each replication before calculating the same frequency statistic. In 3 of the 500 replications, at least 12 counts fall between 1 and 3. The estimated upper-tail proportion for this statistic is \(3/500=0.006\). Because unusually many central counts are the direction associated with concentration, this result flags a possible underdispersion pattern: the observed data place all 12 counts in the central range, while that occurs rarely in the replications.
This is a targeted diagnostic, not a stand-alone conclusion. The central range was selected to represent concentration around the fitted mean; other ranges or statistics could produce a different result. The counts at 0 or at 4 and above should also be compared with their replicated frequencies to see whether the apparent concentration corresponds to deficient tails. The fitted mean, simulation scheme, and any choices made after looking at the data should be reported when interpreting such a check.
Simulation Choices and a Short Justification
The replication scheme determines what “the model predicts” means. In a conditional replication, fitted random effects are held fixed and new counts are generated conditional on them. This asks whether the observed outcomes are consistent with the fitted cluster effects and the conditional count model. In an unconditional replication, new random effects are drawn from the fitted random-effects distribution as well as new counts. This asks about predictions for newly sampled clusters. These answer different questions; neither is universally the right choice.
When fitting uncertainty matters, a parametric bootstrap can generate data under the fitted model and refit the model to each replicate before recalculating the statistic. A posterior predictive check instead uses draws from the posterior distribution of model parameters and random effects. In either case, the statistic for observed data and replications must be defined consistently. Uncertainty in estimated parameters and random effects can otherwise make the reference distribution too narrow.
The direction of the check follows directly from how the statistic is constructed. Let \(T_{\mathrm{obs}}\) be the statistic computed from observed counts, and \(T^{(r)}\) its value for replicate \(r\), for \(r=1,\ldots,B\). If smaller values mean greater concentration, the empirical lower-tail proportion is
A small \(\widehat{q}_{\mathrm{low}}\) means few simulated statistics are as small as the observed statistic; it is evidence that the observed data are unusually concentrated under the chosen model and replication scheme. For a frequency statistic that increases with central concentration, the corresponding evidence is in the upper tail. This reversal is important: “underdispersion” does not always mean “look at the lower tail.” The tail must match the statistic’s definition.
Specify whether the check concerns observations conditional on fitted clusters or predictions that include new random effects.
Use a dispersion statistic for overall spread, and a frequency statistic when the suspected issue is excess concentration in particular ranges.
Generate counts under the fitted model, calculate the same statistics, and inspect the relevant tail and observed-versus-replicated frequencies.
Describe the mismatch and its direction; do not treat a simulation tail proportion as the probability that the model is correct.
Assumptions, Alternatives, and Failure Modes
A meaningful check assumes that the simulated outcomes follow the stated fitted model: the count distribution, mean structure, offsets where applicable, cluster structure, and random-effects specification must all be represented in the simulation. The statistic must be computable for both observed and replicated data in the same way. If the model has not captured systematic predictors or grouping structure, an apparent concentration pattern may point to that misspecification rather than to a need for a different conditional count variance.
The Pearson statistic is a compact alternative to examining the full outcome distribution, but it can conceal where a discrepancy occurs. Frequency plots or prespecified summaries of zeros, singletons, central counts, and upper-tail counts show more of the pattern; the earlier material on predictive checks for empty and singleton cells and tail-count checks gives related examples. A general GLM analysis can provide related count-model tools, but a GLMM check must also make its treatment of random effects explicit.
Diagnostics can fail in several ways. Very few replications give imprecise tail estimates. Choosing many discrepancies after inspecting the data makes an extreme result easier to find by chance. A conditional check can look satisfactory while predictions for new clusters are poor, or vice versa. And a low variability statistic alone does not identify its cause: omitted covariates, dependence, constraints in the data-generating process, or a misspecified random-effects distribution may contribute. Simulation evidence flags where model and data disagree; it does not determine the remedy by itself.
A common mistake is to assume that only overdispersion matters because Poisson counts have variance equal to their mean. The correction is to recognize that this equality is a conditional model property, not a universal benchmark for a heterogeneous GLMM data set. Compare the observed outcome pattern with replicated outcomes under the fitted model, and check both unusually large and unusually small discrepancies.
Check Your Understanding
Answer each question, then compare your reasoning with the brief solutions.
- If an observed Pearson discrepancy is smaller than 95% of simulated discrepancies, what direction of mismatch does this suggest, assuming smaller values indicate concentration?
- Why can a variance-to-mean comparison be misleading when fitted means differ across observations?
- A statistic counts observations in a prespecified central range, and larger values indicate more concentration. Which tail should be inspected for underdispersion?
- What question does an unconditional replication answer that a conditional replication does not?
- Why is a simulation-based tail proportion not the probability that the fitted model is true?
Solutions. (1) The observed data are more concentrated than most replicates, suggesting underdispersion relative to the model. (2) Different fitted means create variability even when the conditional count distributions are correctly specified; the model-based simulation accounts for that structure. (3) Inspect the upper tail, because large central counts indicate more concentration. (4) Unconditional replication includes new random effects and therefore concerns outcomes for newly sampled clusters; conditional replication holds the fitted cluster effects fixed. (5) The proportion compares a chosen observed statistic with replicated statistics under a specified model and simulation scheme. It does not assign probability to the model itself.