Comparing Poisson, Negative-Binomial, and Zero-Inflated Fits

Why Compare Count Likelihoods?

A count model can reproduce an average response rate while still missing important features of the data. Counts may be more variable than a Poisson model allows, zeros may occur more often than its fitted distribution predicts, or the model may miss the upper tail. Comparing candidate likelihoods is therefore not a contest to find the largest log likelihood: more flexible models usually fit at least as well, and the practical question is whether their additional complexity is supported by the data and improves relevant predictions.

This comparison assumes the candidate models describe the same response observations, use the same exposure definition and covariates where intended, and account for dependence at the same level. In repeated-measures data, for example, a likelihood that accounts for within-subject dependence should not be compared casually with one that treats all records as independent. Likelihood criteria and diagnostics complement one another; neither replaces a scientific assessment of whether the model’s data-generating assumptions are plausible.

Candidate Count Models

In these models, \(\mu_i\) is the mean of the Poisson count component, not generally the mean of the observed ZIP response. The ZIP mean is \((1-\pi_i)\mu_i\). For the NB2 model, \(\alpha\) governs extra-Poisson dispersion: as \(\alpha\) approaches zero, its variance approaches \(\mu_i\), and the distribution approaches the Poisson model. The negative-binomial model can generate extra zeros through its broader count distribution; it does not require a separate structural-zero state.

A useful comparison begins with the scientific and data structure, then examines fitted likelihoods, residual patterns, predictive behavior, and uncertainty in parameters. The models need not be nested for information criteria to be compared, but the likelihoods must be computed for the same data under comparable likelihood conventions.

Information Criteria: What They Do and Do Not Say

AIC balances fit against parameter count. It is useful for ranking candidate models aimed at predictive adequacy, under the same response data and likelihood basis. It does not test a null hypothesis or establish that the model with the lowest value is correct. The small-sample correction is \(\mathrm{AICc}=\mathrm{AIC}+\frac{2k(k+1)}{n-k-1}\), when \(n>k+1\). It is often preferred when the number of independent observational units is not large relative to \(k\). For clustered or repeated data, the relevant effective sample size and likelihood formulation require care; the raw number of rows is not automatically the right \(n\).

Another criterion is \(\mathrm{BIC}=k\log(n)-2\ell\), with \(n\) denoting the sample size under the chosen likelihood formulation. BIC imposes a different complexity penalty and has a different large-sample rationale than AIC. AIC and BIC can rank candidates differently; report which criterion was used and why rather than selecting whichever supports a preferred model. Neither criterion diagnoses a poor fit, and neither corrects a misspecified dependence structure.

The Akaike-weight calculation has a simple justification. AIC differences approximate relative expected information loss; exponentiating minus half a difference converts that scale into relative support. Dividing each support value by their sum makes the weights sum to one:

This normalization is conditional on the candidate set. Adding or removing plausible models changes the weights. They should not be interpreted as posterior model probabilities unless a separate Bayesian model specification justifies that interpretation.

Residual and Predictive Diagnostics

Residual checks identify the type and location of model failure. For count data, inspect residuals against fitted means, exposure, concentration, time, and relevant subject-level summaries. Patterns can reveal a missing concentration relationship, unmodeled heterogeneity, an inappropriate exposure offset, or dependence left in the residuals. A general overdispersion statistic cannot by itself establish that zero inflation is the cause: negative-binomial variation, omitted predictors, and correlation can also produce more zeros than a Poisson fit predicts.

Useful checks include observed versus predicted frequencies for zero and positive counts, grouped count frequencies, and upper-tail counts, along with randomized quantile residuals or simulation-based residual checks. Compare the observed statistics with distributions generated under each fitted model, refitting parameters within each simulation when the goal is to account for estimation uncertainty. A residual pattern is evidence about model adequacy, not a mechanical instruction to add a parameter. Inspect whether discrepancies are systematic and scientifically meaningful.

A good predictive check should reflect the intended use. If the model will predict a total burden over a dosing interval, check that aggregate prediction; if it will predict individual intervals, assess interval-level distributions and repeated-measure structure. A fit can have favorable AIC yet fail a clinically important predictive check. Conversely, an isolated discrepancy in a statistic that was not prespecified should be interpreted cautiously, particularly when many checks are examined.

Parameter Uncertainty and Interpretation

Model choice based on point estimates alone can be misleading. In a negative-binomial fit, examine uncertainty in \(\alpha\); an estimate near zero with a wide interval may indicate that the data do not precisely distinguish Poisson-like dispersion from extra-Poisson dispersion. In ZIP, uncertainty in \(\pi_i\) matters, as do correlations between zero-process and count-process parameters. When the estimate of \(\pi\) is near zero, ordinary symmetric standard errors can be unreliable because the parameter is close to a boundary. Profile likelihood intervals or a bootstrap can better show asymmetry or weak identification, though they do not eliminate information limitations in the data.

Intervals for regression coefficients should also be interpreted on their model’s scale and with the exposure held appropriately defined. A coefficient from the Poisson count component of ZIP describes that component’s rate relationship; it is not automatically the same as a marginal observed-count contrast because the observed mean also depends on \(\pi_i\). Model-specific parameter estimates are not interchangeable when likelihoods represent different distributions.

If candidate models have similar information criteria but produce materially different predictions or parameter estimates, report that model uncertainty. A model-averaged prediction can sometimes summarize this uncertainty using weights, but it requires predictions on the same target scale and should not conceal poor diagnostics or an implausible model. In a confirmatory analysis, model selection and uncertainty procedures should be specified with the scientific question in mind, rather than chosen after inspecting whichever result is most favorable.

Common Pitfalls and Practical Comparison

• Treating every excess zero as structural. Overdispersion, omitted covariates, and dependence can increase zero counts. Compare the full predicted distribution and assess plausible causes.
• Choosing the largest likelihood. A more complex model can increase maximized likelihood simply by having more parameters. Use a complexity-aware criterion and diagnostics.
• Using a routine chi-square test at a boundary. For a zero-inflation parameter tested against zero, the usual likelihood-ratio reference may not apply. Use a justified boundary result or, commonly, a parametric bootstrap.
• Comparing noncomparable likelihoods. Criteria require the same response data and compatible likelihood conventions. Conditional and marginal likelihoods, or different missing-data subsets, are not directly interchangeable without justification.
• Reading an AIC weight as certainty. It is relative support within the candidate set, not the probability that a model is true.
• Ignoring convergence and identification. Warning signs include estimates at parameter limits, large standard errors, a nearly singular Hessian, and materially different estimates from different starting values. Such fits need investigation before their criteria are trusted.

A practical comparison reports the candidate distributions and their assumptions, likelihood criterion and parameter counts, key predictive or residual checks, and relevant parameter uncertainty. The Poisson model is more parsimonious; the negative-binomial model allows extra-Poisson variance; ZIP allows a separate structural-zero process. These are different explanations, and fit statistics alone cannot determine which scientific mechanism generated the observations.