Why Equidispersion Matters
A Poisson regression describes counts through a conditional mean. For observation \(i\), let \(Y_i\) be the observed count and let \(\mu_i\) be its model-predicted mean. Under the Poisson model, the conditional variance is also \(\mu_i\). This equality is called equidispersion. It matters because it determines how much observation-to-observation variability the model expects after accounting for predictors and exposure.
In a pharmacodynamic setting, \(Y_i\) might be the number of responses in a defined observation period, while \(\mu_i\) is the predicted count for that subject and period. The relevant comparison is conditional: a fitted model may assign different means to different concentrations or exposure times, but, at each set of modeled covariates, the Poisson variance equals that conditional mean. Variation among different \(\mu_i\) values is not itself a violation of equidispersion.
The Poisson regression mean may depend on concentration, other covariates, and exposure time. For example, with a log link and exposure \(T_i\), the model can be written \(\log(\mu_i)=\log(T_i)+\eta_i\), where \(\eta_i\) is the modeled log rate. The diagnostic question here is not how to choose that mean model; it is whether the observed counts vary around its predictions at about the Poisson level.
Why the Poisson Variance Equals Its Mean
The equality follows from the Poisson probability mass function. If \(Y\) has a Poisson distribution with mean \(\mu\), then \(P(Y=y)=e^{-\mu}\mu^y/y!\) for nonnegative integers \(y\). The first moment is
For the second factorial moment, use \(y(y-1)/y!=1/(y-2)!\) for \(y\geq2\). It follows that \(E\{Y(Y-1)\}=\mu^2\). Since \(Y^2=Y(Y-1)+Y\), we obtain \(E(Y^2)=\mu^2+\mu\), and therefore
This is a property of the specified Poisson distribution, not an empirical law that every count data set must obey. Real observations can be more variable or less variable than the model predicts. A diagnostic compares that variability with what the fitted model expects; it does not prove that a particular alternative model is correct.
Worked Example: Equidispersion at a Predicted Mean
Suppose a hypothetical observation period has a Poisson-predicted mean of 2 counts. What variance and standard deviation does the model imply, and how do they compare with the mean?
For a Poisson variable with \(\mu=2\), the variance is \(\operatorname{Var}(Y)=\mu=2\) counts squared, and the standard deviation is \(\sqrt{2}=1.414\) counts. The expected count is 2 counts. Thus the variance and mean have the same numerical value, although their units differ: variance is in squared counts and the mean is in counts. Equidispersion refers to equality of the numerical parameters of the Poisson distribution, not equality of physical units.
For example, the probability of zero counts is \(P(Y=0)=e^{-2}=0.1353\). The model specifies both that probability and the full count distribution; its mean–variance equality is only one feature of that distribution.
Dispersion Statistics for a Fitted Model
A practical summary compares each observed count \(y_i\) with its fitted mean \(\hat{\mu}_i\). The Pearson residual is the difference divided by the Poisson standard deviation:
Squaring and summing these residuals gives the Pearson chi-square statistic, \(X_P^2=\sum_i (y_i-\hat{\mu}_i)^2/\hat{\mu}_i\). If the model is appropriate, observations are independent, and the usual large-sample approximation is reasonable, this statistic is often compared with the residual degrees of freedom \(df=n-q\), where \(q\) is the number of fitted regression parameters, including the intercept. The ratio \(X_P^2/df\) is the Pearson dispersion statistic. A value near 1 is broadly consistent with the Poisson variance assumption; a value substantially above 1 is a possible overdispersion warning, and a value substantially below 1 is a possible underdispersion warning.
A related summary uses the model deviance. For a Poisson observation, its deviance contribution compares the fitted model with a saturated model that fits that observation exactly:
When \(y_i=0\), the term \(y_i\log(y_i/\hat{\mu}_i)\) is defined by its limit as zero, so \(d_i=2\hat{\mu}_i\). The signed deviance residual is \(r_{Di}=\operatorname{sign}(y_i-\hat{\mu}_i)\sqrt{d_i}\). Its sign indicates whether the observation is above or below its fitted mean; its magnitude indicates the size of its deviance contribution. The ratio \(D/df\) is another dispersion summary. It is useful alongside, not interchangeable with, \(X_P^2/df\): the two statistics weight discrepancies differently.
Worked Example: Pearson Dispersion and Deviance Residuals
Consider six hypothetical, independent count observations with fitted means \((1.2,1.8,1.0,2.5,2.0,2.5)\) and observed counts \((1,2,0,4,3,1)\). Suppose the fitted model estimates \(q=2\) parameters. Calculate Pearson dispersion and examine the deviance residuals.
The residual degrees of freedom are \(df=n-q=6-2=4\). The six Pearson contributions are
Their sum is \(X_P^2=3.356\), so the Pearson dispersion statistic is \(3.356/4=0.839\). This small example does not show an elevated Pearson dispersion; the ratio is below 1, but with only four residual degrees of freedom it is not strong evidence of underdispersion.
For the zero count with fitted mean 1.0, \(d_i=2(1.0)=2.000\), so \(r_{Di}=-\sqrt{2.000}=-1.414\). For the count of 4 with fitted mean 2.5,
and \(r_{Di}=+\sqrt{0.7600}=0.8718\). The positive sign reflects a count above its fitted mean. Calculating the six contributions gives \(D\approx4.417\), hence \(D/df\approx1.104\). This is not identical to the Pearson ratio, and neither ratio should be treated as a decisive test in such a small data set.
Compare the Count Distribution, Not Just One Number
A single dispersion ratio can hide where a model fails. A complementary diagnostic compares observed count frequencies with frequencies expected under the fitted Poisson distributions. If all observations share the same fitted mean, expected frequency in count category \(k\) is \(nP(Y=k)\). If the fitted means differ, the expected frequency is instead \(\sum_i P(Y_i=k\mid\hat{\mu}_i)\); substituting one average mean can give the wrong reference distribution when means vary substantially.
Worked Example: Observed and Predicted Count Frequencies
In a hypothetical sample of 20 independent equal-exposure observation periods, the observed frequencies are 5 zeros, 6 ones, 5 twos, 3 threes, and 1 count of four or more. The total is \(0(5)+1(6)+2(5)+3(3)+4(1)=29\), so the fitted common Poisson mean is \(\hat{\mu}=29/20=1.45\) counts per period.
For each category, calculate the Poisson probability at \(\hat{\mu}=1.45\) and multiply by 20. For zero, \(P(Y=0)=e^{-1.45}=0.2346\), giving an expected frequency of \(20(0.2346)=4.691\). The recurrence \(P(Y=k)=P(Y=k-1)(1.45/k)\) gives \(P(Y=1)=0.3401\), \(P(Y=2)=0.2466\), and \(P(Y=3)=0.1192\). The probability of four or more is the remainder, \(1-(0.2346+0.3401+0.2466+0.1192)=0.0595\).
| Count category | Observed frequency | Expected frequency |
|---|---|---|
| 0 | 5 | 4.691 |
| 1 | 6 | 6.803 |
| 2 | 5 | 4.932 |
| 3 | 3 | 2.384 |
| 4 or more | 1 | 1.191 |
The observed frequencies are broadly close to their model expectations in this small example. The expected frequency in the tail is low, so it would be unwise to treat this table as a precise formal goodness-of-fit test without combining categories appropriately and accounting for estimation of the mean. The table is still informative: a persistent excess of zeros, or a tail much heavier than predicted across suitable data, can reveal a pattern that one overall dispersion ratio does not explain.
Reading Diagnostics and Avoiding Overinterpretation
Dispersion summaries are diagnostics, not automatic decision rules. Ratios near 1 do not establish that the mean structure, independence assumption, or full count distribution is correct. Conversely, a high ratio does not identify its cause. It can arise when the mean model misses a concentration-response pattern, when relevant covariates or exposure differences are omitted or incorrectly represented, or when observations are dependent. Unmodeled variation among individuals can also produce extra variability. A low ratio can signal an overly flexible mean model, dependence, data constraints, or other departures from the Poisson assumptions.
Look at the pattern as well as the magnitude. Plot or tabulate signed deviance residuals against fitted means, concentration, exposure, and other important predictors. A trend or cluster of large residuals may indicate that predictions are systematically too high or too low in part of the covariate range. A few large residuals deserve inspection for data errors or influential observations, but should not be deleted simply because they are inconvenient. For count-frequency comparisons, account for differing fitted means by summing the observation-specific predicted probabilities.
The Pearson and deviance ratios provide compact summaries. Residual plots and observed-versus-predicted frequencies provide more detail about the form of misfit. These approaches complement one another: the ratios assess aggregate discrepancy, while plots and frequency comparisons can locate it. If a discrepancy appears, first check the data, exposure definition, independence, and conditional mean specification. Depending on the problem, a revised mean model or a different count model may be appropriate. The negative-binomial model is one alternative for overdispersed counts, but a dispersion warning alone is not enough to select it.
Common pitfall: “The Pearson ratio is above 1, so the Poisson model is definitely wrong and the counts must follow a negative-binomial distribution.” This conclusion overstates what the statistic shows. The ratio is a warning about variation relative to the fitted Poisson model; sampling variability, dependence, and a misspecified mean can also produce it. Check residual patterns and model assumptions, then choose a response that addresses the diagnosed source rather than treating the ratio as a model-selection test.
Check Your Understanding
Answer each question, then compare your reasoning with the brief solutions.
- If the fitted conditional mean for a count is 4, what variance and standard deviation does the Poisson model imply?
- A model has \(X_P^2=18\), \(n=25\), and \(q=5\). Calculate its Pearson dispersion statistic and interpret it cautiously.
- For an observation with \(y_i=0\) and \(\hat{\mu}_i=0.7\), calculate its deviance contribution and signed deviance residual.
- Why can using a Poisson distribution with the average fitted mean misrepresent expected count frequencies when individual fitted means vary?
Solutions. 1. The variance is 4 counts squared and the standard deviation is \(\sqrt{4}=2\) counts. 2. \(df=25-5=20\), so \(X_P^2/df=18/20=0.900\). This is not an elevated dispersion ratio, but it does not verify all model assumptions. 3. For a zero count, \(d_i=2\hat{\mu}_i=1.4\), and \(r_{Di}=-\sqrt{1.4}=-1.183\), with the negative sign because the observation is below its fitted mean. 4. Different means imply different probabilities for each count category. The correct expected frequency sums each observation’s category probability, \(\sum_i P(Y_i=k\mid\hat{\mu}_i)\); a single distribution at the average mean generally need not give the same result.