Mean-Dependent Dispersion in Count GLMMs

Why the Mean Matters for Count Dispersion

In a count GLMM, a fitted mean is not enough to determine the model’s uncertainty unless the variance relationship is also specified. Negative binomial models allow the conditional variance to exceed the conditional mean, but they do not all make that excess grow in the same way as the mean changes. This matters when predicted counts vary substantially across observations: two models can agree at one fitted mean and imply quite different variances elsewhere.

The focus here is the relationship between the negative binomial mean and dispersion parameters, and how to calculate predicted variances at different fitted means. The earlier material on Negative Binomial Parameterizations and Dispersion established that the label “negative binomial” alone does not uniquely identify a variance function. The formulas below make the particular parameterizations explicit.

Definition: For observation \(i\), let \(Y_i\) be a count and let \(\mu_i=E(Y_i\mid b_i)\) be its mean conditional on the random effects in the model. In this tutorial, NB1 means \(Var(Y_i\mid b_i)=\mu_i(1+\alpha)\), and NB2 means \(Var(Y_i\mid b_i)=\mu_i+\alpha\mu_i^2\), with \(\alpha\geq 0\). At \(\alpha=0\), each variance function reduces to the Poisson variance \(\mu_i\). These formulas describe conditional dispersion; they do not, by themselves, include variation from integrating over random effects.

For NB1, the variance is proportional to the mean. Its variance-to-mean ratio, also called the conditional Fano factor, is \(1+\alpha\), constant across fitted means. The extra-Poisson component is \(Var(Y_i\mid b_i)-\mu_i=\alpha\mu_i\). Some software instead writes the NB1 variance as \(\phi\mu_i\), where \(\phi=1+\alpha\); the two forms are equivalent under that conversion.

For NB2, the extra-Poisson component is \(\alpha\mu_i^2\), so it grows quadratically with the mean. Its variance-to-mean ratio is \(1+\alpha\mu_i\), which increases with the fitted mean whenever \(\alpha>0\). In a common NB2 parameterization, software reports a size parameter \(\theta\) such that \(\alpha=1/\theta\); always check the reported parameterization before substituting a value.

How Predicted Variance Changes Across Means

The variance functions show their distinct growth rates directly. Treating \(\alpha\) as fixed, the derivative of the conditional variance with respect to the conditional mean is \(1+\alpha\) for NB1, and \(1+2\alpha\mu_i\) for NB2. The second derivative is zero for NB1 and \(2\alpha\) for NB2. Thus NB1 variance increases linearly, whereas NB2 variance curves upward when \(\alpha>0\).

\begin{aligned} V_{\text{NB1}}(\mu) &= \mu(1+\alpha), &\frac{dV_{\text{NB1}}}{d\mu}&=1+\alpha, &\frac{d^2V_{\text{NB1}}}{d\mu^2}&=0,\\ V_{\text{NB2}}(\mu) &= \mu+\alpha\mu^2, &\frac{dV_{\text{NB2}}}{d\mu}&=1+2\alpha\mu, &\frac{d^2V_{\text{NB2}}}{d\mu^2}&=2\alpha. \end{aligned}

These derivatives describe how the model’s conditional variance changes as its fitted conditional mean changes; they are not derivatives of an observed count. The mean \(\mu_i\) can depend on predictors, an offset, and the random effects. Once a fitted value of \(\mu_i\) is obtained on the count scale, substitute it into the relevant variance function. A fitted variance is in squared count units, while its square root is the model-implied conditional standard deviation in count units.

Worked Example: NB2 Variance at Several Fitted Means

Suppose a hypothetical NB2 GLMM has estimated \(\alpha=0.15\). Consider observations whose fitted conditional means are 2, 4, and 8 events per observation. Calculate the conditional variance and variance-to-mean ratio at each fitted mean.

For NB2, \(V(\mu)=\mu+0.15\mu^2\). At \(\mu=2\), \(V(2)=2+0.15(2^2)=2+0.60=2.60\), and the variance-to-mean ratio is \(2.60/2=1.30\).

At \(\mu=4\), \(V(4)=4+0.15(4^2)=4+2.40=6.40\), with ratio \(6.40/4=1.60\). At \(\mu=8\), \(V(8)=8+0.15(8^2)=8+9.60=17.60\), with ratio \(17.60/8=2.20\).

Thus, as the fitted conditional mean increases from 2 to 8 events, the conditional variance increases from 2.60 to 17.60 squared events, and the variance-to-mean ratio increases from 1.30 to 2.20. The same \(\alpha\) applies at each mean; it is the NB2 variance function that makes the relative dispersion increase.

Comparing NB1 and NB2 at a Shared Mean

A dispersion parameter cannot be compared across NB1 and NB2 by its numerical value alone. It appears in a different function of the mean in each model. A clear comparison evaluates the implied variances at the same fitted means, and, when useful, at a reference mean where the models have been made to agree.

Worked Example: Matching at One Mean, Diverging Elsewhere

Consider two hypothetical models. Let NB1 have \(\alpha=1\), so its variance is \(V_{\text{NB1}}(\mu)=2\mu\). Let NB2 have \(\alpha=0.20\), so \(V_{\text{NB2}}(\mu)=\mu+0.20\mu^2\). Calculate both variances at fitted means 2, 5, and 10 events.

At \(\mu=2\), NB1 gives \(2(2)=4\), while NB2 gives \(2+0.20(2^2)=2.80\). At \(\mu=5\), NB1 gives \(2(5)=10\), while NB2 gives \(5+0.20(5^2)=5+5=10\). The models match at this reference mean.

At \(\mu=10\), NB1 gives \(2(10)=20\), while NB2 gives \(10+0.20(10^2)=10+20=30\). The variance predictions therefore agree at 5 events but not at means of 2 or 10. NB1 maintains a constant variance-to-mean ratio of 2; NB2’s ratio is \(1+0.20\mu\), which equals 1.40, 2.00, and 3.00 at the three means.

The comparison concerns conditional variances in squared events, holding the relevant random effects fixed. It illustrates why a single reported dispersion value—or agreement at one fitted mean—does not establish that the models imply similar uncertainty across the full range of fitted means.

Conditional Dispersion and Random-Effect Variation

In a GLMM, the conditional variance function is only one part of the variance story. It describes variation in \(Y_i\) after conditioning on the random effects. If the goal is instead to describe the distribution after averaging over random effects, use the law of total variance:

Var(Y_i)=E\{Var(Y_i\mid b_i)\}+Var\{E(Y_i\mid b_i)\}.

The first term averages the conditional sampling variance over the random-effects distribution. The second term is variation in the conditional mean across random effects. Omitting the second term generally understates marginal variance. This distinction is especially important when reporting a variance for a population-level prediction rather than a prediction conditional on a particular cluster’s estimated random effect.

Worked Example: Marginal NB2 Variance With a Normal Random Intercept

Suppose a hypothetical count GLMM has a random intercept \(b\sim N(0,0.25)\), and its conditional mean is \(\mu(b)=m\exp(b)\). Let \(m=2\), the mean at \(b=0\), and let the conditional NB2 dispersion parameter be \(\alpha=0.20\). Calculate the marginal mean and variance over the random-intercept distribution.

For a normal random variable with mean 0 and variance \(\sigma^2\), \(E\{\exp(b)\}=\exp(\sigma^2/2)\) and \(E\{\exp(2b)\}=\exp(2\sigma^2)\). Here \(\sigma^2=0.25\), so the marginal mean is \(M=E\{\mu(b)\}=2\exp(0.125)\approx2.266\).

Also, \(E\{\mu(b)^2\}=m^2\exp(2\sigma^2)=4\exp(0.50)\approx6.595\). Therefore, \(Var\{\mu(b)\}=E\{\mu(b)^2\}-M^2\approx6.595-(2.266)^2\approx1.459\).

Conditional on \(b\), the NB2 variance is \(\mu(b)+0.20\mu(b)^2\). Its expectation over \(b\) is \(E\{Var(Y\mid b)\}=M+0.20E\{\mu(b)^2\}\approx2.266+0.20(6.595)=3.585\). Applying the law of total variance gives \(Var(Y)\approx3.585+1.459=5.044\) squared events.

For comparison, at \(b=0\) the conditional mean is 2 and the conditional variance is \(2+0.20(2^2)=2.80\). That is a conditional variance at one random-effect value, not the marginal variance of about 5.044. The calculation assumes the stated normal random-intercept distribution and the NB2 variance function conditional on that intercept.

Assumptions, Diagnostics, and Limits

The variance calculations require a model that defines a nonnegative count response, a specified conditional mean, and one of the stated conditional variance functions. In a GLMM, the mean and variance formulas apply conditional on the modeled random effects. If the model also assumes conditional independence of observations given those effects, that assumption concerns their joint behavior; a correct marginal variance for each observation does not by itself establish conditional independence.

Useful checks focus on whether the fitted model captures the observed mean-variance pattern and the dependence structure. Examine residual summaries or simulation-based residual diagnostics across the range of fitted means, not only an overall dispersion summary. Patterns in residual variability that grow with the fitted mean can signal that the chosen variance relationship is inadequate. Also inspect whether a few large counts dominate the apparent pattern, whether the mean model or random-effects structure is misspecified, and whether fitted means cover the range where predictions will be used.

A mismatch is not automatically evidence that the dispersion function alone is wrong: omitted predictors, dependence not captured by the random effects, and an unsuitable mean structure can also produce systematic residual patterns. The observation-level random-effects approach and negative binomial models are distinct ways of representing extra-Poisson variation; the earlier material on Observation-Level Random Effects Versus Negative Binomial Models addresses that comparison. Here the key point is narrower: NB1 and NB2 imply different changes in conditional variance as the mean changes.

Do not read \(\alpha\) as a variance. It controls the variance function; the predicted variance also depends on \(\mu\).
Do not compare dispersion parameters across parameterizations without conversion. For example, NB1’s \(\phi\) in \(V=\phi\mu\) equals \(1+\alpha\) under the NB1 convention used here.
Do not report conditional variance as marginal variance without qualification. The random-effects contribution must be included when the target is marginal.
Do not extrapolate a fitted variance relationship casually. NB2’s quadratic term can imply rapidly increasing variance at large means, so predictions far beyond the observed mean range may be unstable.

Key takeaway: NB1 has a constant conditional variance-to-mean ratio, while NB2’s ratio increases linearly with the conditional mean. In a GLMM, both statements concern variance conditional on random effects; marginal variance also includes variation in the conditional mean across those effects.

Check Your Understanding

Answer each question, then compare with the solutions below.

An NB1 model has \(\alpha=0.5\). What are its conditional variance and variance-to-mean ratio at a fitted mean of 6 counts?
An NB2 model has \(\alpha=0.10\). Calculate its conditional variance at fitted means of 3 and 9 counts. How does the variance-to-mean ratio change?
For NB1 with \(\alpha=0.8\) and NB2 with \(\alpha=0.16\), calculate both variances at \(\mu=5\). Explain what can and cannot be concluded from agreement at that mean.
Why is \(Var(Y\mid b)\) not generally the marginal variance \(Var(Y)\) in a GLMM?

Solutions. (1) \(V=6(1+0.5)=9\) squared counts, and the variance-to-mean ratio is \(9/6=1.5\). (2) At \(\mu=3\), \(V=3+0.10(9)=3.90\), with ratio \(1.30\). At \(\mu=9\), \(V=9+0.10(81)=17.10\), with ratio \(1.90\); the ratio increases with the mean. (3) NB1 gives \(5(1+0.8)=9\). NB2 gives \(5+0.16(25)=9\). They agree at \(\mu=5\), but this does not imply agreement at other means because their variance functions have different shapes. (4) The marginal variance equals \(E\{Var(Y\mid b)\}+Var\{E(Y\mid b)\}\); the second term represents variation in the conditional mean across random effects and is generally nonzero.