Deriving the Lognormal-Poisson Mixture Variance

Why Derive the Marginal Variance?

A Poisson model has equal conditional mean and variance. Yet a count can be Poisson conditional on a latent observation-level effect and still be more variable across repeated observations after that effect is integrated out. The distinction matters: conditional variance describes variation around the rate for a fixed latent effect, while marginal variance also includes variation in the rate itself.

Here the observation-level random effect is normal on the log-rate scale. Exponentiating that effect makes the Poisson rate lognormal, producing a lognormal-Poisson mixture. The goal is to derive its marginal moments, not to assume that a conditional Poisson variance remains the variance after the random effect is averaged over.

Model and Assumptions

Definition: For observation $i$, let $Y_i$ be a count and let $b_i$ be an observation-level random effect. Conditional on $b_i$, suppose \[ Y_i\mid b_i\sim \operatorname{Poisson}(\lambda_i),\qquad \log(\lambda_i)=\eta_i+b_i,\qquad b_i\sim N(0,\sigma^2). \] Here $\eta_i$ is a fixed linear predictor, which may include an offset, and $\sigma^2\geq 0$ is the variance of the observation-level effects. For the marginal-independence result below, also assume the counts are conditionally independent given the full vector of random effects and that the $b_i$ are mutually independent.

The first two assumptions for the marginal moments are that the conditional distribution is Poisson with the stated log-rate and that the random effect has the stated normal distribution. They imply $\operatorname{E}(Y_i\mid b_i)=\operatorname{Var}(Y_i\mid b_i)=\lambda_i$. The additional independence assumptions are not needed to derive one observation’s mean and variance; they are needed for the conclusion about dependence between different observations.

Write $\mu_i=\exp(\eta_i)$ for the rate that would apply at $b_i=0$. This is a conditional reference rate, not generally the marginal mean. Counts and rates are expressed in counts per observation; variances are in squared-count units.

Deriving the Marginal Mean and Variance

A normal random variable $b_i\sim N(0,\sigma^2)$ has moment-generating function $\operatorname{E}[\exp(t b_i)]=\exp(t^2\sigma^2/2)$. In particular, $\operatorname{E}[\exp(b_i)]=\exp(\sigma^2/2)$ and $\operatorname{E}[\exp(2b_i)]=\exp(2\sigma^2)$. Since $\lambda_i=\mu_i\exp(b_i)$, these identities give its first two moments.

\operatorname{E}(\lambda_i)=\mu_i\exp(\sigma^2/2),\qquad \operatorname{E}(\lambda_i^2)=\mu_i^2\exp(2\sigma^2).

The law of total expectation gives the marginal count mean. The law of total variance separates the expected conditional variance from the variance of the conditional mean:

\operatorname{E}(Y_i)=\operatorname{E}\{\operatorname{E}(Y_i\mid b_i)\} =\operatorname{E}(\lambda_i) =\mu_i\exp(\sigma^2/2),

\begin{aligned} \operatorname{Var}(Y_i) &=\operatorname{E}\{\operatorname{Var}(Y_i\mid b_i)\} +\operatorname{Var}\{\operatorname{E}(Y_i\mid b_i)\}\\ &=\operatorname{E}(\lambda_i)+\operatorname{Var}(\lambda_i). \end{aligned}

Using $\operatorname{Var}(\lambda_i)=\operatorname{E}(\lambda_i^2)-\{\operatorname{E}(\lambda_i)\}^2$, the rate variance is $\mu_i^2\{\exp(2\sigma^2)-\exp(\sigma^2)\}$. Let $m_i=\operatorname{E}(Y_i)=\mu_i\exp(\sigma^2/2)$. Then $\operatorname{Var}(\lambda_i)=m_i^2\{\exp(\sigma^2)-1\}$, so the marginal variance can be written directly in terms of the marginal mean.

Result: Under the stated lognormal-Poisson model, \[ \operatorname{E}(Y_i)=m_i=\mu_i\exp(\sigma^2/2), \qquad \operatorname{Var}(Y_i)=m_i+m_i^2\{\exp(\sigma^2)-1\}. \] The first term is the expected Poisson variation conditional on the rate. The second is the added variation from the random rate.

When $\sigma^2=0$, the random effect is absent, $m_i=\mu_i$, and the variance reduces to the Poisson value $m_i$. For $\sigma^2>0$, the variance exceeds the mean. The excess is quadratic in the marginal mean, with multiplier $\exp(\sigma^2)-1$. Thus, with a common random-effect variance, greater expected counts also have greater absolute excess variance.

Worked Example: Computing Both Marginal Moments

Suppose a hypothetical observation has $\eta_i=\log(2)$ and $\sigma^2=0.25$. The reference rate is $\mu_i=\exp\{\log(2)\}=2$ counts per observation. Find the marginal mean and variance.

The marginal mean is \[ m_i=2\exp(0.25/2)=2\exp(0.125)=2(1.13315)=2.2663 \] counts per observation.

For the variance, $\exp(0.25)-1=1.28403-1=0.28403$, and $m_i^2\approx 2.2663^2=5.1361$. Therefore, \[ \operatorname{Var}(Y_i)=2.2663+(5.1361)(0.28403)=2.2663+1.4588=3.7251 \] squared counts.

The marginal variance exceeds the marginal mean by about $1.459$ squared counts. The calculation uses the marginal mean in the quadratic term; substituting the reference rate $2$ there would not implement the displayed variance formula.

Reading the Formula in Context

The added term is $\operatorname{Var}(\lambda_i)$: it measures variation in the Poisson rate across hypothetical realizations of the observation-level effect. It is nonnegative and is zero exactly when $\sigma^2=0$. The conditional Poisson variance has not disappeared; its expectation is the $m_i$ term. The total variance adds the variability of the latent rates.

The marginal mean exceeds the reference rate whenever $\sigma^2>0$, because the exponential function is convex and $\operatorname{E}\{\exp(b_i)\}=\exp(\sigma^2/2)>1$. Consequently, $\exp(\eta_i)$ is not the population-average count under this model. It is the rate at random effect zero. The marginal mean includes the upward adjustment $\exp(\sigma^2/2)$.

Worked Example: A Larger Random-Effect Variance

For a second hypothetical setting, let $\mu_i=3$ counts per observation and $\sigma^2=0.64$. First calculate the marginal mean: \[ m_i=3\exp(0.64/2)=3\exp(0.32)=3(1.37713)=4.1314 \] counts per observation.

Next, $\exp(0.64)-1=1.89648-1=0.89648$, and $m_i^2\approx 17.0683$. Thus, \[ \operatorname{Var}(Y_i)=4.1314+(17.0683)(0.89648) =4.1314+15.3014=19.4328 \] squared counts.

The large variance is consistent with the model: variation in the latent rate adds substantially to the conditional Poisson variation. It does not mean that a particular count is expected to be $19.4328$; the expected count is $4.1314$, and $19.4328$ is its marginal variance.

Independence Requires More Than Independent Effects

The marginal mean and variance for one count do not settle whether different counts are independent. To establish marginal independence under the observation-level model, assume both that the random effects are independent across observations and that the counts are conditionally independent given the full vector of random effects. Then the joint probability of a count vector factors after integrating over those effects:

\begin{aligned} P(Y_1=y_1,\ldots,Y_n=y_n) &=\operatorname{E}_{\mathbf b}\left\{\prod_{i=1}^n P(Y_i=y_i\mid\mathbf b)\right\}\\ &=\prod_{i=1}^n \operatorname{E}_{b_i}\{P(Y_i=y_i\mid b_i)\}\\ &=\prod_{i=1}^n P(Y_i=y_i). \end{aligned}

The second equality uses both conditional independence of the counts and independence of the random effects. Under those assumptions, the marginal counts are independent. Independent effects alone do not establish this result: without conditional independence, the conditional joint distribution could still couple the counts.

This conclusion is specific to independent observation-level effects without other shared random effects. If counts also depend on a shared group-level random intercept, they can be conditionally independent given all effects yet dependent marginally after that shared intercept is integrated out. In that case, the derivation above still applies to each count’s moments conditional on the shared effects, but does not establish unconditional independence across observations.

Worked Example: Checking the Independence Argument

Consider two hypothetical observations with independent effects $b_1$ and $b_2$. Suppose $Y_1$ and $Y_2$ are conditionally independent given $(b_1,b_2)$, with each conditional count following its specified Poisson distribution. For any nonnegative integers $y_1,y_2$, \[ P(Y_1=y_1,Y_2=y_2) =\operatorname{E}\{P(Y_1=y_1\mid b_1)P(Y_2=y_2\mid b_2)\}. \]

Because $b_1$ and $b_2$ are independent, the expectation factors: \[ P(Y_1=y_1,Y_2=y_2) =\operatorname{E}\{P(Y_1=y_1\mid b_1)\} \operatorname{E}\{P(Y_2=y_2\mid b_2)\} =P(Y_1=y_1)P(Y_2=y_2). \] Thus these two marginal counts are independent under the stated assumptions. If conditional independence were omitted, the first equality would not be enough to justify the factorization, even though the two random effects remained independent.

Diagnostics, Alternatives, and Limitations

The formula is a model implication, not a guarantee that the model fits observed data. A useful check is whether the observed count variation, after accounting for the predictors and any group structure, is broadly compatible with the mean-dependent excess variance. Conditional or simulated residual checks can help reveal remaining patterns, such as systematic changes in residual spread with fitted counts or clusters of unusually large counts. Excess zeros may be worth investigating, but their presence alone does not establish a separate zero-generating process.

The lognormal-Poisson mixture imposes a particular shape on the distribution: it comes from Poisson counts whose rates vary lognormally. The same mean and variance do not prove that this shape is appropriate. Poor fit, influential observations, unstable estimates of $\sigma^2$, or residual dependence can signal problems. With only one count per observation, the observation-level effects are latent; estimation depends on the assumed distribution and the information across observations, not on repeated measurements that directly reveal each individual effect.

The main alternatives make different assumptions. A Poisson model without an observation-level effect requires equality of conditional mean and variance and has no such extra-Poisson term. Negative binomial models also allow variance beyond the mean, but specify a different mixing structure and variance relationship; matching a mean and variance does not make their full count distributions identical to this mixture. Quasi-Poisson approaches can represent mean-dependent variance without specifying the same full probability distribution, but do not provide the same likelihood-based distributional model. Choosing among these options requires considering the data structure, substantive assumptions, and model fit.

Common Mistake

Common mistake: “The conditional variance is $\lambda_i$, so the marginal variance is just the marginal mean.” This ignores that $\lambda_i$ itself varies with $b_i$. The correction is the law of total variance: \[ \operatorname{Var}(Y_i)=\operatorname{E}(\lambda_i)+\operatorname{Var}(\lambda_i). \] The first term is the marginal mean; the second is the nonnegative extra variation from the random rate.

Check Your Understanding

Try each question before reading the solutions.

For $\mu_i=4$ and $\sigma^2=0.2$, calculate the marginal mean and variance.
What happens to the marginal variance formula when $\sigma^2=0$?
If the marginal mean is $5$ and the marginal variance is $30$, what value of $\sigma^2$ is implied by this model?
Why do independent observation-level effects alone not establish marginal independence of the counts?

Solutions. (1) The mean is $m_i=4\exp(0.1)=4.4207$ counts per observation. The variance is $$ m_i=4\exp(0.1)=4.4206836723,\qquad m_i^2=16\exp(0.2)=19.54244413 $$ $$ \operatorname{Var}(Y_i)=m_i+m_i^2\{\exp(0.2)-1\} =4.4206836723+(19.54244413)(0.221402758) =8.74743470\approx8.7474 $$ squared counts. (2) Since $\exp(0)-1=0$, the variance is $m_i$, the Poisson value. (3) From $30=5+5^2\{\exp(\sigma^2)-1\}$, we obtain $1=\exp(\sigma^2)-1$, so $\sigma^2=\log(2)=0.6931$. (4) Conditional independence of the counts is also needed. Independent effects do not rule out dependent counts conditional on those effects; with conditional independence and independent effects, the joint marginal probability factors.