Marginal Variance In A Poisson-Lognormal Model

Why the Marginal Variance Changes

A Poisson GLMM can have Poisson variation conditional on its random effect and still have a marginal variance larger than its marginal mean. The reason is that observations in clusters with different random intercepts have different conditional rates. Averaging over that unobserved cluster-to-cluster variation adds variability to the count distribution.

Here, “marginal” means that the random intercept has been integrated out, while the covariates are held fixed. Thus the marginal mean and variance describe repeated outcomes at the same covariate values across the population of clusters represented by the random-effects distribution. They are not obtained simply by setting the random effect to zero.

Model: For observation $i$ in cluster $j$, suppose $Y_{ij}\mid b_j\sim\operatorname{Poisson}(\lambda_{ij})$, where $\log(\lambda_{ij})=\eta_{ij}+b_j$, $\eta_{ij}=x_{ij}^{T}\beta$ (including any offset), and $b_j\sim N(0,\sigma_b^2)$. Random intercepts are independent across clusters. Conditional on the random effects, observations are independent. The covariates and fixed effects are held fixed when taking marginal moments.

The conditional Poisson model gives $E(Y_{ij}\mid b_j)=\lambda_{ij}$ and $\operatorname{Var}(Y_{ij}\mid b_j)=\lambda_{ij}$. The normal distribution for $b_j$ implies that $\exp(b_j)$ is lognormal: $E[\exp(b_j)]=\exp(\sigma_b^2/2)$ and $E[\exp(2b_j)]=\exp(2\sigma_b^2)$. These moments are the ingredients for the marginal calculations.

Marginal Mean and Variance

Write $a_{ij}=\exp(\eta_{ij})$, the rate that would result at $b_j=0$. The conditional rate is $\lambda_{ij}=a_{ij}\exp(b_j)$. By the law of total expectation, the marginal mean is the expectation of the conditional mean over the distribution of $b_j$.

\begin{aligned} E(Y_{ij}) &=E\{E(Y_{ij}\mid b_j)\}\\ &=E(\lambda_{ij})\\ &=a_{ij}E\{\exp(b_j)\}\\ &=\exp(\eta_{ij}+\sigma_b^2/2). \end{aligned}

For the variance, the law of total variance states that $\operatorname{Var}(Y)=E\{\operatorname{Var}(Y\mid b)\}+\operatorname{Var}\{E(Y\mid b)\}$. The first term is the average conditional Poisson variance; the second is the variance of the conditional mean across clusters. Both terms matter.

Result: Under the stated Poisson-lognormal random-intercept model, with $m_{ij}=E(Y_{ij})=\exp(\eta_{ij}+\sigma_b^2/2)$, $$ \operatorname{Var}(Y_{ij})=m_{ij}+m_{ij}^{\,2}\{\exp(\sigma_b^2)-1\}. $$ When $\sigma_b^2=0$, this reduces to the Poisson equality $\operatorname{Var}(Y_{ij})=m_{ij}$. For $\sigma_b^2>0$, the marginal variance exceeds the marginal mean.

The derivation makes clear why the expression uses the marginal mean $m_{ij}$, rather than the rate $a_{ij}$ at a zero random intercept. The expected conditional variance is $E(\lambda_{ij})=m_{ij}$. The variance of the conditional mean is $a_{ij}^2\{\exp(2\sigma_b^2)-\exp(\sigma_b^2)\}$. Since $m_{ij}^2=a_{ij}^2\exp(\sigma_b^2)$, this second term equals $m_{ij}^2\{\exp(\sigma_b^2)-1\}$.

Worked Calculations

Worked Example: Marginal Variance With a Small Random-Intercept Variance

Suppose a hypothetical count outcome follows the stated model at a fixed covariate pattern with $\eta_{ij}=\log(2)$ and $\sigma_b^2=0.09$. Calculate its marginal mean and variance.

The conditional rate at $b_j=0$ is $a_{ij}=\exp\{\log(2)\}=2$. Integrating over the normal random intercept gives $m_{ij}=2\exp(0.09/2)=2\exp(0.045)\approx2.092056$ counts.

The excess-variance factor is $\exp(0.09)-1\approx0.0941743$. Therefore

\begin{aligned} \operatorname{Var}(Y_{ij}) &=2.092056+(2.092056)^2(0.0941743)\\ &\approx2.092056+0.412173\\ &\approx2.50423. \end{aligned}

Thus the marginal variance is about $2.504$ counts squared, compared with a marginal mean of about $2.092$ counts. The variance is larger because the conditional rate varies with the random intercept. The calculation is consistent with a conditionally Poisson outcome; it does not imply extra-Poisson variation conditional on $b_j$.

Worked Example: A Clear Contrast With a Poisson Distribution

For a hypothetical covariate pattern, let $\eta_{ij}=0$ and $\sigma_b^2=\log(2)$. Find the marginal mean and variance, then compare the variance with that of a Poisson variable having the same mean.

Here $a_{ij}=\exp(0)=1$, and $\exp(\sigma_b^2/2)=\exp\{\log(2)/2\}=\sqrt{2}$. Hence $m_{ij}=\sqrt{2}\approx1.4142$ counts. Also, $\exp(\sigma_b^2)-1=2-1=1$, so

\operatorname{Var}(Y_{ij}) =\sqrt{2}+(\sqrt{2})^2(1) =\sqrt{2}+2 \approx3.4142.

A Poisson distribution with mean $1.4142$ would have variance $1.4142$ counts squared. The Poisson-lognormal marginal variance is about $3.4142$ counts squared. The distinction is between a Poisson model conditional on the cluster intercept and the mixture obtained after integrating that intercept out.

Integration also creates dependence among observations in the same cluster. Conditional independence does not imply marginal independence: two observations share $b_j$, so their conditional means vary together as the intercept varies.

Same-cluster covariance: If $i\ne k$, and $Y_{ij}$ and $Y_{kj}$ are conditionally independent given $b_j$, then $\operatorname{Cov}(Y_{ij},Y_{kj})=\operatorname{Cov}(\lambda_{ij},\lambda_{kj})$. Under the shared normal random intercept, $$ \operatorname{Cov}(Y_{ij},Y_{kj}) =m_{ij}m_{kj}\{\exp(\sigma_b^2)-1\}. $$ Observations in different clusters have covariance zero under independent random intercepts and conditional independence.

Worked Example: Covariance for Two Counts in One Cluster

Suppose two hypothetical observations in the same cluster have fixed predictors satisfying $\eta_{1j}=\eta_{2j}=\log(2)$, with $\sigma_b^2=0.09$. Their marginal means are both approximately $2.092056$ counts. Calculate their covariance and correlation.

Using $\exp(0.09)-1\approx0.0941743$, their covariance is

\operatorname{Cov}(Y_{1j},Y_{2j}) =(2.092056)^2(0.0941743) \approx0.412173.

Each marginal variance is approximately $2.50423$ counts squared, as in the first example. The correlation is therefore $\operatorname{Cov}(Y_{1j},Y_{2j})/\sqrt{\operatorname{Var}(Y_{1j})\operatorname{Var}(Y_{2j})}\approx0.412173/2.50423\approx0.1646$. This is positive dependence induced by the shared random intercept, not by conditional dependence between the two counts.

Interpretation, Diagnostics, and Alternatives

The ratio of marginal variance to marginal mean is $\operatorname{Var}(Y_{ij})/m_{ij}=1+m_{ij}\{\exp(\sigma_b^2)-1\}$. It exceeds one when $\sigma_b^2>0$, and its excess over one grows with both the random-intercept variance and the marginal mean. The model therefore predicts a particular mean-dependent pattern of overdispersion, rather than a constant variance-to-mean ratio.

This result is useful when interpreting count variation across a population of clusters. A marginal variance larger than the mean is not, by itself, evidence that the conditional Poisson assumption fails: a random intercept can produce that pattern through mixing. Conversely, observing overdispersion does not establish that this model is appropriate. The formula depends on the log link, a normal random intercept, conditional Poisson variation, and the specified dependence structure.

Diagnostics should assess the model rather than compare one observed variance with one mean. Examine residual behavior and whether the fitted model reproduces the distribution of counts, including the frequency of zeros and upper-tail counts. Check whether unexplained variation or patterns remain across clusters and covariate values. A mismatch can reflect an incorrect mean structure, omitted predictors, an unsuitable random-effects distribution, dependence not represented by a shared intercept, or conditional variation beyond Poisson. These possibilities are not distinguished by the marginal variance formula alone.

A Poisson GLM without a random intercept imposes the Poisson variance-mean equality and does not represent cluster-level heterogeneity. The Poisson-lognormal GLMM instead specifies that heterogeneity through a normal random intercept and also implies within-cluster dependence. A negative-binomial model is another common approach to overdispersed counts, but its variance relationship and dependence assumptions are not automatically identical to those of this GLMM. A quasi-Poisson specification can adjust a variance relationship without defining the same full probability distribution. These approaches are alternatives to consider against the scientific design and diagnostics; the variance formula alone does not select among them.

Common pitfall: It is incorrect to say, “The conditional distribution is Poisson, so the marginal variance must equal the marginal mean.” That equality holds conditional on $b_j$, not after averaging over it. Apply the law of total variance: average the conditional variance and add the variance of the conditional mean. Also do not substitute $a_{ij}=\exp(\eta_{ij})$ for the marginal mean in the final formula; the latter is $a_{ij}\exp(\sigma_b^2/2)$.

Limits of the Result

The derivation is exact for the specified model, but that does not make the assumptions harmless. If the random intercept is not normally distributed, its exponential moments may differ, so the displayed expressions need not hold. If the conditional outcome is not Poisson, then the first term in the total-variance calculation changes. If multiple observations share the intercept, their marginal outcomes are dependent, even though they are independent conditional on it.

The formula is also not an estimation method by itself. In an analysis, $\beta$ and $\sigma_b^2$ are estimated from data, and uncertainty in those estimates matters for inference about marginal quantities. The calculation here describes the model-implied moments at fixed parameter values. It does not supply a confidence interval or a test for overdispersion.

Check Your Understanding

Use the Poisson-lognormal random-intercept model and show the key substitution in each answer.

If $\eta=0$ and $\sigma_b^2=\log(4)$, find the marginal mean and variance.
If the marginal mean is $4$ and $\sigma_b^2=0.1$, calculate the marginal variance.
Two counts in one cluster have marginal means $2$ and $3$, and $\sigma_b^2=0.2$. Find their covariance.
Explain why conditional independence within a cluster does not imply marginal independence.

Solutions. (1) The mean is $\exp\{0+\log(4)/2\}=2$. Since $\exp(\sigma_b^2)-1=4-1=3$, the variance is $2+2^2(3)=14$ counts squared. (2) The variance is $4+4^2\{\exp(0.1)-1\}\approx4+16(0.10517)=5.683$ counts squared. (3) The covariance is $2(3)\{\exp(0.2)-1\}\approx6(0.221403)=1.328$ counts squared. (4) Conditional on $b_j$, the counts are independent, but both conditional means depend on the same $b_j$. Integrating over that shared random effect gives covariance equal to the covariance of the conditional means, which is positive when $\sigma_b^2>0$.