Mixed Poisson Distributions

Why Random Rates Change a Poisson Count

A Poisson model assumes that the event rate is fixed across the counts being described. In some settings, however, that rate varies across days, locations, patients, or other units. Even if a count follows a Poisson distribution once its rate is specified, variation in the rate changes the marginal distribution of the count. The resulting model is called a mixed Poisson distribution.

The key distinction is between variation in a count at a fixed rate and variation in the rate itself. A Poisson count with fixed rate \(\lambda\) has mean and variance both equal to \(\lambda\). When the rate is random, the overall variance includes both sources of variation. We derive that result and identify the additional variance precisely.

Definition: Let \(\Lambda\) be a nonnegative, finite almost surely random variable. A random variable \(N\) has a mixed Poisson distribution with mixing variable \(\Lambda\) if, conditional on \(\Lambda\), it has a Poisson distribution with rate \(\Lambda\): \[ N\mid \Lambda \sim \operatorname{Poisson}(\Lambda). \] Thus, for each integer \(k\geq 0\), \[ P(N=k\mid\Lambda)=e^{-\Lambda}\frac{\Lambda^k}{k!}. \] When \(\Lambda=0\), this conditional distribution assigns probability one to \(N=0\).

The mixing variable describes variation in the rate across units or occasions. The model does not say that a single realized rate fluctuates during the generation of one count: conditional on the realized value of \(\Lambda\), the count is Poisson. The marginal law averages over the possible values of that rate.

Marginal Probability Mass Function and PGF

Conditioning on the rate gives the marginal probability of each count by averaging the conditional Poisson probability. The probability-generating function (PGF) likewise averages the conditional Poisson PGF.

Theorem: If \(N\mid\Lambda\sim\operatorname{Poisson}(\Lambda)\), then for \(k=0,1,2,\ldots\), \[ P(N=k)=E\left[e^{-\Lambda}\frac{\Lambda^k}{k!}\right]. \] For \(0\leq s\leq 1\), the PGF \(G_N(s)=E[s^N]\) is \[ G_N(s)=E\left[e^{\Lambda(s-1)}\right]. \]

For the mass function, apply the law of total probability conditional on \(\Lambda\):

P(N=k)=E[P(N=k\mid\Lambda)] =E\left[e^{-\Lambda}\frac{\Lambda^k}{k!}\right].

Each term is nonnegative. Moreover, summing the probabilities over \(k\) gives

\sum_{k=0}^{\infty}P(N=k) =E\left[\sum_{k=0}^{\infty}e^{-\Lambda} \frac{\Lambda^k}{k!}\right] =E[1]=1.

The exchange of sum and expectation is valid by Tonelli's theorem, since all terms are nonnegative. For the PGF, condition on \(\Lambda\) and use the Poisson series:

G_N(s)=E[E[s^N\mid\Lambda]] =E\left[\sum_{k=0}^{\infty}s^k e^{-\Lambda} \frac{\Lambda^k}{k!}\right] =E\left[e^{-\Lambda}e^{s\Lambda}\right] =E[e^{\Lambda(s-1)}].

This expression is well-defined for \(0\leq s\leq1\), because \(0\leq e^{\Lambda(s-1)}\leq1\). At \(s=1\), it equals one, as every PGF must. Values \(s>1\) require a finite exponential moment of \(\Lambda\) at \(s-1\); they are not guaranteed for an arbitrary mixing distribution. \(\square\)

Worked Example: A Two-Rate Poisson Mixture

Suppose \(\Lambda=1\) with probability \(1/2\) and \(\Lambda=4\) with probability \(1/2\), and \(N\mid\Lambda\sim\operatorname{Poisson}(\Lambda)\). Find the marginal mass function, PGF, mean, and variance.

For each \(k\geq0\), average the two conditional Poisson probabilities:

P(N=k)=\frac12 e^{-1}\frac{1^k}{k!} +\frac12 e^{-4}\frac{4^k}{k!} =\frac{e^{-1}+4^k e^{-4}}{2k!}.

The PGF is the same mixture of conditional PGFs:

G_N(s)=\frac12 e^{s-1}+\frac12 e^{4(s-1)},\qquad 0\leq s\leq1.

The first two moments of the rate are \(E[\Lambda]=(1+4)/2=5/2\) and \(E[\Lambda^2]=(1+16)/2=17/2\). Hence \(\operatorname{Var}(\Lambda)=17/2-(5/2)^2=9/4\). The count therefore has mean \(5/2\) and variance \(5/2+9/4=19/4\). For example, its first two probabilities are \(P(N=0)=(e^{-1}+e^{-4})/2\) and \(P(N=1)=(e^{-1}+4e^{-4})/2\), directly from the mass-function formula.

Mean, Variance, and Excess Variance

The mean and variance separate the count's conditional Poisson variation from the variation in its random rate. The following result assumes the corresponding moments of the rate exist.

Theorem: If \(N\mid\Lambda\sim\operatorname{Poisson}(\Lambda)\) and \(E[\Lambda]<\infty\), then \[ E[N]=E[\Lambda]. \] If, in addition, \(E[\Lambda^2]<\infty\), then \[ \operatorname{Var}(N)=E[\Lambda]+\operatorname{Var}(\Lambda). \] Consequently, \(\operatorname{Var}(N)-E[N]=\operatorname{Var}(\Lambda)\geq0\).

To derive the result, first calculate the conditional factorial moments. For a fixed rate \(\lambda\geq0\), the Poisson series gives

E[N\mid\Lambda=\lambda] =\sum_{k=1}^{\infty}k e^{-\lambda}\frac{\lambda^k}{k!} =\lambda,

where the equality follows by writing \(k/k!=1/(k-1)!\). Similarly,

E[N(N-1)\mid\Lambda=\lambda] =\sum_{k=2}^{\infty}k(k-1)e^{-\lambda}\frac{\lambda^k}{k!} =\lambda^2.

Since \(N^2=N(N-1)+N\), it follows that \(E[N^2\mid\Lambda=\lambda]=\lambda^2+\lambda\). These identities hold also at \(\lambda=0\). Taking expectations over \(\Lambda\), with finite moments as assumed, yields

E[N]=E[\Lambda],\qquad E[N^2]=E[\Lambda^2]+E[\Lambda].

Subtract the square of the mean from the second moment:

\operatorname{Var}(N) =E[N^2]-(E[N])^2 =E[\Lambda^2]+E[\Lambda]-(E[\Lambda])^2 =E[\Lambda]+\operatorname{Var}(\Lambda).

The excess variance over the mean is therefore exactly the variance of the rate, not an unspecified correction. Equality of the count's variance and mean holds if and only if \(\operatorname{Var}(\Lambda)=0\), which, under the finite-moment condition, is equivalent to \(\Lambda\) being constant almost surely. If \(E[\Lambda]\) is finite but \(E[\Lambda^2]\) is infinite, then \(E[N^2]\) and \(\operatorname{Var}(N)\) are infinite. \(\square\)

Worked Example: A Continuously Varying Rate

Let \(\Lambda\) have the uniform distribution on \([0,2]\), with \(N\mid\Lambda\sim\operatorname{Poisson}(\Lambda)\). Find the PGF, mean, variance, and the probabilities of zero and one.

The rate density is \(1/2\) on \([0,2]\), so the PGF is

G_N(s)=\frac12\int_0^2 e^{\lambda(s-1)}\,d\lambda =\frac{e^{2(s-1)}-1}{2(s-1)},\qquad 0\leq s<1.

At \(s=1\), the expression has the limiting value \(1\), agreeing with the defining PGF. The rate mean is \(E[\Lambda]=\frac12\int_0^2\lambda\,d\lambda=1\), and its variance is \(\operatorname{Var}(\Lambda)=(2-0)^2/12=1/3\). Thus \(E[N]=1\) and \(\operatorname{Var}(N)=1+1/3=4/3\).

For any \(k\geq0\), the mass function is

P(N=k)=\frac{1}{2k!}\int_0^2 e^{-\lambda}\lambda^k\,d\lambda.

For \(k=0\), integrating \(e^{-\lambda}\) gives \(P(N=0)=(1-e^{-2})/2\). For \(k=1\), integration by parts gives \(\int_0^2\lambda e^{-\lambda}\,d\lambda =[-(\lambda+1)e^{-\lambda}]_0^2=1-3e^{-2}\), and hence \(P(N=1)=(1-3e^{-2})/2\). The probabilities are positive, and the variance exceeds the mean by \(1/3\), the rate variance.

Interpretation and a Zero-Rate Example

A mixed Poisson distribution is a mixture of Poisson laws, not generally another Poisson law. Its mean is the average rate, while its variance also reflects how widely the rate varies. Units with unusually high rates produce more large counts; units with low rates produce more small counts, including zeros. This can make the marginal distribution more dispersed than a Poisson distribution with the same mean.

Worked Example: A Rate That Is Sometimes Zero

Suppose \(\Lambda=0\) with probability \(0.8\) and \(\Lambda=3\) with probability \(0.2\). Conditional on this rate, \(N\) is Poisson. Calculate the probability of zero, the mean, the variance, and the excess variance.

The marginal mass function and PGF are

P(N=k)=0.8\,\mathbf{1}_{\{k=0\}}+ 0.2e^{-3}\frac{3^k}{k!},\qquad G_N(s)=0.8+0.2e^{3(s-1)}.

Here \(\mathbf{1}_{\{k=0\}}\) equals one when \(k=0\) and zero otherwise. Thus \(P(N=0)=0.8+0.2e^{-3}\), while for \(k\geq1\), \(P(N=k)=0.2e^{-3}3^k/k!\). The rate moments are \(E[\Lambda]=0.2(3)=0.6\) and \(E[\Lambda^2]=0.2(9)=1.8\), so \(\operatorname{Var}(\Lambda)=1.8-(0.6)^2=1.44\). Therefore \(E[N]=0.6\), \(\operatorname{Var}(N)=0.6+1.44=2.04\), and the excess variance is \(1.44\).

The extra probability at zero comes from the possibility that the rate itself is zero. The other zero counts arise from the Poisson distribution when the rate equals three.

Common Mistake: Averaging the Rate and Stopping

A tempting shortcut is to replace the random rate by its mean and conclude that \(N\) is Poisson with rate \(E[\Lambda]\). That replacement gets the mean right, but it generally gets the distribution and variance wrong. In the two-rate example, the mean rate is \(5/2\), but the variance of the mixed count is \(19/4\), not \(5/2\). The discrepancy is \(9/4=\operatorname{Var}(\Lambda)\).

A mixed Poisson count is an ordinary Poisson count only when its rate is constant almost surely. If the rate is nonconstant and has finite second moment, its variance is strictly greater than its mean. When comparing a count model with a Poisson model, check whether a single fixed rate is plausible across the observations; apparent excess variance may indicate rate heterogeneity.

Key takeaway: Mixing over a nonnegative random rate \(\Lambda\) gives \(P(N=k)=E[e^{-\Lambda}\Lambda^k/k!]\) and \(G_N(s)=E[e^{\Lambda(s-1)}]\). With finite second moment, \(E[N]=E[\Lambda]\) and \(\operatorname{Var}(N)-E[N]=\operatorname{Var}(\Lambda)\).

Check Your Understanding

Answer each question using the conditional-rate definition of a mixed Poisson distribution.

Write the marginal probability \(P(N=k)\) in terms of the mixing variable \(\Lambda\).
For \(0\leq s\leq1\), express the PGF of \(N\) as an expectation involving \(\Lambda\).
If \(E[\Lambda]=4\) and \(\operatorname{Var}(\Lambda)=2\), what are \(E[N]\), \(\operatorname{Var}(N)\), and the excess variance?
Under finite second moment, when does a mixed Poisson count have variance equal to its mean?
Why does substituting \(E[\Lambda]\) for \(\Lambda\) preserve the mean but generally fail to preserve the variance?

What You'll Learn