Normal Limits for Poisson and Gamma Sums

Why Poisson and Gamma Variables Become Nearly Normal

Poisson counts and gamma waiting times are often approximated by normal distributions when their means or shape parameters are large. The approximation is useful because it replaces a discrete or skewed distribution with a familiar, symmetric one. Its accuracy depends on the scale of the parameter: a normal limit describes what happens as that scale grows, not a guarantee that every moderate-parameter probability is already close.

The earlier material on the central limit theorem for sample means and sums gives a route to normal limits for sums of independent variables. Here we connect that limit to the exact forms of Poisson and gamma distributions. In particular, the Poisson limit can be derived directly from its characteristic function, and a standardized gamma density can be compared explicitly with the standard normal density.

The Large-Mean Poisson Limit

A Poisson variable with parameter \(\lambda>0\) has probability mass function \(P(X_\lambda=k)=e^{-\lambda}\lambda^k/k!\) for \(k=0,1,2,\ldots\), mean \(\lambda\), and variance \(\lambda\). The natural centered and scaled variable is therefore \(Z_\lambda\). As \(\lambda\) increases, its lattice spacing, \(1/\sqrt{\lambda}\), shrinks, while its standardized variance remains one.

Proof. For real \(t\), the characteristic function of a Poisson variable is \(E[e^{iuX_\lambda}]=\exp\{\lambda(e^{iu}-1)\}\). Substituting \(u=t/\sqrt{\lambda}\) and centering gives

Taylor's formula for the exponential gives \(e^{ix}=1+ix-x^2/2+r(x)\), where \(|r(x)|\le |x|^3/6\) for real \(x\). Thus the exponent equals \(-t^2/2+\lambda r(t/\sqrt{\lambda})\), and

For every fixed \(t\), the characteristic function therefore tends to \(e^{-t^2/2}\), the characteristic function of \(N(0,1)\). This limit is continuous at zero, so the continuity theorem for characteristic functions gives the stated convergence in distribution. \(\square\)

The limit suggests the approximation \(X_\lambda\approx N(\lambda,\lambda)\). Because a Poisson variable takes integer values, probabilities for integer cutoffs are often approximated with a continuity correction: for example, \(P(X_\lambda\le k)\) is approximated by \(\Phi((k+0.5-\lambda)/\sqrt{\lambda})\), where \(\Phi\) is the standard normal distribution function. The correction accounts for the width of the lattice cell at the boundary; it does not alter the limiting theorem.

Exact Distributions of Exponential and Gamma Sums

Use the shape-rate parameterization: a \(\operatorname{Gamma}(a,\beta)\) variable has density \(f(x)=\beta^a x^{a-1}e^{-\beta x}/\Gamma(a)\) for \(x>0\), where \(a>0\) is the shape and \(\beta>0\) is the rate. Its mean is \(a/\beta\) and its variance is \(a/\beta^2\). An exponential variable with rate \(\beta\) is the special case \(a=1\).

If independent gamma variables share the same rate, their shapes add. In particular, if \(X_1,\ldots,X_n\) are independent and each has distribution \(\operatorname{Gamma}(a,\beta)\), then their sum \(S_n\) has distribution \(\operatorname{Gamma}(na,\beta)\). This exact identity lets us compare the normal approximation with the full distribution rather than only its moments.

Derivation. The moment generating function of \(X_i\) is \(E[e^{tX_i}]=(\beta/(\beta-t))^a\) for \(t<\beta\). Independence gives the moment generating function of the sum:

This is the moment generating function of \(\operatorname{Gamma}(na,\beta)\), establishing the exact sum distribution. For the limit, write \(S_n-na/\beta=\sum_{i=1}^n(X_i-a/\beta)\). The summands are independent and identically distributed with finite mean \(a/\beta\) and finite, positive variance \(a/\beta^2\). The classical iid central limit theorem therefore gives

which is the claimed standardization. \(\square\)

There is also a direct density comparison. Put \(m=na\) and \(Y=\beta S_n\), so \(Y\sim\operatorname{Gamma}(m,1)\). Then \(W_n=(Y-m)/\sqrt m\) has density

For each fixed real \(w\), the support condition holds once \(m\) is sufficiently large. To see the normal shape in the density, rewrite it as

Stirling's formula, \(\Gamma(m)\sim\sqrt{2\pi}\,m^{m-1/2}e^{-m}\), makes the prefactor tend to \(1/\sqrt{2\pi}\). For fixed \(w\), the Taylor expansion \(\log(1+u)=u-u^2/2+O(u^3)\) as \(u\to0\) gives

Consequently \(g_m(w)\to (2\pi)^{-1/2}e^{-w^2/2}\) at each fixed \(w\). This density calculation agrees with the central limit theorem; pointwise convergence of densities alone is not, in general, a substitute for a proof of convergence in distribution.

What the Approximation Captures—and What It Does Not

The standardized gamma variable \(W_n\) has mean zero and variance one, just like the standard normal. But its exact distribution is not symmetric: it is bounded below by \(-\sqrt{na}\), while a normal variable has support on the whole real line. The gamma skewness is \(2/\sqrt{na}\), which decreases as the total shape grows. Thus the normal limit is supported both by the shrinking relative skewness and by the central limit theorem.

For Poisson variables, the corresponding standardized skewness is \(1/\sqrt{\lambda}\). Both families become less skewed on their standardized scales as their effective size increases. This is why normal approximations generally improve for large \(\lambda\) or large total gamma shape, while tails and probabilities near a boundary can remain sensitive to discreteness or skewness.