The Poisson Distribution in Exponential-Family Form

Why Write the Poisson Model in Exponential-Family Form?

The Poisson distribution describes a count $Y$ taking values $0,1,2,\ldots$. Its usual parameter, $\lambda$, is both its mean and its variance. Writing the distribution in exponential-family form makes that structure explicit: the parameter multiplying the observed count is the natural parameter, and derivatives of the log-partition function give the mean and variance.

This representation is useful because it identifies the pieces that recur across exponential-family models. Here the goal is specific: express the Poisson mass function in that form and derive its first two moments from its log-partition function. The count is one response; no predictors or group effects are needed for this calculation.

Definition: If $Y\sim\operatorname{Poisson}(\lambda)$, then $\lambda>0$, the support is $y=0,1,2,\ldots$, and $$ P(Y=y)=\frac{e^{-\lambda}\lambda^y}{y!}. $$ The Poisson distribution in exponential-family form is $$ P(Y=y)=\exp\{y\eta-b(\eta)+c(y)\}, $$ where $\eta=\log\lambda$ is the natural parameter, $b(\eta)=e^\eta$ is the log-partition function, and $c(y)=-\log(y!)$. Its natural-parameter space is all real numbers.

Rewriting the Poisson Mass Function

Take the logarithm of the usual Poisson probability and then express $\lambda$ through its logarithm:

\begin{aligned} \log P(Y=y) &=-\lambda+y\log\lambda-\log(y!)\\ &=y\eta-e^\eta-\log(y!), \qquad \eta=\log\lambda. \end{aligned}

Exponentiating gives $P(Y=y)=\exp\{y\eta-e^\eta-\log(y!)\}$. Thus the statistic depending on the observation is $y$; the natural parameter multiplying it is $\eta$; and the term $e^\eta$ normalizes the probabilities. The remaining term, $-\log(y!)$, depends on the observation but not on the parameter.

Because $\lambda>0$, taking $\eta=\log\lambda$ allows any finite real value of $\eta$, and conversely $\lambda=e^\eta>0$. The support remains the nonnegative integers for every such parameter. At $\lambda=0$, the count would be zero with probability one, but that degenerate boundary case is not represented by a finite natural parameter; it is approached as $\eta$ tends to negative infinity.

Worked Example: Rewriting a Poisson Model With Mean 3.2

Suppose a hypothetical count $Y$ follows a Poisson distribution with $\lambda=3.2$. Find its natural parameter, write its mass function in exponential-family form, and calculate $P(Y=4)$.

The natural parameter is $\eta=\log(3.2)\approx1.16315081$. The log-partition function evaluated at this parameter is $b(\eta)=e^\eta=3.2$. Therefore,

P(Y=y)=\exp\{1.16315081y-3.2-\log(y!)\},\qquad y=0,1,2,\ldots.

For $y=4$, use $4!=24$. Carrying full precision through the exponent gives

\begin{aligned} P(Y=4) &=\exp\{4\log(3.2)-3.2-\log(24)\}\\ &=\exp\{4.65260324-3.2-3.17805383\}\\ &=\exp(-1.72545059)\\ &\approx 0.1781. \end{aligned}

Thus, under this hypothetical Poisson model, the probability of observing exactly four counts is about 0.1781. The displayed exponent and probability agree when intermediate values are not rounded prematurely.

Deriving the Mean and Variance

For an exponential-family mass function of the form $\exp\{y\eta-b(\eta)+c(y)\}$, the mean is the first derivative $b'(\eta)$, and the variance is the second derivative $b''(\eta)$, provided the derivatives exist and the support does not vary with $\eta$. For the Poisson model the support is fixed, and $b(\eta)=e^\eta$ has derivatives of every order.

Differentiate the Poisson log-partition function explicitly:

b'(\eta)=e^\eta,\qquad b''(\eta)=e^\eta.

Consequently,

E(Y)=b'(\eta)=e^\eta=\lambda,\qquad \operatorname{Var}(Y)=b''(\eta)=e^\eta=\lambda.

The equality of mean and variance is a property of this distribution, not a statement that a particular observed data set must have its sample mean exactly equal to its sample variance. The mean is measured in counts; the variance is measured in squared counts, though both have the same numerical value under the Poisson model.

Justification from the moment-generating function: The moment-generating function is $M_Y(t)=E(e^{tY})$. Using the exponential-family form and the normalization of the Poisson probabilities, $$ M_Y(t)=\exp\{b(\eta+t)-b(\eta)\} =\exp\{e^\eta(e^t-1)\}. $$ Therefore $M'_Y(0)=e^\eta$. Also, $\operatorname{Var}(Y)=\left.\frac{d^2}{dt^2}\log M_Y(t)\right|_{t=0}=e^\eta$. This independently confirms the mean and variance obtained from $b'(\eta)$ and $b''(\eta)$.

Worked Example: Recovering Moments From a Natural Parameter

Consider a hypothetical Poisson count with natural parameter $\eta=\log(2.5)$. Find its mean and variance from the log-partition function, and calculate the probability of zero counts.

Here $b(\eta)=e^\eta$, so

E(Y)=b'(\eta)=e^{\log(2.5)}=2.5,\qquad \operatorname{Var}(Y)=b''(\eta)=e^{\log(2.5)}=2.5.

For zero counts, $0!=1$, so the mass function gives

P(Y=0)=\exp\{0\cdot\log(2.5)-2.5-\log(1)\} =e^{-2.5}\approx0.08208.

The model assigns probability about 0.08208 to zero counts. This calculation also illustrates why the factor $y!$ matters in the mass function: it supplies the correct probability for each count, including the boundary value $y=0$.

What the Variance Implication Does and Does Not Say

The Poisson model imposes equidispersion: its variance equals its mean. This is a distributional assumption, not an identity that can be checked from a single count. With repeated observations under a common Poisson distribution, the sample variance may differ from the sample mean because of sampling variation. Persistent and substantial disagreement can, however, be a warning that the Poisson model does not describe the observed variation well.

Worked Example: A Small-Sample Equidispersion Check

Suppose six hypothetical, comparable counting occasions produce counts $0,1,2,3,4,8$. Calculate the sample mean and sample variance, and describe what they suggest about a common Poisson model.

The sum is $18$, so the sample mean is

\bar{x}=\frac{0+1+2+3+4+8}{6}=\frac{18}{6}=3.

The squared deviations from 3 are $9,4,1,0,1,25$, which sum to 40. The sample variance is therefore

s^2=\frac{40}{6-1}=8.

The sample mean is 3 and the sample variance is 8, whereas a Poisson distribution with mean 3 has population variance 3. This difference is a reason to inspect whether a common Poisson model is adequate, not proof from six observations that it is false: sample variance is itself variable, particularly in a small sample. The calculation also uses $s^2$, the sample variance with denominator $n-1$; it should not be confused with the population variance specified by the model.

Comparison, Diagnostics, and Limits

The ordinary parameterization and the exponential-family parameterization describe the same distribution but emphasize different features. The ordinary form makes the mean parameter $\lambda$ immediately visible. The exponential-family form separates the natural parameter, the normalizing function, and the count-dependent factorial term; it also makes moment derivations systematic.

Representation	Expression	What it highlights
Usual Poisson form	$e^{-\lambda}\lambda^y/y!$	The positive mean parameter $\lambda$ and count probabilities
Exponential-family form	$\exp\{y\eta-b(\eta)+c(y)\}$	The natural parameter $\eta=\log\lambda$, log-partition function $b(\eta)=e^\eta$, and statistic $y$

Several checks help catch errors in algebra or application:

Check the support. A Poisson response is a nonnegative integer count. Negative values and noninteger measurements do not fit this mass function.
Check the parameter domain. The mean must be positive for a nondegenerate Poisson distribution. The natural parameter can be any real number because $\lambda=e^\eta$.
Check normalization. Summing the exponential-family probabilities over the count support gives $e^{-e^\eta}\sum_{y=0}^{\infty}e^{y\eta}/y!=1$, using the exponential series.
Check variation as well as average level. A count process with variation much greater than its mean may not be adequately represented by a single Poisson distribution. Dependence among observations or differing underlying rates can also make a simple common-distribution assumption inappropriate.

A common mistake is to read “variance equals mean” as an empirical claim that every sample must have matching sample statistics. The correction is that $E(Y)=\operatorname{Var}(Y)=\lambda$ describes the population distribution; observed summaries fluctuate around their population counterparts. Another mistake is to treat $\eta$ as the mean. It is the logarithm of the mean: $\eta=\log\lambda$, while the mean is $e^\eta$.

The Poisson form has limitations. It only assigns probability to nonnegative integer counts and, for a single distribution, ties its variance to its mean. A poor fit may reflect this variance restriction, an unsuitable count distribution, or a failure of the assumption that observations follow the same distribution. The exponential-family rewrite does not by itself resolve those issues; it makes the model’s restrictions easier to state and examine.

Check Your Understanding

Answer each question, then compare your reasoning with the concise solutions below.

For a Poisson count with mean $\lambda=5$, state the natural parameter and log-partition function value.
Starting from $b(\eta)=e^\eta$, derive the mean and variance in terms of $\lambda$.
For $\lambda=1.5$, calculate $P(Y=0)$ to four significant figures.
A sample has sample mean 4 and sample variance 4. Does this establish that the Poisson model is correct? Explain.

Solutions. (1) The natural parameter is $\eta=\log(5)\approx1.6094$; $b(\eta)=e^\eta=5$. (2) Since $b'(\eta)=b''(\eta)=e^\eta$ and $\lambda=e^\eta$, both $E(Y)$ and $\operatorname{Var}(Y)$ equal $\lambda$. (3) $P(Y=0)=e^{-1.5}/0!=e^{-1.5}\approx0.2231$. (4) No. The matching sample summaries are consistent with equidispersion, but sample summaries vary, and other features such as the support and distribution of the counts must also be considered.

Representation	Expression	What it highlights
Usual Poisson form	\(e^{-\lambda}\lambda^y/y!\)	The positive mean parameter \(\lambda\) and count probabilities
Exponential-family form	\(\exp\{y\eta-b(\eta)+c(y)\}\)	The natural parameter \(\eta=\log\lambda\), log-partition function \(b(\eta)=e^\eta\), and statistic \(y\)