Why Write the Poisson Distribution in Exponential Form?
A count model is often first described by its rate: for example, a hypothetical process may produce an average of 3.5 events during a specified observation period. In an exponential-family representation, the same distribution is described using a natural parameter and a cumulant function. Recognizing those pieces makes it possible to obtain the mean and variance systematically and to compare the Poisson model with other distributions in the family.
This tutorial focuses on the Poisson probability mass function (pmf). The count \(Y\) is assumed to take values \(y=0,1,2,\ldots\), with rate parameter \(\lambda>0\). Under the Poisson model, the probability of observing \(y\) is determined by this rate. No exposure adjustment is included here: the rate \(\lambda\) refers to the particular fixed observation interval in the model.
Rewriting the Probability Mass Function
The key algebraic step is to express the power \(\lambda^y\) using a logarithm. Since \(\theta=\log\lambda\), we have \(\lambda=e^\theta\) and \(\lambda^y=e^{y\theta}\). Also, \(e^{-\lambda}=e^{-e^\theta}\). Substituting these identities into the pmf gives
In the standard unit-dispersion exponential-family form
the Poisson components are therefore \(\theta=\log\lambda\), \(b(\theta)=e^\theta\), and \(c(y)=-\log(y!)\). The support consists of the nonnegative integers and does not depend on \(\theta\). The rate \(\lambda\) and the natural parameter \(\theta\) are related one-to-one, but they are not the same quantity: \(\lambda=e^\theta\).
| Exponential-family component | Poisson value | Role |
|---|---|---|
| Response | \(Y=y\) | Observed count |
| Natural parameter | \(\theta=\log\lambda\) | Parameter multiplying \(y\) in the exponent |
| Cumulant function | \(b(\theta)=e^\theta\) | Function that normalizes the distribution and determines its moments |
| Response-only term | \(c(y)=-\log(y!)\) | Term that depends on the count, not on \(\theta\) |
| Dispersion factor | \(1\) | The ordinary Poisson form has no separate free dispersion parameter |
A quick structural check is useful. The coefficient of \(y\) in the exponent is \(\theta\), so it identifies the natural parameter. The term subtracted from \(y\theta\) is \(e^\theta\), so it is \(b(\theta)\). The factorial term is independent of the parameter and belongs in \(c(y)\). These assignments are specific to the displayed convention; changing the general exponential-family convention can change how terms are written, so the convention should be stated before matching components.
Why the Cumulant Function Gives the Mean and Variance
For an exponential-family distribution in this unit-dispersion form, the moment-generating function can be found by inserting \(e^{tY}\) inside the expectation. The \(e^{tY}\) factor adds \(t\) to the coefficient of \(Y\), changing \(\theta\) to \(\theta+t\). Taking the ratio of the resulting normalizing factors gives
Taking the logarithm gives the cumulant-generating function \(K_Y(t)=b(\theta+t)-b(\theta)\). Its first derivative at zero is the mean, and its second derivative at zero is the variance. Thus, when the needed derivatives exist,
For the Poisson distribution, \(b(\theta)=e^\theta\), so both derivatives are \(e^\theta\). Since \(e^\theta=\lambda\), the mean and variance are both \(\lambda\). The same result follows directly from the probability-generating calculation: \(M_Y(t)=\exp\{e^\theta(e^t-1)\}\), whose first two cumulants are \(e^\theta\).
The normalization also verifies that the proposed form is a probability distribution. Summing over the allowed count values yields
The second equality uses the power-series expansion of the exponential function. This calculation explains why the term \(b(\theta)=e^\theta\) is needed: it offsets the parameter-dependent sum and makes the probabilities add to one.
Worked Example: Identify the Components and Calculate a Count Probability
A hypothetical monitoring system records the number \(Y\) of alert events during a fixed shift. Assume \(Y\sim\operatorname{Poisson}(\lambda)\) with \(\lambda=3.5\). Write the model in exponential-family form, state its mean and variance, and calculate \(P(Y=2)\). Use four decimals for the final probability.
Method. Convert the rate to the natural parameter, substitute into the exponential-family form, and calculate the requested probability from the original Poisson pmf.
Natural parameter and cumulant function. The natural parameter is \(\theta=\log(3.5)\approx1.252763\). The cumulant function is \(b(\theta)=e^\theta\), which here equals \(3.5\). The full representation is
Mean and variance. Since \(E[Y]=\lambda\) and \(\operatorname{Var}(Y)=\lambda\), both are \(3.5\) alerts per shift (the variance is in squared count units).
Probability. Substituting \(y=2\) into the pmf gives
Rounded to four decimals, \(P(Y=2)=0.1850\). The probability is between zero and one, as required. The common mistake to avoid is using \(3.5\) as \(\theta\); it is the rate \(\lambda\), while \(\theta=\log(3.5)\).
Worked Example: Convert a Natural Parameter Back to a Rate
Suppose a hypothetical count variable has the exponential-family representation \(p(y)=\exp\{y\theta-e^\theta-\log(y!)\}\), with \(\theta=\log(2.4)\). Identify the Poisson rate, cumulant value, mean, and variance, then calculate \(P(Y=4)\). Give the final probability to four decimals.
Identify the model. Comparing terms with \(\exp\{y\theta-b(\theta)+c(y)\}\), we have \(b(\theta)=e^\theta\) and \(c(y)=-\log(y!)\). The rate is \(\lambda=e^\theta=2.4\), so \(Y\sim\operatorname{Poisson}(2.4)\). Consequently, \(E[Y]=2.4\) and \(\operatorname{Var}(Y)=2.4\).
Calculate the requested probability.
Rounded to four decimals, \(P(Y=4)=0.1254\). The arithmetic is checked by noting that the factorial denominator is \(4!=24\), and the resulting probability is valid. A common error is to report \(\theta=\log(2.4)\) as the mean; the mean is \(e^\theta=2.4\), not the logarithm.
Worked Example: Use the Exponential Form for a Cumulative Probability
A hypothetical instrument produces a count \(Y\) in a fixed inspection period, with model \(p(y)=\exp\{y\theta-e^\theta-\log(y!)\}\) and \(\theta=\log(1.5)\). Find \(P(Y\leq1)\), identify the cumulant value, and check the mean and variance. Report the probability to four decimals.
Recover the rate and moments. The rate is \(\lambda=e^\theta=1.5\). The cumulant value is \(b(\theta)=e^\theta=1.5\). Therefore \(E[Y]=1.5\) and \(\operatorname{Var}(Y)=1.5\).
Add the probabilities for the permitted counts. The event \(Y\leq1\) consists of \(Y=0\) and \(Y=1\), so
Rounded to four decimals, \(P(Y\leq1)=0.5578\). This result is below one and includes exactly the two possible values in the event. The common mistake is to calculate only \(P(Y=1)\); a cumulative probability must include the probability at zero as well.
Common Traps and a Reliable Check
The most important distinction is between the ordinary rate and the natural parameter. For \(\lambda>0\), the rate is \(\lambda\), while the natural parameter is \(\theta=\log\lambda\). The cumulant function is \(b(\theta)=e^\theta\), which equals \(\lambda\) after conversion. If a solution labels \(\lambda\) as the natural parameter, it has confused the two parameter scales.
A second trap is to omit the factorial term or put it into the cumulant function. The term \(-\log(y!)\) depends on the observed count but not on the parameter, so it belongs in \(c(y)\). By contrast, \(-e^\theta\) depends on the parameter and is the negative cumulant term. A practical check is to sum the proposed probabilities over \(y=0,1,\ldots\); a correct representation must sum to one.
- Wrong: \(\theta=\lambda\). Correction: \(\theta=\log\lambda\), so \(\lambda=e^\theta\).
- Wrong: \(b(\theta)=\theta\). Correction: \(b(\theta)=e^\theta\), as required by the Poisson normalizing term.
- Wrong: \(E[Y]=\log\lambda\). Correction: \(E[Y]=b'(\theta)=e^\theta=\lambda\).
- Wrong: Include negative or fractional values of \(y\) in the support. Correction: A Poisson variable takes only nonnegative integer values.
Check Your Understanding
For each problem, identify the requested exponential-family quantities before calculating any probability. Final answers are provided for checking.
- A hypothetical counter has \(Y\sim\operatorname{Poisson}(4)\). State \(\theta\), \(b(\theta)\), the mean, and the variance. (Answer: \(\theta=\log 4\), \(b(\theta)=4\), \(E[Y]=4\), and \(\operatorname{Var}(Y)=4\).)
- A count model is written as \(\exp\{y\theta-e^\theta-\log(y!)\}\), where \(\theta=\log(0.8)\). State the rate and calculate \(P(Y=0)\) to four decimals. (Answer: \(\lambda=0.8\), \(P(Y=0)=e^{-0.8}\approx0.4493\).)
- For \(Y\sim\operatorname{Poisson}(2)\), write the pmf in exponential-family form and calculate \(P(Y=3)\) to four decimals. (Answer: \(\theta=\log2\), \(b(\theta)=e^\theta=2\), \(P(Y=y)=\exp\{y\log2-2-\log(y!)\}\), and \(P(Y=3)=e^{-2}2^3/3!\approx0.1804\).)
- A proposed representation is \(\exp\{y\lambda-\lambda-\log(y!)\}\), with \(\lambda=1.2\). Explain whether \(\lambda\) is the natural parameter in this expression and state the correct natural parameter for a Poisson rate of 1.2. (Answer: As written, the coefficient of \(y\) is the natural parameter, but the expression does not use the Poisson rate 1.2 consistently as its cumulant term; for rate 1.2 the correct natural parameter is \(\theta=\log(1.2)\), and the exponent is \(y\log(1.2)-1.2-\log(y!)\).)
- For \(Y\sim\operatorname{Poisson}(1.5)\), calculate \(P(Y\leq2)\) to four decimals by summing the relevant pmf terms. (Answer: \(e^{-1.5}\{1+1.5+(1.5)^2/2\}=0.8088\).)