The Canonical Exponential-Family Form For Poisson Responses

Why the Canonical Form Matters

The Poisson probability mass function is familiar in the form \(P(Y=y)=e^{-\mu}\mu^y/y!\), where \(\mu>0\) and \(y\) is a nonnegative integer. The same distribution can be written in a standard exponential-family form. That rewrite is more than a change of notation: it makes clear which part of the model is the natural parameter, which part normalizes the probabilities, and which part depends only on the observed response.

For a Poisson response with a conditional mean specified by a log link, the natural parameter turns out to be the log of that mean. The mean itself is not the natural parameter. This distinction is especially useful when connecting the Poisson distribution to the general theory of generalized linear models.

Definition: A one-parameter exponential-family model, relative to a specified measure, can be written as \(f(y;\eta)=h(y)\exp\{T(y)\eta-A(\eta)\}\). Here \(T(y)\) is the sufficient statistic associated with the parameter, \(\eta\) is the natural parameter, \(A(\eta)\) is the log-partition function (also called the log-normalizer), and \(h(y)\) is the base measure or base-measure factor. In the Poisson case, the sample space is the nonnegative integers and the reference measure is counting measure.

Putting the Poisson Mass Function in Canonical Form

For \(y\in\{0,1,2,\ldots\}\), the Poisson probability mass function is \(f(y;\mu)=e^{-\mu}\mu^y/y!\), with \(\mu>0\). Take logarithms of the factors in this expression and use \(\eta=\log\mu\), so that \(\mu=e^\eta\):

\begin{aligned} f(y;\mu) &= \frac{1}{y!}\exp\{-\mu\}\exp\{y\log\mu\}\\ &= \frac{1}{y!}\exp\{y\eta-e^\eta\}\\ &= h(y)\exp\{T(y)\eta-A(\eta)\}, \end{aligned}

Thus \(T(y)=y\), \(h(y)=1/y!\), and \(A(\eta)=e^\eta\). Equivalently, the response-dependent base-measure term can be written inside the exponent as \(c(y)=-\log(y!)\), giving \(f(y;\eta)=\exp\{y\eta-e^\eta-\log(y!)\}\). These are two notational ways of displaying the same factorization.

Canonical Poisson form: For \(y=0,1,2,\ldots\) and \(\eta\in\mathbb{R}\), \(f(y;\eta)=\dfrac{1}{y!}\exp\{y\eta-e^\eta\}\). The natural parameter is \(\eta=\log\mu\), the sufficient statistic is \(T(y)=y\), the log-partition function is \(A(\eta)=e^\eta\), and the base measure is \(h(y)=1/y!\), relative to counting measure.

Worked Example: Identifying the Components at a Fixed Mean

Suppose \(Y\) has a Poisson distribution with mean \(\mu=3.2\), and consider the probability of observing \(y=2\). First calculate the natural parameter: \(\eta=\log(3.2)\). The components in the canonical form are \(T(2)=2\), \(A(\eta)=e^\eta=3.2\), and \(h(2)=1/2!=1/2\).

Substituting them into the exponential-family expression gives \(f(2;\eta)=\frac{1}{2}\exp\{2\log(3.2)-3.2\}\). Since \(\exp\{2\log(3.2)\}=(3.2)^2=10.24\), this becomes \(\frac{1}{2}\times 10.24\times e^{-3.2}=5.12e^{-3.2}\), approximately \(0.2087\).

The original mass-function calculation is \(e^{-3.2}(3.2)^2/2!=5.12e^{-3.2}\), the same result. In particular, the factorial appears in the base measure; it does not change the natural parameter or the log-partition function.

Conditional Poisson Models

In a conditional Poisson response model, the distribution of \(Y\) is specified given covariates \(x\). If its conditional mean is \(\mu(x)>0\), then its conditional mass function has exactly the same decomposition:

f(y\mid x) = \frac{1}{y!} \exp\{y\log\mu(x)-\mu(x)\}.

The natural parameter may therefore depend on the covariates: \(\eta(x)=\log\mu(x)\). Under a log-link specification \(\log\mu(x)=x^\mathsf{T}\beta\), this natural parameter is the linear predictor. The response support and base measure remain unchanged as \(x\) varies; only the parameter-dependent terms change.

Worked Example: A Linear Predictor as the Natural Parameter

Suppose the conditional model is \(Y\mid x\sim\operatorname{Poisson}(\mu(x))\), with \(\log\mu(x)=\beta_0+\beta_1x\). At \(x=2\), let \(\beta_0=-1\) and \(\beta_1=0.4\). The linear predictor, and hence the natural parameter, is \(\eta(2)=-1+0.4(2)=-0.2\).

The conditional mean corresponding to this parameter is \(\mu(2)=e^{-0.2}\), approximately \(0.8187\). For \(y=1\), the base-measure factor is \(h(1)=1/1!=1\), the sufficient statistic is \(T(1)=1\), and the log-partition function is \(A(-0.2)=e^{-0.2}\). Thus \(f(1\mid x=2)=\exp\{-0.2-e^{-0.2}\}\), approximately \(0.361\).

This substitution identifies the canonical components at the covariate value in question. It does not make the natural parameter equal to the conditional mean: here the natural parameter is \(-0.2\), whereas the mean is \(e^{-0.2}\).

Normalization and the Natural Parameter Space

The log-partition function is not an arbitrary extra term. It is the term that makes the probabilities sum to one. For the Poisson family, normalization also shows that every real number is an allowed natural parameter.

Theorem: For the canonical Poisson expression with \(T(y)=y\) and \(h(y)=1/y!\), the normalizing sum is finite and equal to \(\exp(e^\eta)\) for every \(\eta\in\mathbb{R}\). Consequently the log-partition function is \(A(\eta)=e^\eta\), and the natural parameter space is all of \(\mathbb{R}\).

Proof. For any fixed real \(\eta\), each term in the sum is nonnegative. The exponential power series converges for every finite real argument, so

\sum_{y=0}^{\infty}h(y)e^{y\eta} = \sum_{y=0}^{\infty}\frac{(e^\eta)^y}{y!} = \exp(e^\eta) < \infty.

A probability mass function proportional to \(h(y)e^{y\eta}\) must divide by this sum. Its normalizing factor is therefore \(\exp(-e^\eta)\), which can be written as \(\exp\{-A(\eta)\}\) with \(A(\eta)=e^\eta\). Since the sum is finite for every real \(\eta\), every real value is permitted; conversely, the natural parameter is defined here as a real number, so the natural parameter space is exactly \(\mathbb{R}\). \(\square\)

This argument also explains why \(A\) is called the log-partition function. The partition sum is \(\sum_y h(y)e^{y\eta}=\exp(e^\eta)\); taking its logarithm gives \(A(\eta)=e^\eta\). More generally, once the statistic \(T\) and base factor \(h\) are fixed, normalization determines \(A\).

Result: With \(T(y)=y\) and \(h(y)=1/y!\) fixed, the normalizing term in the exponent is uniquely determined by \(A(\eta)=\log\sum_{y=0}^{\infty}h(y)e^{y\eta}\), whenever that sum is finite and positive. For the Poisson family, this gives \(A(\eta)=\log\{\exp(e^\eta)\}=e^\eta\).

Justification. If \(h(y)\exp\{y\eta-A(\eta)\}\) is a normalized mass function, summing over the support gives

1 = \sum_{y=0}^{\infty}h(y)e^{y\eta-A(\eta)} = e^{-A(\eta)}\sum_{y=0}^{\infty}h(y)e^{y\eta}.

The sum is positive. Rearranging and taking logarithms therefore forces \(A(\eta)=\log\sum_{y=0}^{\infty}h(y)e^{y\eta}\). For the Poisson base factor, the exponential-series calculation above evaluates this logarithm as \(e^\eta\). \(\square\)

Worked Example: A Positive Exposure in the Canonical Form

Suppose a count has a Poisson mean equal to exposure \(t\) times a positive rate, with \(t=3\) and rate \(\exp(\beta_0+\beta_1x)\). At \(x=2\), take \(\beta_0=-1\) and \(\beta_1=0.5\). The rate is \(\exp(-1+0.5(2))=\exp(0)=1\), so the mean is \(\mu=3(1)=3\).

The natural parameter is the log of the full mean, not the log of the rate alone: \(\eta=\log\mu=\log 3\). Equivalently, \(\eta=\log t+\beta_0+\beta_1x=\log 3\). For an observed count \(y=4\), the canonical expression is \(\frac{1}{4!}\exp\{4\log 3-e^{\log 3}\}\).

Because \(4!=24\), \(e^{\log 3}=3\), and \(\exp(4\log 3)=3^4=81\), this probability is \(\frac{81}{24}e^{-3}=3.375e^{-3}\), approximately \(0.1680\). Exposure changes the natural parameter through the full mean, but the support and base factor \(1/y!\) are unchanged.

Reading the Factorization Correctly

The factorization separates terms by their roles, not merely by their appearance. The term \(y\eta\) combines the observed response with the parameter. The term \(-A(\eta)=-e^\eta\) depends on the parameter and normalizes the distribution. The term \(-\log(y!)\) depends on the response but not on the parameter; in multiplicative form it is the base factor \(1/y!\).

The natural parameter is particularly important because the canonical link for the Poisson family relates the conditional mean to this parameter through \(\eta=\log\mu\). In a regression model using the log link, the linear predictor is therefore the natural parameter. This identification does not by itself derive any moments of the distribution; those are separate calculations.

Common Mistakes

Calling \(\mu\) the natural parameter. The canonical form contains \(y\eta\), with \(\eta=\log\mu\). The mean is recovered as \(\mu=e^\eta\), not by setting \(\eta=\mu\).
Putting the factorial into the log-partition function. The term \(-\log(y!)\) varies with the observed response, while \(A(\eta)\) depends only on the natural parameter. The factorial belongs in \(h(y)\).
Forgetting the support or reference measure. The Poisson mass is defined for nonnegative integers and is expressed relative to counting measure. The factor \(1/y!\) is its base-measure factor in this representation.
Using only the log rate when the mean includes exposure. If \(\mu=t\lambda\), then \(\eta=\log(t\lambda)=\log t+\log\lambda\). The canonical parameter is the log of the full conditional mean.

Key takeaway: For a Poisson response, the canonical exponential-family representation is \(f(y;\eta)=\frac{1}{y!}\exp\{y\eta-e^\eta\}\). Thus \(T(y)=y\), \(\eta=\log\mu\), \(A(\eta)=e^\eta\), and \(h(y)=1/y!\).

Check Your Understanding

Use the canonical form to answer each question.

For a Poisson response with mean \(\mu=5\), what are the natural parameter and log-partition function evaluated at that parameter?
In \(f(y;\eta)=h(y)\exp\{T(y)\eta-A(\eta)\}\), identify \(T(y)\) and \(h(y)\) for the Poisson family.
Why does the exponential-series identity show that every real \(\eta\) is allowed?
If \(\mu=t\exp(x^\mathsf{T}\beta)\) for fixed positive \(t\), write the natural parameter in terms of \(t\), \(x\), and \(\beta\).
Which term in the Poisson mass function depends on the response but not on \(\eta\), and where does it belong in the canonical form?