Poisson Counts and Log-Rate Parameters

From a Poisson Rate to a Log-Rate

A Poisson model describes a nonnegative integer count through a positive rate parameter. For exponential-family calculations, it is useful to replace that rate by its logarithm. This turns the rate-dependent power in the mass function into a product of a parameter and the observed count, while keeping the same probability model.

Let $Y$ have a Poisson distribution with rate $\lambda>0$. Its support is $y\in\{0,1,2,\ldots\}$, and its probability mass function is

P(Y=y)=f(y;\lambda)=\frac{e^{-\lambda}\lambda^y}{y!}.

Define the log-rate parameter $\eta=\log(\lambda)$. Since $\lambda>0$, this transformation is well defined, and its inverse is $\lambda=e^\eta$. In particular, a finite real $\eta$ always corresponds to a positive rate.

Definition: The Poisson mass function in log-rate form is $$ f(y;\eta)=\exp\left\{y\eta-e^\eta-\log(y!)\right\},\qquad y=0,1,2,\ldots,\quad \eta\in\mathbb{R}. $$ The count statistic is $T(y)=y$, the log-partition function is $A(\eta)=e^\eta$, and the base measure is $h(y)=1/y!$ on the nonnegative integers.

The form displayed above separates the roles of the observation, parameter, and support. The statistic $T(y)$ is the part of the observation multiplied by the natural parameter. The log-partition function depends on the parameter but not on the observed count. The base measure depends on the count but not on the parameter. Since the natural parameter itself is $\eta$, the natural-parameter domain is the set of real values for which the corresponding normalizing sum is finite.

Deriving the Canonical Form

Start with the Poisson mass function and substitute $\lambda=e^\eta$. For a nonnegative integer $y$, the power term satisfies $\lambda^y=(e^\eta)^y=e^{y\eta}$, including $y=0$. Therefore

f(y;\eta) =\frac{e^{-e^\eta}e^{y\eta}}{y!} =\exp\left\{y\eta-e^\eta-\log(y!)\right\}.

This gives a canonical one-parameter exponential-family representation with a parameter-free carrier term. In the usual form $h(y)\exp\{\eta T(y)-A(\eta)\}$, the carrier is $h(y)=1/y!$, and taking its logarithm gives the term $-\log(y!)$ inside the exponent. It is important that the factorial term remains: it is not part of the log-partition function because it does not vary with $\eta$.

Derived result: For the Poisson model with rate $\lambda>0$, setting $\eta=\log\lambda$ gives $$ f(y;\eta)=h(y)\exp\{\eta T(y)-A(\eta)\}, \qquad T(y)=y,\quad h(y)=\frac{1}{y!},\quad A(\eta)=e^\eta. $$ This follows by substituting $\lambda=e^\eta$ into the mass function and collecting the count-dependent and parameter-dependent terms.

Worked Example: Rewriting a Poisson Probability

Suppose $Y$ has a Poisson distribution with rate $\lambda=2.5$, and find $P(Y=3)$ in log-rate form.

The natural parameter is $\eta=\log(2.5)$, so $e^\eta=2.5$. The observed count is $y=3$, and $3!=6$. Substitution in the canonical mass function gives

P(Y=3) =\exp\{3\log(2.5)-2.5-\log(6)\} =\frac{(2.5)^3e^{-2.5}}{6}.

Since $(2.5)^3=15.625$, the probability is

P(Y=3)=\frac{15.625}{6}e^{-2.5} \approx 2.60417(0.082085) \approx 0.21376.

The same calculation shows the roles of the components: $3$ is the count statistic, $\log(2.5)$ is multiplied by that statistic, $A(\eta)=2.5$, and the base-measure contribution is $1/3!=1/6$.

The Log-Partition Function and Natural-Parameter Domain

The log-partition function is determined by normalization: summing the mass function over every possible count must give one. This provides a direct check of the proposed representation and identifies the natural-parameter domain.

Theorem: For the Poisson representation with $T(y)=y$ and $h(y)=1/y!$, the normalizing sum is finite for every $\eta\in\mathbb{R}$, and its logarithm is $A(\eta)=e^\eta$. Thus the natural-parameter domain is all of $\mathbb{R}$.

Proof. For a fixed finite real $\eta$, the normalizing sum is

Z(\eta) =\sum_{y=0}^{\infty}h(y)e^{\eta T(y)} =\sum_{y=0}^{\infty}\frac{e^{y\eta}}{y!}.

Set $x=e^\eta$. For finite real $\eta$, $x$ is finite and strictly positive. The exponential series converges for every finite $x$ and gives

Z(\eta)=\sum_{y=0}^{\infty}\frac{x^y}{y!} =e^x =\exp(e^\eta).

Consequently $Z(\eta)$ is finite and positive for every real $\eta$, and its logarithm is

A(\eta)=\log Z(\eta)=e^\eta.

Every finite real value is therefore admissible. The parameterization $\eta=\log\lambda$ with $\lambda>0$ also shows that no finite real value is missing: for any $\eta\in\mathbb{R}$, $e^\eta$ is a valid positive Poisson rate. Hence the natural-parameter domain is exactly $\mathbb{R}$. $\square$

This calculation is more than a formal check. The log-partition function is precisely what makes the exponential-family expression sum to one. Omitting it, or using a different function in its place, would generally produce weights that do not form a probability distribution.

Worked Example: Checking the Normalizer at a Specific Parameter

Take $\eta=\log(4)$, corresponding to rate $\lambda=4$. The unnormalized terms in the exponential-family sum are $4^y/y!$. Their total is

\sum_{y=0}^{\infty}\frac{4^y}{y!}=e^4.

The log-partition function at this parameter is $A(\eta)=e^\eta=4$, so the normalized mass for any count $y$ is

\frac{4^y/y!}{e^4} =\exp\{y\log(4)-4-\log(y!)\}.

For $y=0$, this gives $P(Y=0)=e^{-4}$, since $0!=1$. For $y=5$, it gives

P(Y=5)=\frac{4^5e^{-4}}{5!} =\frac{1024}{120}e^{-4} \approx 8.53333(0.0183156) \approx 0.15629.

Both probabilities use the same normalizer $e^4$; the count changes the statistic and base-measure contribution, not the log-partition function.

Using the Representation for Several Counts

The same factorization can be applied observation by observation. If $Y_1,\ldots,Y_n$ are independent Poisson counts with the same rate and observed values $y_1,\ldots,y_n$, their joint likelihood is the product of their individual mass functions. Multiplying the terms adds their exponents, giving

L(\eta) =\prod_{i=1}^n \frac{\exp\{y_i\eta-e^\eta\}}{y_i!} =\exp\left\{\eta\sum_{i=1}^n y_i-ne^\eta-\sum_{i=1}^n\log(y_i!)\right\}.

This is a direct product-likelihood calculation: the parameter appears through $\eta$ and $e^\eta$, while the factorial terms depend only on the observed counts. The number of observations $n$ matters because the same log-partition contribution occurs once for each observation.

Worked Example: A Likelihood for Three Independent Counts

Suppose three independent counts have a common Poisson rate $\lambda=2$, and the observed values are $1,0,3$. Then $n=3$, $\eta=\log(2)$, and $\sum_i y_i=1+0+3=4$. Substitution into the product likelihood gives

L(\eta) =\exp\{4\log(2)-3(2)-[\log(1!)+\log(0!)+\log(3!)]\}.

Because $1!=1$, $0!=1$, and $3!=6$, the expression reduces to

L(\eta) =\frac{2^4e^{-6}}{1!\,0!\,3!} =\frac{16}{6}e^{-6} =\frac{8}{3}e^{-6} \approx \frac{8}{3}(0.00247875) \approx 0.006610.

The count-statistic contributions add to $4$, and the log-partition contribution is $3e^\eta=6$, one term for each observation. The product of base measures is $1/(1!\,0!\,3!)=1/6$.

Common Mistakes and Why the Components Matter

A frequent error is to treat the rate $\lambda$ itself as the natural parameter in the canonical expression. The original mass function contains $\lambda^y$, but after taking the exponential-family form, the coefficient multiplying $y$ is $\log\lambda$, not $\lambda$. For example, with rate $4$, the term involving the observed count is $y\log(4)$, not $4y$.

Another error is to call $-\log(y!)$ part of the log-partition function. This term varies with the observed count and stays fixed as the parameter changes. The log-partition function must instead be $e^\eta$, which depends on the parameter and normalizes the sum over all possible counts.

Wrong: “The natural parameter is $\lambda$, so the exponent contains $y\lambda$.”
Correction: Rewriting $\lambda^y$ as $\exp\{y\log\lambda\}$ shows that $\eta=\log\lambda$ multiplies $T(y)=y$.
Wrong: “The factorial is the normalizer because it appears in the denominator.”
Correction: The factorial is part of the base measure; normalization over the support yields $Z(\eta)=\exp(e^\eta)$, so $A(\eta)=e^\eta$.

These distinctions matter because the canonical representation separates the model’s parameter dependence from its count-dependent carrier term. Once the pieces are identified correctly, the normalization and the full real natural-parameter domain follow from the exponential series.

Key takeaway: For a Poisson count, the log-rate $\eta=\log\lambda$ is the natural parameter, the observed count $y$ is the count statistic, $1/y!$ is the base measure, and $e^\eta$ is the log-partition function. The natural-parameter domain is $\mathbb{R}$.

Check Your Understanding

Use the canonical representation to answer each question.

For a Poisson count with rate $\lambda=7$, state the natural parameter and write the mass function in log-rate form.
Identify the count statistic, base measure, and log-partition function for an observed count $y$.
Why is the natural-parameter domain all real numbers, rather than only positive real numbers?
For two independent Poisson counts $y_1$ and $y_2$ with a common rate, write the product likelihood in terms of $\eta$.
Explain why $-\log(y!)$ belongs to the base-measure contribution rather than the log-partition function.