Fisher Information in Poisson Samples

Why Poisson Information Depends on the Parameter Scale

For a Poisson model, information comes from the curvature of the log-likelihood: a sharply curved likelihood indicates that the data distinguish nearby parameter values more strongly. The curvature can depend on the observed counts, and its numerical value also depends on how the parameter is expressed. We will calculate both observed and expected information for a common rate, then repeat the calculation for its logarithm and for observations collected over different known exposures.

Throughout, counts are independent, exposure amounts are fixed and known, and the rate is positive. We use \(\lambda\) for a rate per unit exposure and \(\alpha=\log\lambda\) for its logarithm. The exposure \(t_i\) scales the mean count; it is not an estimated parameter.

A Common Poisson Rate

Let \(Y_1,\ldots,Y_n\) be independent and identically distributed with \(Y_i\sim\operatorname{Poisson}(\lambda)\), where \(\lambda>0\). Write \(S=\sum_{i=1}^n Y_i\). The joint probability mass function is the product of the individual Poisson mass functions, so the log-likelihood, up to no omitted terms that depend on \(\lambda\), is

Differentiating with respect to the rate gives the score and observed information:

The observed information is random because \(S\) is random. In particular, a sample with a small total count has less curvature on the rate scale than a sample with a large total count, at the same value of \(\lambda\).

To obtain the expected information, use \(E_\lambda[S]=n\lambda\). Thus

The score-variance calculation agrees. Since the counts are independent, \(\operatorname{Var}(S)=n\lambda\), and therefore

This agreement can also be checked by differentiating the log-likelihood twice and taking the negative expectation. The derivatives exist for every \(\lambda>0\), and the Poisson count has finite first and second moments, so the displayed expectations are finite. No interchange of differentiation and an infinite sum is needed for these direct calculations.

Known Unequal Exposures

Now suppose \(Y_1,\ldots,Y_n\) are independent, with \(Y_i\sim\operatorname{Poisson}(t_i\lambda)\). Here each \(t_i>0\) is a known exposure, such as the duration of observation or the amount of population monitored. Define \(T=\sum_{i=1}^n t_i\) and again let \(S=\sum_{i=1}^nY_i\). Independence gives the log-likelihood

The terms involving \(\log t_i\) do not depend on \(\lambda\), since the exposures are known. Differentiating with respect to \(\lambda\) yields

Each count has mean and variance \(t_i\lambda\). Consequently, \(E[S]=\lambda T\) and \(\operatorname{Var}(S)=\lambda T\), by independence. It follows that

Thus known unequal exposures enter the expected information through their sum \(T\). This is not a claim that all observations have the same distribution: their means differ when their exposures differ. It follows directly from the likelihood, which combines the counts through \(S\) and the known exposures through \(T\).

For completeness, the equality between the two information calculations follows explicitly. The score has mean \(E[S]/\lambda-T=0\), and its variance is \(\operatorname{Var}(S)/\lambda^2=T/\lambda\). The expected negative second derivative is also \(E[S]/\lambda^2=T/\lambda\). The Poisson moments are finite, and the score and second derivative are finite for every possible finite count and every \(\lambda>0\).

Reparameterizing to the Log Rate

Let \(\alpha=\log\lambda\), so \(\lambda=e^\alpha\) and \(\alpha\) can be any real number. For the unequal-exposure model, the log-likelihood as a function of \(\alpha\), omitting terms constant in \(\alpha\), is

Its first and second derivatives give the score and observed information on this parameter scale:

Unlike the observed information on the rate scale, this observed information is not random: for a fixed \(\alpha\), it depends only on the known exposures. Its expectation is therefore the same quantity:

The score-variance check gives the same answer because \(\operatorname{Var}(S)=\lambda T\) and \(U(\alpha)=S-\lambda T\). For a common rate with unit exposures, take \(T=n\): \(J(\alpha;Y)=n e^\alpha\) and \(I(\alpha)=n e^\alpha=n\lambda\).

The numerical information on the rate scale and log-rate scale is not generally the same. For example, in the unequal-exposure model the expected information is \(T/\lambda\) for \(\lambda\), but \(\lambda T\) for \(\alpha\). These are curvatures with respect to different coordinates. The calculation must match the parameter being differentiated; one should not carry a curvature formula unchanged from one scale to another.

Common Mistakes and Why the Distinction Matters

• Confusing observed information with its expectation. For the rate, \(S/\lambda^2\) is the observed information and \(T/\lambda\) is its expectation. They need not match in a particular sample.
• Dropping the exposures from the likelihood. For known unequal exposures, the rate term is \(-\lambda T\), not \(-n\lambda\). The score and expected information must therefore involve \(T\).
• Using the rate formula for the log rate. Differentiating with respect to \(\alpha\) produces \(J(\alpha;Y)=e^\alpha T\), not \(S/e^{2\alpha}\). Re-derive the derivatives in the parameter actually used.
• Assuming a zero count forces zero information in every parameterization. In the rate parameterization, a total count \(S=0\) gives observed curvature \(J(\lambda;Y)=0\). In the log-rate parameterization, \(J(\alpha;Y)=e^\alpha T>0\) whenever \(T>0\).

These distinctions matter because information summarizes local likelihood curvature, and that curvature is used to describe how strongly data constrain a parameter. A calculation that omits known exposure or uses the wrong parameter scale can give a numerically incorrect account of that curvature.