Deriving the Poisson Log-Link Score

Why Differentiate With Respect to the Linear Predictor?

In a Poisson model, the response is a count and the conditional mean must be positive. The log link meets that requirement by expressing the mean as the exponential of a linear predictor. Differentiating the log-likelihood with respect to that predictor gives a compact quantity: the observed count minus the fitted mean. Its sign tells us whether a small upward or downward change in the predictor would locally improve the likelihood for that observation.

The focus here is a single observed count and a derivative with respect to its linear predictor. This is a local statement about the likelihood, not a complete model-fitting procedure or a test of a hypothesis. The derivation also makes clear why the score is not itself the change in log-likelihood: a score must be multiplied by a step size to give a first-order approximation to that change.

Model and assumptions: Let $Y$ be one count with observed value $y$, where $y$ is a nonnegative integer. Assume $Y\sim\operatorname{Poisson}(\mu)$, so $P(Y=y)=e^{-\mu}\mu^y/y!$, with $\mu>0$. Under the log link, the linear predictor is $\eta=\log(\mu)$, equivalently $\mu=e^\eta$, where $\eta$ can be any finite real number. The likelihood calculation treats the observed count as arising from this Poisson distribution with the specified mean.

Deriving the Log-Likelihood and Score

For the observed value $y$, substitute $\mu=e^\eta$ into the Poisson probability mass function. The likelihood is the probability assigned to that observed count, viewed as a function of $\eta$. Taking its natural logarithm gives the log-likelihood.

L(\eta;y)=\frac{\exp(-e^\eta)(e^\eta)^y}{y!}, \qquad \ell(\eta;y)=-e^\eta+y\eta-\log(y!).

The factorial term depends on the observed count but not on $\eta$. It therefore contributes to the log-likelihood value but disappears when differentiating with respect to $\eta$. The score is this derivative.

Definition: The Poisson log-link score for one observation, differentiated with respect to its linear predictor, is $$ U(\eta;y)=\frac{\partial \ell(\eta;y)}{\partial\eta} =y-e^\eta=y-\mu. $$ It measures the instantaneous rate of change of the log-likelihood per unit increase in $\eta$, evaluated at the stated $\eta$ and observed count $y$.

The derivative follows directly: the derivative of $-e^\eta$ is $-e^\eta$, that of $y\eta$ is $y$, and that of the constant $-\log(y!)$ is zero. Because $e^\eta=\mu$, the score is the observed count minus the mean specified by the model at that predictor.

\frac{\partial}{\partial\eta}\{-e^\eta+y\eta-\log(y!)\} =-e^\eta+y =y-\mu.

This sign has a direct local interpretation. If $y>\mu$, then $U(\eta;y)>0$, so a sufficiently small increase in $\eta$ increases the log-likelihood to first order. If $y<\mu$, the score is negative, so a small decrease in $\eta$ increases it to first order. If $y=\mu$, the first derivative is zero: infinitesimal changes have no first-order effect. The predictor acts on a log scale, so changing $\eta$ changes the mean multiplicatively, not by adding the same number of counts.

For a small predictor step $h$, differentiability gives $\ell(\eta+h;y)-\ell(\eta;y)\approx U(\eta;y)h$. The score alone is a slope; the product of the slope and the step is the first-order approximation to the change. This approximation is most useful for small $h$, and it is not generally equal to the exact change for a finite step.

Worked Examples

Worked Example: A Count Above the Fitted Mean

Suppose a hypothetical observation is $y=7$ events, and the current predictor is $\eta=\log(5)$. The implied fitted mean is $e^\eta=5$ events. The model assumptions for this calculation are that the observation is a nonnegative count and its likelihood is Poisson with this mean.

The log-likelihood and score are $$ \ell(\log 5;7)=7\log(5)-5-\log(7!) \approx -2.25910, $$ $$ U(\log 5;7)=7-e^{\log 5}=7-5=2. $$

The score is positive because seven observed events exceed the fitted mean of five. For an upward step of $h=0.1$, the first-order approximation to the change in log-likelihood is $$ U(\log 5;7)(0.1)=2(0.1)=0.2000. $$ This is the score at the starting predictor multiplied by the step; it is not the score evaluated at $0.1$. The exact change is $$ \ell(\log 5+0.1;7)-\ell(\log 5;7) =7(0.1)-5(e^{0.1}-1) \approx 0.1741. $$ The positive exact change agrees with the score’s local direction, while differing from the first-order approximation because the step is finite.

Worked Example: A Zero Count Above a Positive Mean

For a hypothetical count $y=0$, let the current predictor be $\eta=\log(2)$, giving fitted mean $e^\eta=2$ events. Zero is a valid Poisson count, so the probability mass function applies; here $0!=1$.

The log-likelihood and score are $$ \ell(\log 2;0)=-2+0\log(2)-\log(0!)=-2, \qquad U(\log 2;0)=0-2=-2. $$

The negative score means the log-likelihood locally increases if the predictor is decreased. For a downward step of $h=-0.2$, the first-order approximation to the change is $$ U(\log 2;0)(-0.2)=(-2)(-0.2)=0.4000. $$ The exact change is $$ \ell(\log 2-0.2;0)-\ell(\log 2;0) =-2e^{-0.2}+2 \approx 0.3625. $$ It is positive, as the score predicts. This example also points to an edge case: for $y=0$, the log-likelihood is $-e^\eta$, which increases toward zero as $\eta$ decreases without bound. If $\eta$ is unrestricted, there is no finite predictor that maximizes this one-observation log-likelihood.

Worked Example: A Zero Score at the Fitted Mean

Suppose a hypothetical observation is $y=4$ events and the predictor is $\eta=\log(4)$, so the fitted mean is four events. The count is a nonnegative integer and the mean is positive, as required by the Poisson model.

The log-likelihood and score are $$ \ell(\log 4;4)=4\log(4)-4-\log(4!) \approx 4(1.38629)-4-3.17805=-1.6329, $$ $$ U(\log 4;4)=4-e^{\log 4}=4-4=0. $$

The score is zero because the observed count equals the fitted mean. To determine the local curvature, differentiate the score once more: $$ \frac{\partial^2\ell(\eta;y)}{\partial\eta^2}=-e^\eta. $$ At $\eta=\log(4)$, this second derivative is $-4$, so the log-likelihood is locally concave there. For this single observation, it is also the finite maximum: the score is positive when $e^\eta<4$ and negative when $e^\eta>4$. Thus the log-likelihood increases up to $\log(4)$ and decreases after it.

Concavity and What the Score Establishes

The second derivative is negative for every finite $\eta$, since $e^\eta>0$. Therefore the single-observation log-likelihood is strictly concave as a function of the predictor. This gives a useful justification for interpreting the score’s sign: the score decreases as $\eta$ increases, and any finite point where the score is zero is the unique maximizer.

\frac{\partial^2\ell(\eta;y)}{\partial\eta^2}=-e^\eta<0.

Solving $U(\eta;y)=0$ requires $e^\eta=y$. If $y>0$, this equation has the finite solution $\eta=\log(y)$; strict concavity makes it the unique maximum. If $y=0$, the equation has no finite solution because $e^\eta>0$ for every finite $\eta$. In that case the likelihood approaches its supremum as $\eta$ tends to negative infinity, consistent with the zero-count example.

For multiple conditionally independent Poisson observations with predictors $\eta_i$ and observed counts $y_i$, the log-likelihood is the sum of the individual log-likelihoods, so its derivative with respect to the predictor for observation $i$ is $y_i-e^{\eta_i}$. This statement concerns derivatives with respect to the separate predictors. Differentiating with respect to model coefficients would require accounting for how those predictors depend on the coefficients; that is a distinct derivative.

Interpretation, Checks, and Common Pitfalls

The score is a signed discrepancy between the count and its fitted mean, expressed as a derivative on the predictor scale. A positive value does not mean that the model is wrong; it says that, for this observation at this current predictor, a small upward move would improve its log-likelihood to first order. Likewise, a negative value gives a local direction, not a guarantee that an arbitrary large move will improve the likelihood.

The Poisson assumption is substantive. The response must be a count, and the probability model assigns probabilities according to a Poisson distribution with mean $\mu$. In particular, this distribution has variance $\mu$. If the observations or their conditional variability are inconsistent with the assumed model, the algebraic derivative remains the derivative of the assumed likelihood, but its model-based interpretation may not be adequate. The score for one observation cannot diagnose those issues by itself.

Useful checks include confirming that the response is a nonnegative integer, that the mean used in the calculation is positive, and that $\eta$ and $e^\eta$ are not accidentally interchanged. In numerical work, very large positive predictors can make $e^\eta$ overflow computationally; this is a numerical warning, not a change in the formula. A very negative predictor yields a mean near zero, and a zero observed count may then have a score close to zero without having a finite maximizing predictor in the one-observation unrestricted case.

Confusing score and log-likelihood. The score is $\partial\ell/\partial\eta$, a slope, not the log-likelihood value.
Confusing score and change. For a small step $h$, the first-order change is $U(\eta;y)h$, not $U$ evaluated at $h$. In the first example, the approximation is $U(\log 5;7)(0.1)=2(0.1)=0.2000$.
Reading the sign as a global guarantee. The sign gives the local direction at the stated predictor. For a finite move, calculate the exact log-likelihood difference or use the local approximation cautiously.
Treating zero score as proof of a good model. A zero score means the first derivative with respect to this predictor is zero. It does not establish that the Poisson assumptions fit the data or that an entire model is adequate.

The basic comparison is with the Bernoulli logit score from the earlier material on the Bernoulli logit score: both scores are derivatives with respect to a predictor, but they arise from different response distributions. Here the count is compared with a positive fitted mean, and the log-link score takes the particularly direct form $y-\mu$. A finite-difference calculation can approximate the derivative by comparing nearby log-likelihood values, but the derivative formula gives the exact local slope and its direction without choosing a step size.

Key takeaway: Under a Poisson model with log link, one observed count $y$ has log-likelihood $\ell(\eta;y)=-e^\eta+y\eta-\log(y!)$ and score $U(\eta;y)=y-e^\eta=y-\mu$. The score is a local slope; multiply it by a small predictor step to obtain a first-order approximation to the log-likelihood change.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

A hypothetical count is $y=3$, with $\eta=\log(2)$. Find the fitted mean and score. What local direction does the score indicate?
For $y=5$ and $\eta=\log(4)$, calculate the first-order approximation to the log-likelihood change for an upward step of $0.05$.
Why does a zero score at $y=6$ occur at $\eta=\log(6)$, and what does the second derivative establish there?
For $y=0$, explain why there is no finite predictor that maximizes the one-observation log-likelihood when $\eta$ is unrestricted.

Solutions. (1) The fitted mean is $e^{\log(2)}=2$, and $U=3-2=1$. The positive score indicates that a sufficiently small increase in $\eta$ raises the log-likelihood to first order. (2) The score is $5-4=1$, so the approximation is $Uh=1(0.05)=0.0500$. (3) At $\eta=\log(6)$, the mean is six, so $U=6-6=0$. The second derivative is $-e^\eta=-6<0$, establishing a strict local maximum; strict concavity makes it the unique maximum for this observation. (4) When $y=0$, $\ell(\eta;0)=-e^\eta$. This increases toward zero as $\eta$ tends to negative infinity, but no finite $\eta$ makes $e^\eta=0$, so the supremum is not attained at a finite predictor.