Zero Counts In The Poisson Likelihood

Why Zero Counts Deserve Careful Attention

A count of zero is a familiar outcome in clinical and other count data: no events occurred during an observation period, or no cases were recorded in a group. In a Poisson regression, zero is an ordinary possible response, but its likelihood contribution can look unfamiliar because it contains no \(y\log(\mu)\) term. Making that contribution explicit shows how zeros affect estimation and why they require no separate modeling rule.

Assume the responses \(Y_1,\ldots,Y_n\) are conditionally independent given fixed covariate vectors \(x_i\), with \(Y_i\) having a Poisson distribution with mean \(\mu_i\). In the log-link regression model, \(\mu_i=\exp(x_i^\mathsf{T}\beta)\), where \(\beta\) is the coefficient vector. The mean is positive for every finite coefficient vector.

Definition: For an observed response \(y_i\), the Poisson likelihood contribution is \(f(y_i;\mu_i)=\exp(-\mu_i)\mu_i^{y_i}/y_i!\), for \(y_i=0,1,2,\ldots\). Its log-likelihood contribution is \(\ell_i(\beta)=y_i\log(\mu_i)-\mu_i-\log(y_i!)\), with \(\mu_i=\exp(x_i^\mathsf{T}\beta)\). At \(y_i=0\), the convention \(0\log(\mu_i)=0\) gives \(\ell_i(\beta)=-\mu_i\).

Deriving the Zero-Count Contribution

For a zero response, substitute \(y_i=0\) into the Poisson probability mass function. Since \(0!=1\) and \(\mu_i^0=1\), the probability of observing zero is \(\exp(-\mu_i)\). Taking its logarithm gives the zero-count log-likelihood contribution.

P(Y_i=0\mid x_i)=\frac{\exp(-\mu_i)\mu_i^0}{0!} =\exp(-\mu_i), \qquad \ell_i(\beta)=\log P(Y_i=0\mid x_i)=-\mu_i =-\exp(x_i^\mathsf{T}\beta).

This term is finite for every finite \(\beta\). In particular, there is no need to take the logarithm of the observed response: the logarithm in the Poisson log-likelihood is \(\log(\mu_i)\), not \(\log(y_i)\). The product \(y_i\log(\mu_i)\) is zero when \(y_i=0\), while \(\log(\mu_i)\) itself is well-defined because \(\mu_i>0\).

Worked Example: One Zero Response at a Specified Mean

Suppose an observation has \(Y_i=0\) and fitted mean \(\mu_i=2.5\). Its likelihood contribution is \(P(Y_i=0\mid x_i)=\exp(-2.5)\approx0.0821\). Its log-likelihood contribution is therefore \(\log(0.0821)\approx-2.5\), exactly \(-\mu_i\). The factorial term adds nothing because \(\log(0!)=\log(1)=0\).

Now suppose this observation has a scalar covariate value \(x_i=2\), and the linear predictor is \(x_i\beta\) with \(\beta=0.3\). Then \(\mu_i=\exp(2(0.3))=\exp(0.6)\approx1.8221\), so the log-likelihood contribution is approximately \(-1.8221\). Its derivative with respect to \(\beta\), derived below, is \(-x_i\mu_i\approx-3.6442\). Thus, this zero count contributes both a finite likelihood term and a nonzero score.

The Score and Curvature from a Zero

The response-specific score is the derivative of \(\ell_i(\beta)\) with respect to the coefficient vector. Let \(\eta_i=x_i^\mathsf{T}\beta\), so that \(\mu_i=\exp(\eta_i)\) and \(\partial\mu_i/\partial\beta=\mu_i x_i\). Differentiating the general Poisson contribution first makes the zero case immediate.

Result: For a fixed covariate vector \(x_i\) and a finite coefficient vector \(\beta\), the score contribution is \(\partial\ell_i(\beta)/\partial\beta=x_i(y_i-\mu_i)\). Its Hessian is \(\partial^2\ell_i(\beta)/\partial\beta\partial\beta^\mathsf{T} =-\mu_i x_i x_i^\mathsf{T}\). When \(y_i=0\), these become \(-\mu_i x_i\) and \(-\mu_i x_i x_i^\mathsf{T}\), respectively.

To verify the result, write the response-specific log likelihood in terms of \(\eta_i\): \(\ell_i=y_i\eta_i-\exp(\eta_i)-\log(y_i!)\). Its derivative with respect to \(\eta_i\) is \(y_i-\exp(\eta_i)=y_i-\mu_i\). Since \(\partial\eta_i/\partial\beta=x_i\), the chain rule gives the stated score. Differentiating that score once more gives the stated Hessian. For a zero response, the score can also be obtained directly from \(\ell_i=-\exp(x_i^\mathsf{T}\beta)\): its gradient is \(-\exp(x_i^\mathsf{T}\beta)x_i=-\mu_i x_i\), as claimed. \(\square\)

The score points in a direction determined by \(x_i\) and the fitted mean. A zero response contributes a negative multiple of \(x_i\); it therefore tends to pull the linear predictor downward along the covariate direction represented by that observation. This does not mean that every coefficient must decrease: the effect on each coefficient depends on the full design and on how the score equations balance across observations.

How Zeros Enter the Regression Score Equations

Because the observations are conditionally independent, their log-likelihood contributions add. Summing the response-specific scores yields the full score. The next derivation states precisely how zeros appear in those equations.

Result: Let \(X\) be the \(n\)-by-\(p\) design matrix whose \(i\)th row is \(x_i^\mathsf{T}\), let \(y=(y_1,\ldots,y_n)^\mathsf{T}\), and let \(\mu=(\mu_1,\ldots,\mu_n)^\mathsf{T}\). The Poisson regression score is \(U(\beta)=X^\mathsf{T}(y-\mu)\). At any finite maximum-likelihood estimate \(\hat\beta\), it satisfies \(X^\mathsf{T}y=X^\mathsf{T}\hat\mu\), where \(\hat\mu_i=\exp(x_i^\mathsf{T}\hat\beta)\). A zero observation contributes zero to the observed-response side \(X^\mathsf{T}y\), but contributes its fitted mean times its covariate to the fitted side.

For the derivation, sum the individual scores:

U(\beta)=\sum_{i=1}^n x_i(y_i-\mu_i) =\sum_{i=1}^n x_i y_i-\sum_{i=1}^n x_i\mu_i =X^\mathsf{T}y-X^\mathsf{T}\mu.

If a finite maximum occurs at an interior point of the coefficient space, differentiability implies that its score is zero. Setting the displayed expression to zero gives \(X^\mathsf{T}y=X^\mathsf{T}\hat\mu\). This condition applies to all observations, including those with \(y_i=0\): their individual score terms are \(-x_i\hat\mu_i\), not zero. The result is conditional on a finite maximum; it does not assert that such a maximum exists for every data set. \(\square\)

When the model includes an intercept, one component of \(x_i\) is \(1\) for every observation. The corresponding score equation is the balance between the total observed count and the total fitted count. This is one useful implication of the full score equations, not a rule that zeros should be omitted.

Worked Example: Zeros in an Intercept-Only Model

Consider the observations \(0,0,2,4\) in an intercept-only Poisson model. The common mean is \(\mu=\exp(\beta_0)\). The log likelihood is \(\ell(\beta_0)=\sum_i[y_i\log(\mu)-\mu-\log(y_i!)]\). Here \(\sum_i y_i=6\), and \(\sum_i\log(y_i!)=\log(2!)+\log(4!)=\log(48)\), so \(\ell(\beta_0)=6\log(\mu)-4\mu-\log(48)\).

Differentiating with respect to \(\beta_0\), using \(d\mu/d\beta_0=\mu\), gives \(U(\beta_0)=6-4\mu\). Setting this equal to zero gives \(\hat\mu=6/4=1.5\), and hence \(\hat\beta_0=\log(1.5)\). The observed total is six, and the fitted total is \(4(1.5)=6\), as required by the intercept score equation.

At this estimate, each zero contributes \(-1.5\) to the log likelihood, so together the two zeros contribute \(-3\). If the zeros were incorrectly deleted, the remaining observations \(2,4\) would give an intercept-only estimate of \((2+4)/2=3\), not \(1.5\). Deleting zeros changes both the likelihood being fitted and the estimated mean.

Zeros Across Covariate Profiles

With predictors, a zero’s contribution is weighted by its covariates in the score equations. For a binary predictor, the intercept equation balances counts and fitted means across both groups, while the predictor equation balances them within the group coded \(1\). This makes the role of zero observations visible without introducing a separate zero-count formula.

Worked Example: A Binary Predictor with Zeros in Both Groups

Suppose three observations have predictor \(z=0\), with counts \(0,1,2\), and three have predictor \(z=1\), with counts \(0,2,4\). Use the model \(\log(\mu_i)=\beta_0+\beta_1z_i\). In each group the fitted mean is constant. The observed group means are \(3/3=1\) for \(z=0\) and \(6/3=2\) for \(z=1\), so the fitted values are obtained at \(\hat\beta_0=\log(1)=0\) and \(\hat\beta_1=\log(2)-\log(1)=\log(2)\).

Check the score equations directly. In the \(z=0\) group, the fitted mean is \(1\), so the sum of residuals is \((0-1)+(1-1)+(2-1)=-1+0+1=0\). In the \(z=1\) group, the fitted mean is \(2\), giving \((0-2)+(2-2)+(4-2)=-2+0+2=0\). The intercept score is the sum of both group residual totals, \(0+0=0\). The \(\beta_1\) score is the residual total for \(z=1\), also zero.

The zero in the \(z=0\) group contributes \(-1\) to that group’s residual total; the zero in the \(z=1\) group contributes \(-2\). The other observations in each group balance those negative terms. The zeros are not ignored or assigned an exceptional score: each contributes \(x_i(0-\hat\mu_i)\), exactly as the general score formula specifies.

Common Mistakes and Practical Interpretation

A zero may be visually striking in a data set, but in an ordinary Poisson regression its likelihood treatment is direct. The model assigns probability \(\exp(-\mu_i)\) to zero, and the resulting score and curvature follow from the same likelihood used for positive counts.

Taking \(\log(y_i)\) at zero. The Poisson log likelihood contains \(y_i\log(\mu_i)\), not \(\log(y_i)\). At \(y_i=0\), that product is zero.
Assuming a zero has no score contribution. Although it contributes zero to the observed count total, its score is \(-x_i\mu_i\), which is generally nonzero.
Deleting zeros before fitting. This changes the sample and the likelihood. In an intercept-only model, it can change the fitted mean from the full-sample average to the average among positive observations.
Claiming every coefficient must decrease when a zero is added. The zero contributes in the direction \(-x_i\mu_i\), but the fitted coefficients solve a system of score equations; their changes depend on all covariates and observations.

The central practical point is that zeros inform the fitted means. For a given covariate profile, a zero makes a negative contribution to the score, while observed positive counts may offset it. The fitted regression balances all these contributions at once. This explains why routine Poisson regression includes zero responses directly, without a special zero-specific adjustment to its likelihood formula.

Check Your Understanding

Use the Poisson log-link model and score formulas developed above.

For a response \(y_i=0\) with fitted mean \(\mu_i\), write its likelihood and log-likelihood contributions.
For a zero response with covariate vector \(x_i\), derive its score contribution with respect to \(\beta\).
In an intercept-only model with \(n\) observations, explain the score equation connecting the observed and fitted total counts.
Why does deleting zero responses generally change the fitted intercept-only mean?
In a regression with a binary predictor, which observations contribute to the score for the binary-predictor coefficient?