Why the Poisson Likelihood Matters
A Poisson regression model describes counts whose expected values depend on observed predictors. Estimation begins with a probability model for each count, combines those probabilities into a likelihood, and differentiates the log-likelihood with respect to the regression coefficients. The resulting score equations identify parameter values at which the likelihood has zero slope in every coefficient direction.
A zero score does not mean that every fitted count equals its observed count. Instead, the equations impose particular aggregate relationships between observed counts and fitted means. Understanding those relationships helps distinguish a valid maximum-likelihood calculation from a tempting but incorrect attempt to fit every observation exactly.
The model assumes that, conditional on the predictor values, each \(Y_i\) follows a Poisson distribution with its specified mean, and that the observations are independent. The predictor values are treated as fixed in the likelihood. Independence is what allows the joint probability to factor into the product of the individual Poisson probabilities. If counts are dependent, or the conditional distribution is not Poisson, this likelihood and its score equations need not be appropriate.
From the Poisson Probability to the Score
For a nonnegative integer \(y_i\), the Poisson probability mass function with mean \(\mu_i>0\) is \(P(Y_i=y_i)=e^{-\mu_i}\mu_i^{y_i}/y_i!\). Multiplying these probabilities across independent observations gives the likelihood. Taking logarithms turns the product into a sum, which is easier to differentiate.
The factorial term depends on the observed data, not on the coefficients. It may be omitted when maximizing the likelihood, but retaining it gives the full log-likelihood. To obtain the score, differentiate each observation's contribution. The chain rule gives \((y_i/\mu_i-1)\,\partial\mu_i/\partial\beta_j\): the derivative of \(y_i\log\mu_i\) is \(y_i(\partial\mu_i/\partial\beta_j)/\mu_i\), while the derivative of \(-\mu_i\) is \(-\partial\mu_i/\partial\beta_j\).
An interior maximum-likelihood estimate \(\widehat{\boldsymbol{\beta}}\) must satisfy \(U_j(\widehat{\boldsymbol{\beta}})=0\) for every coefficient \(j\). These are necessary first-order conditions, not by themselves proof that a solution is a maximum. A candidate must also be in the parameter space and give a maximum rather than a minimum or saddle point. Some data sets have no finite estimate, as the all-zero example below illustrates.
Regression Scores and Fitted-Count Constraints
A useful special case specifies the means as \(\mu_i=\exp(\mathbf{x}_i^\mathsf{T}\boldsymbol{\beta})\), where \(\mathbf{x}_i\) is the predictor vector for observation \(i\). This makes \(\partial\mu_i/\partial\beta_j=\mu_i x_{ij}\), with \(x_{ij}\) the \(j\)-th predictor entry. Substitution into the general score gives a particularly simple equation for each coefficient.
Thus, at an interior estimate, the fitted-minus-observed residuals satisfy \(\mathbf{X}^{\mathsf T}(\mathbf{y}-\widehat{\boldsymbol{\mu}})=\mathbf{0}\), or equivalently \(\mathbf{X}^{\mathsf T}(\mathbf{y}-\widehat{\boldsymbol{\mu}})=\mathbf{0}\) with residuals written as observed minus fitted. The sign convention changes if residuals are defined in the opposite order; the zero condition does not. More directly, each predictor column has a zero weighted sum of observed-minus-fitted counts.
If the model includes an intercept, its predictor column consists of ones. The intercept score therefore implies \(\sum_i y_i=\sum_i\widehat{\mu}_i\): total observed counts equal total fitted counts at an interior solution. With additional predictor columns, further weighted residual sums are zero. These constraints do not generally imply \(y_i=\widehat{\mu}_i\) for every observation. Exact agreement observation by observation occurs only in a sufficiently flexible model that can fit the data that way.
The score formula also shows why a score equation depends on how each mean changes with the coefficient. For a general mean specification, the factor \(\partial\mu_i/\partial\beta_j\) remains in the equation. It is only for the exponential specification above that this derivative simplifies to \(x_{ij}\mu_i\), producing the residual form \(\sum_i x_{ij}(y_i-\mu_i)\).
Worked Example: Estimating a Common Poisson Mean
A hypothetical monitoring system records event counts \(2,3,4,3\) on four independent, equal-duration intervals. Assume each count has the same Poisson mean \(\mu>0\). Find the maximum-likelihood estimate, write the score equation, and check the solution.
Method and likelihood. This is an intercept-only count model with a common mean. The likelihood is \(L(\mu)=\prod_i e^{-\mu}\mu^{y_i}/y_i!\), so the log-likelihood is \(\ell(\mu)=12\log\mu-4\mu-\sum_i\log(y_i!)\), because the total count is \(2+3+4+3=12\).
Score and solution. Differentiating gives \(U(\mu)=12/\mu-4\). Setting the score to zero yields \(12/\mu=4\), so \(\widehat{\mu}=3\). The second derivative is \(-12/\mu^2\), which is negative for every \(\mu>0\), confirming that this stationary point is a maximum.
Likelihood check. At \(\widehat{\mu}=3\), \(\ell(3)=12\log 3-12-[\log(2!)+\log(3!)+\log(4!)+\log(3!)]\). Using \(\log 3\approx1.098612\), the first two terms give \(13.183344-12=1.183344\). The factorial-log sum is approximately \(0.693147+1.791759+3.178054+1.791759=7.454719\). Thus \(\ell(3)\approx-6.271375\). The score solution also matches the sample mean, \(12/4=3\). The log-likelihood is rounded to four decimal places only at the end: \(-6.2714\).
Common mistake avoided. Setting each fitted mean equal to its own observed count would demand four different values of one common parameter. The model has only one mean, and its score instead equates the total observed count with the total fitted count.
Worked Example: Two Groups and Two Regression Coefficients
A hypothetical study records independent counts for three units in each of two settings. The counts are \(2,3,1\) in setting \(x=0\) and \(3,6,3\) in setting \(x=1\). Assume \(\mu_i=\exp(\beta_0+\beta_1x_i)\). Find the maximum-likelihood coefficients and verify both score equations.
Method and group means. The observed totals are \(6\) for the first setting and \(12\) for the second, with three observations in each. Write \(\mu_0=e^{\beta_0}\) and \(\mu_1=e^{\beta_0+\beta_1}\). The coefficient score equations can be written as \(U_0=(6-3\mu_0)+(12-3\mu_1)\) and \(U_1=12-3\mu_1\), since the predictor is 0 for the first group and 1 for the second.
Solve the equations. Setting \(U_1=0\) gives \(12-3\mu_1=0\), hence \(\mu_1=4\). Then \(U_0=0\) gives \(6-3\mu_0+12-3(4)=0\), or \(6-3\mu_0=0\), hence \(\mu_0=2\). Therefore \(\beta_0=\log 2\) and \(\beta_1=\log 4-\log 2=\log 2\). Rounded to four decimals at the final step, \(\widehat{\beta}_0=0.6931\) and \(\widehat{\beta}_1=0.6931\).
Check. Substitution gives \(U_1=12-3(4)=0\) and \(U_0=(6-3(2))+(12-3(4))=0+0=0\). Each fitted mean equals the corresponding group's observed average here because the two-parameter model assigns a separate mean to each of the two groups. It does not fit each individual count exactly.
Common mistake avoided. The intercept score combines both groups; it is not just the first group's total-minus-fitted total. The slope score isolates the second group through its predictor value of 1.
Worked Example: A One-Coefficient Mean Model
For a third hypothetical count model, suppose independent observations have means \(\mu_i=\beta x_i\), where \(\beta>0\). The predictor values are \(x=(1,2,4)\), and the observed counts are \(y=(1,2,5)\). Estimate \(\beta\) by maximum likelihood and verify the result.
Method and score. Here \(\partial\mu_i/\partial\beta=x_i\). The general score is \(U(\beta)=\sum_i(y_i/\mu_i-1)x_i\). Since \(\mu_i=\beta x_i\), this simplifies to \(U(\beta)=\sum_i(y_i/\beta-x_i)=(1+2+5)/\beta-(1+2+4)=8/\beta-7\).
Solve and check. The score equation \(8/\beta-7=0\) gives \(\widehat{\beta}=8/7\). The fitted means are \(8/7\), \(16/7\), and \(32/7\), with total \(56/7=8\), matching the observed total \(1+2+5=8\). The second derivative of the log-likelihood is \(-8/\beta^2<0\), so the score root is a maximum. Rounded to four decimals only at the final step, \(\widehat{\beta}=1.1429\), and the fitted means are approximately \(1.1429,2.2857,4.5714\).
Common mistake avoided. The model does not have a separate free mean for each observation. Its fitted means must retain the specified proportions \(1:2:4\); they cannot be set independently to \(1,2,5\).
Interpreting and Checking Score Equations
For the exponential mean specification, the log-likelihood is concave in the coefficients when the design matrix is fixed. Its Hessian is \(-\mathbf{X}^{\mathsf T}\operatorname{diag}(\mu_1,\ldots,\mu_n)\mathbf{X}\), which is negative semidefinite because for any vector \(\mathbf{a}\), \(\mathbf{a}^{\mathsf T}\mathbf{X}^{\mathsf T}\operatorname{diag}(\boldsymbol{\mu})\mathbf{X}\mathbf{a} =\sum_i\mu_i(\mathbf{x}_i^\mathsf{T}\mathbf{a})^2\geq0\). If the design matrix has full column rank, the Hessian is negative definite, giving strict concavity and at most one finite score root. A root that exists is then the unique maximum. If the design matrix lacks full rank, coefficients may not be separately identifiable.
A score equation can also fail to have a finite solution. For example, suppose there are \(n\) independent observations in a common-mean model and all observed counts are zero. Then \(\ell(\mu)=-n\mu\), apart from data constants, and \(U(\mu)=-n\), which is never zero for \(\mu>0\). The likelihood increases as \(\mu\) approaches zero, so its supremum occurs at the excluded boundary \(\mu=0\); there is no finite interior maximum-likelihood estimate. This is why checking the parameter space and the likelihood behavior matters, rather than solving formal equations mechanically.
Check Your Understanding
For each problem, use the stated Poisson mean model and distinguish an individual fitted mean from a score-imposed total or weighted total.
- Four independent counts are \(0,2,4,6\), with one common Poisson mean \(\mu>0\). Find the maximum-likelihood estimate. (Answer: \(\widehat{\mu}=3\).)
- Two groups each contain three independent observations. The first group's counts are \(2,3,1\); the second group's counts are \(3,6,3\). Under \(\mu_i=\exp(\beta_0+\beta_1x_i)\), with \(x=0\) and \(x=1\) for the respective groups, give the fitted group means and coefficients. (Answer: \(\widehat{\mu}_0=2\), \(\widehat{\mu}_1=4\), \(\widehat{\beta}_0=\log 2\), and \(\widehat{\beta}_1=\log 2\).)
- Suppose \(\mu_i=\beta x_i\), \(\beta>0\), with \(x=(1,3,2)\) and observed counts \(y=(0,4,2)\). Find the score root and state whether the fitted means sum to the observed total. (Answer: \(\widehat{\beta}=1\); fitted means are \(1,3,2\), whose total \(6\) equals the observed total \(6\).)
- For \(\mu_i=\exp(\beta_0+\beta_1x_i)\), let \(x=(0,1,2)\), \(y=(1,1,5)\), and evaluate both score components at \(\beta_0=0,\ \beta_1=\log 2\). (Answer: \((U_0,U_1)=(0,1)\).)
- In a common-mean Poisson model, five independent observations are all zero. Does the score equation give a finite maximum-likelihood estimate in the parameter space \(\mu>0\)? (Answer: No; \(U(\mu)=-5\) for all \(\mu>0\), and the likelihood supremum is approached as \(\mu\) tends to zero.)