When Poisson Regression Estimates Are Finite

Why Finiteness Depends on the Observed Data

Poisson regression specifies a positive conditional mean for each covariate pattern, but that alone does not guarantee that the likelihood reaches its largest value at finite coefficients. Sometimes the data permit a coefficient to move farther and farther in one direction, improving the fit at each step. The likelihood can approach a supremum without attaining it.

To make this precise, consider independent responses \(Y_1,\ldots,Y_n\) with fixed design rows \(x_i^T\in\mathbb{R}^p\) and model \(Y_i\sim\operatorname{Poisson}(\mu_i)\), where \(\log\mu_i=x_i^T\beta\). The observed counts are \(y_i\), and \(\beta\in\mathbb{R}^p\) is the coefficient vector. The issue is not simply how many counts are zero or positive: it is how the outcomes line up with the covariate patterns.

Definition: A finite maximum-likelihood estimate exists if the likelihood has a maximizing coefficient vector \(\hat\beta\in\mathbb{R}^p\). This does not require the vector to be unique. A supremum approached only as one or more coefficients diverge is not a finite maximum-likelihood estimate.

The Poisson log-likelihood, with terms that do not depend on \(\beta\) retained for clarity, is

\ell(\beta)=\sum_{i=1}^n\left\{y_i x_i^T\beta-\exp(x_i^T\beta)-\log(y_i!)\right\}.

It is convenient to minimize the negative log-likelihood after dropping the same constant terms:

Q(\beta)=\sum_{i=1}^n\left\{\exp(x_i^T\beta)-y_i x_i^T\beta\right\}.

Maximizing \(\ell\) is equivalent to minimizing \(Q\). A direction \(d\) in coefficient space changes the linear predictor for observation \(i\) at rate \(x_i^T d\). The signs of these changes, together with whether each observed count is positive, control what happens along the path \(\beta+td\) as \(t\) increases.

The Finite-Estimate Criterion

Theorem: For the fixed-design Poisson regression model above, a finite maximum-likelihood estimate exists if and only if there is no vector \(d\in\mathbb{R}^p\) such that

\(x_i^T d=0\) for every observation with \(y_i>0\);
\(x_i^T d\leq 0\) for every observation with \(y_i=0\); and
\(x_i^T d<0\) for at least one observation with \(y_i=0\).

A vector satisfying these conditions gives a direction of strict likelihood improvement at every finite coefficient vector. Directions that leave every linear predictor unchanged are only non-identifiability; by themselves, they do not prevent a finite maximum.

The criterion separates two issues. A direction can lower some fitted means without changing the linear predictors for observations with positive counts. If it lowers at least one fitted mean associated with a zero count, the likelihood improves as that mean moves toward zero. In contrast, if every linear predictor is unchanged, the likelihood is unchanged too: different coefficient vectors represent the same fit.

Why the Criterion Is Necessary and Sufficient

We prove the theorem by examining the negative log-likelihood along every coefficient direction. First, restrict attention to the identifiable subspace \(V=(\ker X)^\perp\), where \(X\) is the design matrix with rows \(x_i^T\). Every coefficient vector has a representative in \(V\) with the same linear predictors, and \(X d=0\) for \(d\in V\) implies \(d=0\). Thus minimizing over \(V\) is enough to decide whether a finite maximizing fit exists.

For a direction \(d\), write \(a_i=x_i^T d\). The change in the \(i\)th summand of \(Q\), divided by \(t\), is

\frac{\exp(x_i^T\beta+t a_i)-y_i(x_i^T\beta+t a_i) -\{\exp(x_i^T\beta)-y_i x_i^T\beta\}}{t}.

If \(a_i>0\), this expression tends to \(+\infty\), since the exponential term grows faster than the linear term. If \(a_i=0\), it is zero. If \(a_i<0\), the exponential term tends to zero and the expression tends to \(-y_i a_i\), which is nonnegative and is strictly positive exactly when \(y_i>0\). Summing the finitely many terms shows that the limiting slope of \(Q(\beta+td)\) is zero exactly when

a_i\leq 0\text{ for all }i,\qquad a_i=0\text{ whenever }y_i>0.

If these conditions hold and \(d\in V\) is nonzero, at least one \(a_i\) must be nonzero. It must therefore be negative for an observation with \(y_i=0\). Conversely, a direction meeting the theorem’s three conditions has limiting slope zero.

We now justify why the absence of such a direction ensures a minimum is attained. The following elementary fact about continuous convex functions on finite-dimensional spaces is useful: if a continuous convex function \(Q\) has strictly positive limiting slope in every nonzero direction, then it attains its minimum. To see this, suppose the sublevel set \(\{b:Q(b)\leq Q(0)\}\) were unbounded. Choose \(b_k\) in it with \(r_k=\|b_k\|\to\infty\). A subsequence of \(u_k=b_k/r_k\) converges to a unit vector \(u\). For any fixed \(s>0\) and all sufficiently large \(k\), convexity gives

Q(su_k)\leq \left(1-\frac{s}{r_k}\right)Q(0)+\frac{s}{r_k}Q(b_k) \leq Q(0).

Continuity gives \(Q(su)\leq Q(0)\). Consequently, the limiting slope along \(u\) cannot be positive, a contradiction. The sublevel set is therefore bounded; it is also closed and nonempty. In finite dimensions it is compact, so the continuous function \(Q\) attains a minimum on it.

For the Poisson objective, the direction calculation shows that the limiting slope is nonnegative, and it is strictly positive in every nonzero direction in \(V\) precisely when no direction in the theorem exists. The convex-function argument then gives a minimizer in \(V\), hence a finite maximizing coefficient vector. In the other case, along a direction satisfying the three conditions, positive-count contributions stay unchanged and zero-count contributions with \(x_i^T d<0\) strictly decrease in \(Q\) for every \(t>0\). No finite coefficient vector can minimize \(Q\), because it can always be improved by moving farther along that direction. This proves both implications. \(\square\)

Worked Examples

Worked Example: A Finite Intercept-Only Estimate

Suppose three independent counts are \(y_1=1\), \(y_2=2\), and \(y_3=0\), with an intercept-only model. Each design row is \(x_i^T=1\), so \(\mu_i=\exp(\beta_0)\).

For a direction \(d\in\mathbb{R}\), every linear predictor changes at rate \(d\). Since \(y_1\) and \(y_2\) are positive, the criterion would require \(d=0\). There is therefore no direction satisfying its conditions with a strict inequality, and a finite estimate exists.

Direct calculation confirms the result. The log-likelihood, apart from constants, is \(3\beta_0-3\exp(\beta_0)\). Its derivative is \(3-3\exp(\beta_0)\), which is zero at \(\hat\beta_0=0\). Its second derivative is \(-3\exp(\beta_0)<0\), so this is the unique maximum. The fitted mean is \(\exp(0)=1\), equal to the observed average count \((1+2+0)/3=1\).

Worked Example: Two Covariate Patterns with a Finite Fit

Consider a binary predictor with two observations at each level. At level \(0\), the counts are \(1\) and \(2\); at level \(1\), they are \(0\) and \(3\). Use \(\log\mu_i=\beta_0+\beta_1 x_i\). The two distinct design rows are \((1,0)\) and \((1,1)\).

Both patterns have at least one positive count. Any direction satisfying the theorem’s requirements must have \(d_0=0\) for the positive count at level \(0\), and \(d_0+d_1=0\) for the positive count at level \(1\). These equations imply \(d_0=d_1=0\). No strict improving direction exists.

For a direct check, let \(\lambda_0=\exp(\beta_0)\) and \(\lambda_1=\exp(\beta_0+\beta_1)\). The level-\(0\) log-likelihood terms are \(3\log\lambda_0-2\lambda_0\), up to constants, and the level-\(1\) terms are \(3\log\lambda_1-2\lambda_1\). Each is maximized at its observed mean: \(\hat\lambda_0=3/2\) and \(\hat\lambda_1=3/2\). Thus \(\hat\beta_0=\log(3/2)\) and \(\hat\beta_1=0\), both finite.

Worked Example: A Likelihood Supremum That Is Not Attained

Now observe \(y=2\) at predictor value \(x=0\) and \(y=0\) at \(x=1\), again with \(\log\mu=\beta_0+\beta_1x\). The direction \(d=(0,-1)^T\) leaves the first linear predictor unchanged and decreases the second by one unit per step. It meets all three conditions in the theorem.

The likelihood contribution from the first observation, omitting its constant, is \(2\beta_0-\exp(\beta_0)\). The second contribution is \(-\exp(\beta_0+\beta_1)\). For any fixed \(\beta_0\), decreasing \(\beta_1\) strictly increases the likelihood. As \(\beta_1\to-\infty\), the second contribution tends to zero, but it is strictly negative at every finite \(\beta_1\). The first contribution is maximized at \(\beta_0=\log 2\), yet for every finite pair \((\beta_0,\beta_1)\) the second contribution can still be improved. The likelihood has a supremum, but no finite maximum-likelihood estimate.

Worked Example: Nonunique but Finite Coefficients

Suppose the model includes an intercept and two identical predictor columns, so every design row has the form \((1,z_i,z_i)\). Let two observations have \(z_1=0,z_2=1\) and counts \(y_1=1,y_2=2\). The linear predictors are \(\beta_0\) and \(\beta_0+\beta_1+\beta_2\).

The two distinct patterns each have a positive count. The theorem’s conditions force \(d_0=0\) and \(d_0+d_1+d_2=0\), so there is no direction that strictly improves the likelihood. The model can fit the two means as \(1\) and \(2\); for example, \(\beta_0=0,\beta_1=\log 2,\beta_2=0\) gives those fitted means. But \(\beta_1\) and \(\beta_2\) are not separately determined: any pair with \(\beta_1+\beta_2=\log 2\) gives the same likelihood. This is non-uniqueness, not failure of a finite maximum.

Common Mistake and Practical Importance

Common mistake: “A zero count means the Poisson regression estimate must be infinite.” This conclusion ignores the covariate pattern. In the two-pattern finite example, one pattern includes a zero but also includes a positive count, so its mean cannot be driven toward zero while leaving the positive-count observation’s linear predictor unchanged. The correct question is whether there is a direction that leaves every positive-count predictor unchanged and decreases at least one zero-count predictor without increasing any others.

This check matters before interpreting coefficients or using model-based uncertainty calculations. A finite fitted model needs a maximum, while a diverging coefficient direction means the data support increasingly extreme fitted means along that path. A separate rank check is also useful: redundant predictors can make coefficients nonunique even when the likelihood does attain its maximum.

Key takeaway: Finiteness is determined jointly by the design and observed counts. Look for a coefficient direction that holds all positive-count linear predictors fixed and moves at least one zero-count predictor downward, with none moving upward. If such a direction exists, no finite maximum is attained; if not, a finite maximum exists, though its coefficients may not be unique.

Check Your Understanding

Use the direction criterion and, where helpful, the negative log-likelihood to answer these questions.

In an intercept-only model with counts \(0,0,2\), does a finite maximum-likelihood estimate exist? Identify the relevant direction or explain why none exists.
For a model with an intercept and a binary predictor, suppose the counts at predictor level \(0\) are \(0,2\), and the counts at level \(1\) are \(0,1\). Does the criterion rule out a finite estimate?
Explain why a direction that leaves every design-row linear predictor unchanged can create nonunique coefficients without preventing a finite maximum.
What must be true of the linear predictor changes for observations with positive counts along a direction that strictly improves the likelihood?
Why does a zero count at one covariate pattern not, by itself, determine whether the maximum-likelihood estimate is finite?