Zero and One Responses Without Invalid Arithmetic

Why Zero and Full Counts Need Care

A binomial response can be zero, equal the number of trials, or lie between those values. These are ordinary outcomes, not exceptional data. Yet formulas written in the wrong form can produce undefined expressions: for example, a score written with a response divided by a fitted probability can contain \(0/0\) when both are zero. A log-likelihood written as a sum of terms such as \(y\log(y/m)\) can also appear to contain \(0\log 0\).

The remedy is to use the likelihood in a form that respects its underlying probability model, and to distinguish a zero coefficient multiplying a log probability from a probability that is itself zero. The former has a well-defined limiting contribution of zero. The latter can make the likelihood zero and the log-likelihood negative infinity.

For a logistic GLM, observation \(i\) has \(m_i\) trials, count \(y_i\), linear predictor \(\eta_i=\mathbf{x}_i^\top\boldsymbol{\beta}\), and probability \(p_i=1/(1+\exp(-\eta_i))\). At every finite \(\eta_i\), \(0<p_i<1\), even when \(y_i=0\) or \(y_i=m_i\). The issue is therefore not that the data values are invalid; it is choosing expressions that do not manufacture undefined arithmetic.

Log-Likelihood Forms That Respect the Data

For an interior probability, the binomial log-likelihood is already safe if its two coefficients are handled as coefficients: when \(y=0\), omit the \(y\log p\) term; when \(y=m\), omit the \((m-y)\log(1-p)\) term. For logistic regression, there is also a useful form in terms of the softplus function, defined by \(\operatorname{softplus}(t)=\log(1+\exp(t))\):

This identity follows from \(\log p_i=-\operatorname{softplus}(-\eta_i)\) and \(\log(1-p_i)=-\operatorname{softplus}(\eta_i)\). Its response-weighted terms make the endpoint counts explicit: for \(y_i=0\), the first term after the binomial coefficient is absent; for \(y_i=m_i\), the second is absent. In computation, an absent term should be omitted rather than evaluated as zero times a possibly infinite quantity.

The score can also be written without subtracting two quantities tied to a success probability: \(y_i(1-p_i)-(m_i-y_i)p_i\). These expressions are algebraically equal because \(y_i(1-p_i)-(m_i-y_i)p_i=y_i-m_ip_i\). The first makes the endpoint cases especially clear: the score is \(-m_ip_i\) for zero successes and \(m_i(1-p_i)\) for all successes. Neither case requires dividing by \(p_i\) or \(1-p_i\).

Deriving the Score Without Zero-over-zero Terms

It is tempting to differentiate the log-likelihood with respect to \(p\) and then substitute endpoint counts. That approach is valid for \(0<p<1\), but unsimplified expressions can be needlessly fragile. Differentiating with respect to the logistic predictor gives a direct derivation that is valid for every count.

The division in the first line is only by \(p\) and \(1-p\), both strictly positive at finite \(\eta\); it is not division by \(y\) or \(m-y\). The final equality follows by expanding the preceding line. It remains valid when \(y=0\) or \(y=m\), giving respectively \(-mp\) and \(m(1-p)\). For \(\eta=\mathbf{x}^\top\boldsymbol{\beta}\), the chain rule multiplies this scalar derivative by \(\mathbf{x}\), so the coefficient-score contribution is \(\mathbf{x}(y-mp)\). \(\square\)

What the Limiting Convention Does—and Does Not—Say

The rule \(x\log x\to0\) as \(x\) decreases to zero explains why zero-count contributions in deviance-style expressions have finite limiting values. It does not say that every expression involving a zero and a logarithm is zero. In particular, at \(p=0\), a binomial outcome with \(y>0\) has probability zero and log-likelihood \(-\infty\); at \(p=1\), an outcome with \(y<m\) has probability zero and log-likelihood \(-\infty\).

For a finite logistic predictor, the fitted probability is strictly between zero and one in exact arithmetic, so the likelihood and score formulas above are interior formulas. If a computation rounds a fitted probability to exactly zero or one, that is not evidence that the mathematical predictor has reached a boundary. Evaluate log probabilities from a stable predictor-based expression, and compute endpoint scores from \(-mp\) or \(m(1-p)\), using the small tail probability directly when possible.

Common Mistakes and Why These Cases Matter

• Evaluating \(0\log 0\) as an ordinary arithmetic operation. The limiting contribution is zero, but a programming language may return a non-finite value for the intermediate logarithm. Omit the zero-coefficient term or use the response-aware formula.
• Using a score formula that divides by the observed count. Expressions such as \(y/p\) are safe at finite logistic predictors, but dividing by \(y\) is not safe for an all-failure group. Simplify the score to \(y-mp\) before handling endpoint counts.
• Clipping a rounded probability and treating it as the exact model value. A rounded \(p=1\) can make \(\log(1-p)\) appear undefined even when the finite predictor implies a positive failure probability. Compute log probabilities from the predictor instead.
• Extending an interior score formula blindly to an exact boundary. The score with respect to \(\eta\) at finite predictors has a well-defined limit in some endpoint cases, but \(\eta=\pm\infty\) is not a finite model parameter. A boundary likelihood value is not automatically an ordinary finite-parameter score.

An all-success group has log-likelihood increasing toward its supremum as its predictor increases; an all-failure group has the corresponding tendency as its predictor decreases. This matters in fitting because a software routine may move toward increasingly extreme predictors. The finite-predictor likelihood and score must still be evaluated correctly throughout that process; endpoint-safe arithmetic does not by itself guarantee a finite maximizing coefficient.