Why Log-Link Computation Needs Care
With a log link, the mean is defined by \(\log(\mu_i)=\eta_i\), so \(\mu_i=\exp(\eta_i)\). This is mathematically straightforward, but computer arithmetic has a finite range. A predictor that is finite as a floating-point number can still be too large for its exponential to be represented, or so negative that the computed exponential rounds to zero. The model remains well-defined in both cases; the numerical calculation may not.
The practical distinction is important: the predictor \(\eta_i\) is already the log mean. Quantities such as \(\log(\mu_i)\) can therefore be evaluated from \(\eta_i\) directly, without calculating \(\mu_i\) and then taking its logarithm. For likelihood calculations, this avoids avoidable overflow, underflow, and invalid operations.
Representable Exponentials
A floating-point format represents only a finite set of numbers. In a typical IEEE 754 binary64 calculation, the largest finite positive number is about \(1.798\times10^{308}\). The smallest positive subnormal number is about \(4.94\times10^{-324}\); positive subnormal values are representable below the smallest positive normal value, but with reduced precision. Consequently, \(\exp(\eta)\) can overflow for a sufficiently large \(\eta\), and can round to zero for a sufficiently negative \(\eta\).
For orientation, the logarithm of the largest finite binary64 number is about \(709.78\), while the logarithm of the smallest positive subnormal number is about \(-744.44\). These are useful diagnostics, not universal hard-coded cutoffs: the exact behavior at boundaries depends on the format, rounding, and implementation. Some systems flush subnormal results to zero. A robust implementation should obtain limits from its arithmetic environment and leave a margin when the result will be used in further operations.
Worked Example: Checking Three Predictors Before Exponentiating
Consider binary64 arithmetic and predictors \(\eta=3\), \(\eta=-720\), and \(\eta=710\). We check each predictor against the approximate limits above before interpreting the computed mean.
For \(\eta=3\), the predictor is well within the range. The mean is \(\exp(3)\approx20.0855\), a normal, representable value.
For \(\eta=-720\), the predictor is above the approximate lower limit \(-744.44\), so its exponential can be represented as a positive subnormal value. In fact, \(\exp(-720)\approx2.032\times10^{-313}\). It is positive but has less precision than a normal value.
For \(\eta=710\), the predictor exceeds the approximate upper limit \(709.78\). The mathematical mean is finite, but is larger than the largest finite binary64 number, so a binary64 exponential overflows.
The predictor itself can be stored in all three cases. That does not imply that its exponential can be stored. It is therefore useful to retain \(\eta\) and check the exponential operation separately.
Evaluate Poisson Likelihood Contributions from the Predictor
For independent Poisson observations \(Y_i=y_i\) with means \(\mu_i=\exp(\eta_i)\), the probability mass function gives the log-likelihood contribution
The second expression is the key computational form. It uses \(\eta_i\) directly for the log-mean term, rather than computing \(\log(\exp(\eta_i))\). It still contains \(\exp(\eta_i)\), because the Poisson likelihood includes the mean itself. When that exponential is out of range, however, the likelihood contribution may also be out of range; replacing it with a clipped value does not make the original calculation exact.
To verify the identity, the Poisson mass is \(P(Y=y)=\exp(-\mu)\mu^y/y!\), for \(\mu>0\). Taking logarithms gives \(\log P(Y=y)=-\mu+y\log(\mu)-\log(y!)\). Substituting \(\mu=\exp(\eta)\) yields \(\log(\mu)=\eta\), and hence the stated expression. This derivation is an identity over the real numbers; the computational advantage comes from using that identity without an unnecessary exponential-then-logarithm calculation.
Worked Example: A Positive Count with an Underflowing Mean
Suppose \(y=3\) and \(\eta=-720\). In binary64 arithmetic, \(\exp(-720)\approx2.032\times10^{-313}\) is subnormal. The log-likelihood contribution is
Since \(\log(6)\approx1.791759\), this is approximately
The tiny mean term makes no visible difference at the scale of this floating-point result, but the \(y\eta\) term is essential. If an implementation instead underflows the mean to zero and then computes \(y\log(\mu)\), it attempts \(3\log(0)\) and produces negative infinity. That is not the finite log-likelihood contribution given by the model.
For \(y=0\) at the same predictor, the contribution is \(-\exp(-720)\), since \(0\eta=0\) and \(\log(0!)=0\). A calculation that rounds the mean to zero may return zero for this contribution. That is a representational approximation, not an exact identity; whether the lost term matters depends on the required precision and subsequent calculation.
Range Checks Are Not Clipping
A range check asks whether a particular operation is safe in the available arithmetic. Its threshold should match that operation. Checking whether \(\exp(\eta)\) fits in the floating-point format is different from checking whether a weighted mean, a sum of means, or a full likelihood can be represented: each may have a different range requirement. A check can warn, reject a computation, or select a suitable alternative calculation.
Clipping replaces a value outside a chosen interval by an endpoint, for example using \(\min(\eta,700)\) in place of \(\eta\). That changes the predictor and therefore changes the implied mean and likelihood. Clipping can be part of an explicitly defined approximation, but it is not a numerically neutral way to evaluate the original model.
Worked Example: Why an Arbitrary Ceiling Changes the Mean
Suppose a program replaces every predictor above \(700\) by \(700\). For an original predictor \(\eta=709\), the model mean is \(\exp(709)\), while the clipped mean is \(\exp(700)\). Their ratio is
Thus clipping changes the mean by a factor of about 8,103; it does not merely protect the last few digits of a calculation. For a zero Poisson count, the original log-likelihood contribution is \(-\exp(709)\), whereas the clipped contribution is \(-\exp(700)\). For a positive count, the \(y\eta\) term also changes from \(709y\) to \(700y\). Both changes alter the likelihood.
A safer response to a predictor whose exponential is out of range is to flag that fact and avoid pretending the clipped value is the model mean. Depending on the task, one might reject the current parameter candidate, use an alternative calculation that keeps the relevant quantity in a representable form, or report that the requested likelihood value is outside the arithmetic range. The choice should preserve the meaning of the computation.
When Log-Domain Work Helps—and Its Limits
Keeping the predictor available is valuable whenever the calculation involves a log mean. In particular, compute \(\log(\mu_i)\) as \(\eta_i\), not by forming \(\mu_i\) first. For Poisson data, use \(y_i\eta_i-\exp(\eta_i)-\log(y_i!)\) so that a rounded-zero mean does not lead to \(\log(0)\) when \(y_i>0\).
Log-domain calculation does not make every quantity representable. The Poisson contribution still includes \(-\exp(\eta_i)\). If the mean exceeds the arithmetic range, this contribution may be too large in magnitude to store. Similarly, individually representable means can have a sum that overflows. A range check must therefore consider the actual operation, not only whether each predictor is finite.
Do not interpret a floating-point overflow result as proof that the mathematical likelihood is exactly negative infinity. For finite \(\eta\) and finite \(y\), the real-valued contribution is finite, although it may be too large in magnitude for the chosen format. Conversely, if underflow turns a small positive exponential into zero, do not use that rounded zero as evidence that the model mean is mathematically zero: \(\exp(\eta)>0\) for every finite real \(\eta\).
Compute and retain \(\eta_i\), the log mean, rather than treating the exponentiated mean as the only representation.
Before exponentiating, compare \(\eta_i\) with limits appropriate to the format and to any later multiplication or summation.
For a Poisson contribution, compute \(y_i\eta_i\) directly and do not form \(\log(\exp(\eta_i))\).
Flag overflow or underflow as a numerical limitation; do not silently replace the predictor with a clipped value.
Common Mistake
A frequent error is to say, “The exponential overflowed, so I will cap the predictor at the largest value that gives a finite exponential; the likelihood is now safe.” This reasoning confuses representability with the statistical model. The cap changes the mean, and thus changes the likelihood. A range check can establish that a calculation cannot be performed as requested in the current format; it cannot justify substituting a different predictor without acknowledging the approximation.
Check Your Understanding
Use the log-link and Poisson formulas above to answer these questions.
- Why can a finite predictor have an exponential that overflows in binary64 arithmetic?
- For a Poisson observation with \(y=2\), write the log-likelihood contribution directly in terms of \(\eta\), without using \(\log(\exp(\eta))\).
- If \(\eta\) is so negative that its exponential rounds to zero, what happens if a program then evaluates \(y\log(\mu)\) for \(y>0\)?
- Explain why capping \(\eta\) at a fixed upper value changes a Poisson likelihood contribution.
- Why should a check for the range of each mean be distinguished from a check for the range of their sum?