Comparing Binary Links by Tail Behavior

Why Compare Binary Links by Their Tails?

Binary-link choices can look similar when the predicted probability is near the middle of its range, yet differ substantially when a linear predictor is far from zero. That difference matters whenever a model is used to predict rare events or near-certain outcomes: predictions that are both close to zero may still differ by an order of magnitude, and predictions near one may have very different remaining probabilities of failure. The link therefore affects more than the shape of a curve around its midpoint.

Here the comparison holds the linear predictor fixed. It isolates the mathematical behavior of the inverse links; it does not claim that three separately fitted models will have identical coefficient estimates or identical predictor values. The logit, probit, and complementary log-log links were introduced earlier; the focus here is how their inverse links behave at extreme predictor values.

Definition: Let $Y_i$ be a Bernoulli response with conditional success probability $p_i=P(Y_i=1\mid x_i)$, and let $\eta_i=x_i^\mathsf{T}\beta$ be its linear predictor. For a binary inverse link $F$, the predicted probability is $p_i=F(\eta_i)$. The three inverse links compared here are $$ F_{\mathrm{logit}}(\eta)=\frac{e^\eta}{1+e^\eta},\qquad F_{\mathrm{probit}}(\eta)=\Phi(\eta),\qquad F_{\mathrm{cloglog}}(\eta)=1-\exp\{-e^\eta\}, $$ where $\Phi$ is the standard normal cumulative distribution function.

Each formula maps every finite real $\eta$ to a probability strictly between zero and one. In the usual Bernoulli GLM interpretation, observations are conditionally Bernoulli given their predictors, the conditional mean is correctly represented by the chosen inverse link, and the same predictor value is used when comparing links. This last condition is essential: if one model uses $\eta=2$ and another uses $\eta=1$, their probability difference cannot be attributed solely to the link shape.

Same Predictor, Different Probabilities

The following calculations use identical values of $\eta$. They show that the links agree at $\eta=0$ for logit and probit, but not for complementary log-log, and that their predictions can separate at more extreme values. Values are rounded to four significant figures.

Worked Example: Comparing Predictions at Three Predictor Values

Suppose the same linear predictor is evaluated at $\eta=-3$, $0$, and $2$. For each value, substitute into the three inverse-link formulas.

At $\eta=-3$. The logit probability is $e^{-3}/(1+e^{-3})=0.049787/(1.049787)=0.04743$. The probit probability is $\Phi(-3)=0.001350$. The complementary log-log probability is $1-\exp(-e^{-3})=1-\exp(-0.049787)=0.04857$. The probit prediction is much smaller than either of the other two.

At $\eta=0$. Logit gives $1/(1+1)=0.5000$, and probit gives $\Phi(0)=0.5000$. Complementary log-log gives $1-\exp(-1)=0.6321$. Thus, even at the same predictor value, the three inverse links do not share a common midpoint probability.

At $\eta=2$. Logit gives $e^2/(1+e^2)=7.389/(8.389)=0.8808$. Probit gives $\Phi(2)=0.9772$. Complementary log-log gives $1-\exp(-e^2)=1-\exp(-7.389)=0.9994$. The predicted probabilities now span nearly 0.12, despite using the same predictor.

These are comparisons of mathematical response curves, not evidence that one model fits better. For actual predictions, each model's predictor is estimated from data and should be assessed in the context of the application.

What the Tail Approximations Show

The lower tail corresponds to $\eta$ tending to negative infinity, so that $p$ tends to zero. The upper tail corresponds to $\eta$ tending to positive infinity, so that $p$ tends to one. It is useful to examine both $p$ and the remaining probability $1-p$: a model can predict a probability extremely close to one while implying a very different rate of non-events from another model.

For the logit inverse link, write $x=e^\eta$. When $\eta$ is very negative, $x$ is small and $p=x/(1+x)$ is approximately $x=e^\eta$. In the upper tail, $1-p=1/(1+e^\eta)$ is approximately $e^{-\eta}$. For the complementary log-log inverse link, $p=1-\exp(-x)$. When $x$ is small, expansion of the exponential gives $p$ approximately $x=e^\eta$; in the upper tail, however, the exact complement is $\exp(-e^\eta)$, which decreases much faster than $e^{-\eta}$.

For probit, the normal tail behaves differently. The standard normal density is $\phi(\eta)=\exp(-\eta^2/2)/\sqrt{2\pi}$. For large positive $z$, the normal tail satisfies the Mills approximation $1-\Phi(z)$ approximately $\phi(z)/z$. Symmetry gives $\Phi(-z)=1-\Phi(z)$. Thus probit probabilities near either boundary have a leading exponential term involving the square of the predictor, rather than a term linear in the predictor.

\begin{array}{ll} \eta\to-\infty: & F_{\mathrm{logit}}(\eta)\sim e^\eta,\qquad F_{\mathrm{cloglog}}(\eta)\sim e^\eta,\qquad F_{\mathrm{probit}}(\eta)\sim \dfrac{\phi(\eta)}{-\eta},\\[6pt] \eta\to+\infty: & 1-F_{\mathrm{logit}}(\eta)\sim e^{-\eta},\qquad 1-F_{\mathrm{probit}}(\eta)\sim \dfrac{\phi(\eta)}{\eta},\qquad 1-F_{\mathrm{cloglog}}(\eta)=\exp(-e^\eta). \end{array}

Here $a(\eta)\sim b(\eta)$ means that the ratio $a(\eta)/b(\eta)$ tends to one in the stated limit. At the lower boundary, probit probability becomes negligible faster than either logit or complementary log-log probability. Logit and complementary log-log share the same leading lower-tail approximation, but they are not identical. At the upper boundary, the complementary log-log complement shrinks fastest, followed by the probit complement, while the logit complement shrinks slowest.

Deriving the Lower-Tail Comparisons

A short expansion makes the lower-tail comparison more precise. Set $x=e^\eta$, so $x\to0$ as $\eta\to-\infty$. For logit, the geometric expansion of $1/(1+x)$ gives $x/(1+x)=x-x^2+x^3-\cdots$. For complementary log-log, the exponential expansion gives $1-e^{-x}=x-x^2/2+x^3/6-\cdots$. Both have leading term $x$, but their second-order terms differ.

F_{\mathrm{cloglog}}(\eta)-F_{\mathrm{logit}}(\eta) =\left(x-\frac{x^2}{2}+O(x^3)\right) -\left(x-x^2+O(x^3)\right) =\frac{x^2}{2}+O(x^3).

Because $x^2/2$ is positive and dominates the remaining higher-order terms for sufficiently small positive $x$, complementary log-log probabilities exceed logit probabilities sufficiently far into the lower tail. The same leading-order argument does not order probit against them by a constant ratio: its lower-tail probability is on the smaller, quadratic-exponent scale.

Worked Example: Checking the Tail Approximations Numerically

Take $\eta=-4$, for which $x=e^{-4}=0.0183156$. The exact logit probability is $x/(1+x)=0.0183156/1.0183156=0.0179862$, compared with the leading approximation $x=0.0183156$. Its absolute approximation error is $0.0003294$.

The exact complementary log-log probability is $1-\exp(-0.0183156)=0.0181489$, compared with $x=0.0183156$, an absolute error of $0.0001667$. It is slightly larger than the logit probability, as the second-order calculation predicts.

For probit, $\Phi(-4)=0.00003167$. The Mills approximation gives $\phi(4)/4$, where $\phi(4)=\exp(-8)/\sqrt{2\pi}=0.00013383$, so $\phi(4)/4=0.00003346$. This is a useful tail approximation, but it is not exact: it exceeds the actual probability by about $0.00000179$. All three exact probabilities are positive, yet the probit value is about one-six hundredth of the logit value at this predictor.

Worked Example: Comparing the Remaining Probability Near One

At $\eta=4$, the predicted probabilities are all close to one, so compare their complements directly. For logit, $1-p=1/(1+e^4)=1/55.598=0.01799$. For probit, $1-p=1-\Phi(4)=0.00003167$. For complementary log-log, $1-p=\exp(-e^4)=\exp(-54.598)=1.94\times10^{-24}$.

The probability of a non-event is therefore much smaller under complementary log-log than under probit at this same predictor; the logit complement is larger still. Reporting only that all three predicted probabilities are “near one” would conceal this substantial difference in the implied non-event probability.

Interpreting and Checking a Link Comparison

A useful comparison must distinguish a fixed-predictor calculation from a comparison of fitted models. If the predictor is held fixed, the numerical differences above follow directly from the inverse-link formulas. If the three links are fitted separately, their coefficient estimates can differ, and thus so can the resulting $\eta_i$ values. The predictor scale is tied to the link: the same numerical $\eta$ is a controlled mathematical comparison, not a claim that equal coefficients have equal substantive meaning across links.

For example, a probability of 0.95 is reached at different predictor values under the three links. Solving the inverse-link equations gives a transparent illustration of their different response curves.

Worked Example: Finding the Predictor for a 0.95 Probability

For logit, solve $0.95=e^\eta/(1+e^\eta)$. Rearranging gives $e^\eta=0.95/0.05=19$, so $\eta=\log(19)=2.944$.

For probit, solve $\Phi(\eta)=0.95$, which gives $\eta=\Phi^{-1}(0.95)=1.645$.

For complementary log-log, solve $0.95=1-\exp(-e^\eta)$. Then $\exp(-e^\eta)=0.05$, so $e^\eta=-\log(0.05)=2.996$ and $\eta=\log(2.996)=1.097$.

These are three link-scale values for the same response probability; they should not be interpreted as directly comparable effects in the original application. The result reinforces why model comparisons should be based on fitted probabilities and relevant predictive performance, not on a raw comparison of link-scale coefficients alone.

In practice, inspect predictions on the response scale, especially where the fitted predictor is extreme. Useful checks include whether a few observations dominate predictions near zero or one, whether there is enough data in those regions to support such predictions, and whether observed event frequencies among cases with similar predicted probabilities are compatible with the predictions. Calibration summaries or plots can make systematic over- or under-prediction visible. These checks diagnose the adequacy of predictions; they do not prove that a link is correct.

When the goal is to select among fitted links, fit each candidate to the same response observations and predictor information, then compare an appropriate likelihood-based criterion or out-of-sample predictive performance. Calibration and prediction error are often especially relevant when boundary probabilities matter. The choice should reflect the scientific setting and predictive purpose, not just which curve has a preferred tail shape on paper.

Common Pitfalls and Limitations

Calling the links interchangeable because they meet near the center. Logit and probit both give 0.5 at $\eta=0$, but their slopes and tails differ, and complementary log-log gives about 0.632 there. Agreement at one point does not imply agreement elsewhere.
Comparing fitted coefficients as if the link scales were identical. Coefficients are expressed on the chosen link scale. Compare response-scale predictions or meaningful contrasts after fitting, rather than assuming equal numerical coefficients represent equal changes in probability.
Looking only at probabilities rounded to 0 or 1. Rounding can hide large relative differences in small probabilities or in $1-p$. Retain sufficient precision when rare events or near-certain outcomes matter.
Choosing a link from a tail formula alone. The tails describe the mathematical behavior of the inverse links; they do not establish which link best represents a particular data-generating process. Check model fit, calibration, and predictive performance.

Tail behavior is a property of the response curve, not a guarantee of accurate extrapolation. If observed data provide little information in extreme-predictor regions, a model can produce sharply different boundary predictions without those differences being empirically well determined. The comparison here also holds $\eta$ fixed; it does not account for changes in predictions when models are fitted or when additional model structure is introduced.

Key takeaway: At the same linear predictor, logit and complementary log-log have the same leading lower-tail order, while probit's lower tail is thinner. Near one, complementary log-log approaches one fastest, probit next, and logit slowest. These differences matter most for boundary probabilities, but link choice should be evaluated with fitted, response-scale predictions and data-based diagnostics.

Check Your Understanding

Answer each question using the inverse-link formulas and tail results above. Short solutions follow the questions.

At $\eta=-2$, logit gives approximately 0.1192, probit approximately 0.02275, and complementary log-log approximately 0.1266. Which link gives the smallest probability, and does that ordering agree with the lower-tail behavior?
At $\eta=3$, which inverse link has the smallest complement $1-p$? Explain using the upper-tail formulas.
For a very negative predictor, why are logit and complementary log-log probabilities of the same leading order but not exactly equal?
Why is it not valid to infer that one separately fitted link predicts a higher probability merely because its inverse link gives a higher probability at the same numerical $\eta$?
Solve for the complementary log-log predictor that gives $p=0.50$, to three significant figures.

Solutions. (1) Probit is smallest at 0.02275. This agrees with the eventual lower-tail ordering: its probability shrinks on a quadratic-exponent scale, faster than the $e^\eta$ scale of logit and complementary log-log. (2) Complementary log-log has the smallest complement, since $1-p=\exp(-e^\eta)$; at $\eta=3$, this is $\exp(-20.086)=1.90\times10^{-9}$, smaller than the probit complement and the logit complement. (3) With $x=e^\eta$ small, logit is $x-x^2+O(x^3)$, whereas complementary log-log is $x-x^2/2+O(x^3)$. They share leading term $x$, but their second-order terms differ. (4) Separately fitted links can produce different coefficient estimates and different predictor values. The fixed-$\eta$ comparison isolates inverse-link shape; it does not compare complete fitted predictions. (5) Set $0.50=1-\exp(-e^\eta)$. Then $e^\eta=-\log(0.50)=0.6931$, so $\eta=\log(0.6931)=-0.3665$, or $-0.367$ to three significant figures.