Comparing Link Slopes On The Probability Scale

Why Compare Slopes on the Probability Scale?

A coefficient describes a change in the linear predictor, not directly a change in probability. For a binary-response model with mean probability $p$, write $p=g^{-1}(\eta)$, where $\eta$ is the linear predictor and $g^{-1}$ is the inverse link. A one-unit increase in a covariate with coefficient $\beta$ changes $\eta$ by $\beta$, but the corresponding probability change depends on the inverse link and on the starting value of $\eta$.

This tutorial compares probit, complementary log-log, and log links by holding a predictor shift $\delta$ fixed. That comparison separates the scale of the predictor change from the scale of the probability response. It also shows why the direction and size of a link’s probability effect cannot be inferred from a coefficient alone.

Definition: For an inverse link $g^{-1}$, the local probability slope at predictor value $\eta$ is $\frac{d}{d\eta}g^{-1}(\eta)$. For a finite predictor shift $\delta$, the finite probability change is $g^{-1}(\eta+\delta)-g^{-1}(\eta)$. These are different quantities unless the inverse link is linear over the interval.

In a regression model, if a covariate changes by $h$ while the other predictors remain fixed, then $\delta=\beta h$. A local derivative describes the effect of an infinitesimal change; a finite change gives the actual probability difference across the specified shift. The comparisons below use the same $\delta$ for all three links.

Local Probability Slopes

The inverse links considered here are $p=\Phi(\eta)$ for probit, $p=1-\exp\{-\exp(\eta)\}$ for complementary log-log, and $p=\exp(\eta)$ for the log link. For the log link, the formula defines a probability only where $\eta\leq 0$.

Theorem: At a baseline probability $0<p<1$, the local probability slopes with respect to $\eta$ are $$ \begin{aligned} s_{\text{probit}}(p)&=\phi\!\left(\Phi^{-1}(p)\right),\\ s_{\text{cloglog}}(p)&=(1-p)\bigl[-\log(1-p)\bigr],\\ s_{\text{log}}(p)&=p. \end{aligned} $$ For the log link, the derivative is a probability-scale slope only at predictor values where the model is feasible.

Proof. For probit, differentiating $p=\Phi(\eta)$ gives $dp/d\eta=\phi(\eta)$. Since $\eta=\Phi^{-1}(p)$, this is $\phi(\Phi^{-1}(p))$. For complementary log-log, differentiating $p=1-\exp\{-\exp(\eta)\}$ gives

\frac{dp}{d\eta}=\exp(\eta)\exp\{-\exp(\eta)\}.

The link equation implies $\exp(\eta)=-\log(1-p)$ and $\exp\{-\exp(\eta)\}=1-p$; substitution gives the stated slope. For the log link, differentiating $p=\exp(\eta)$ gives $dp/d\eta=\exp(\eta)=p$. This proves all three formulas. $\square$

These expressions already show that the probability effect depends on the baseline risk. For example, at $p=0.20$, the probit slope is approximately $0.280$, the complementary log-log slope is $0.8[-\log(0.8)]\approx0.179$, and the log-link slope is $0.200$. At $p=0.80$, the corresponding slopes are approximately $0.280$, $0.2[-\log(0.2)]\approx0.322$, and $0.800$. The ordering changes: there is no single link that has the largest local slope at every baseline probability.

Worked Example: Local Slopes at Two Baseline Risks

Compare the local slopes of all three links at baseline probabilities $p=0.20$ and $p=0.80$.

At $p=0.20$. The probit quantile is $\Phi^{-1}(0.20)\approx-0.8416$, so its slope is $\phi(-0.8416)\approx0.2800$. The complementary log-log slope is $0.80[-\log(0.80)]=0.80(0.2231)\approx0.1785$. The log-link slope is $0.20$. Thus, locally, probit has the largest slope and complementary log-log the smallest.

At $p=0.80$. By symmetry, $\phi(\Phi^{-1}(0.80))=\phi(0.8416)\approx0.2800$. The complementary log-log slope is $0.20[-\log(0.20)]=0.20(1.6094)\approx0.3219$, while the log-link slope is $0.80$. Here the log link has the largest slope and probit the smallest.

These are derivatives per unit change in $\eta$, not finite probability differences for a specified predictor shift.

Finite Changes Under a Matched Predictor Shift

A finite shift gives a direct comparison for a specific covariate contrast. The following formulas express the changed probability in terms of the baseline probability $p$, so that the same starting risk and shift can be used across links.

Theorem: Let $0<p<1$ be the baseline probability and let $\delta$ be a predictor shift. The resulting probabilities are $$ \begin{aligned} p_{\text{probit}}'&=\Phi\!\left(\Phi^{-1}(p)+\delta\right),\\ p_{\text{cloglog}}'&=1-(1-p)^{\exp(\delta)},\\ p_{\text{log}}'&=p\exp(\delta). \end{aligned} $$ The log-link result is a valid probability only if $p\exp(\delta)\leq1$.

Derivation. Under probit, the baseline predictor is $\eta=\Phi^{-1}(p)$; adding $\delta$ and applying $\Phi$ gives the first expression. Under complementary log-log, the baseline relation is $\exp(\eta)=-\log(1-p)$. After the shift, $\exp(\eta+\delta)=\exp(\delta)[-\log(1-p)]$. Therefore,

p_{\text{cloglog}}' =1-\exp\!\left[-\exp(\delta)\{-\log(1-p)\}\right] =1-(1-p)^{\exp(\delta)}.

For the log link, $p=\exp(\eta)$, so $\exp(\eta+\delta)=p\exp(\delta)$. This is a valid probability precisely when it is at most one. The formulas establish the finite-change result. $\square$

Worked Example: Equal Predictor Shifts at Low and High Risk

Compare the three links for the same positive shift $\delta=0.20$ at baseline risks $p=0.20$ and $p=0.80$. Here $\exp(0.20)\approx1.2214$.

At $p=0.20$. Under probit, $p'=\Phi(\Phi^{-1}(0.20)+0.20)=\Phi(-0.6416)\approx0.2606$, a change of approximately $0.0606$. Under complementary log-log, $p'=1-0.8^{1.2214}\approx1-\exp(-0.2725)\approx0.2386$, a change of approximately $0.0386$. Under the log link, $p'=0.20(1.2214)=0.2443$, a change of approximately $0.0443$.

At $p=0.80$. Under probit, $p'=\Phi(0.8416+0.20)=\Phi(1.0416)\approx0.8512$, a change of approximately $0.0512$. Under complementary log-log, $p'=1-0.2^{1.2214}\approx1-\exp(-1.9658)\approx0.8600$, a change of approximately $0.0600$. Under the log link, $p'=0.80(1.2214)=0.9771$, a change of approximately $0.1771$. This is still feasible because $0.9771\leq1$.

The same predictor shift produces different probability changes at the two baselines, and the relative sizes differ by link. For instance, the probit change is larger at $p=0.20$ than at $p=0.80$, whereas the complementary log-log and log-link changes are larger at $p=0.80$ in these calculations.

Feasibility and Interpretation

The log link has a special restriction in a binary model: its inverse link is $\exp(\eta)$, which can exceed one. A positive shift from baseline $p$ remains feasible exactly when $p\exp(\delta)\leq1$, or equivalently when $\delta\leq\log(1/p)$. Thus, an effect comparison using the log link must check the predicted probability, rather than treating the exponential formula as automatically valid.

Worked Example: Checking a Log-Link Shift

Suppose the baseline probability is $p=0.80$. The largest positive shift that remains feasible satisfies $0.80\exp(\delta)\leq1$. Taking logarithms gives $\delta\leq\log(1/0.80)=\log(1.25)\approx0.2231$.

A shift of $0.20$ is feasible: $0.80\exp(0.20)\approx0.9771$. A shift of $0.25$ is not: $0.80\exp(0.25)\approx1.0272$, which is greater than one and therefore cannot be a probability. By contrast, the probit and complementary log-log inverse links map every finite predictor value to a probability strictly between zero and one.

The choice of comparison matters. A matched predictor shift answers, “What probability change results from the same change in $\eta$?” It does not force the models to have the same estimated coefficients for a covariate, nor does it establish which model fits data better. Those are different questions. Here the baseline probability is also held fixed; this makes the probability-scale effect differences attributable to the inverse-link shape at that baseline and shift.

Common Pitfall: Treating a Coefficient as a Risk Difference

Incorrect reasoning: “The coefficient is $0.20$, so a one-unit increase in the covariate raises the probability by $0.20$, regardless of link.” This confuses a change in the linear predictor with a probability difference. Under probit and complementary log-log, the conversion depends on the baseline predictor; even under a log link, the finite change is $p[\exp(\delta)-1]$, not $\delta$, and the result must remain at most one.

Correction: First identify the baseline probability, link, and predictor shift. Then use the corresponding finite-change formula to compute the new probability and subtract the baseline. For an infinitesimal change, use the local slope at that baseline instead. Stating which quantity is being reported prevents the local derivative and finite contrast from being mistaken for one another.

Key takeaway: Matching the linear-predictor shift does not match probability effects. Probit, complementary log-log, and log links have different baseline-dependent slopes and finite-change formulas; for a binary log-link model, feasibility must also be checked.

Check Your Understanding

Use the formulas in this tutorial to answer the following.

At $p=0.50$, write the local probability slope for each of the three links in terms of standard functions or $p$.
For complementary log-log, explain why a positive predictor shift increases the probability, and give the finite-change formula.
A log-link model has baseline risk $p=0.60$ and predictor shift $\delta=0.30$. Is the resulting probability feasible? Show the calculation.
Why can a local slope not generally be used as the exact probability change for a predictor shift of $0.20$?
In a matched-shift comparison, what is held fixed, and what does the comparison not establish about model fit?