Inverse-Link Derivatives And Probability Effects

From a Linear Predictor to a Probability Effect

A coefficient in a binary-response generalized linear model describes a change in the linear predictor, not directly a fixed change in response probability. To obtain a probability-scale effect, one must account for how the inverse link converts that predictor into a probability. The derivative of the inverse link supplies that conversion locally.

Let $Y$ be binary and let $\mu(x)=P(Y=1\mid X=x)$. For a covariate vector $x$, write the linear predictor as $\eta(x)=\beta_0+x^\mathsf{T}\beta$, and let $h=g^{-1}$ denote the inverse link. Thus $\mu(x)=h(\eta(x))$. The following calculation concerns a covariate that enters the predictor linearly and continuously, with the other covariates held fixed.

Definition: The local probability effect of a continuous covariate $x_j$ at a specified covariate profile is the partial derivative $\partial \mu(x)/\partial x_j$, with the remaining covariates fixed. In a model with $\eta(x)=\beta_0+x^\mathsf{T}\beta$, this effect is $\beta_j h'(\eta(x))$, whenever $h$ is differentiable at $\eta(x)$.

The derivative has two parts. The coefficient $\beta_j$ determines how quickly the linear predictor changes with $x_j$; the inverse-link derivative $h'(\eta)$ determines how that predictor change translates into probability at the current profile. The probability effect can therefore vary across profiles even when the coefficient is constant.

Deriving the Probability-Scale Slope

For a differentiable inverse link, the chain rule gives the fundamental derivative formula. In a one-covariate model, write $\eta(x)=\beta_0+\beta x$, so $\mu(x)=h(\beta_0+\beta x)$.

Result: If $h$ is differentiable at $\eta(x)$, then $$ \frac{d\mu(x)}{dx} = \beta h'(\eta(x)). $$ For a vector of covariates with a linear predictor, the corresponding partial derivative is $\partial \mu(x)/\partial x_j=\beta_j h'(\eta(x))$.

To verify the result, take a nonzero increment $t$. The difference quotient is

\frac{\mu(x+t)-\mu(x)}{t} = \frac{h(\eta(x)+\beta t)-h(\eta(x))}{\beta t}\,\beta

when $\beta\ne 0$. As $t\to 0$, differentiability of $h$ makes the fraction tend to $h'(\eta(x))$, proving the formula. If $\beta=0$, then $\mu(x)=h(\beta_0)$ is constant in $x$, and both sides of the formula are zero. The vector result follows by changing $x_j$ while holding the other coordinates fixed.

The derivative describes an infinitesimal change. For an actual increment $\Delta$, the exact probability change is

\mu(x+\Delta)-\mu(x) = h(\eta(x)+\beta\Delta)-h(\eta(x)).

This is generally not equal to $\beta\Delta h'(\eta(x))$. That expression is the first-order local approximation, suitable when the increment is small and the inverse-link slope does not change much over the corresponding predictor interval.

Result: Suppose $h$ is continuously differentiable on an interval containing $\eta$ and $\eta+\beta\Delta$. Then $$ h(\eta+\beta\Delta)-h(\eta) = \beta\Delta\int_0^1 h'(\eta+s\beta\Delta)\,ds. $$ If $\beta\Delta\ne 0$, the mean value theorem also gives a point $\eta^*$ between $\eta$ and $\eta+\beta\Delta$ such that the change equals $\beta\Delta h'(\eta^*)$.

For the integral identity, define $q(s)=h(\eta+s\beta\Delta)$ on $0\leq s\leq 1$. The chain rule gives $q'(s)=\beta\Delta h'(\eta+s\beta\Delta)$. The fundamental theorem of calculus then yields $q(1)-q(0)=\int_0^1 q'(s)\,ds$, which is the stated identity. For nonzero $\beta\Delta$, continuity of $h$ on the interval and differentiability in its interior satisfy the hypotheses of the mean value theorem. Thus the exact change equals the predictor change times an inverse-link slope somewhere along the interval. $\square$

Three Inverse Links and Their Derivatives

For comparison, the inverse links used here are probit, complementary log-log, and log. Let $\phi$ and $\Phi$ denote the density and cumulative distribution function of $N(0,1)$, respectively. The log link requires particular attention for a binary mean: since probabilities cannot exceed one, a log-link model is valid only at profiles where $\exp(\eta)\leq 1$, or equivalently $\eta\leq 0$. Local two-sided changes are considered at interior profiles with $\eta<0$.

Link	Inverse link $h(\eta)$	Inverse-link derivative $h'(\eta)$
Probit	$\Phi(\eta)$	$\phi(\eta)$
Complementary log-log	$1-\exp\{-\exp(\eta)\}$	$\exp\{\eta-\exp(\eta)\}$
Log	$\exp(\eta)$	$\exp(\eta)$

For probit, the derivative of the cumulative distribution function is its density, so $h'(\eta)=\phi(\eta)$. For the complementary log-log inverse link, differentiating $1-\exp\{-\exp(\eta)\}$ gives $\exp\{-\exp(\eta)\}\exp(\eta)$, which equals $\exp\{\eta-\exp(\eta)\}$. For the log link, differentiating $\exp(\eta)$ gives $\exp(\eta)$. These derivatives are positive, so in each case a positive coefficient gives a positive local probability effect at a valid profile; the magnitude depends on the profile through $\eta$.

Worked Example: A Local Probit Probability Effect

Suppose a probit model has $\eta(x)=-0.2+0.5x$. Evaluate the local effect of increasing $x$ at $x=0.4$, and compare it with the probability change for an increase of $0.1$.

At $x=0.4$, the predictor is $\eta=-0.2+0.5(0.4)=0$, and the fitted probability is $\Phi(0)=0.5$. The local probability effect is

\frac{d\mu}{dx} = 0.5\phi(0) = 0.5\left(\frac{1}{\sqrt{2\pi}}\right) \approx 0.19947.

The local approximation for a $0.1$-unit increase is therefore $0.19947(0.1)=0.019947$, or about 1.995 percentage points. The exact change shifts the predictor from $0$ to $0.05$:

\Phi(0.05)-\Phi(0) \approx 0.5199388-0.5000000 = 0.0199388.

The local approximation is close to, but not exactly, the finite change. The inverse-link slope is evaluated at the starting predictor for the approximation, whereas it varies slightly over the interval from $0$ to $0.05$.

Worked Example: A Local Complementary Log-Log Probability Effect

Consider a complementary log-log model with $\eta(x)=-0.3+0.4x$. At $x=0.75$, calculate the probability and its local slope with respect to $x$, then approximate the change for $\Delta=0.1$.

The predictor is $\eta=-0.3+0.4(0.75)=0$. Hence

\mu=1-\exp\{-\exp(0)\}=1-e^{-1}\approx 0.63212.

The inverse-link derivative at this predictor is $\exp\{0-\exp(0)\}=e^{-1}$. Multiplying by the covariate coefficient gives

\frac{d\mu}{dx} = 0.4e^{-1} \approx 0.14715.

The local approximation for a $0.1$-unit increase is $0.14715(0.1)=0.014715$. For comparison, the exact predictor increment is $0.4(0.1)=0.04$, so the exact probability change is

\left[1-\exp\{-\exp(0.04)\}\right]-\left[1-\exp\{-\exp(0)\}\right] \approx 0.64683-0.63212 \approx 0.01471.

The exact change and the local approximation are close in this case. As with probit, the derivative is a local slope, not an exact rule for every finite covariate shift.

Worked Example: A Local Log-Link Probability Effect

Suppose a log-link binary mean has $\eta(x)=\log(0.2)+0.6x$, evaluated at $x=0$. Calculate the local effect and the exact change for a $0.1$-unit increase, checking that the probabilities remain valid.

At $x=0$, $\eta=\log(0.2)\approx -1.60944<0$, and $\mu=\exp(\eta)=0.2$. The derivative is

\frac{d\mu}{dx} = 0.6\exp(\eta) = 0.6(0.2) = 0.12.

Thus the local probability change for $\Delta=0.1$ is $0.12(0.1)=0.012$, or 1.2 percentage points. The new predictor is $\log(0.2)+0.06\approx -1.54944<0$, so the new fitted probability remains below one. Its exact value and the exact change are

\mu(x+0.1)=0.2e^{0.06}\approx 0.21237, \qquad \mu(x+0.1)-\mu(x) = 0.2(e^{0.06}-1) \approx 0.01237.

Here too, the exact change differs from the local approximation. The log-link derivative at the starting point equals the starting probability, so the local slope is $0.6$ times that probability; it is not a constant probability increase across all profiles.

Interpreting and Reporting the Effect

A probability effect should be reported with its covariate profile and increment in mind. Under all three links, the coefficient controls a change on the linear-predictor scale. The derivative then rescales that change according to the current predictor. Consequently, a statement such as “a one-unit increase in $x$ raises probability by $\beta$” is generally incorrect. A local statement instead reports $\beta h'(\eta)$ at a specified profile; a finite-change statement calculates $h(\eta+\beta\Delta)-h(\eta)$.

The derivative formula applies to a continuous covariate that enters the predictor linearly. For a binary covariate, a derivative is not the appropriate contrast: compare the two fitted probabilities directly, holding other covariates fixed. Likewise, if the predictor depends nonlinearly on a continuous covariate, its derivative must include the derivative of that predictor rather than simply using a coefficient. More generally, for differentiable $\eta(x)$, the chain rule gives $d\mu/dx=h'(\eta(x))\,d\eta/dx$.

Common Mistake

A frequent error is to differentiate the link $g$ and use $g'(\mu)$ as though it were the probability-scale slope. The model gives $\mu=h(\eta)$, so the relevant derivative is $h'(\eta)$. For differentiable inverse functions, $h'(\eta)=1/g'(h(\eta))$ when the derivative in the denominator is nonzero; these quantities are not interchangeable. Another mistake is to report the coefficient alone as a probability change. The correction is to evaluate the inverse-link derivative at the predictor of interest, multiply it by the predictor's rate of change, and distinguish that local effect from an exact finite contrast.

Key takeaway: A covariate's local probability effect is the derivative of the inverse link at the current linear predictor multiplied by the covariate's contribution to that predictor. For a finite covariate change, use the two fitted probabilities rather than treating the local slope as an exact difference.

Check Your Understanding

Use the inverse-link derivative at the stated profile for local effects, and distinguish local approximations from exact changes.

In a probit model with $\eta=-0.4+0.8x$, what expression gives the local probability effect at $x=0.5$?
For the complementary log-log inverse link, differentiate $1-\exp\{-\exp(\eta)\}$ with respect to $\eta$.
A log-link binary model has $\eta=-2$ and covariate coefficient $0.3$. What is the local probability effect per unit increase in that covariate?
Why is $\beta\Delta h'(\eta)$ usually not the exact probability change for a finite increment $\Delta$?
Why should a direct probability contrast, rather than a derivative, be used for a binary covariate?

Link	Inverse link \(h(\eta)\)	Inverse-link derivative \(h'(\eta)\)
Probit	\(\Phi(\eta)\)	\(\phi(\eta)\)
Complementary log-log	\(1-\exp\{-\exp(\eta)\}\)	\(\exp\{\eta-\exp(\eta)\}\)
Log	\(\exp(\eta)\)	\(\exp(\eta)\)