Link Derivatives And Local Sensitivity

Why the Inverse-Link Derivative Matters

In a generalized linear model, the linear predictor $\eta$ is mapped to the mean $\mu$ through the inverse link. Write that mapping as $\mu=h(\eta)$. The derivative $h'(\eta)$ measures how quickly the mean changes when the predictor changes near a particular value. This is local sensitivity: it describes the effect of a small predictor change, not necessarily the effect of a large one.

The derivative can depend on the predictor value. Consequently, an equal change in $\eta$ can produce different mean changes at different starting points. This tutorial calculates that dependence for the identity, log, and logit links. The derivative is a mathematical property of the inverse link; its interpretation as a mean change applies where the model's mean is admissible.

Definition: Let $\mu=h(\eta)$ be a differentiable inverse-link mapping. The local sensitivity of the mean to the linear predictor at $\eta$ is $h'(\eta)=d\mu/d\eta$. For a small predictor increment $\Delta\eta$, the corresponding first-order approximation to the mean change is $h'(\eta)\Delta\eta$. The absolute value $|h'(\eta)|$ measures the size of the local change without its direction.

Derivatives for Three Common Links

For the identity link, $g(\mu)=\mu$, so its inverse is $h(\eta)=\eta$. For the log link, $g(\mu)=\log(\mu)$, so $h(\eta)=e^\eta$. For the logit link, $g(\mu)=\log\{\mu/(1-\mu)\}$, so $h(\eta)=1/(1+e^{-\eta})$. Differentiating each inverse link gives the corresponding local sensitivity.

\begin{aligned} \text{Identity:}\quad & h(\eta)=\eta, & h'(\eta)&=1,\\ \text{Log:}\quad & h(\eta)=e^\eta, & h'(\eta)&=e^\eta=h(\eta),\\ \text{Logit:}\quad & h(\eta)=\frac{1}{1+e^{-\eta}}, & h'(\eta)&=\frac{e^{-\eta}}{(1+e^{-\eta})^2} =h(\eta)\{1-h(\eta)\}. \end{aligned}

The identity-link derivative is constant: a predictor increment produces the same first-order mean increment at every point. With a log link, the derivative equals the current mean, so the same predictor increment has a larger absolute effect when the mean is larger. With a logit link, the derivative depends on the fitted probability and is largest at the middle of its range.

Worked Example: Calculating Sensitivity at Several Predictor Values

Calculate the local sensitivity for each inverse link at $\eta=-1, 0,$ and $2$. For the logit link, also calculate the corresponding mean.

Identity link. Since $h'(\eta)=1$ everywhere, the sensitivity is $1$ at all three predictor values.

Log link. Here $h'(\eta)=e^\eta$, so $h'(-1)=e^{-1}\approx0.367879$, $h'(0)=1$, and $h'(2)=e^2\approx7.389056$. The absolute sensitivity increases with $\eta$.

Logit link. At $\eta=-1$, the mean is $h(-1)=1/(1+e)\approx0.268941$, and the sensitivity is $0.268941(1-0.268941)\approx0.196612$. At $\eta=0$, the mean is $0.5$, and the sensitivity is $0.5(1-0.5)=0.25$. At $\eta=2$, the mean is $1/(1+e^{-2})\approx0.880797$, and the sensitivity is $0.880797(1-0.880797)\approx0.104994$.

Thus the logit sensitivity is larger at $\eta=0$ than at either $-1$ or $2$. The log-link calculation instead shows why its absolute sensitivity depends strongly on the current predictor value.

What a Derivative Says About a Small Change

A derivative is a local statement. To make the approximation precise, let $\Delta\eta$ be a predictor increment. Differentiability of $h$ at $\eta$ means that the change in $h$ can be written as its linear term plus a remainder that is small relative to the increment.

Theorem: If $h$ is differentiable at $\eta$, then, as $\Delta\eta\to0$, $$ h(\eta+\Delta\eta)-h(\eta) =h'(\eta)\Delta\eta+o(\Delta\eta). $$ Here $o(\Delta\eta)$ denotes a remainder whose ratio to $\Delta\eta$ tends to zero.

Proof. By the definition of differentiability, for nonzero $\Delta\eta$, $$ \frac{h(\eta+\Delta\eta)-h(\eta)}{\Delta\eta} \longrightarrow h'(\eta) \quad\text{as }\Delta\eta\to0. $$ Equivalently, the difference between this ratio and $h'(\eta)$ tends to zero. Multiplying that difference by $\Delta\eta$ gives $$ h(\eta+\Delta\eta)-h(\eta)-h'(\eta)\Delta\eta=o(\Delta\eta), $$ which is the stated expansion. $\square$

In a model, a predictor change may result from changing a covariate. If, for example, $\eta=\beta_0+\beta_1x$, then a change $\Delta x$ changes the predictor by $\beta_1\Delta x$. The first-order mean change is therefore $h'(\eta)\beta_1\Delta x$. The factor $h'(\eta)$ is the link-dependent conversion from predictor scale to mean scale at the current predictor value.

Worked Example: Comparing Changes from the Same Predictor Increment

Take a predictor increment $\Delta\eta=0.1$, starting at $\eta=0$. Calculate the first-order and exact mean changes under each link.

Identity link. The approximation is $1(0.1)=0.1$. The exact change is $h(0.1)-h(0)=0.1-0=0.1$, so the approximation is exact.

Log link. At $\eta=0$, $h(0)=1$ and $h'(0)=1$, giving the first-order change $1(0.1)=0.1$. The exact change is $e^{0.1}-e^0\approx1.10517019-1=0.10517019$. The approximation is close, while the exact increase is slightly larger.

Logit link. At $\eta=0$, $h(0)=0.5$ and $h'(0)=0.25$, giving the first-order change $0.25(0.1)=0.025$. The exact change is $h(0.1)-h(0)\approx0.52497919-0.50000000=0.02497919$. These nearby values illustrate the local approximation, not a claim that it is exact for every nonzero increment.

Absolute and Relative Sensitivity

The log-link derivative gives a useful distinction. Its absolute sensitivity is $h'(\eta)=\mu$, so absolute changes grow with the mean. But the exact effect of a predictor increment has a simple relative form: $$ \frac{h(\eta+\Delta\eta)-h(\eta)}{h(\eta)} =\frac{e^{\eta+\Delta\eta}-e^\eta}{e^\eta} =e^{\Delta\eta}-1. $$ This ratio does not depend on the starting predictor $\eta$. For small $\Delta\eta$, it is approximately $\Delta\eta$, since the derivative of $e^u$ at $u=0$ is $1$. Thus equal predictor increments give the same exact proportional change under the log link, although their absolute changes differ.

Theorem: For the log inverse link $h(\eta)=e^\eta$, a predictor increment $\Delta\eta$ multiplies the mean by $e^{\Delta\eta}$, regardless of the starting value $\eta$.

Proof. The new mean is $h(\eta+\Delta\eta)=e^{\eta+\Delta\eta}$. Factoring this as $e^\eta e^{\Delta\eta}$ shows that it equals the old mean $h(\eta)=e^\eta$ multiplied by $e^{\Delta\eta}$. Dividing the new mean by the old mean gives the same factor. $\square$

Worked Example: Same Proportional Change, Different Absolute Changes

Compare the effect of $\Delta\eta=0.1$ under the log link when starting at $\eta=0$ and $\eta=2$.

At $\eta=0$, the starting mean is $e^0=1$, and the new mean is $e^{0.1}\approx1.10517019$. The absolute change is $1.10517019-1=0.10517019$.

At $\eta=2$, the starting mean is $e^2\approx7.38905610$, and the new mean is $e^{2.1}\approx8.16616991$. The absolute change is $8.16616991-7.38905610=0.77711381$.

The ratios of new to starting means are, respectively, $1.10517019/1\approx1.10517019$ and $8.16616991/7.38905610\approx1.10517019$. Both correspond to the same proportional increase, $e^{0.1}-1\approx0.10517019$, despite the very different absolute changes.

Why Logit Sensitivity Peaks in the Middle

For a logit link, write $p=h(\eta)$, so $0<p<1$ for every finite $\eta$, and $h'(\eta)=p(1-p)$. This derivative is positive, but its size varies across the predictor scale.

Result: The logit inverse-link derivative satisfies $0<h'(\eta)\leq1/4$ for every finite $\eta$. Its maximum, $1/4$, occurs at $\eta=0$, where $h(\eta)=1/2$.

Proof. Since $p=h(\eta)$ lies strictly between zero and one, the identity $$ p(1-p)=\frac14-\left(p-\frac12\right)^2 $$ shows that $0<p(1-p)\leq1/4$. Equality in the upper bound holds exactly when $p=1/2$. The logit inverse satisfies $h(0)=1/2$, and because it is strictly increasing, $h(\eta)=1/2$ only at $\eta=0$. Therefore the derivative reaches its maximum at zero. As $\eta$ tends to either positive or negative infinity, $p$ tends to one or zero, respectively, and $p(1-p)$ tends to zero. $\square$

This means equal small predictor shifts have the greatest local absolute effect on the mean near probability $1/2$, and smaller effects near the ends of the probability range. It does not mean the effect is zero at any finite predictor value: for finite $\eta$, $0<p<1$ and $p(1-p)>0$.

Common Mistake: Treating a Derivative as an Exact Change

Wrong reasoning: “The logit derivative at $\eta=2$ is about $0.105$, so increasing $\eta$ by $0.1$ increases the mean by exactly $0.0105$.” This multiplies the derivative by the increment and treats a first-order approximation as an exact finite change.
Correction: The derivative gives the local approximation. At $\eta=2$, the derivative is about $0.10499359$, so the approximation is $0.10499359(0.1)\approx0.01049936$. The exact values are $h(2.1)\approx0.89090318$ and $h(2)\approx0.88079708$; their difference is approximately $0.01010610$, or $0.01011$ to five decimal places.

A second error is to say that the log link has “constant sensitivity” without specifying the scale. Its relative change for a given predictor increment is constant across starting values, but its absolute derivative $e^\eta$ is not.

Key takeaway: To translate a small predictor change into a local mean change, multiply it by the inverse-link derivative at the starting predictor. Identity links have constant absolute sensitivity, log links have mean-dependent absolute sensitivity but a predictor-independent proportional change, and logit links are most sensitive near a mean of one half.

Check Your Understanding

Use the inverse-link derivatives and distinguish local approximations from exact changes.

For the log link, calculate the local sensitivity at $\eta=1$. What first-order mean change does $\Delta\eta=-0.02$ imply at that point?
For the logit link, calculate the mean and local sensitivity at $\eta=0$. Why is this sensitivity the largest possible value?
Under the identity link, what is the exact mean change for a predictor increment of $0.3$, wherever the mean is admissible?
Explain why a log-link predictor increment of $0.1$ gives the same proportional mean change at $\eta=0$ and $\eta=2$, but not the same absolute change.
At a finite logit predictor value, can the derivative be zero? Use $h'(\eta)=h(\eta)\{1-h(\eta)\}$ to justify your answer.