Why Compare Alternative Links on the Same Data?
For a binary response, a generalized linear model relates a covariate to a success probability through a link function. The link determines how a linear predictor becomes a probability. Consequently, two models can describe the same observed risks while assigning different meanings to their coefficients and behaving differently at covariate values not represented in the data.
This comparison is easiest to see in a small dataset where the covariate has two values. With an intercept and one binary covariate, the model can fit a separate probability at each value. If both observed proportions lie strictly between zero and one, all four links considered here can reproduce them exactly. That makes the example useful for distinguishing what the data determine—the fitted risks at those two covariate values—from what depends on the chosen link.
The corresponding inverse links are \(\Phi(\eta)\), \(1-\exp\{-\exp(\eta)\}\), \(\exp(\eta)\), and \(\tfrac12+\arctan(\eta)/\pi\), respectively. The probit, complementary log-log, and cauchit inverse links return probabilities strictly between zero and one for every finite \(\eta\). The log inverse link returns a valid probability when \(\eta\leq 0\); it returns a probability strictly between zero and one only when \(\eta<0\). A positive log-link predictor gives a value greater than one, which is not a valid probability.
Worked Dataset and Saturated Fits
Suppose 20 independent Bernoulli records have a binary covariate \(x\). Ten records have \(x=0\), with two responses equal to 1 and eight equal to 0. Ten have \(x=1\), with six responses equal to 1 and four equal to 0. Thus the observed success proportions are \(0.2\) at \(x=0\) and \(0.6\) at \(x=1\). The table summarizes the individual records; the modeling target is the probability for an individual with the stated covariate.
| Covariate value | Number of records | Number with \(Y=1\) | Observed proportion |
|---|---|---|---|
| \(x=0\) | 10 | 2 | 0.2 |
| \(x=1\) | 10 | 6 | 0.6 |
Worked Example: Fitting All Four Links
For each link, take the inverse link of the observed proportion at \(x=0\) to obtain \(\hat\beta_0\), and the difference between the two inverse-linked proportions to obtain \(\hat\beta_1\). This works because \(\eta(0)=\beta_0\) and \(\eta(1)=\beta_0+\beta_1\).
Probit. The fitted probabilities give \( \hat\beta_0=\Phi^{-1}(0.2)\approx-0.84162 \) and \( \hat\beta_1=\Phi^{-1}(0.6)-\Phi^{-1}(0.2)\approx0.25335-(-0.84162)=1.09497 \).
Complementary log-log. At the two covariate values, the link values are \( \log\{-\log(0.8)\}\approx-1.49994 \) and \( \log\{-\log(0.4)\}\approx-0.08742 \). Thus \(\hat\beta_0\approx-1.49994\) and \(\hat\beta_1\approx-0.08742-(-1.49994)=1.41252\).
Log. The link values are \(\log(0.2)\approx-1.60944\) and \(\log(0.6)\approx-0.51083\), so \(\hat\beta_0\approx-1.60944\) and \(\hat\beta_1\approx-0.51083-(-1.60944)=1.09861\). Both predictors are negative at the observed covariate values.
Cauchit. The inverse link on the link scale is \(g(p)=\tan\{\pi(p-\tfrac12)\}\). Hence \(g(0.2)=\tan(-0.3\pi)\approx-1.37638\) and \(g(0.6)=\tan(0.1\pi)\approx0.32492\). Therefore \(\hat\beta_0\approx-1.37638\) and \(\hat\beta_1\approx0.32492-(-1.37638)=1.70130\).
In every case, applying the inverse link at \(x=0\) and \(x=1\) recovers \(0.2\) and \(0.6\). These calculations give maximum-likelihood estimates, as justified below; they are not merely a way to match the sample proportions without regard to the likelihood.
To prove the result, let \(s_j\) be the number of successes among \(m_j\) individual records at covariate value \(j\), for \(j=0,1\). The Bernoulli log-likelihood contribution from that set of records, as a function of its common probability \(p_j\), is
Differentiating gives
The derivative is zero at \(p_j=s_j/m_j\), positive below that value, and negative above it. Also,
Thus this is the unique maximum for that covariate value. Since the two probabilities can be chosen independently by the intercept and slope, maximizing the full likelihood amounts to maximizing each contribution separately. In this dataset, the maxima are \(p_0=2/10=0.2\) and \(p_1=6/10=0.6\).
It remains to check that the link parameterization can attain those maxima. The probit, complementary log-log, and cauchit links each map \((0,1)\) onto \(\mathbb{R}\). For the log link, the range on \((0,1)\) is \((-\infty,0)\), not \(\mathbb{R}\). Here both observed proportions are in \((0,1)\), so their log-link predictors are negative. For any one of the four links, setting \(\beta_0=g(p_0)\) and \(\beta_1=g(p_1)-g(p_0)\) therefore attains both fitted probabilities. This completes the justification. \(\square\)
What the Coefficients and Risks Say
The estimates differ across links, but the coefficient sizes cannot be compared as if they were on a common scale. Each coefficient measures a change in a different transformed probability. In this example, the covariate is binary, so the directly interpretable probability-scale contrast at the observed values is the risk difference \(0.6-0.2=0.4\). It is the same in all four fits because all four reproduce the same two fitted risks.
Worked Example: Comparing Link-Scale and Probability-Scale Effects
For the dataset above, the estimated slope coefficients are approximately \(1.095\) (probit), \(1.413\) (complementary log-log), \(1.099\) (log), and \(1.701\) (cauchit). Their numerical differences do not imply different fitted risks at \(x=0\) and \(x=1\): all give the same fitted risk change, \(0.4\).
A derivative illustrates why a slope is not itself a risk difference. If \(p(x)=G(\beta_0+\beta_1x)\), then \(dp(x)/dx=G'(\eta(x))\beta_1\), where \(G\) is the inverse link. At \(x=0\), the probit derivative is \(\phi(-0.84162)(1.09497)\approx0.2800(1.09497)\approx0.307\), where \(\phi\) is the standard normal density.
For the complementary log-log fit, \(G'(\eta)=\exp(\eta)\exp\{-\exp(\eta)\}\). At \(x=0\), \(\exp(\eta)=0.22314\) and \(\exp\{-\exp(\eta)\}=0.8\), so the derivative is approximately \(0.22314(0.8)(1.41252)=0.252\). For the log fit, \(G'(\eta)=\exp(\eta)=p\), giving \(0.2(1.09861)=0.220\). For the cauchit fit, \(G'(\eta)=1/\{\pi(1+\eta^2)\}\), giving approximately \(1.70130/\{\pi(1+1.37638^2)\}=0.187\).
These are local derivatives, not the discrete change from \(x=0\) to \(x=1\). In particular, \(x\) only takes the values zero and one in this dataset, so interpreting a derivative as an observed-unit risk difference would be inappropriate. The finite risk difference, \(0.4\), is the appropriate contrast for these two covariate values.
The equality of the fitted risks here does not show that the links are interchangeable in general. This model has two fitted probabilities and two coefficients: it is saturated for a binary covariate. With a continuous covariate, or with additional covariate patterns but a more restricted linear predictor, alternative links need not fit the same risks. Their probability curves can differ in curvature and asymmetry even when their coefficients have been adjusted to describe similar regions of the data.
Prediction Validity and Extrapolation
A model may fit every observed covariate value and still behave poorly elsewhere. This is especially important for the log link, whose inverse link is not automatically confined to the probability interval. The next calculation extends the fitted linear predictors to \(x=2\), even though the dataset contains no such records. These values are illustrations of model behavior, not evidence that any of the extrapolations are reliable.
Worked Example: Checking a Prediction Beyond the Data
For the log-link fit, the predictor at \(x=2\) is \(\hat\eta(2)=-1.60944+2(1.09861)=0.58778\). The inverse link gives \(\hat p(2)=\exp(0.58778)=1.8\), which is not a valid probability. At \(\eta=0\), the log inverse link gives \(p=1\), a valid boundary probability; for a probability strictly between zero and one, \(\eta<0\) is required. It is only when \(\eta>0\) that the log inverse link exceeds one.
For comparison, the other fitted predictors at \(x=2\) are approximately \(1.34832\) (probit), \(1.32510\) (complementary log-log), and \(2.02622\) (cauchit). Their inverse links give probabilities of approximately \(0.911\), \(0.977\), and \(0.854\), respectively. These are in \((0,1)\) because those inverse links have that range for every finite predictor. But being a valid probability does not make an extrapolated prediction trustworthy: the data contain no observations at \(x=2\) with which to assess it.
Thus prediction checks have two parts. First, determine whether the inverse link returns a valid probability at the proposed predictor value. Second, ask whether the covariate value and the assumed link behavior are supported by the data and scientific context. The first check is essential for a log-link model; neither check can be skipped merely because the fitted risks within the observed range look sensible.
How to Compare Alternative Links Responsibly
For this dataset, the maximized likelihood is identical across the four models: each fits risks of \(0.2\) and \(0.6\) at the two observed covariate values. That equality follows from the saturated parameterization, not from evidence that all four links describe the underlying response process equally well. There is no difference in fitted likelihood here that could choose one link over another.
A practical comparison should keep the target of interpretation clear. If the scientific question concerns risks or risk differences at the observed covariate values, report those quantities directly. If the question concerns how a covariate acts on a particular transformed scale, state the link and interpret coefficients on that scale. For predictions, inspect the relevant covariate range and check the probability constraints, especially under the log link. With a less saturated model, compare the fitted risk patterns over the range of interest rather than comparing raw coefficient magnitudes across links.
A common mistake is to rank the models by the size of their slope estimates and conclude that the cauchit fit has the largest covariate effect because \(1.701\) is larger than the other slopes. That reasoning compares quantities on different link scales. The correction is to define the effect of interest on a common scale—for example, a predicted risk difference between specified covariate values—and calculate that quantity from each fitted model. In this saturated example, that contrast is \(0.4\) in every model. In other datasets or less saturated fits, the corresponding contrasts can differ and should be compared at clearly stated covariate values.
Check Your Understanding
Use the fitted dataset and link definitions above to answer the following.
- Why do all four models fit risks of \(0.2\) and \(0.6\) at \(x=0\) and \(x=1\)?
- What probability-scale effect is directly represented by moving from \(x=0\) to \(x=1\) in this dataset?
- Why is it not valid to interpret the largest of the four estimated slopes as the largest probability-scale effect?
- For a log-link model, what conditions on \(\eta\) give a valid probability in \([0,1]\), and what condition gives a probability strictly between zero and one?
- Why do valid inverse-link predictions at \(x=2\) not establish that those extrapolations are reliable?