Noncanonical Links and Their Consequences

Why Link Choice Matters

For a binomial response, the response family specifies the possible outcomes and their conditional distribution. A link specifies how the conditional mean is connected to predictors. Holding the response family fixed while changing the link therefore preserves the kind of outcome being modeled, but changes the form imposed on the mean as predictors vary.

The logit and probit links are both common choices for binomial responses. Each maps probabilities between zero and one to a real-valued linear predictor, but they use different transformations. Consequently, the same numerical coefficient values generally produce different probabilities. Nor is there a single constant multiplier that makes the two models identical at every predictor value. These distinctions matter when specifying a model, comparing its predictions with data, and communicating what a fitted model implies.

Definition: For independent observations \(Y_i\mid x_i\sim\operatorname{Binomial}(m_i,p_i)\), a binomial regression model specifies \(g(p_i)=x_i^\mathsf{T}\beta\). The logit link is \(g(p)=\log\{p/(1-p)\}\), with inverse \(p=\{1+\exp(-\eta)\}^{-1}\). The probit link is \(g(p)=\Phi^{-1}(p)\), with inverse \(p=\Phi(\eta)\), where \(\Phi\) is the standard normal cumulative distribution function. The binomial canonical link is the logit; the probit is noncanonical.

Here \(m_i\) is the number of trials for observation \(i\), \(p_i\) is the success probability on each trial in that observation, and \(\eta_i=x_i^\mathsf{T}\beta\) is its linear predictor. A Bernoulli response is the special case \(m_i=1\). The binomial model assumes that, conditional on the predictors, the trials within an observation share the stated success probability and are independent, and that observations are independent of one another. The regression assumption is that the selected link of \(p_i\) has the specified linear form in the predictors.

What Is Canonical, and What Changes?

In the binomial exponential-family representation, the natural parameter for the success probability is \(\log\{p/(1-p)\}\). A link is canonical when it makes the linear predictor equal to this natural parameter (under the usual binomial parameterization). Thus, using the logit means that \(x_i^\mathsf{T}\beta\) is the binomial natural parameter. With the probit link, the linear predictor instead equals \(\Phi^{-1}(p_i)\); it is not the natural parameter.

This distinction concerns the parameterization, not the response distribution. Both models still describe \(Y_i\) as binomial with mean \(m_i p_i\) and variance \(m_i p_i(1-p_i)\), conditional on the predictors. Both produce probabilities strictly between zero and one for every finite linear predictor. What differs is the assumption about how those probabilities change with the predictors: the logit of the probability is linear in one model, and the standard-normal quantile of the probability is linear in the other.

Canonical links have a particularly direct relationship to the exponential-family natural parameter and often give convenient likelihood calculations. That convenience does not make a noncanonical link invalid. A probit model is a legitimate binomial regression model when its assumptions and probability predictions are appropriate for the application. It does, however, impose a different mean structure, so model fit and conclusions can differ.

Worked Example: Same Coefficients, Different Probabilities

Suppose \(x\) is a centered, unitless predictor and, purely for illustration, both candidate models use \(\beta_0=-1\) and \(\beta_1=0.8\), so \(\eta=-1+0.8x\). The binomial response assumptions are the same for both candidates; only the link changes. Calculate the probabilities at \(x=0,1,2\).

At \(x=0\), \(\eta=-1\). The logit model gives \(p=1/(1+\exp(1))\approx0.2689\), while the probit model gives \(p=\Phi(-1)\approx0.1587\).

At \(x=1\), \(\eta=-0.2\). The logit model gives \(p=1/(1+\exp(0.2))\approx0.4502\), while the probit model gives \(p=\Phi(-0.2)\approx0.4207\).

At \(x=2\), \(\eta=0.6\). The logit model gives \(p=1/(1+\exp(-0.6))\approx0.6457\), while the probit model gives \(p=\Phi(0.6)\approx0.7257\).

The probabilities differ despite identical numerical coefficients. In particular, the logit probability is higher at \(x=0\), whereas the probit probability is higher at \(x=2\). This example is a direct comparison of parameterizations, not evidence that either set of coefficients is a fitted result.

A Local Conversion Is Not a Global Conversion

Because both links are increasing, their linear predictors can be related at any one probability. But the relation between the predictors changes with the probability. A useful way to see this is to compare their local rates of change around \(p=0.5\), where both inverse links give a probability of one-half at \(\eta=0\).

Let \(p_L(\eta)=\{1+\exp(-\eta)\}^{-1}\) be the inverse-logit probability. Differentiating gives \(dp_L/d\eta=p_L(1-p_L)\), which equals \(0.25\) at \(\eta=0\). For the probit inverse \(p_P(\eta)=\Phi(\eta)\), differentiation gives \(dp_P/d\eta=\phi(\eta)\), where \(\phi\) is the standard normal density; at zero, \(\phi(0)=1/\sqrt{2\pi}\approx0.3989\). To match the local probability slopes at one-half, a probit predictor must therefore change at about \(0.25/0.3989\) times the logit predictor’s rate.

\left.\frac{dp_L}{d\eta}\right|_{\eta=0}=0.25,\qquad \left.\frac{dp_P}{d\eta}\right|_{\eta=0}=\phi(0)\approx0.3989,\qquad \frac{0.25}{0.3989}\approx0.6267

This ratio is a local approximation, not a universal conversion for coefficients. More generally, if \(\eta_L\) is a logit predictor and the models give the same probability \(p\), then \(\eta_P=\Phi^{-1}\{(1+\exp(-\eta_L))^{-1}\}\). The derivative of this transformation is \(p(1-p)/\phi\{\Phi^{-1}(p)\}\), which varies with \(p\). Therefore, a constant rescaling cannot make the two linear predictors agree across the whole probability range.

Worked Example: Matching the Slope Near One-Half

Suppose a logit model has \(\eta_L=0.8x\), with no intercept, so both models have \(p=0.5\) at \(x=0\). To make the probit model’s probability change at approximately the same rate at \(x=0\), set \(\eta_P=b x\) and choose \(b\) using the derivative ratio.

The logit probability slope with respect to \(x\) at zero is \(0.8(0.25)=0.2000\). The probit slope is \(b\phi(0)\), so equating the slopes gives \(b(0.3989)=0.2000\), or \(b\approx0.5014\). Equivalently, \(b=0.8(0.6267)\approx0.5014\).

At \(x=1\), the logit model gives \(p_L=1/(1+\exp(-0.8))\approx0.6900\), and the locally matched probit model gives \(p_P=\Phi(0.5014)\approx0.6920\). The predictions are close. At \(x=2\), however, they give \(p_L=1/(1+\exp(-1.6))\approx0.8320\) and \(p_P=\Phi(1.0028)\approx0.8420\). The match is not exact, and the discrepancy can change as \(x\) moves farther from the point of local matching.

The derivative calculation justifies using a rescaling as a rough comparison near a central probability. It does not justify treating logit and probit coefficients as interchangeable throughout a data range.

Comparing Candidate Models with Binomial Data

A link comparison should hold the response definition, trials, and predictors fixed. For each candidate, calculate probabilities from its own link and linear predictor, then evaluate how well those probabilities account for the observed binomial counts. The binomial log-likelihood contribution for an observation with \(y_i\) successes in \(m_i\) trials is

\ell_i=\log {m_i\choose y_i}+y_i\log(p_i)+(m_i-y_i)\log(1-p_i).

The binomial coefficient is identical across link candidates for the same observations. Differences in their log-likelihoods therefore come from the probability terms. If coefficients are estimated separately for each link, the maximized likelihoods can be compared as one part of model assessment; a higher maximized likelihood alone does not establish that the link is the best choice. The fits must also be checked for adequate predictions, and comparisons should account for model complexity when it differs.

Worked Example: Comparing Binomial Likelihood Contributions

Consider hypothetical independent groups, each with \(m=10\) trials, at predictor values \(x=0,1,2\). Suppose the observed success counts are \(y=3,5,7\). Compare the candidate probabilities from the first worked example, which used \(\eta=-1+0.8x\) for both links. These coefficients are held fixed for the calculation; they are not claimed to be fitted estimates.

For the logit candidate, the probabilities are approximately \(0.2689,0.4502,0.6457\). Omitting the common binomial coefficients, the three log-likelihood contributions are \(3\log(0.2689)+7\log(0.7311)\approx-6.133\), \(5\log(0.4502)+5\log(0.5498)\approx-6.981\), and \(7\log(0.6457)+3\log(0.3543)\approx-6.175\). Their sum is approximately \(-19.29\).

For the probit candidate, the probabilities are approximately \(0.1587,0.4207,0.7257\). The corresponding contributions are \(3\log(0.1587)+7\log(0.8413)\approx-6.732\), \(5\log(0.4207)+5\log(0.5793)\approx-7.058\), and \(7\log(0.7257)+3\log(0.2743)\approx-6.127\). Their sum is approximately \(-19.92\).

For these particular fixed coefficient values and observations, the logit candidate has the larger log-likelihood (by about \(0.63\)). Because the coefficients were not separately estimated to maximize each likelihood, this calculation does not show which fitted model would be preferred. It illustrates a fair link comparison’s basic ingredient: evaluate both using the same binomial outcomes and likelihood definition.

Assumptions, Diagnostics, and Limitations

Before interpreting either model, check that the response is genuinely a binary outcome or a count of successes out of known trials, and that the binomial trial assumptions are reasonable. The predictor structure must also be plausible on the selected link scale: for example, a model with a single linear predictor term assumes that the logit or probit of the probability changes linearly with that predictor. A link choice does not repair dependence between observations, unmodeled variation in success probabilities, or an unsuitable predictor structure.

Useful checks include comparing observed success proportions with fitted probabilities across predictor ranges, inspecting residual patterns, and checking whether groups with similar fitted probabilities show more variation than the binomial model allows. Repeated or clustered observations require attention to their dependence; treating correlated outcomes as independent can make uncertainty estimates unreliable. Extreme observed proportions can also provide limited information about how either link behaves near zero or one, so predictions in sparsely observed regions deserve particular caution.

The logit and probit are alternatives, not transformations that guarantee identical fitted curves. A canonical link can be convenient for likelihood-based calculations, while a noncanonical link may be selected for modeling or scientific reasons. The choice should be checked against the data and the intended prediction task. It does not change the binomial support or conditional variance formula, but it can change fitted probabilities, parameter estimates, and resulting uncertainty calculations.

A Common Mistake

Common mistake: “Both models have a linear predictor, so the same coefficient values must give the same probability curve.” This is false: the logit inverse is the logistic cumulative distribution function, whereas the probit inverse is the standard normal cumulative distribution function. For example, at \(\eta=-1\), they give approximately \(0.2689\) and \(0.1587\), respectively. Compare the predicted probabilities or fitted likelihoods, rather than assuming equal coefficients mean equal predictions.

Key takeaway: For a binomial response, logit is canonical and probit is noncanonical. Both preserve the binomial response family, but they impose different linear structures on the probability. Their coefficients are not generally interchangeable, and their predictions should be compared on the response scale as well as through model fit.

Check Your Understanding

Answer each question, then check the brief solutions below.

Which link is canonical for a binomial response, and what quantity does its linear predictor equal?
Under a probit link, calculate the probability for \(\eta=0.5\), using \(\Phi(0.5)\approx0.6915\).
Near \(p=0.5\), what probit slope approximately matches a logit slope of \(0.6\) with respect to the same predictor?
Two candidate links are evaluated on the same binomial observations, with the same number of coefficients. What does a larger maximized log-likelihood tell you, and what does it not establish by itself?

Solutions. (1) The logit is canonical, and its linear predictor equals the binomial natural parameter \(\log\{p/(1-p)\}\). (2) \(p=\Phi(0.5)\approx0.6915\). (3) Multiply by the local ratio \(0.6267\): \(0.6(0.6267)\approx0.3760\). (4) It indicates a better maximized likelihood for those data among the candidates, but does not by itself establish that the model is adequate or that its link is preferable for the scientific or prediction goal.