Canonical Is a Special Property, Not a Requirement
A generalized linear model (GLM) relates a response distribution to predictors through a linear predictor and a link function. Some links are canonical: they connect the conditional mean to the natural parameter of the response distribution’s exponential-family form. That connection can simplify likelihood calculations. It does not mean that every valid GLM must use that link.
The practical requirement is that the link and its inverse be compatible with the response family. For example, a Bernoulli conditional mean is a probability and must lie between 0 and 1. A link is unsuitable for an unrestricted linear predictor if its inverse can produce values outside that range. The earlier material on noncanonical links introduces their consequences; here the central question is the condition that makes a link usable in the first place.
A link must be one-to-one over the relevant mean domain so that it has an inverse there. Its inverse must return means allowed by the response distribution for the predictor values the model can produce. With an unrestricted linear predictor, that usually means checking \(g^{-1}(\eta)\) for every real \(\eta\). If the inverse is only valid over part of the real line, a model can still be used with explicit restrictions that keep all predictors in that part, but the restrictions must be justified and maintained.
For Bernoulli responses, the allowable means are probabilities in \([0,1]\); ordinary smooth links map real predictors into the interior \((0,1)\), approaching the boundaries only as limits. For Poisson responses, the mean must be positive. A Gamma mean is also positive. Gaussian means, by contrast, may be any real number. The response family determines the relevant mean space, not whether the link is canonical.
Why Canonical Links Are Convenient
Canonical links can produce compact likelihood and score equations. That convenience can be useful in derivations, computation, and interpretation. It is not a condition for defining a GLM likelihood: for any chosen link that produces valid means, the response distribution can be evaluated at those means and its likelihood can be formed.
A short score calculation makes the distinction clear. Suppose the responses are conditionally independent and belong to an exponential-family model with variance \(\operatorname{Var}(Y_i\mid x_i)=\phi V(\mu_i)\), where \(\phi\) is a dispersion parameter and \(V\) is the variance function. The contribution of observation \(i\) to the log likelihood has derivative with respect to \(\mu_i\) equal to \((Y_i-\mu_i)/[\phi V(\mu_i)]\). Applying the chain rule to \(\mu_i=g^{-1}(x_i^\mathsf{T}\beta)\) gives the following score:
The expression applies to a differentiable link whose inverse is defined at the fitted predictors. For the canonical parameterization, \(d\eta_i/d\mu_i=1/V(\mu_i)\), so \(d\mu_i/d\eta_i=V(\mu_i)\); the variance-function factors cancel. With a noncanonical link, the derivative generally remains in the score. Thus a canonical link may simplify the mathematics, but the general likelihood and score calculation still work with a valid noncanonical link.
Valid Noncanonical Links for Bernoulli Means
For a Bernoulli response, \(\mu=P(Y=1\mid x)\). The logit is the canonical link. The probit and complementary log-log links are noncanonical alternatives that also map every real predictor to a probability strictly between 0 and 1:
Here \(\Phi\) is the standard normal cumulative distribution function. For every finite real \(\eta\), \(\Phi(\eta)\) is between 0 and 1. Also \(\exp(\eta)>0\), so \(\exp[-\exp(\eta)]\) is between 0 and 1, making the complementary log-log inverse a probability in \((0,1)\). Both inverses satisfy the response-space condition for an unrestricted linear predictor. They do not, however, impose the same relationship between predictors and probabilities.
Worked Example: Comparing Three Valid Bernoulli Links
Suppose a hypothetical Bernoulli outcome has a linear predictor of \(\eta=0.8\) at a specified covariate value. Compute the fitted probability under the logit, probit, and complementary log-log links.
For the logit link, \(g^{-1}(\eta)=1/(1+\exp(-\eta))\):
For the probit link, the fitted probability is the standard normal cumulative probability:
For the complementary log-log link:
All three fitted values are valid probabilities. They differ because the links map the same linear predictor to different probabilities. This calculation compares the inverse-link transformations at a common \(\eta\); it does not imply that fitting the three models to data would give the same estimated coefficients or the same fitted probabilities.
Validity Depends on the Predictor Range
Some familiar links are valid only when the linear predictor is restricted. The identity link makes the mean equal to the predictor. For a Bernoulli response, that can be appropriate only if every predictor value lies in \([0,1]\). The fact that the identity link is valid for unrestricted Gaussian means does not make it valid for unrestricted Bernoulli probabilities.
The same issue arises for positive means. The inverse link \(g(\mu)=1/\mu\) has inverse \(g^{-1}(\eta)=1/\eta\), which requires \(\eta>0\). It cannot accept arbitrary real predictors. In contrast, the Gamma log link has inverse \(g^{-1}(\eta)=\exp(\eta)\), which is positive for every real \(\eta\). The commonly used Gamma canonical link is the inverse link, with the sign depending on the natural-parameter convention; either sign convention requires a restricted predictor range. The log link is a noncanonical alternative that can be used with unrestricted real predictors.
Worked Example: Checking Gamma Mean Predictions
Consider hypothetical Gamma conditional means modeled with \(\eta=-0.4\), \(0\), and \(0.4\). Check whether the inverse and log links return allowable positive means at each predictor value.
With the log link, \(\mu=\exp(\eta)\), giving:
All three means are positive, as required for a Gamma response, and the log-link inverse is defined for every real predictor.
With the inverse link, \(\mu=1/\eta\). At \(\eta=-0.4\), this gives \(-2.5\), which is not a valid Gamma mean. At \(\eta=0\), the inverse is undefined. At \(\eta=0.4\), it gives \(2.5\), a positive mean. Thus the inverse link can be used only if the model ensures that every fitted predictor is positive. Merely choosing a canonical link does not guarantee valid means.
When a Link Fails, and How to Diagnose It
A link can fail because its inverse is undefined at a predictor value, because it returns a mean outside the response family’s allowable space, or because it is not one-to-one over the proposed mean domain. Those are mathematical problems, not just awkward interpretations. A fitting algorithm may also have numerical difficulty near a boundary even when the inverse link is theoretically valid for all finite predictors.
For example, under a Bernoulli identity-link model with \(\eta=0.2+0.6x\), the predicted probabilities at \(x=0,1,2\) are \(0.2,0.8,1.4\). The last value cannot be a Bernoulli mean. By comparison, the probit inverse returns valid probabilities for these same predictor values:
Worked Example: Detecting Invalid Identity-Link Probabilities
For the hypothetical Bernoulli model above, calculate the identity-link fitted mean at each covariate value and assess whether the model is valid across the stated values.
At \(x=0\), \(\eta=0.2+0.6(0)=0.2\), so the fitted mean is \(0.2\), a valid probability. At \(x=1\), \(\eta=0.2+0.6(1)=0.8\), also a valid probability. At \(x=2\), \(\eta=0.2+0.6(2)=1.4\), which exceeds 1 and is invalid.
The identity-link specification therefore does not produce a valid Bernoulli mean at every stated covariate value. It is not enough that some fitted means fall between 0 and 1. One must check the complete range of predictors for which the model will be used. Replacing the identity link with a link whose inverse maps real values into \((0,1)\), such as probit or logit, avoids this particular failure.
Useful checks include calculating fitted means across the observed covariate values, examining the range of the linear predictor, and checking predictions at the covariate values where the model will be applied. With restricted-range inverse links, verify the restriction directly; do not assume that a numerical fitting routine enforced it. Also inspect for estimates near boundaries, extreme predictors, convergence warnings, and sensitivity to reasonable changes in the link. A model can meet the mathematical range condition and still fit poorly or extrapolate implausibly.
Choosing Among Valid Links
Canonical and noncanonical links offer different advantages. A canonical link may simplify score equations and estimation. A noncanonical link may provide a useful relationship between predictors and the mean or suit a scientific modeling aim. Both choices still require the response distribution to be appropriate and the link to produce allowable means. Link choice is not a substitute for checking the data, model assumptions, fit, or predictions.
| Choice | Potential advantage | Important check |
|---|---|---|
| Canonical link | Often simplifies likelihood and score calculations | Its inverse must still give allowable means for the predictor range |
| Noncanonical link with unrestricted valid inverse | Can provide a different, valid mean-predictor relationship | Check fit, interpretation, and predictions for the intended use |
| Link with restricted inverse domain | May be usable when restrictions are substantively and mathematically justified | Ensure every relevant predictor satisfies the restriction |
There is no universal rule that the canonical link is best for every purpose. Nor is a noncanonical link automatically preferable. Compare plausible choices using the scientific meaning of the model, whether their assumptions are credible, and how well they behave for the intended prediction range. Some links can be computationally harder to fit or numerically unstable in particular data sets. A valid inverse link guarantees allowable means, not good fit, correct specification, or reliable extrapolation.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions.
- For a Bernoulli response, why is the probit inverse valid with an unrestricted real linear predictor?
- Under the Gamma log link, what mean corresponds to \(\eta=-0.2\)? Is it allowable?
- A Bernoulli identity-link model gives \(\eta=0.1+0.5x\). At \(x=2\), is the fitted mean valid?
- Why might a canonical link be convenient, and why does that convenience not make it mandatory?
Solutions. (1) The inverse is \(\Phi(\eta)\). For every finite real \(\eta\), this is strictly between 0 and 1, so it is an allowable Bernoulli mean. (2) The inverse is \(\exp(-0.2)\approx 0.8187\), which is positive and therefore allowable for a Gamma mean. (3) The predictor is \(0.1+0.5(2)=1.1\). An identity link gives \(\mu=1.1\), which exceeds 1 and is not a valid Bernoulli probability. (4) A canonical link can simplify likelihood and score calculations because of its connection to the natural parameter. A valid noncanonical link still defines a GLM likelihood when it maps predictors to allowable means; the extra algebra does not invalidate the model.