Why Canonical Links Matter
A generalized linear model separates the response distribution from the way predictors describe its conditional mean. The link function is the connection between those pieces. Some links are called canonical because they connect the conditional mean directly to the natural parameter of the response distribution. This is more than a naming convention: with a canonical link, the linear predictor is itself the natural parameter, which simplifies the likelihood score and often the algebra used in estimation.
A canonical link is not the only link a generalized linear model can use, and choosing one does not guarantee a good fit or successful numerical optimization. Its defining feature is a precise relationship to the exponential-family representation, not a general claim that it is always the best modeling choice.
Definition and Conditions
Consider a one-parameter exponential-family model for a response \(Y\), conditional on predictors \(x\), in the standard scalar-response form
Here \(\theta\) is the natural parameter, \(\phi\) is a dispersion parameter, and \(a(\phi)\) is the scale factor in the exponent. The response support is assumed not to depend on \(\theta\), and the distribution must be regular enough that the cumulant function \(b\) can be differentiated on the relevant parameter domain. In this form, the conditional mean is \(\mu=E[Y\mid x]=b'(\theta)\).
The definition presumes that the natural parameter multiplies \(y\) in the exponent. More generally, an exponential-family representation may have a sufficient statistic \(T(y)\) in place of \(y\). In the standard GLM setup above, \(T(y)=y\), so the parameter paired with the response is the one that the canonical link maps to. If a distribution is written in a different parameterization, identify its natural parameter in the exponent before naming its canonical link.
A GLM can use a different valid link \(g\), provided it maps the allowed means to a suitable predictor scale. In that case, the linear predictor is \(g(\mu)\), but it is generally not the natural parameter \(\theta\). The defining distinction is therefore:
| Feature | Canonical link | Other valid link |
|---|---|---|
| Predictor scale | \(\eta=\theta\) | \(\eta=g(\mu)\), generally different from \(\theta\) |
| Connection to the exponential-family form | Directly uses the natural parameter | Does not have to use the natural parameter |
| Likelihood score | Has a direct residual form | Includes a derivative factor relating \(\theta\) to \(\eta\) |
Why the Score Simplifies
Suppose \(Y_1,\ldots,Y_n\) are conditionally independent given predictor vectors \(x_1,\ldots,x_n\), share dispersion \(\phi\), and follow the stated exponential-family form. Let \(\mu_i=E[Y_i\mid x_i]\), \(\theta_i\) be the natural parameter for observation \(i\), and \(\eta_i=x_i^\mathsf{T}\beta\). The model is assumed to have no additional parameter-dependent terms omitted from this likelihood representation.
For the canonical link, \(\theta_i=\eta_i=x_i^\mathsf{T}\beta\). The log-likelihood, up to terms not depending on \(\beta\), is
Differentiating with respect to \(\beta\) yields
The last equality follows from the mean identity \(b'(\theta_i)=\mu_i\). This shows exactly what the canonical link removes: there is no extra derivative converting from the linear predictor to the natural parameter.
Conditional independence makes the joint log-likelihood a sum of the individual log-likelihood contributions.
Set \(\theta_i=\eta_i=x_i^\mathsf{T}\beta\), so the natural parameter is linear in the regression coefficients.
The derivative of \(y_i\theta_i-b(\theta_i)\) with respect to \(\theta_i\) is \(y_i-b'(\theta_i)=y_i-\mu_i\).
Because \(\partial\eta_i/\partial\beta=x_i\), each score contribution is \(x_i(y_i-\mu_i)/a(\phi)\).
For comparison, if \(\eta_i=g(\mu_i)\) is a noncanonical link, then \(\theta_i\) is generally a function of \(\eta_i\). Where this relationship is differentiable, the corresponding score has the form
The extra factor is the chain-rule derivative. The canonical link makes \(d\theta_i/d\eta_i=1\), giving the simpler score. This derivation concerns the likelihood score; it does not say that a fitted model has zero residuals, nor that its coefficients can generally be found by solving a closed-form equation.
Worked Examples
Worked Example: A Bernoulli Model and Its Canonical Predictor
A hypothetical model describes whether a policyholder files a claim during a specified period. Conditional on a covariate \(z\), \(Y\) is Bernoulli with success probability \(p\). The model uses \(\eta=\beta_0+\beta_1z\), with \(\beta_0=\log 3\) and \(\beta_1=-\log 3\). For observations at \(z=0,1,2\), the recorded outcomes are \(1,0,0\). Find the fitted means, identify the canonical-link relationship, and compute the score vector at these coefficient values.
Method. A Bernoulli response has probability mass function \(p^y(1-p)^{1-y}\). Its exponent can be written \(y\log\{p/(1-p)\}+\log(1-p)\), so its natural parameter is \(\theta=\log\{p/(1-p)\}\). The canonical predictor sets \(\eta=\theta\), hence \(p=1/(1+\exp(-\eta))\).
At \(z=0\), \(\eta=\log 3\), so \(p=3/(1+3)=0.75\). At \(z=1\), \(\eta=\log 3-\log 3=0\), so \(p=1/(1+1)=0.50\). At \(z=2\), \(\eta=\log 3-2\log 3=-\log 3\), so \(p=(1/3)/(1+1/3)=0.25\). Thus the fitted means are \(0.75,0.50,0.25\).
For the Bernoulli model, \(a(\phi)=1\). The residuals \(y_i-\mu_i\) are \(1-0.75=0.25\), \(0-0.50=-0.50\), and \(0-0.25=-0.25\). With predictor vectors \(x_i=(1,z_i)^\mathsf{T}\), the intercept score is \(0.25-0.50-0.25=-0.50\). The slope score is \(0(0.25)+1(-0.50)+2(-0.25)=-1.00\). The score vector is therefore \((-0.50,-1.00)^\mathsf{T}\).
Check. The probabilities are all between zero and one, as required for Bernoulli means. The canonical score matches the direct residual calculation. Common mistake avoided: treating \(\eta\) as the probability \(p\); here \(\eta\) is the log-odds, not the response mean.
Worked Example: A Poisson Model and the Residual Score
For a hypothetical service portfolio, let \(Y_i\) be the number of reported incidents in a fixed one-year period for exposure profiles \(z=0,1,2\). Assume independent Poisson responses with means \(\mu_i\), and use the canonical predictor \(\eta_i=\beta_0+\beta_1z_i\), where \(\beta_0=\log 2\) and \(\beta_1=\log 2\). The observed counts are \(1,5,6\). Compute the means and the score vector.
Method. For a Poisson response with mean \(\mu\), the probability mass function is \(\exp(-\mu)\mu^y/y!\). Its exponential-family exponent contains \(y\log\mu-\mu\), so \(\theta=\log\mu\) and \(b(\theta)=\exp(\theta)\). The canonical predictor is \(\eta=\theta=\log\mu\), which gives \(\mu=\exp(\eta)\).
At \(z=0\), \(\eta=\log 2\), so \(\mu_1=2\). At \(z=1\), \(\eta=2\log 2=\log 4\), so \(\mu_2=4\). At \(z=2\), \(\eta=3\log 2=\log 8\), so \(\mu_3=8\). The residuals are \(1-2=-1\), \(5-4=1\), and \(6-8=-2\).
Here \(a(\phi)=1\). The intercept score is the sum of residuals: \(-1+1-2=-2\). The slope score is the predictor-weighted sum: \(0(-1)+1(1)+2(-2)=-3\). Thus the score vector is \((-2,-3)^\mathsf{T}\).
Check. The fitted means are positive, as required for Poisson means. Substituting them into the score formula reproduces both components. Common mistake avoided: using the count \(y_i\) in place of its fitted mean in the predictor; the model specifies \(\log\mu_i=x_i^\mathsf{T}\beta\), not \(\log y_i=x_i^\mathsf{T}\beta\).
Worked Example: A Normal Model and the Role of Dispersion
Suppose independent measurements \(Y_i\), conditional on \(z_i=0,1,2\), follow normal distributions with means \(\mu_i=\beta_0+\beta_1z_i\) and known variance \(\sigma^2=4\). Use \(\beta_0=2\), \(\beta_1=1\), and observed values \(1,4,2\). Verify the canonical-link relationship and compute the score vector.
Method. The normal density with mean \(\mu\) and variance \(\sigma^2\) has exponent \(\{y\mu-\mu^2/2\}/\sigma^2\), apart from terms not depending on \(\mu\). Thus \(\theta=\mu\), \(b(\theta)=\theta^2/2\), and \(a(\phi)=\sigma^2=4\). Since \(b'(\theta)=\theta=\mu\), the canonical link is \(g(\mu)=\mu\), the identity link. Therefore \(\eta=\mu=x^\mathsf{T}\beta\).
The fitted means at \(z=0,1,2\) are respectively \(2+1(0)=2\), \(2+1(1)=3\), and \(2+1(2)=4\). The residuals are \(1-2=-1\), \(4-3=1\), and \(2-4=-2\). The intercept score is \((-1+1-2)/4=-2/4=-0.50\). The slope score is \(\{0(-1)+1(1)+2(-2)\}/4=(1-4)/4=-0.75\). The score vector is \((-0.50,-0.75)^\mathsf{T}\).
Check. The dispersion divides the residual score by \(4\); omitting it would multiply both score components by \(4\). Common mistake avoided: confusing the variance \(\sigma^2=4\) with the standard deviation \(\sigma=2\), or incorrectly treating the canonical link as depending on the numerical value of the variance in this parameterization.
Interpretation and Common Traps
The practical benefit of the canonical choice is clearest in the likelihood equations: the score is a weighted sum of observed-minus-fitted responses, without a further derivative of the natural parameter with respect to the predictor. This structure is useful in deriving and understanding estimation procedures. It does not imply that the coefficients are obtained by an ordinary least-squares calculation; the fitted means still depend on \(\beta\), and the score equations may require numerical solution.
- Do not equate “canonical” with “the only valid link.” A noncanonical link can define a valid GLM when it maps the allowed conditional means appropriately.
- Do not identify the natural parameter by its name alone. Read the coefficient of the response statistic in the exponential form, including any scale factor convention.
- Do not confuse mean and natural-parameter scales. The canonical link maps \(\mu\) to \(\theta\); it need not be the identity function.
- Do not drop the dispersion factor. In the stated likelihood form, the score includes \(1/a(\phi)\). For Bernoulli and Poisson models it is one; for the normal example it is \(1/4\).
- Do not assume the inverse exists everywhere. The natural parameter and mean have allowable domains. The canonical mapping is used only on the corresponding mean range.
Check Your Understanding
For each problem, identify the relevant natural-parameter relationship or apply the canonical score carefully.
- A Bernoulli model has fitted mean \(p=0.20\). What is its canonical predictor value? Give the answer to four decimals. (Answer: \(\log(0.20/0.80)=-1.3863\))
- A Poisson model has canonical predictor \(\eta=\log 5\). Find the conditional mean. Then state whether a fitted mean of \(-1\) could be consistent with this model. (Answer: \(\mu=5\); no, a Poisson mean must be positive)
- For two independent Poisson observations with predictor vectors \(x_1=(1,0)^\mathsf{T}\), \(x_2=(1,1)^\mathsf{T}\), observed counts \(y_1=2,y_2=1\), and fitted means \(\mu_1=1,\mu_2=3\), calculate the canonical score vector. (Answer: \((2-1,\,0(2-1)+1(1-3))^\mathsf{T}=(1,-2)^\mathsf{T}\))
- A normal response has conditional variance \(9\) and mean \(\mu\), with exponent parameterized as \(\{y\mu-\mu^2/2\}/9\). Identify \(\theta\), \(b(\theta)\), and the canonical link. (Answer: \(\theta=\mu\), \(b(\theta)=\theta^2/2\), and \(g(\mu)=\mu\))
- For a Bernoulli model at a given coefficient vector, three observations have predictor vectors \((1,0)^\mathsf{T},(1,1)^\mathsf{T},(1,2)^\mathsf{T}\), fitted means \(0.50,0.75,0.25\), and outcomes \(1,1,0\). Compute the score vector under the canonical link. (Answer: \((0.50+0.25-0.25,\;0(0.50)+1(0.25)+2(-0.25))^\mathsf{T}=(0.50,-0.25)^\mathsf{T}\))