Recognizing Canonical Links In Common Families

From a Response Family to Its Canonical Link

A link function maps a response mean to the scale on which a model specifies its linear predictor. Some links are especially closely tied to the response distribution: they turn the mean into the distribution’s natural parameter. These are the canonical links. Recognizing them for common response families is a matter of matching two expressions: the natural parameter in the family’s exponential form, and the mean of the response.

The three standard matches in this tutorial are Bernoulli with the logit link, Poisson with the log link, and normal with the identity link. The normal case requires care: the usual GLM convention represents the variance through a dispersion parameter, which determines how the canonical link is written.

This definition uses the common GLM exponential-dispersion convention. The representation must be valid for the family under consideration, and the mean-to-natural-parameter relationship is considered on the interior range where it is one-to-one. We will obtain that relationship explicitly for each family below. No regression coefficients or fitted model are needed to identify the link.

Why the Mean Is the Derivative of the Log-Partition Function

The function \(b(\theta)\) in exponential-family form determines the mean. The next result justifies the matching procedure. For a discrete response, the argument uses a sum; for a continuous response, it uses an integral.

Proof. Since \(f\) is a normalized density or mass function, its integral or sum over the support equals 1. Differentiate this identity with respect to \(\theta\). The stated interchange condition permits differentiation inside the integral or sum. From the exponential form, \(\partial f(y;\theta,\phi)/\partial\theta = f(y;\theta,\phi)[y-b'(\theta)]/a(\phi)\). Therefore,

For a discrete response, replace the integrals by sums. Since \(a(\phi)>0\), the last equality implies \(E[Y]=b'(\theta)\). \(\square\)

In the regular one-parameter cases here, \(b'(\theta)\) can be inverted to express \(\theta\) as a function of \(\mu\). That inverse is the canonical link. The next derivation gives all three matches directly from the family distributions.

The Three Common Family Matches

For a Bernoulli response \(Y\in\{0,1\}\) with \(P(Y=1)=p\), the mean is \(\mu=p\). Its mass function is \(f(y)=p^y(1-p)^{1-y}\). Taking logarithms and collecting terms gives \(\log f(y)=y\log\{p/(1-p)\}+\log(1-p)\). With \(\theta=\log\{p/(1-p)\}\), the second term is \(-\log(1+\exp(\theta))\). Thus \(b(\theta)=\log(1+\exp(\theta))\), and \(b'(\theta)=\exp(\theta)/[1+\exp(\theta)]=p\). Solving for \(\theta\) in terms of \(\mu=p\) gives the canonical link \(g(\mu)=\log\{\mu/(1-\mu)\}\), the logit link.

For a Poisson response with mean \(\mu>0\), the mass function is \(f(y)=\exp(-\mu)\mu^y/y!\), for nonnegative integer \(y\). Its log is \(y\log(\mu)-\mu-\log(y!)\). Set \(\theta=\log(\mu)\), so \(\mu=\exp(\theta)\). The mass function becomes \(\exp\{y\theta-\exp(\theta)-\log(y!)\}\). Hence \(b(\theta)=\exp(\theta)\), and \(b'(\theta)=\exp(\theta)=\mu\). The canonical link is therefore \(g(\mu)=\log(\mu)\), the log link.

For a normal response \(Y\sim N(\mu,\phi)\), where \(\phi>0\) is the variance, expand the log density: \(\log f(y)=-\tfrac12\log(2\pi\phi)-(y-\mu)^2/(2\phi)\). Rearranging,

This is exponential-dispersion form with \(\theta=\mu\), \(b(\theta)=\theta^2/2\), \(a(\phi)=\phi\), and \(c(y,\phi)=-y^2/(2\phi)-\tfrac12\log(2\pi\phi)\). Since \(b'(\theta)=\theta=\mu\), the canonical link is \(g(\mu)=\mu\), the identity link.

Worked Checks

A Scaling Pitfall and Why the Match Matters

A common mistake is to write the normal natural parameter as \(\mu/\sigma^2\) and conclude that the normal canonical link must be \(g(\mu)=\mu/\sigma^2\). That expression can arise if a normal density with fixed variance \(\sigma^2\) is written in a different one-parameter exponential-family convention, with the variance factor absorbed into the natural parameter. It is not the canonical-link convention used here for GLMs.

In the GLM exponential-dispersion form, the normal variance is \(a(\phi)=\phi\), outside the term \(y\theta-b(\theta)\). The density expansion above therefore gives \(\theta=\mu\) and the identity canonical link. For example, with \(\mu=2.5\) and \(\phi=9\), the GLM natural parameter is \(2.5\), not \(2.5/9\). Always inspect where the scale factor appears in the chosen exponential-family representation before identifying the canonical parameter.

Canonical links are useful because they give a direct correspondence between the mean and the response family’s natural parameter. For recognition, the procedure is concrete: express the distribution in the stated exponential form, identify \(\theta\), and express it as a function of \(\mu\). The result is a family-specific link; it is not selected by the response support alone.