Deriving Canonical Links

Why Derive a Canonical Link?

A generalized linear model (GLM) specifies a relationship between a response mean and a linear predictor. A link function $g$ expresses that relationship as $g(\mu)=\eta$, where $\mu$ is the mean on the response scale and $\eta$ is the linear predictor. Some links are chosen for interpretation or practical reasons; a canonical link is the one that makes the linear predictor equal to the natural parameter in the response distribution’s exponential-family representation.

Deriving the link rather than memorizing a list makes it easier to check which mean is being modeled, especially for grouped binomial data. It also prevents a subtle but important error: the coefficient of the observed response in a log density is not necessarily the natural parameter. The dispersion factor must be accounted for first.

Definition: Suppose a response distribution has exponential-family form $$ f(y;\theta,\phi)=\exp\left\{\frac{y\theta-b(\theta)}{a(\phi)}+c(y,\phi)\right\}. $$ Here $\theta$ is the natural parameter, $\phi$ is a dispersion parameter, and $a(\phi)$ is the factor scaling the bracket. If the GLM uses linear predictor $\eta$, its canonical link is the link $g$ for which $g(\mu)=\theta=\eta$, with $\mu=E[Y]$. The function $b$, and sometimes the model’s known trial count, determine the relationship between $\theta$ and $\mu$.

The defining match is between $\theta$ and $\eta$, not between $\eta$ and an arbitrary coefficient extracted from the log likelihood. If the coefficient of $y$ in the fully expanded log density is $q$, comparison with the displayed form gives $q=\theta/a(\phi)$, so $\theta=a(\phi)q$. For families with $a(\phi)=1$, the coefficient happens to equal the natural parameter. For a Gaussian response, it generally does not.

A Derivation Strategy

The response mean is obtained from the exponential-family function $b$: under the stated parameterization, $E[Y]=b'(\theta)$, where $b'$ denotes the derivative of $b$. Thus, to derive a canonical link, identify $\theta$, express the mean in terms of $\theta$, and then set $\eta=\theta$. The result is a link written as a function of the mean. This procedure is valid for the chosen distribution and parameterization; it does not imply that a canonical link is the only valid link for that family.

Write the log likelihood or density.
Keep any dispersion or scale factor visible rather than absorbing it into the response coefficient.

Match the exponential-family form.
Factor the log likelihood into $[y\theta-b(\theta)]/a(\phi)$ plus terms not involving $\theta$. Read $\theta$ from inside the bracket.

Relate the mean and natural parameter.
Use $\mu=b'(\theta)$, or the corresponding distributional mean, and solve for $\theta$ in terms of $\mu$.

Set the natural parameter equal to the predictor.
Use $\eta=\theta$. The resulting expression for $\eta$ is the canonical link $g(\mu)$.

Bernoulli and Binomial Responses

For a Bernoulli response $Y$, assume $Y$ takes values 0 and 1, with $P(Y=1)=p$ and $P(Y=0)=1-p$. Its mean is $\mu=p$. The probability mass function is $p^y(1-p)^{1-y}$. Taking logarithms and collecting terms gives

\log f(y)=y\log\left(\frac{p}{1-p}\right)+\log(1-p).

Set $\theta=\log\{p/(1-p)\}$. Since $p=e^\theta/(1+e^\theta)$, the remaining term is $\log(1-p)=-\log(1+e^\theta)$. Therefore the exponential-family form has $a(\phi)=1$, $b(\theta)=\log(1+e^\theta)$, and $\theta=\log\{p/(1-p)\}$. Equating $\eta$ with $\theta$ gives the canonical link for a Bernoulli mean.

g(\mu)=\log\left(\frac{\mu}{1-\mu}\right), \qquad 0<\mu<1.

For a grouped binomial response $Y$, assume there are $m$ independent Bernoulli trials, each with the same success probability $p$, and $Y$ is the number of successes. Its mean is $\mu=mp$. The probability mass function includes the binomial coefficient $\binom{m}{y}$, which does not involve $p$. The terms involving $p$ can be written

y\log\left(\frac{p}{1-p}\right)+m\log(1-p) = y\theta-m\log(1+e^\theta).

Thus $\theta=\log\{p/(1-p)\}$, and $b(\theta)=m\log(1+e^\theta)$. Because the modeled mean count is $\mu=mp$, its canonical link is $g(\mu)=\log\{\mu/(m-\mu)\}$. If the modeled response is instead the success proportion $Y/m$, its mean is $p$, and the canonical link for that mean is $\log\{p/(1-p)\}$. Keep the modeled response and its mean explicit: the formula in terms of a count mean and the formula in terms of a proportion mean use different quantities.

Worked Example: Bernoulli Mean and Natural Parameter

A hypothetical claim-review indicator $Y$ is 1 if a file is referred for a second review and 0 otherwise. Assume $Y$ is Bernoulli with $P(Y=1)=p=0.20$. Derive the canonical link value at this mean, then verify that it returns the specified probability.

Method. For a Bernoulli response, the mean is $\mu=p$ and the natural parameter is $\theta=\log\{p/(1-p)\}$. The canonical link sets $\eta=\theta$.

First, $1-p=1-0.20=0.80$, so $p/(1-p)=0.20/0.80=0.25$. Therefore

\eta=g(\mu)=\log(0.25)=-1.386294\ldots

To check, invert the relationship: $p=e^\eta/(1+e^\eta)$. Since $e^\eta=0.25$, the recovered probability is $0.25/(1+0.25)=0.25/1.25=0.20$, as required. Rounded to four decimal places, the link value is $-1.3863$. The common mistake this avoids is using $\log(p)$ alone; the natural parameter contains the odds $p/(1-p)$, not just the probability.

Worked Example: Grouped Binomial Count Versus Proportion

A hypothetical audit samples $m=8$ records independently, each having probability $p=0.25$ of requiring correction. Let $Y$ be the number requiring correction. Find the mean count, the canonical link for that count mean, and the canonical link for the corresponding mean proportion. Check that both links use the same underlying probability.

Method. The count is binomial, so $\mu=mp=8(0.25)=2$. For a count mean, the canonical link is $\log\{\mu/(m-\mu)\}$.

The numerator is $2$, and the denominator is $8-2=6$. Hence the count-mean link is $\log(2/6)=\log(1/3)=-1.098612\ldots$. For the proportion response $Y/8$, the mean is $p=0.25$, and its canonical link is $\log\{0.25/0.75\}=\log(1/3)=-1.098612\ldots$.

As a check, the inverse transformation of the count-mean link gives $e^\eta/(1+e^\eta)=(1/3)/(1+1/3)=1/4$, which is the success probability $p$. Multiplying by $m=8$ gives the count mean $2$. Rounded to four decimal places, both link values are $-1.0986$. The trap is to apply the proportion formula directly to the count mean, or to apply the count formula to a proportion without using its trial count.

Poisson Responses

For a Poisson response $Y$, assume $Y$ is a nonnegative integer-valued count with mean $\mu>0$ and probability mass function $e^{-\mu}\mu^y/y!$. The log probability is

\log f(y)=y\log(\mu)-\mu-\log(y!).

Let $\theta=\log(\mu)$, so that $\mu=e^\theta$. Then $y\log(\mu)-\mu=y\theta-e^\theta$. The exponential-family representation has $a(\phi)=1$, $b(\theta)=e^\theta$, and the natural parameter $\theta=\log(\mu)$. Equating the linear predictor with $\theta$ gives the canonical link:

g(\mu)=\log(\mu), \qquad \mu>0.

Worked Example: Poisson Mean and Canonical Link

A hypothetical monitoring unit records the number $Y$ of equipment alerts in a fixed observation period. Assume $Y$ has a Poisson distribution with mean $\mu=4.5$. Derive its canonical link value and verify the mean recovered from the predictor.

Method. For a Poisson response, $\theta=\log(\mu)$ and the canonical link is $g(\mu)=\log(\mu)$. Therefore $\eta=\log(4.5)=1.504077\ldots$, or $1.5041$ to four decimal places.

The inverse relationship is $\mu=e^\eta$. Using the unrounded value $\eta=\log(4.5)$, we obtain $e^\eta=e^{\log(4.5)}=4.5$, which verifies the stated mean. The common error is to use $\log(1+\mu)$, which is not the canonical link for a Poisson mean. A link must follow from the response distribution’s natural parameter, not from a convenient transformation chosen without derivation.

Gaussian Responses and Dispersion Scaling

For a Gaussian response, assume $Y\sim N(\mu,\sigma^2)$, with mean $\mu$ and positive variance $\sigma^2$. The density is

f(y)=\frac{1}{\sqrt{2\pi\sigma^2}}\exp\left\{-\frac{(y-\mu)^2}{2\sigma^2}\right\}.

Expand the square in the exponent:

-\frac{(y-\mu)^2}{2\sigma^2} = \frac{y\mu-\mu^2/2}{\sigma^2}-\frac{y^2}{2\sigma^2}.

Comparing this with $[y\theta-b(\theta)]/a(\phi)+c(y,\phi)$, we identify $a(\phi)=\sigma^2$, $\theta=\mu$, and $b(\theta)=\theta^2/2$. The term $-y^2/(2\sigma^2)$, together with the density’s normalizing term, belongs to $c(y,\phi)$ because it does not involve $\theta$. Differentiating $b$ gives $b'(\theta)=\theta=\mu$. Therefore the canonical link is the identity:

g(\mu)=\mu.

Derivation check: In the expanded Gaussian log density, the coefficient multiplying $y$ is $\mu/\sigma^2$. But the exponential-family bracket is scaled by $1/a(\phi)=1/\sigma^2$, so the coefficient has the form $\theta/a(\phi)$. Consequently $\theta=\sigma^2(\mu/\sigma^2)=\mu$, not $\mu/\sigma^2$. Setting the natural parameter equal to the predictor gives $\eta=\mu$, regardless of the positive value of $\sigma^2$.

Worked Example: Gaussian Dispersion Must Be Factored Out

A hypothetical continuous administrative measure $Y$ is modeled as Gaussian with mean $\mu=120$ and variance $\sigma^2=400$. Identify the coefficient of $y$ in the log density, recover the natural parameter, and state the canonical predictor value.

Method. For $Y\sim N(\mu,\sigma^2)$, the expanded log density contains the term $y\mu/\sigma^2$. Substituting the given values, its coefficient is $120/400=0.3$. This coefficient is $\theta/a(\phi)$, where $a(\phi)=\sigma^2=400$.

Recover the natural parameter by multiplying the response coefficient by the dispersion factor: $\theta=400(0.3)=120$. The Gaussian canonical link is the identity, so the canonical predictor value is $\eta=\theta=120$. The inverse mean relationship is also $\mu=\eta$, which returns $120$ and checks the result. No rounding is needed for these exact values. The common mistake is to call $0.3$ the natural parameter and conclude that the canonical predictor is $0.3$; this confuses the coefficient in the expanded density with $\theta$ inside the scaled exponential-family bracket.

Summary and Common Exam Traps

The canonical links for the four response cases follow directly from their natural parameters. Bernoulli and grouped binomial models use the log odds of the success probability; Poisson models use the logarithm of the mean count; Gaussian models use the identity link for the mean. For grouped binomial data, distinguish the mean count $mp$ from the mean proportion $p$. For Gaussian data, distinguish the natural parameter $\mu$ from the coefficient $\mu/\sigma^2$ that appears when the log density is expanded.

Response and modeled mean	Natural parameter	Canonical link
Bernoulli, $\mu=p$	$\log\{p/(1-p)\}$	$\log\{\mu/(1-\mu)\}$
Binomial count, $\mu=mp$	$\log\{p/(1-p)\}$	$\log\{\mu/(m-\mu)\}$
Poisson count, $\mu>0$	$\log(\mu)$	$\log(\mu)$
Gaussian, $Y\sim N(\mu,\sigma^2)$	$\mu$	$\mu$

A useful final check is to verify that the proposed link maps the valid mean range to a valid real-valued predictor and that the inverse relationship recovers the original mean. For example, the Bernoulli mean must lie strictly between 0 and 1 for its finite log-odds link; the Poisson mean must be positive for its logarithm. Boundary means may be approached as limits, but they do not give finite canonical predictor values. A Gaussian mean can be any real number, consistent with the identity link.

Do not skip the exponential-family scaling. Read $\theta$ from inside the bracket after factoring out $1/a(\phi)$.
Do not confuse a count with a proportion. In grouped binomial models, state whether the response mean is $mp$ or $p$.
Do not assume canonical means mandatory. A GLM may use another valid link; “canonical” identifies the link matching the natural parameter.

Check Your Understanding

For each problem, identify the modeled mean before applying the canonical-link relationship.

A Bernoulli outcome has success probability $0.40$. Find its canonical link value to four decimal places. (Answer: $\log(2/3)=-0.4055$)
A grouped binomial response counts successes in $m=10$ trials, each with probability $0.30$. Find the count mean and its canonical link value to four decimal places. (Answer: mean $=3$; link $=\log(3/7)=-0.8473$)
For the same $m=10$ binomial setting, model the proportion of successes instead of the count. Find the mean proportion and its canonical link. (Answer: mean $=0.30$; link $=\log(3/7)=-0.8473$)
A Poisson response has mean $2$. State its canonical predictor value and verify the inverse mean. (Answer: $\eta=\log(2)=0.6931$; $e^\eta=2$)
A Gaussian response has mean $75$ and variance $25$. In the expanded log density, what is the coefficient of $y$, and what is the natural parameter? (Answer: coefficient $=75/25=3$; natural parameter $=25(3)=75$)

Response and modeled mean	Natural parameter	Canonical link
Bernoulli, \(\mu=p\)	\(\log\{p/(1-p)\}\)	\(\log\{\mu/(1-\mu)\}\)
Binomial count, \(\mu=mp\)	\(\log\{p/(1-p)\}\)	\(\log\{\mu/(m-\mu)\}\)
Poisson count, \(\mu>0\)	\(\log(\mu)\)	\(\log(\mu)\)
Gaussian, \(Y\sim N(\mu,\sigma^2)\)	\(\mu\)	\(\mu\)

What You'll Learn