Why Canonicality Depends On The Response Family

Why Canonicality Is Relative to the Response

A link function transforms the mean of a response into the scale of a linear predictor. But a link is not canonical simply because it is familiar, convenient, or appropriate for the range of possible means. Canonicality has a more specific meaning: the link must map the mean to the natural parameter of the chosen response distribution. Change the response family, and that natural-parameter mapping can change.

This distinction matters even when two response models describe the same numerical mean. For example, a mean of $0.4$ is possible for both a Bernoulli response and a Poisson response, but those families have different natural parameters at that mean. As a result, the logit link is canonical for the Bernoulli family, while the log link is canonical for the Poisson family.

Definition in Exponential-Dispersion Form

Consider a one-parameter exponential-dispersion family with density or mass function

f(y;\theta,\phi) = \exp\left\{ \frac{y\theta-b(\theta)}{a(\phi)} +c(y,\phi) \right\}.

Here $\theta$ is the natural parameter, $\phi$ is a dispersion parameter, and $a(\phi)>0$. The function $b$ determines the mean as a function of $\theta$. The response variable is $Y$, and its mean is denoted by $\mu=E[Y]$. In a generalized linear model, the link $g$ relates that mean to the linear predictor: $g(\mu)=\eta$.

Definition: A link $g$ is canonical for a specified response family when $g(\mu)=\theta$, where $\theta$ is that family’s natural parameter expressed as a function of the mean $\mu$. In the exponential-dispersion form above, the factor $a(\phi)$ is separated from $\theta$; the canonical link is $\theta$, not $\theta/a(\phi)$.

To express the definition as a link on the mean scale, the relationship between $\mu$ and $\theta$ must be invertible on the relevant mean domain. Under standard regularity conditions, $b'(\theta)=\mu$; if $b'$ is one-to-one, then the canonical link is $g(\mu)=(b')^{-1}(\mu)$. The regularity conditions and this relationship are stated explicitly below.

Why the Mean Determines the Natural Parameter

Theorem: Suppose the exponential-dispersion density or mass function is normalized for $\theta$ in an open interval, and its support does not depend on $\theta$. Suppose also that differentiation of its normalizing integral or sum with respect to $\theta$ is valid. Then $$ E[Y]=b'(\theta). $$ If, in addition, $b'$ is one-to-one on that interval, the canonical link on the corresponding mean range is $g(\mu)=(b')^{-1}(\mu)$.

Proof. Write $a=a(\phi)$ and hold $\phi$ fixed. The density or mass function integrates or sums to one. Differentiating that normalization with respect to $\theta$ is valid by the stated assumption, so

0 = \int \frac{\partial}{\partial\theta}f(y;\theta,\phi)\,dy = \int f(y;\theta,\phi)\frac{y-b'(\theta)}{a}\,dy.

For a discrete response, the integrals in this argument are sums. The derivative in the integrand follows by differentiating the exponent, and the support does not vary with $\theta$. Since $a>0$, the last equality gives

0=\frac{E[Y]-b'(\theta)}{a}, \qquad\text{so}\qquad E[Y]=b'(\theta).

If $b'$ is one-to-one, solving $\mu=b'(\theta)$ for $\theta$ gives $\theta=(b')^{-1}(\mu)$. By definition, that function of $\mu$ is the canonical link. $\square$

This result makes the family dependence explicit. Each response family has its own function $b$, and therefore may have a different mapping from a mean to a natural parameter. The canonical link is obtained from that mapping; it is not assigned to a link function independently of a family.

Worked Example: Bernoulli and Poisson Means of 0.4

Compare the natural parameters at mean $0.4$ for a Bernoulli response and a Poisson response. The comparison uses the same numerical mean, but the distributions are different.

Bernoulli response. For $Y\in\{0,1\}$ with success probability $p$, the mass function is $p^y(1-p)^{1-y}$. Set $\theta=\log\{p/(1-p)\}$. Then

p^y(1-p)^{1-y} = \exp\left\{y\theta-\log(1+e^\theta)\right\}.

Thus $b(\theta)=\log(1+e^\theta)$, and $b'(\theta)=e^\theta/(1+e^\theta)=p$. The natural parameter as a function of the mean is $g(p)=\log\{p/(1-p)\}$. At $p=0.4$, it is

\theta_{\mathrm{Bernoulli}} = \log(0.4/0.6) = \log(2/3) \approx -0.4055.

Poisson response. For $Y$ with a Poisson distribution of mean $\mu$, its mass function can be written

\frac{e^{-\mu}\mu^y}{y!} = \exp\left\{y\log\mu-\mu-\log(y!)\right\}.

Here $\theta=\log\mu$, $b(\theta)=e^\theta$, and $b'(\theta)=e^\theta=\mu$. Its canonical link is $g(\mu)=\log\mu$. At $\mu=0.4$,

\theta_{\mathrm{Poisson}} = \log(0.4) \approx -0.9163.

The two natural parameters differ: $-0.4055\ne -0.9163$. Therefore the logit link is canonical for the Bernoulli response, not the Poisson response; the log link is canonical for the Poisson response, not the Bernoulli response. This comparison demonstrates why a mean value alone does not determine canonicality.

Canonical Links for Common Response Families

Several familiar canonical links follow directly from the family-specific mapping between the mean and the natural parameter. These examples also show that “the link” must be identified precisely: a link applied to a success probability is not quite the same expression as a link applied to a binomial count mean.

Worked Example: The Identity Link for a Normal Response

Let $Y\sim N(\mu,\sigma^2)$, with known variance $\sigma^2>0$. Its density can be written as

f(y;\mu) = \exp\left\{ \frac{y\mu-\mu^2/2}{\sigma^2} -\frac{y^2}{2\sigma^2} -\frac12\log(2\pi\sigma^2) \right\}.

This is exponential-dispersion form with $\theta=\mu$, $b(\theta)=\theta^2/2$, and $a(\phi)=\sigma^2$. Differentiating gives $b'(\theta)=\theta=\mu$. Consequently, the natural parameter expressed as a function of the mean is $g(\mu)=\mu$: the canonical link is the identity link.

Now compare with a Bernoulli response whose mean is $p=0.4$. The identity link gives $g(p)=0.4$, whereas the Bernoulli natural parameter is $\log(0.4/0.6)\approx-0.4055$. Since these are unequal, the identity link is not canonical for the Bernoulli family, even though it is canonical for the normal family.

Worked Example: Binomial Counts and Their Mean

Let $Y$ count successes in $m$ trials, with success probability $p$. The binomial mass function has exponential-family form

\binom{m}{y}p^y(1-p)^{m-y} = \exp\left\{ y\theta-m\log(1+e^\theta)+\log\binom{m}{y} \right\}, \qquad \theta=\log\left(\frac{p}{1-p}\right).

The mean count is $\mu=mp$. Thus, if the link is defined on the count mean $\mu$, the canonical link is

g(\mu) = \log\left(\frac{\mu/m}{1-\mu/m}\right) = \log\left(\frac{\mu}{m-\mu}\right), \qquad 0<\mu<m.

For example, if $m=10$ and $p=0.3$, then $\mu=10(0.3)=3$. The natural parameter is $\log(0.3/0.7)=\log(3/7)\approx-0.8473$. Using the count-mean expression gives the same result:

g(3)=\log\left(\frac{3}{10-3}\right) =\log(3/7) \approx -0.8473.

When a model uses the success probability $p$ as its mean-scale quantity, the familiar canonical link is $g(p)=\log\{p/(1-p)\}$. When it uses the count mean $\mu$, the trials count $m$ must appear in the conversion $\mu=mp$. The natural parameter is the same; the expression changes because the mean quantity has been defined differently.

A General Consequence of the Definition

Result: If a link is canonical for two response families on a common mean domain, then the two families’ natural-parameter functions must agree at every mean in that domain.

Proof. Let the natural parameter functions, expressed as functions of the mean, be $\theta_1(\mu)$ and $\theta_2(\mu)$. If $g$ is canonical for the first family, the definition gives $g(\mu)=\theta_1(\mu)$. If it is canonical for the second, the same definition gives $g(\mu)=\theta_2(\mu)$. Therefore $\theta_1(\mu)=\theta_2(\mu)$ wherever both statements apply. If the functions differ at even one mean in the common domain, the link cannot be canonical for both families there. $\square$

The Bernoulli and Poisson calculation above is a direct application: at $0.4$, their natural-parameter functions disagree, so no link can equal both at that mean. Conversely, the result is only a condition for a shared link to be canonical: the natural-parameter functions must agree across the domain, not merely at one isolated mean.

Why the Distinction Matters

A link’s suitability and its canonical status answer different questions. Suitability concerns whether the link can relate the predictor to the response mean in the model being considered. Canonicality asks whether that link is exactly the response family’s natural-parameter function. The preceding material on link domains and inverse-link ranges helps assess whether a mean mapping is mathematically appropriate; it does not, by itself, establish canonicality.

Canonicality is therefore a relationship among three ingredients: the response family, its mean parameterization, and the link. Saying only “the logit link is canonical” is incomplete unless the response family and the modeled mean are understood. For a Bernoulli response, logit maps the success probability to its natural parameter. For a Poisson response, the natural parameter is the log of the mean, so logit does not perform that mapping.

Common Mistake

Wrong reasoning: “The logit link is canonical because it is the standard link for probabilities, so it must be canonical for any response with a mean between zero and one.”
Correction: A mean’s range does not determine a response family’s natural parameter. For a Bernoulli response, the natural parameter is $\log\{p/(1-p)\}$. Canonicality follows because the logit equals that parameter. A different response family with a mean in the same numerical range would require its own natural-parameter calculation.
Another common mix-up: Using a binomial count mean $\mu$ as though it were the success probability. If there are $m$ trials, $p=\mu/m$, so the canonical link on the count mean is $\log\{\mu/(m-\mu)\}$, not $\log\{\mu/(1-\mu)\}$.

Key takeaway: To check whether a link is canonical, identify the response family, write its natural parameter in terms of the modeled mean, and compare that function with the proposed link. Canonicality belongs to this family–mean–link combination, not to the link name alone.

Check Your Understanding

Use the response family and the definition of its natural parameter in each question.

For a Poisson response with mean $\mu=2$, calculate the natural parameter. Is the identity link canonical for this family?
A Bernoulli response has mean $p=0.7$. Write its canonical link value at that mean and evaluate it approximately.
For a binomial count with $m=8$ trials and mean count $\mu=2$, calculate the canonical link on the count-mean scale.
Explain why a link being compatible with the range of possible means does not establish that it is canonical.
State the condition that must hold for one link to be canonical for two families on a common mean domain.