Why Canonical Links Depend on the Response Distribution
In a generalized linear model, a link function maps the response mean to a scale on which a linear predictor can be defined. Some links are especially natural because they connect that mean directly to the distribution’s natural parameter. These are called canonical links. For three commonly used response distributions, the canonical links are the identity link for the normal distribution, the logit link for the Bernoulli distribution, and the log link for the Poisson distribution.
The aim here is to derive those correspondences from the distributions themselves. We do not fit a regression model: there are no predictors or estimated coefficients. Instead, for each distribution, we identify its natural parameter, express the mean in terms of that parameter, and then invert the relationship.
The exact role of the dispersion parameter depends on the distribution. For a normal response with variance \(\sigma^2\), it scales the part of the exponent containing \(\theta\). For Bernoulli and Poisson responses, the corresponding scale factor is one. The mean-to-natural-parameter relationship, and hence the canonical link, can be derived separately for each case.
Why the Mean Is the Derivative of the Cumulant Function
A brief derivation explains the link between the mean and the natural parameter. In the exponential-family form above, the density or probability mass function must sum or integrate to one. Differentiating that normalization identity with respect to \(\theta\) gives
Since \(a(\phi)\) is positive, this implies \(E[Y]=b'(\theta)\). The canonical link therefore follows by solving \(\mu=b'(\theta)\) for \(\theta\). For the distributions below, that inversion gives three familiar functions.
Normal Response: The Identity Link
Assume \(Y\sim N(\mu,\sigma^2)\), where \(\mu\) is unrestricted and \(\sigma^2>0\). The variance is fixed for this distributional derivation. The normal density can be expanded as
Comparing this with the exponential-family form identifies the natural parameter as \(\theta=\mu\), with \(b(\theta)=\theta^2/2\) and \(a(\phi)=\sigma^2\). Differentiating gives \(b'(\theta)=\theta=\mu\). Solving for the natural parameter simply leaves the mean unchanged:
This is the identity link. It is called identity because it maps every mean to itself. The derivation concerns a normal response distribution; it does not require choosing or estimating regression coefficients.
Worked Example: Identifying the Normal Canonical Link
Assume a response \(Y\sim N(8,4)\), so its mean is \(8\) and its variance is \(4\). Rewrite the density exponent in exponential-family form, identify the natural parameter, and evaluate the canonical link at the mean. As a check, find the density at \(y=10\).
Method. Expand the squared deviation and match the terms involving \(y\) to the exponential-family form:
Thus the term involving the natural parameter is \((y\theta-b(\theta))/4\), with \(\theta=8\) and \(b(\theta)=\theta^2/2=32\). The mean is \(b'(\theta)=\theta=8\), so \(g(8)=8\). The inverse relationship is also the identity: a natural parameter of \(8\) corresponds to a mean of \(8\).
For the density check, \(y-\mu=10-8=2\), and the exponent in the original density is \(-2^2/(2\cdot4)=-1/2\). Therefore \(f(10)=\frac{1}{\sqrt{8\pi}}e^{-1/2}\). This is the same value obtained by substituting \(y=10\), \(\theta=8\), and \(b(\theta)=32\) into the exponential-family expression. The method avoids the mistake of treating the variance \(4\) as the natural parameter: it scales the exponent but does not change the identity link.
Bernoulli Response: The Logit Link
Assume \(Y\) is Bernoulli with success probability \(p\), where \(0<p<1\). Thus \(Y\) takes values \(0\) and \(1\), and \(E[Y]=p\). Its probability mass function is \(P(Y=y)=p^y(1-p)^{1-y}\). Taking logarithms of the factors and collecting terms gives
The natural parameter is \(\theta=\log(p/(1-p))\). Since \(\log(1-p)=-\log(1+e^\theta)\), the cumulant function is \(b(\theta)=\log(1+e^\theta)\). Its derivative is
Solving \(\mu=p=e^\theta/(1+e^\theta)\) for \(\theta\) yields \(\theta=\log(\mu/(1-\mu))\). The canonical link is therefore the logit link:
The ratio \(\mu/(1-\mu)\) is the odds of a success, so the logit is the log of those odds. The restriction \(0<\mu<1\) ensures that the odds and their logarithm are finite.
Worked Example: Checking the Bernoulli Logit Link
Assume \(Y\) is Bernoulli with \(p=0.2\). Find the natural parameter, verify the exponential-family expression for both possible outcomes, and check that the inverse relationship returns the original mean.
Method. The mean is \(\mu=p=0.2\). The odds are \(p/(1-p)=0.2/0.8=0.25\), so \(\theta=\log(0.25)\approx-1.3863\), rounded to four decimal places. The cumulant value is \(b(\theta)=\log(1+e^\theta)=\log(1.25)\).
For \(y=1\), the exponential-family probability is \[ \exp\{1\cdot\log(0.25)-\log(1.25)\} =\frac{0.25}{1.25}=0.2. \] For \(y=0\), it is \[ \exp\{0\cdot\log(0.25)-\log(1.25)\} =\frac{1}{1.25}=0.8. \] The two probabilities sum to \(0.2+0.8=1\), as required.
To check the inverse relationship, \(e^\theta=0.25\), so \(e^\theta/(1+e^\theta)=0.25/1.25=0.2\), the original mean. The method avoids the common error of using \(\log(p)\) as the canonical link. That would give \(\log(0.2)\approx-1.6094\), which is not the natural parameter; the correct quantity is the log of the odds, \(\log(p/(1-p))\).
Poisson Response: The Log Link
Assume \(Y\) has a Poisson distribution with parameter \(\lambda>0\). Its possible values are nonnegative integers, and its mean is \(E[Y]=\lambda\). The probability mass function can be written
Matching terms identifies the natural parameter as \(\theta=\log\lambda\) and the cumulant function as \(b(\theta)=e^\theta\). Then \(b'(\theta)=e^\theta=\lambda\). Solving \(\mu=e^\theta\) for \(\theta\) gives the log link:
The positive-mean restriction corresponds to \(\lambda>0\), the parameter range for a nondegenerate Poisson distribution. The log link maps that positive mean to its natural parameter.
Worked Example: Verifying the Poisson Log Link
Assume \(Y\) is Poisson with \(\lambda=3.2\). Find its natural parameter, verify the probability of observing \(y=2\) from the exponential-family form, and check the mean by inverting the link.
Method. The mean is \(\mu=\lambda=3.2\). The natural parameter is \(\theta=\log(3.2)\approx1.1632\), rounded to four decimal places, and \(b(\theta)=e^\theta=3.2\).
For \(y=2\), the exponential-family probability is \[ \exp\{2\log(3.2)-3.2-\log(2!)\} =\frac{3.2^2e^{-3.2}}{2} =5.12e^{-3.2} \approx0.2087, \] rounded to four decimal places. This agrees with the Poisson mass function. To check the mean, \(e^\theta=e^{\log(3.2)}=3.2\), recovering the original parameter and mean.
The method avoids applying the log link to the observed count \(y=2\) when the task is to identify the distribution’s canonical link. The link relates the mean \(\mu\) to the natural parameter; an individual observed value is not substituted for that mean.
Comparing the Three Canonical Links
The derivations can be summarized by starting with the response distribution, identifying its mean and natural parameter, and then writing the natural parameter as a function of the mean.
| Response distribution | Mean \(\mu\) | Natural parameter \(\theta\) | Canonical link \(g(\mu)\) |
|---|---|---|---|
| Normal \(N(\mu,\sigma^2)\) | \(\mu\) | \(\mu\) | Identity: \(\mu\) |
| Bernoulli with probability \(p\) | \(p\) | \(\log(p/(1-p))\) | Logit: \(\log(\mu/(1-\mu))\) |
| Poisson with parameter \(\lambda\) | \(\lambda\) | \(\log\lambda\) | Log: \(\log\mu\) |
These links are determined by the natural-parameter representation of the distribution, not by a numerical data set. A canonical link is a particular link function associated with that representation; identifying it does not mean that any regression coefficients have been fitted. In each case, a quick check is to apply the inverse relationship and confirm that the original mean is recovered.
Common Traps and a Reliable Procedure
The most frequent errors come from skipping the step that relates the natural parameter to the mean. For a Bernoulli response, confusing \(\log(p)\) with \(\log(p/(1-p))\) omits the failure probability from the odds. For a Poisson response, the natural parameter is \(\log(\lambda)\), not \(\lambda\). For a normal response, the variance scales the exponential-family expression but the natural parameter is the mean, not the variance.
A reliable procedure is:
Collect the terms involving the observed response \(y\) and the distribution parameter.
Match the response-dependent term to \(y\theta\), accounting for any dispersion scale.
Use \(\mu=b'(\theta)\), or the distribution’s equivalent mean formula.
Express \(\theta\) as a function of \(\mu\); that function is the canonical link.
This distinction matters because similar-looking transformations can answer different questions. The canonical link is not chosen by taking any convenient function of the response, nor by transforming an observed outcome. It follows from the distribution’s natural parameter and its mean. The normal, Bernoulli, and Poisson derivations provide three standard cases aligned with the SOA Exam SRM (Statistics for Risk Modeling) syllabus.
Check Your Understanding
For each problem, identify the natural parameter and canonical link as requested, and check the relationship by converting back to the mean where appropriate.
- Assume \(Y\sim N(-2,16)\). Identify its natural parameter and canonical link, and state the link value at its mean. (Answer: \(\theta=-2\); identity link; \(g(-2)=-2\).)
- A Bernoulli response has success probability \(p=0.75\). Find its canonical-link value, rounded to four decimal places. (Answer: \(\log(0.75/0.25)=\log 3\approx1.0986\).)
- A Poisson response has mean \(\lambda=0.4\). Find its natural parameter to four decimal places and verify the mean by inversion. (Answer: \(\theta=\log(0.4)\approx-0.9163\); \(e^\theta=0.4000\).)
- A Bernoulli response has natural parameter \(\theta=\log 4\). Find its success probability and the probability of \(Y=0\). (Answer: \(p=4/(1+4)=0.8000\); \(P(Y=0)=0.2000\).)
- A normal response has natural parameter \(\theta=1.5\) and variance \(\sigma^2=2\). Give its mean and canonical-link value, and explain whether the variance changes the canonical link. (Answer: \(\mu=1.5\); \(g(\mu)=1.5\) under the identity link; the variance does not change the canonical link.)