Deriving Canonical Links From Natural Parameters

From a Natural Parameter to a Link

A generalized linear model connects a response distribution to predictors in two steps: the distribution determines a conditional mean, and a link function connects that mean to the linear predictor. The canonical link makes this connection directly through the distribution’s natural parameter. This gives a systematic way to derive familiar links rather than selecting them by inspection.

Use the standard exponential-family representation for an observation \(Y\):

Here \(\theta\) is the natural parameter, \(\phi\) is a dispersion parameter, and \(b\) is the log-partition function. In this representation, the conditional mean is \(\mu=E(Y\mid \mathbf{x})=b'(\theta)\). A GLM specifies a linear predictor \(\eta=\mathbf{x}^{\mathsf T}\boldsymbol{\beta}\) and a link \(g\) satisfying \(g(\mu)=\eta\).

This definition assumes the stated exponential-family representation, with \(Y\) as the statistic multiplied by \(\theta\). It also assumes the mean is in the family’s admissible interior when a finite natural parameter is required. A different response statistic or a different parameterization can change how the natural parameter is written, so the representation should be made explicit before deriving a link.

Why Equating the Parameters Gives the Link

The link has to turn the conditional mean into the quantity that the predictor models. The distribution supplies that quantity through \(b'(\theta)\). The canonical choice simply uses the natural parameter itself as the predictor. The derivation is therefore an inversion of the mean–natural-parameter relationship:

The dispersion factor \(a(\phi)\) affects the variance and likelihood scaling, but under this representation it does not change the equation \(\mu=b'(\theta)\) or the canonical link. The conditional sampling distribution and the linear predictor must still be appropriate for the application; the algebra alone does not establish that a proposed model is scientifically adequate.

Binomial Trial Counts: Link the Probability Carefully

For \(Y\) successes in \(m\) trials, the binomial probability mass function can be written as

Its natural parameter is again \(\theta=\log\{p/(1-p)\}\). The expected count is \(E(Y\mid\mathbf{x})=mp\), while the per-trial mean, or success probability, is \(p\). In the usual binomial GLM with trial counts, the link is applied to \(p\), giving \(g(p)=\log\{p/(1-p)\}\). If instead the mean is written as the expected count \(\mu_{\mathrm{count}}=mp\), then the same natural parameter is \(\log\{\mu_{\mathrm{count}}/(m-\mu_{\mathrm{count}})\}\). The distinction matters: the canonical logit is a link for the probability, not a logit applied to an unrestricted count.

The Poisson Log Link

For a Poisson response with conditional mean \(\lambda>0\), the probability mass function is

The natural parameter is \(\theta=\log(\lambda)\), and the log-partition function is \(b(\theta)=e^\theta\). Therefore \(b'(\theta)=e^\theta=\lambda\), so inverting the mean relationship yields \(g_{\mathrm{can}}(\lambda)=\log(\lambda)\). The inverse link is \(\lambda=e^\eta\), which is positive for every finite predictor.

The Gaussian Identity Link

For a Gaussian response with conditional mean \(\mu\) and known variance \(\sigma^2\), the density can be expressed as

Comparing this with the exponential-family form gives \(\theta=\mu\), \(b(\theta)=\theta^2/2\), and \(a(\phi)=\sigma^2\). Since \(b'(\theta)=\theta=\mu\), the canonical link is \(g_{\mathrm{can}}(\mu)=\mu\), the identity link. This result follows directly from the natural parameter; it does not require the mean to be positive.

Assumptions, Diagnostics, and Choices

A canonical link is derived from a specified conditional response family; it is not a test that the family fits. For ordinary GLM likelihood calculations, the observations are typically modeled as conditionally independent given their predictors, with each conditional distribution belonging to the stated family. The mean must lie in the family’s allowable range, and the predictor structure must be suitable for the question. In a binomial model, for example, probabilities lie in \([0,1]\), while finite logit predictors yield probabilities in \((0,1)\); exact boundary means require infinite logits.

Diagnostics should therefore examine both numerical behavior and the response model. Check that fitted values are in the response family’s allowable range, that the fitting algorithm converged, and that residual patterns or observed variability do not indicate a poor distributional fit. For example, a positive fitted Poisson mean is guaranteed by the log inverse link, but this does not rule out extra variation relative to the Poisson variance assumption. In a binary model, very large coefficient estimates or unstable standard errors can signal separation or sparse information. Such warnings concern estimation or model adequacy, not an error in the algebra deriving the canonical link.

Canonical links are convenient because they align the predictor with the natural parameter and often simplify likelihood calculations. They are not mandatory: another link can be used when it better suits the scientific interpretation or prediction task, provided its mean predictions are valid for the response family. A noncanonical link changes the relationship between predictors and means, not the response distribution itself. Choosing between links should consider the range of possible predictions, interpretability, fit, and numerical stability rather than treating “canonical” as a guarantee of best performance.

The derivation also relies on the stated exponential-family representation. In the standard representation used here, \(Y\) multiplies \(\theta\). When the response is instead recorded as a proportion, a count, or a transformed statistic, identify which mean is being linked and how the natural parameter is written before applying the result. This prevents a correct canonical parameter from being attached to the wrong response-scale quantity.