Canonical Links and Quasi-Score Simplification

Why the Link Can Simplify an Estimating Equation

A quasi-likelihood model specifies a mean and a variance relationship without necessarily specifying a complete probability distribution. Its estimating equation depends on both the variance function and the link: changing the link changes how a change in the regression parameter changes the mean. For a canonical link, these two ingredients can cancel, leaving an equation in the observed-minus-fitted residuals.

That cancellation is useful, but it needs careful interpretation. It is an algebraic property of the mean–variance relationship and link; it does not, by itself, establish that a full likelihood exists. We derive the general equation first, then compare it with the familiar exponential-family case.

Quasi-Score for a Regression Model

Suppose $Y_1,\ldots,Y_n$ are independent responses with means $\mu_i$ and variances $\operatorname{Var}(Y_i)=\phi V(\mu_i)$, where $\phi>0$ is a common dispersion parameter and $V(\mu)>0$ is a specified variance function. Assume the mean model is $g(\mu_i)=\eta_i=x_i^\top\beta$, with a differentiable, invertible link $g$. Here $x_i$ is a fixed covariate vector and $\beta$ is the regression parameter.

Definition: The quasi-likelihood contribution $Q_i(\mu_i;y_i)$, up to an additive term not depending on $\mu_i$, is defined by $$ \frac{\partial Q_i}{\partial \mu_i}=\frac{y_i-\mu_i}{V(\mu_i)}. $$ The quasi-score for $\beta$ is the derivative of $\sum_i Q_i(\mu_i;y_i)/\phi$ with respect to $\beta$. Its estimating equation is $$ U(\beta)=\sum_{i=1}^n \frac{1}{\phi}\frac{d\mu_i}{d\eta_i}\frac{x_i(y_i-\mu_i)}{V(\mu_i)}=0. $$

The equation follows by the chain rule: $\partial\mu_i/\partial\beta=(d\mu_i/d\eta_i)x_i$. Since $g(\mu_i)=\eta_i$, differentiating gives $d\mu_i/d\eta_i=1/g'(\mu_i)$. Thus the same equation can be written

U(\beta)=\sum_{i=1}^n \frac{x_i(y_i-\mu_i)} {\phi V(\mu_i)g'(\mu_i)} =0.

The derivative $g'(\mu_i)$ matters: a residual is weighted not just by its variance, but also by how quickly the link changes with the mean. We assume throughout that the means lie in a region where $V$ is positive and the link is differentiable with nonzero derivative.

Canonical-Link Cancellation

For a regular one-parameter exponential family in canonical form, the natural parameter is $\theta$. A convenient representation is

f(y;\theta,\phi)= \exp\left\{\frac{y\theta-b(\theta)}{\phi}+c(y,\phi)\right\}, \qquad \mu=b'(\theta),\quad V(\mu)=b''(\theta).

In this convention $\operatorname{Var}(Y)=\phi b''(\theta)=\phi V(\mu)$. The canonical link sets $\eta=\theta$. Differentiating the mean with respect to this link gives $d\mu/d\eta=b''(\theta)=V(\mu)$.

Theorem: If the mean model uses the canonical link associated with the variance function, so that $d\mu_i/d\eta_i=V(\mu_i)$, then the quasi-score is $$ U(\beta)=\frac{1}{\phi}\sum_{i=1}^n x_i(y_i-\mu_i). $$ Therefore its estimating equation is $\sum_i x_i(y_i-\mu_i)=0$ when $\phi$ is a common positive scalar.

The same cancellation can be used in a quasi-likelihood model without a full probability law. What is required for the algebra is the stated variance function and a link satisfying $d\mu/d\eta=V(\mu)$, not a density or mass function.

Start with the general quasi-score.
For each observation its contribution is $x_i(y_i-\mu_i)(d\mu_i/d\eta_i)/\{\phi V(\mu_i)\}$.

Apply the canonical-link derivative.
Substitute $d\mu_i/d\eta_i=V(\mu_i)$, which cancels the variance function in the denominator.

Remove a common scale from the root equation.
Because $\phi>0$ is common to all observations, multiplying the equation by $\phi$ does not change its solutions.

For completeness, the cancellation is an immediate consequence of the general equation. Under the stated derivative condition,

\begin{aligned} U(\beta) &=\sum_{i=1}^n \frac{1}{\phi} \frac{d\mu_i}{d\eta_i} \frac{x_i(y_i-\mu_i)}{V(\mu_i)}\\ &=\sum_{i=1}^n \frac{1}{\phi}V(\mu_i) \frac{x_i(y_i-\mu_i)}{V(\mu_i)}\\ &=\frac{1}{\phi}\sum_{i=1}^n x_i(y_i-\mu_i). \end{aligned}

Since $\phi>0$, $U(\beta)=0$ if and only if $\sum_i x_i(y_i-\mu_i)=0$. This proof uses neither a likelihood nor a distributional identity; it uses only the quasi-score definition and the link derivative.

In the exponential-family model above, the same expression is also the ordinary likelihood score for $\beta$. Indeed, differentiating the log density with respect to $\theta_i$ gives $(y_i-b'(\theta_i))/\phi=(y_i-\mu_i)/\phi$; applying the chain rule with $\theta_i=x_i^\top\beta$ gives $\sum_i x_i(y_i-\mu_i)/\phi$. That likelihood interpretation requires the full exponential-family model. The quasi-score cancellation itself does not.

Worked Examples Across Common Variance Functions

Worked Example: Poisson Counts With a Log Link

Suppose independent counts have means $\mu_i$, variance $\phi\mu_i$, and a log-link model $\log\mu_i=x_i^\top\beta$. The Poisson variance function is $V(\mu)=\mu$, and the log link has derivative $g'(\mu)=1/\mu$. Thus $V(\mu)g'(\mu)=1$, so the general quasi-score reduces to $\sum_i x_i(y_i-\mu_i)/\phi$.

For a numerical check, take two observations with $x_1=(1,0)^\top$, $x_2=(1,1)^\top$, responses $y_1=2,y_2=5$, and evaluate the score at $\beta=(\log 2,\log 2)^\top$. The fitted means are $\mu_1=\exp(\log 2)=2$ and $\mu_2=\exp(\log 2+\log 2)=4$. With $\phi=1$, the residuals are $0$ and $1$. Therefore

U(\beta)=x_1(0)+x_2(1) =\begin{pmatrix}1\\1\end{pmatrix}.

Directly from the unsimplified expression, the first contribution is zero, while the second is $x_2(1)(1)/(4(1/4))=x_2$, since $V(4)=4$ and $g'(4)=1/4$. This agrees with the simplified score.

Worked Example: Grouped-Binomial Counts With a Logit Link

Suppose group $i$ has $m_i$ trials and observed success count $K_i$. Write $Y_i=K_i$, so $\mu_i=m_ip_i$ and $\operatorname{Var}(Y_i)=\phi m_ip_i(1-p_i)$. The variance function for the count mean is $V(\mu_i)=m_ip_i(1-p_i)$. With logit link $\eta_i=\log\{p_i/(1-p_i)\}$, differentiation gives $dp_i/d\eta_i=p_i(1-p_i)$, and hence $d\mu_i/d\eta_i=m_ip_i(1-p_i)=V(\mu_i)$.

Take two groups with sizes $m_1=10,m_2=20$, observed counts $K_1=4,K_2=14$, design vectors $x_1=(1,0)^\top,x_2=(1,1)^\top$, and evaluate at $\beta=(0,0)^\top$. Then both fitted probabilities are $p_i=1/2$, giving fitted counts $\mu_1=5,\mu_2=10$. The residuals are $-1$ and $4$. With $\phi=1$,

U(\beta)=x_1(-1)+x_2(4) =\begin{pmatrix}3\\4\end{pmatrix}.

For the second group, the unsimplified scalar multiplier of $x_2(K_2-\mu_2)$ is $(d\mu_2/d\eta_2)/V(\mu_2)=20(1/4)/\{20(1/4)\}=1$. The first group has the same ratio, so both contributions agree with the residual form. If the data are represented as proportions $K_i/m_i$, their variance is $\phi p_i(1-p_i)/m_i$; the corresponding score contribution is weighted by $m_i$, giving the same count-residual equation.

Worked Example: Gamma Variance With the Natural Link

Consider positive responses with $\operatorname{Var}(Y_i)=\phi\mu_i^2$, so $V(\mu)=\mu^2$. For the gamma exponential family, the natural parameter is $\eta=-1/\mu$, and thus $\mu=-1/\eta$ on the domain $\eta<0$. Differentiating gives $d\mu/d\eta=1/\eta^2=\mu^2=V(\mu)$. The canonical-link quasi-score therefore has the residual form.

For a numerical check, let $x_1=1,x_2=2$, $\beta=-1$, and $\eta_i=x_i\beta$. The fitted means are $\mu_1=1$ and $\mu_2=1/2$. Let the observed responses be $y_1=1.5,y_2=1$, and take $\phi=1$. The residuals are $0.5$ and $0.5$, so

U(\beta)=1(0.5)+2(0.5)=1.5.

Checking the original weights, $ (d\mu_1/d\eta_1)/V(\mu_1)=1/1=1$, and $ (d\mu_2/d\eta_2)/V(\mu_2)=(1/4)/(1/4)=1$. The score is again $0.5+2(0.5)=1.5$. The calculation remains valid if only the mean and variance are assumed; calling the model gamma would additionally assert a full distribution.

What Does and Does Not Simplify

The cancellation simplifies the estimating equation, not every aspect of inference. A common $\phi$ multiplies the whole score by $1/\phi$, so it does not change the score’s root. It still affects score magnitudes and uncertainty calculations. If observations have known variance weights, such as $\operatorname{Var}(Y_i)=\phi V(\mu_i)/w_i$, the score includes $w_i$; the residual equation is then weighted rather than an unweighted sum.

The link condition also matters. For a general link, the contribution is weighted by $1/\{V(\mu_i)g'(\mu_i)\}$, and that factor need not equal one. The word “canonical” refers specifically to the link associated with a chosen exponential-family representation. When only a mean–variance specification is available, one can still use a link satisfying $d\mu/d\eta=V(\mu)$, but the algebraic property alone does not establish that a probability distribution with that canonical parameter exists.

Key takeaway: Under a canonical link, the derivative of the mean with respect to the linear predictor equals the variance function, so the quasi-score becomes a scaled sum of covariate-weighted residuals. This simplification needs a compatible mean–variance function and link, not a full likelihood.

Common Mistake

A tempting but incorrect argument is: “The canonical-link quasi-score is the residual score, so the quasi-likelihood model must have the same likelihood score and inference as the corresponding exponential-family model.” The first clause is an algebraic conclusion; the second does not follow. A mean and variance do not specify a full distribution, and the quasi-score identity does not supply one. The correction is to distinguish the estimating equation from its probabilistic interpretation: the residual equation follows from the quasi-score definition and the link derivative, while equality to a likelihood score requires an actual likelihood with the relevant exponential-family form.

Check Your Understanding

Use the general quasi-score to answer these questions.

For a log link and variance function $V(\mu)=\mu$, calculate $V(\mu)g'(\mu)$. What does this imply for the score weight?
For grouped binomial counts, why does the quasi-score use count residuals rather than unweighted proportion residuals?
What link derivative condition produces the canonical-link residual form?
Why does a common positive dispersion parameter leave the estimating equation’s root unchanged?
What additional assumption is needed to interpret a quasi-score as a full likelihood score?