Quasi-Score Equations for Regression

From Scalar Contributions to a Regression Equation

A regression model connects each response’s mean to its covariates. In a quasi-likelihood model, the mean and variance specification provide the ingredients for an estimating equation without requiring a full probability distribution for the response. The key regression step is to combine the contribution from each observation into one vector equation for the coefficient vector.

For observation $i$, the linear predictor is $\eta_i=x_i^T\beta$, where $x_i$ is a column vector of covariates and $\beta$ is the coefficient vector. The mean is $\mu_i=h(\eta_i)$, where $h=g^{-1}$ is the inverse link. Thus, a change in a coefficient changes the mean through both the design vector and the derivative of $h$.

Definition: Suppose $Y_1,\ldots,Y_n$ are independent responses with fixed design vectors $x_i\in\mathbb{R}^p$. For a coefficient vector $\beta$ in an open parameter set, let $\eta_i=x_i^T\beta$ and $\mu_i=h(\eta_i)$. Assume $h$ is differentiable at each $\eta_i$, and specify $E[Y_i]=\mu_i$ and $\operatorname{Var}(Y_i)=\phi V(\mu_i)$, where $\phi>0$ and $V(\mu_i)>0$. The regression quasi-score is $$ U(\beta)=\sum_{i=1}^n x_i\frac{h'(\eta_i)}{\phi V(\mu_i)}(Y_i-\mu_i). $$ A quasi-score estimate is a solution $\hat\beta$ of $U(\beta)=0$, when such a solution exists.

The design vectors make each observation’s contribution a $p$-vector, so their sum has the same dimension as $\beta$. Equivalently, place the row vectors $x_i^T$ into the $n\times p$ design matrix $X$. Write $y=(Y_1,\ldots,Y_n)^T$, $\mu=(\mu_1,\ldots,\mu_n)^T$, and define diagonal matrices $D=\operatorname{diag}\{h'(\eta_1),\ldots,h'(\eta_n)\}$ and $A=\operatorname{diag}\{V(\mu_1),\ldots,V(\mu_n)\}$. Then the same equation is

U(\beta)=\frac{1}{\phi}X^TDA^{-1}(y-\mu)=0.

This form displays the separate roles of the model components: $X^T$ gathers contributions for the coefficients, $D$ carries the change in mean with the linear predictor, and $A^{-1}$ scales by the specified variance function. The vector $\mu$, and therefore both diagonal matrices, depend on $\beta$.

Deriving the Vector Form

For one response, the quasi-score contribution with respect to its mean has the factor $(Y_i-\mu_i)/\{\phi V(\mu_i)\}$. To obtain the contribution for $\beta$, apply the chain rule to $\mu_i=h(x_i^T\beta)$. The gradient of the linear predictor is $x_i$, and hence the gradient of the mean is $h'(\eta_i)x_i$.

Derivation: The contribution from observation $i$ to the coefficient estimating equation is $$ \frac{Y_i-\mu_i}{\phi V(\mu_i)} \frac{\partial\mu_i}{\partial\beta} = \frac{Y_i-\mu_i}{\phi V(\mu_i)}h'(\eta_i)x_i. $$ Summing these $p$-vectors over the independent responses gives $U(\beta)$. Stacking the row vectors into $X$ gives $X^TDA^{-1}(y-\mu)/\phi$, as displayed above. This is a chain-rule construction of the estimating equation; it does not require a likelihood for the full response distribution. $\square$

Independence is a standard setup for this equation: the sum is formed from individual response contributions, each using its own mean and variance. The mean and variance assumptions must hold at the parameter value under consideration, and the displayed equation is defined only where the inverse link and variance terms are defined and the variance function is positive. Independence by itself does not guarantee that a root exists, is unique, or is numerically easy to find.

How to Read a Root of the Equation

Let $a_i(\beta)=h'(\eta_i)/V(\mu_i)$. Since $\phi$ is a positive scalar, it does not change the roots of the equation. The component for coefficient $j$ is $\sum_i x_{ij}a_i(\beta)(Y_i-\mu_i)/\phi$. A root therefore makes each such coefficient-specific sum zero.

Result: At any $\beta$ where the equation is defined, $U(\beta)=0$ if and only if $$ \sum_{i=1}^n x_i a_i(\beta)(Y_i-\mu_i)=0, \qquad a_i(\beta)=\frac{h'(\eta_i)}{V(\mu_i)}. $$ Equivalently, for each design column $j=1,\ldots,p$, $$ \sum_{i=1}^n x_{ij}a_i(\beta)(Y_i-\mu_i)=0. $$

To verify the equivalence, substitute the definition of $a_i(\beta)$ into the sum. The result is $\phi U(\beta)$. Because $\phi>0$, this vector is zero exactly when $U(\beta)$ is zero; equality of vectors holds exactly when all $p$ components are zero. These are weighted residual orthogonality conditions, but their coefficients generally depend on $\beta$ through $\eta_i$ and $\mu_i$. They are not generally ordinary, unweighted least-squares equations.

Under the correctly specified mean, the equation is unbiased at the true coefficient vector $\beta_0$, provided the expectations exist and the displayed terms are defined there. In fact, $E[U(\beta_0)]=\sum_i x_i h'(\eta_i) \{E[Y_i]-\mu_i\}/\{\phi V(\mu_i)\}=0$. This expectation check concerns the equation’s value at the true parameter; it does not say that a solution computed from one sample will equal $\beta_0$.

Worked Regression Examples

Worked Example: Logistic Regression for Binary Responses

Suppose $Y_i$ is Bernoulli with mean $\mu_i$, use the logit link, and set $\phi=1$. Take an intercept and a binary covariate $x$, with design rows $x_1=(1,0)^T$, $x_2=(1,1)^T$, and $x_3=(1,1)^T$. The observed responses are $(0,1,1)$.

For the logit link, $h(\eta)=e^\eta/(1+e^\eta)$, so $h'(\eta)=\mu(1-\mu)$. A Bernoulli response has variance $\mu(1-\mu)$, giving $V(\mu)=\mu(1-\mu)$. Thus $h'(\eta_i)/V(\mu_i)=1$, and the quasi-score simplifies to $U(\beta)=\sum_i x_i(Y_i-\mu_i)$.

Write $\mu_i=h(\beta_0+x_i\beta_1)$. The intercept component is $0-\mu_1+1-\mu_2+1-\mu_3=2-\mu_1-\mu_2-\mu_3$. The slope component uses only the two rows with $x=1$, and is $1-\mu_2+1-\mu_3=2-\mu_2-\mu_3$. Therefore,

U(\beta)= \begin{pmatrix} 2-\mu_1-\mu_2-\mu_3\\ 2-\mu_2-\mu_3 \end{pmatrix}.

At $\beta=(0,0)^T$, all three means equal $1/2$. The equation’s value is then $(1/2,1)^T$, not zero. This calculation illustrates how the observed residuals enter the coefficient equation through the design rows, even when the variance and inverse-link derivative cancel.

Worked Example: Poisson Log-Link Regression With Dispersion

Consider independent count responses with mean $\mu_i$, variance $\phi\mu_i$, and log link. Use an intercept and a numerical covariate with values $0,1,2$, so the design rows are $(1,0)^T,(1,1)^T,(1,2)^T$. Let the observed counts be $1,2,4$.

Here $h(\eta)=e^\eta$, so $h'(\eta_i)=\mu_i$, and $V(\mu_i)=\mu_i$. Each contribution is consequently $x_i(Y_i-\mu_i)/\phi$. At $\beta=(0,0)^T$, the three means are all $1$, and the residuals are $0,1,3$. Therefore the intercept component is $(0+1+3)/\phi=4/\phi$, while the slope component is $(0\cdot0+1\cdot1+2\cdot3)/\phi=7/\phi$. Thus $U(\beta)=(4/\phi,7/\phi)^T$ at this coefficient vector.

To check a root in this particular data set, set $\beta_0=0$ and $\beta_1=\log 2$. The means are then $e^0=1$, $e^{\log 2}=2$, and $e^{2\log 2}=4$. Each residual is zero, so both score components are zero. This verifies a root directly; it does not imply that every data set has such a root.

Worked Example: Identity Link With a Quadratic Mean–Variance Function

Suppose the mean is linked by $h(\eta)=\eta$, the variance is $\phi\mu_i^2$, and all modeled means are positive. Take the same design rows $(1,0)^T,(1,1)^T,(1,2)^T$, responses $(2,3,6)$, and evaluate the equation at $\beta=(2,1)^T$. The fitted means are $2,3,4$, all positive as required for this variance function.

The identity inverse link has derivative $h'(\eta)=1$, and $V(\mu_i)=\mu_i^2$. The residuals are $0,0,2$, so the three scalar factors multiplying the design vectors are $0$, $0$, and $2/(16\phi)=1/(8\phi)$. The last design vector is $(1,2)^T$. Hence

U(\beta)= \begin{pmatrix}1\\2\end{pmatrix}\frac{1}{8\phi} = \begin{pmatrix} 1/(8\phi)\\ 1/(4\phi) \end{pmatrix}.

This example shows why the variance function matters even when the inverse link has derivative one: the residual from a mean of $4$ is divided by $V(4)=16$. The calculation uses a mean–variance specification and does not require assuming a particular response distribution.

Common Mistakes and Practical Checks

Dropping the inverse-link derivative. The derivative $h'(\eta_i)$ enters because the equation is for $\beta$, not directly for $\mu_i$. It may cancel with $V(\mu_i)$ in a particular model, as in the logistic and Poisson examples, but that cancellation must be shown rather than assumed.
Using the variance function without dispersion. If the specified variance is $\phi V(\mu_i)$, the contribution contains $1/\phi$. A common positive $\phi$ does not change the roots, but it is still part of the quasi-score as defined.
Writing an unweighted residual equation for every model. In general, the multiplier $h'(\eta_i)/V(\mu_i)$ varies across observations. The correct condition is weighted residual orthogonality to each design column.
Assuming the equation must have a solution. A correctly formed equation can have no finite root, multiple roots, or a root only at a boundary outside the parameter set. Its algebraic construction does not establish existence or uniqueness.

When checking a proposed regression quasi-score, verify the dimensions first: $X^T$ is $p\times n$, $DA^{-1}(y-\mu)$ is $n\times1$, and $U(\beta)$ is $p\times1$. Then check each observation’s factor against the specified variance and the derivative of the inverse link. These checks often reveal a missing design component or a misplaced derivative.

Check Your Understanding

Use the definition and examples to answer the following questions.

For $p$ regression coefficients, what is the dimension of the quasi-score vector, and what role does $X^T$ play?
With a log link and $V(\mu)=\mu$, simplify the observation-specific factor $h'(\eta)/\{\phi V(\mu)\}$.
Why does $U(\beta)=0$ generally not imply that the unweighted residuals sum to zero?
For a Bernoulli response with a logit link and $\phi=1$, show why the inverse-link derivative cancels the variance function.
Does correct specification of the mean guarantee that a finite quasi-score root exists in every sample? Explain.