Deriving Quasi-Score Estimating Equations

Why a Quasi-Score Uses Both Sensitivity and Variance

A regression mean model describes how the expected response changes with parameters. A variance function describes how much responses vary around those means. An estimating equation should account for both: a residual is more informative when the mean is sensitive to a parameter, and less informative when the response has high variance.

A quasi-score combines these two features without requiring a complete probability distribution for the observations. Its terms are residuals weighted by the derivative of the mean and by the inverse variance. The resulting equation can be used to estimate regression parameters even when only the mean and variance are specified. It is not, in general, the score of a likelihood.

Definition: Suppose $Y_1,\ldots,Y_n$ are independent, with $E[Y_i]=\mu_i(\beta)$ and $\operatorname{Var}(Y_i)=\phi V_i(\mu_i(\beta))$, where $\beta$ is a $p$-dimensional parameter, $\phi>0$ is a common scale factor, and each variance function $V_i$ is positive. Let $d_i(\beta)=\partial\mu_i(\beta)/\partial\beta$ be the $p$-dimensional column vector of derivatives. The quasi-score is the estimating function $$ U(\beta;y)=\frac{1}{\phi}\sum_{i=1}^n \frac{d_i(\beta)}{V_i(\mu_i(\beta))}\{y_i-\mu_i(\beta)\}. $$ The quasi-score estimating equation is $U(\beta;y)=0$.

For a scalar parameter, $d_i(\beta)$ is an ordinary derivative and the equation is scalar. For a vector parameter, the equation has one component for each parameter. In either case, each response contributes its residual $y_i-\mu_i(\beta)$, scaled by its mean's parameter sensitivity and its variance.

The assumptions above specify only means, variances, and independence; they do not specify the full distributions of the $Y_i$. The quasi-score is therefore a principled estimating function under a mean-variance model, not an automatic likelihood score. A shared positive $\phi$ multiplies the entire equation, so it does not change its roots when it is treated as common to all observations.

Deriving the Estimating Equation

The derivative $d_i(\beta)$ measures how much the $i$th fitted mean changes when the parameter changes. The inverse variance $1/\{\phi V_i(\mu_i(\beta))\}$ gives less weight to a residual with greater variability. Multiplying these factors by the residual produces the $i$th contribution to the quasi-score.

This form can also be motivated by asking for a residual-based estimating function that is unbiased at the true parameter. If $\beta_0$ is the true parameter, then $E[Y_i-\mu_i(\beta_0)]=0$. Any fixed coefficient multiplying that residual preserves mean zero. The quasi-score selects coefficients that use the mean derivative and the variance model, rather than treating all residuals as equally variable or equally informative.

Theorem: Under the definition's mean assumptions, $E_{\beta_0}[U(\beta_0;Y)]=0$. If the mean functions are continuously differentiable, the variance functions are positive and differentiable, and the displayed derivatives exist, then the derivative of the expected quasi-score at $\beta_0$ is $$ \left.\frac{\partial}{\partial\beta^T}E_{\beta_0}[U(\beta;Y)]\right|_{\beta=\beta_0} =-\frac{1}{\phi}\sum_{i=1}^n \frac{d_i(\beta_0)d_i(\beta_0)^T}{V_i(\mu_i(\beta_0))}. $$

Proof. At $\beta_0$, each residual has expectation zero, so $$ E_{\beta_0}[U(\beta_0;Y)] =\frac{1}{\phi}\sum_{i=1}^n \frac{d_i(\beta_0)}{V_i(\mu_i(\beta_0))} \{E_{\beta_0}[Y_i]-\mu_i(\beta_0)\}=0. $$ For the derivative statement, write the expected equation at a general $\beta$ while keeping the true distribution fixed: $$ E_{\beta_0}[U(\beta;Y)] =\frac{1}{\phi}\sum_{i=1}^n \frac{d_i(\beta)}{V_i(\mu_i(\beta))} \{\mu_i(\beta_0)-\mu_i(\beta)\}. $$ This is a finite sum of differentiable functions. At $\beta=\beta_0$, the residual factor in each summand is zero. Consequently, differentiating the coefficient contributes zero there, while differentiating the residual contributes $-d_i(\beta_0)^T$. The resulting derivative matrix is the stated sum. If the $d_i(\beta_0)$ span the parameter space, this matrix is nonsingular because each denominator is positive. $\square$

The theorem establishes unbiasedness of the estimating function at the true parameter; it does not assert that a solution $\hat{\beta}$ is unbiased in finite samples. Its derivative also shows how the fitted means respond locally to a change in the parameter. These facts explain why the equation is useful, without requiring a full probability law.

Worked Scalar and Vector Equations

Worked Example: A Scalar Weighted Mean

Suppose three independent responses have a common unknown mean $\mu_i(\theta)=\theta$ and variances $\phi V_i$, with $V_1=1$, $V_2=2$, and $V_3=4$. The observed values are $y_1=1$, $y_2=2$, and $y_3=5$. Find the quasi-score estimate.

Here $d_i(\theta)=1$ for every observation. The quasi-score equation is $$ \frac{1}{\phi}\left\{(1-\theta)+\frac{1}{2}(2-\theta)+\frac{1}{4}(5-\theta)\right\}=0. $$ Since $\phi>0$, multiply through by $\phi$ and collect terms: $$ 1+1+\frac{5}{4}-\left(1+\frac{1}{2}+\frac{1}{4}\right)\theta=0, \qquad \frac{13}{4}-\frac{7}{4}\theta=0. $$ Therefore $\hat{\theta}=13/7$. Direct substitution checks the equation: $$ 1-\frac{13}{7}+\frac{1}{2}\left(2-\frac{13}{7}\right) +\frac{1}{4}\left(5-\frac{13}{7}\right) =-\frac{6}{7}+\frac{1}{14}+\frac{11}{14}=0. $$ The observation with variance factor $4$ receives less residual weight than the one with variance factor $1$.

Worked Example: A Two-Parameter Linear Mean

Let $\mu_i(\beta)=\beta_0+\beta_1x_i$, with observations $(x_i,y_i)=(0,1),(1,2),(2,4)$. Suppose the variance factors are $V_i=1,2,1$, respectively, and take $\phi=1$. Find both quasi-score equations and solve them.

For the intercept, the derivative of each mean is $1$. For the slope, it is $x_i$. Thus the intercept equation is $$ (1-\beta_0)+\frac{1}{2}(2-\beta_0-\beta_1) +(4-\beta_0-2\beta_1)=0, $$ or, collecting each term, $$ 6-\frac{5}{2}\beta_0-\frac{5}{2}\beta_1=0. $$ The constant $6$ is the weighted response sum $1+2/2+4$. For the slope equation, each residual is also multiplied by $x_i$: $$ \frac{1}{2}(2-\beta_0-\beta_1) +2(4-\beta_0-2\beta_1)=0, $$ which expands to $$ 9-\frac{5}{2}\beta_0-\frac{9}{2}\beta_1=0. $$ Subtracting the first expanded equation from the second gives $3-2\beta_1=0$, so $\hat{\beta}_1=3/2$. Substitution in the first gives $6-(5/2)\beta_0-15/4=0$, hence $\hat{\beta}_0=9/10$.

Check both equations: the first has left side $6-9/4-15/4=0$, and the second has left side $9-9/4-27/4=0$. The fitted means are $9/10,12/5,39/10$, giving residuals $1/10,-2/5,1/10$. The weighted intercept residual sum is $1/10-1/5+1/10=0$; the weighted slope residual sum is $-1/5+2/10=0$. Thus the reported coefficients solve the original quasi-score equations.

Worked Example: A Nonlinear Scalar Mean

Suppose three independent responses have mean $\mu_i(\theta)=e^\theta$ and variance $\phi V_i(\mu_i)=\phi\mu_i$. For observed values $1,2,3$, derive and solve the quasi-score equation.

The derivative is $d_i(\theta)=e^\theta=\mu_i(\theta)$. Since $V_i(\mu_i)=\mu_i$, the derivative divided by the variance function is $1$. Therefore $$ U(\theta;y)=\frac{1}{\phi}\sum_{i=1}^3(y_i-e^\theta) =\frac{1}{\phi}(6-3e^\theta). $$ Setting this equal to zero gives $e^{\hat{\theta}}=2$, so $\hat{\theta}=\log 2$. Substitution yields $U(\log 2;y)=\phi^{-1}(6-3\cdot2)=0$. This calculation uses the stated mean and variance model; it does not require choosing a probability distribution for the responses.

Why the Derivative Weight Is Natural

For a scalar parameter, one can make the weighting principle more precise. Consider estimating functions $U_a(\theta)=\sum_i a_i\{Y_i-\mu_i(\theta)\}$, where the coefficients $a_i$ are fixed locally and the derivative $d_i=\partial\mu_i/\partial\theta$ is evaluated at the parameter of interest. Their expected derivative is $-\sum_i a_i d_i$, while their variance is $\sum_i a_i^2\sigma_i^2$, where $\sigma_i^2=\phi V_i(\mu_i)$. The local variance of a root, obtained by linearizing the equation, is therefore proportional to $$ \frac{\sum_i a_i^2\sigma_i^2}{(\sum_i a_i d_i)^2}, $$ provided $\sum_i a_i d_i\ne0$.

Result: Among these scalar linear estimating functions, the ratio above is minimized by coefficients proportional to $d_i/\sigma_i^2$, provided $\sum_i d_i^2/\sigma_i^2>0$.

Proof. Apply the Cauchy–Schwarz inequality to the sequences $a_i\sigma_i$ and $d_i/\sigma_i$: $$ \left(\sum_i a_i d_i\right)^2 \leq \left(\sum_i a_i^2\sigma_i^2\right) \left(\sum_i\frac{d_i^2}{\sigma_i^2}\right). $$ Rearranging gives $$ \frac{\sum_i a_i^2\sigma_i^2}{(\sum_i a_i d_i)^2} \geq \frac{1}{\sum_i d_i^2/\sigma_i^2}. $$ Equality in Cauchy–Schwarz holds when $a_i\sigma_i$ is proportional to $d_i/\sigma_i$, or equivalently $a_i$ is proportional to $d_i/\sigma_i^2$. Since $\sigma_i^2=\phi V_i(\mu_i)$, these coefficients are proportional to $d_i/V_i(\mu_i)$, the scalar quasi-score weights. $\square$

This optimality statement is local and limited to the scalar class of linear residual estimating functions just described. It is a motivation for the quasi-score weights, not a claim that they are optimal among every possible estimator or under every model.

Common Mistake and Practical Interpretation

A frequent error is to weight residuals only by inverse variance and omit the derivative of the mean. That works for an intercept-only mean with derivative $1$, but not generally for regression. In a linear mean, for example, the slope equation must multiply each weighted residual by its covariate $x_i$. Without that factor, one has not formed the slope component of the quasi-score.

A second error is to call every quasi-score a likelihood score. A likelihood score is obtained by differentiating a specified log-likelihood. The quasi-score above is defined from a mean and variance model, which may not determine any full distribution. It can coincide with a likelihood score in particular models, but that requires additional assumptions.

When calculating a vector equation, expand each component separately and check its constants, residual weights, and derivative factors. Then substitute the proposed solution into the unexpanded equations. This is especially useful when the design values are not all equal, because the parameter derivatives differ across observations.

Check Your Understanding

Use the mean-variance assumptions and quasi-score definition to answer the following.

For a scalar mean $\mu_i(\theta)$, what two factors, besides the residual, determine the $i$th quasi-score contribution?
Why does multiplying the whole quasi-score equation by a common positive $\phi$ leave its roots unchanged?
In a linear mean $\mu_i(\beta)=\beta_0+\beta_1x_i$, what additional factor distinguishes the slope equation from the intercept equation?
For the scalar weighted-mean example, verify the estimate by evaluating the weighted residual sum at $\theta=13/7$.
Why is a quasi-score not necessarily a likelihood score?