Why the Quasi-Score Is Unbiased

What “Unbiased” Means for a Quasi-Score

A quasi-score is built from residuals weighted by a specified variance function. Its expectation is governed by a simple but important distinction: the variance function determines the weight, while a correctly specified conditional mean makes the residual have conditional expectation zero. This tutorial makes that argument precise and states the moment conditions needed for it to be valid.

Let \(Y_i\) be response \(i\), let \(X_i\) denote its covariates, and let \(\beta\) be a parameter vector in an open parameter space. Write \(\mu_i(\beta)\) for the modeled conditional mean, not automatically for the actual conditional expectation. Correct specification at \(\beta_0\) means that the two agree at that particular parameter value.

Definition: The conditional mean model is correctly specified at \(\beta_0\) if \(E[Y_i\mid X_i]=\mu_i(\beta_0)\) almost surely for every \(i\). The function \(\mu_i(\beta)\) is the mean proposed by the model at a general parameter value \(\beta\); it need not equal the actual conditional expectation away from \(\beta_0\).

For a positive variance function \(V\), the quasi-score contribution is the derivative of the modeled mean multiplied by the residual and divided by the working variance. We use column vectors for parameter derivatives.

Definition: Put \(D_i(\beta)=\partial\mu_i(\beta)/\partial\beta\), a column vector, and define \(U(\beta)=\sum_{i=1}^n D_i(\beta)\{Y_i-\mu_i(\beta)\}/V\{\mu_i(\beta)\}\). If a common positive dispersion factor \(\phi\) is included in the variance, the quasi-score is \(\phi^{-1}U(\beta)\). Multiplication by this fixed positive scalar does not affect whether its expectation is zero.

This is the quasi-score form derived from the mean–variance specification: the residual is scaled by the modeled variance function, and the derivative accounts for how the mean changes with the parameter. The question here is narrower than how to solve the resulting equations: under what conditions does this vector have expectation zero at the true mean parameter?

The Mean-Zero Result and Its Conditions

The essential requirements are that the mean is correct, the weights and derivatives are well defined, and the weighted residual is integrable. For a random design, integrability matters because the derivative and weight can vary with the covariates.

Theorem: Suppose that, for each \(i\), \(\mu_i(\beta_0)\) is finite and \(0<V\{\mu_i(\beta_0)\}<\infty\) almost surely, and \(D_i(\beta_0)\) exists and is finite almost surely. Suppose also that \(E[Y_i\mid X_i]=\mu_i(\beta_0)\) almost surely and \(E[\|D_i(\beta_0)\|\,E\{|Y_i-\mu_i(\beta_0)|\mid X_i\}/V\{\mu_i(\beta_0)\}]<\infty\). Then \(U(\beta_0)\) is integrable and \(E[U(\beta_0)]=0\). Independence of the responses is not required for this conclusion.

Here \(\|\cdot\|\) can be any norm on the finite-dimensional parameter space. The integrability condition ensures that taking expectations of the weighted residual is legitimate; it is not enough simply to observe that an unweighted residual has mean zero.

Proof. Define the covariate-measurable vector \(A_i=D_i(\beta_0)/V\{\mu_i(\beta_0)\}\). By the stated integrability condition, \(E[\|A_i\|\,|Y_i-\mu_i(\beta_0)|]<\infty\), so the vector contribution \(A_i\{Y_i-\mu_i(\beta_0)\}\) is integrable. Since \(A_i\) is a function of \(X_i\), the conditional expectation rule for an integrable random variable gives

\begin{aligned} E\left[A_i\{Y_i-\mu_i(\beta_0)\}\right] &=E\left[A_i\,E\{Y_i-\mu_i(\beta_0)\mid X_i\}\right]\\ &=E\left[A_i\{E[Y_i\mid X_i]-\mu_i(\beta_0)\}\right]\\ &=E[A_i\cdot 0]=0. \end{aligned}

The finite sum of these integrable contributions is integrable, and expectation is additive. Therefore \(E[U(\beta_0)]=\sum_{i=1}^n E[A_i\{Y_i-\mu_i(\beta_0)\}]=0\). If the quasi-score includes a fixed finite \(\phi^{-1}\), its expectation is \(\phi^{-1}0=0\) as well. \(\square\)

There is also a useful conditional version. If the design is fixed, or if the stronger condition \(E[Y_i\mid X_1,\ldots,X_n]=\mu_i(\beta_0)\) holds for every \(i\), then the same calculation conditional on the full design gives \(E[U(\beta_0)\mid X_1,\ldots,X_n]=0\), provided the corresponding conditional integrability holds. The theorem above needs only the individual conditional-mean assumption to establish the unconditional result.

Which Moments Does the Argument Use?

The proof uses a finite first absolute moment sufficient to make the weighted residual integrable. It does not use a finite second moment, a particular distribution, or independence. A conditional variance model is often part of the motivation for choosing \(V\), but the mean-zero calculation itself uses only the conditional mean and valid, integrable weights.

In a conventional quasi-likelihood model, the working variance is written \(\phi V(\mu)\), with \(\phi>0\), and is intended to describe conditional variability. For that interpretation as an actual variance, the response must have a finite conditional second moment and the variance must be compatible with it. Those additional conditions are not needed merely to show that the quasi-score has expectation zero. The working variance function must still be finite and strictly positive at the modeled mean so the score is defined.

Worked Example: An Overdispersed Count With a Mean-Zero Score

Let \(Y\) take the value \(0\) with probability \(1/2\) and \(4\) with probability \(1/2\). Consider the one-parameter mean model \(\mu(\beta)=\beta\), with \(\beta>0\), and working variance function \(V(\mu)=\mu\). The correct mean parameter is \(\beta_0=2\).

The conditional mean (there are no covariates here) is \(E[Y]=(0)(1/2)+(4)(1/2)=2=\mu(2)\). The derivative is \(D(\beta)=1\), so the quasi-score for one observation is \(U(\beta)=(Y-\beta)/\beta\). At the correct parameter, its two possible values are \(U(2)=(0-2)/2=-1\) and \(U(2)=(4-2)/2=1\). Thus \(E[U(2)]=(-1)(1/2)+(1)(1/2)=0\).

The actual variance is \(E[Y^2]-E[Y]^2=8-4=4\), whereas the working variance at the mean is \(V(2)=2\). They differ, yet the score remains mean zero. The variance mismatch changes the weighting scheme; it does not change the fact that \(E[Y-\mu(2)]=0\).

Regression Example: A Logistic Mean

The same result applies when the derivative and weights depend on covariates. In the Bernoulli logistic model, \(V(\mu)=\mu(1-\mu)\), and the derivative of the mean includes this same factor. The resulting score contribution is especially transparent.

Worked Example: Checking a Two-Parameter Logistic Quasi-Score

Suppose there are two fixed covariate values \(x_1=-1\) and \(x_2=1\), with model \(\mu_i(\beta)=\{1+\exp[-(\beta_0+\beta_1x_i)]\}^{-1}\). Take \(\beta_0=0\) and \(\beta_1=\log 3\). Then \(\mu_1=1/4\) and \(\mu_2=3/4\), since the linear predictors are \(-\log 3\) and \(\log 3\). Let \(Y_i\mid x_i\) be Bernoulli with success probability \(\mu_i\), so the conditional mean model is correct.

Writing \(z_i=(1,x_i)^\mathsf{T}\), differentiation gives \(D_i(\beta)=\mu_i(1-\mu_i)z_i\). Dividing by \(V(\mu_i)=\mu_i(1-\mu_i)\) gives \(z_i\). Hence the total score at the specified parameter is \(U(\beta)=z_1(Y_1-1/4)+z_2(Y_2-3/4)\).

For the intercept component, the expectation is \(E[Y_1-1/4]+E[Y_2-3/4]=0+0=0\). For the slope component, it is \((-1)E[Y_1-1/4]+(1)E[Y_2-3/4]=(-1)(0)+(1)(0)=0\). Therefore both components have expectation zero. This verifies the result directly with nonconstant means and covariate-dependent derivatives.

More generally, the conditional-mean argument applies component by component to any differentiable mean model. No special cancellation between derivatives and the variance function is required: each derivative-weight vector is measurable with respect to the covariates and multiplies a residual whose conditional expectation is zero.

What Unbiasedness Does Not Require

It is easy to attribute the mean-zero property to stronger assumptions than the proof needs. The next example makes two of those unnecessary assumptions visible: independence and a correctly specified variance function.

Worked Example: Dependent Responses and a Wrong Working Variance

Let \(Z=4\) with probability \(1/4\) and \(Z=0\) with probability \(3/4\), and set \(Y_1=Y_2=Z\). The responses are perfectly dependent. Use the mean model \(\mu_i(\beta)=\beta\), the working variance \(V(\mu)=\mu\), and \(\beta_0=1\).

For each response, \(E[Y_i]=E[Z]=4(1/4)+0(3/4)=1=\mu_i(1)\). The score contributions at \(\beta_0\) are \(Y_i-1\), because \(D_i=1\) and \(V(1)=1\). Therefore \(U(1)=(Y_1-1)+(Y_2-1)=2(Z-1)\). It equals \(-2\) with probability \(3/4\) and \(6\) with probability \(1/4\), so \(E[U(1)]=(-2)(3/4)+(6)(1/4)=-3/2+3/2=0\).

For each response, \(\operatorname{Var}(Y_i)=E[Z^2]-1^2=4-1=3\), not \(V(1)=1\). Thus neither independence nor agreement of the working and actual variance is needed for this expectation calculation. Dependence and variance specification can matter for other properties of an estimator, but they do not invalidate this mean-zero result.

When the Mean Is Wrong—and a Common Pitfall

Correct specification at \(\beta_0\) is essential. A modeled mean \(\mu(\beta)\) is a function evaluated at a candidate parameter; it is not a declaration that the actual conditional mean equals that function at every candidate value.

Worked Example: A Nonzero Expected Score Under Mean Misspecification

Suppose \(Y\) is Bernoulli with success probability \(0.7\), but fit an intercept-only logistic mean \(\mu(\beta)=\{1+\exp(-\beta)\}^{-1}\). Evaluate the quasi-score at \(\beta=0\). The modeled mean is \(\mu(0)=0.5\), while the actual mean is \(E[Y]=0.7\), so the mean model is not correct at this candidate parameter.

Here \(D(0)=\mu(0)\{1-\mu(0)\}=0.25\), and \(V\{\mu(0)\}=0.25\). Therefore the score contribution simplifies to \(U(0)=Y-0.5\), and \(E[U(0)]=0.7-0.5=0.2\). For \(n\) such observations, the expected total score is \(0.2n\), not zero. The theorem does not apply because the required conditional-mean equality fails.

Common mistake: “The quasi-score is a weighted residual, and residuals average to zero, so its expectation is always zero.” This reasoning is incomplete. Residuals need not have mean zero at an arbitrary parameter value, and a random weight cannot simply be ignored without integrability. The correction is to check the equality \(E[Y_i\mid X_i]=\mu_i(\beta_0)\), verify that the weight and derivative are defined, and ensure the weighted residual is integrable. Only then does conditional expectation establish the result.

Why This Result Matters

Mean-zero quasi-scores explain why quasi-likelihood methods can target a correctly specified mean even when the response distribution is not fully specified. They also clarify the boundary of that robustness: a wrong variance specification does not by itself destroy the score’s expectation at the true mean parameter, but a wrong mean generally does.

Finally, an unbiased score is not the same thing as an unbiased estimator. The equation \(E[U(\beta_0)]=0\) is a statement about the score evaluated at a fixed parameter. An estimator is typically obtained by solving a random equation such as \(U(\hat\beta)=0\); nonlinearity and sampling variability mean that this does not, by itself, imply \(E[\hat\beta]=\beta_0\). The mean-zero property is a foundation for studying such estimators, not a guarantee of their finite-sample unbiasedness.

Key takeaway: The quasi-score has expectation zero at \(\beta_0\) when the conditional mean is correct and the weighted residual is integrable. The working variance must define valid weights, but it need not equal the actual variance for this particular conclusion.

Check Your Understanding

Use the conditional-mean argument to answer these questions.

Which equality between the actual conditional expectation and the modeled mean is required for the quasi-score to have expectation zero at \(\beta_0\)?
Why must the weighted residual be integrable, even when the unweighted residual has conditional expectation zero?
Does the mean-zero result require the working variance function to equal the actual conditional variance? Explain.
Can dependence among responses prevent the unconditional expectation of the summed quasi-score from being zero under the theorem’s assumptions?
In the Bernoulli example with actual success probability \(0.7\), what is the expected score at the candidate modeled mean \(0.5\), and why does the theorem not apply?