Efficiency and the Choice of Working Variance

Why the Working Variance Matters

A quasi-score uses a variance function to weight residuals: observations treated as more variable receive less weight. If the mean model is correct, changing those weights need not change the parameter being estimated. It can, however, change how much information the estimator extracts from the data. The central comparison is therefore between target and precision: a wrong working variance may leave the target intact while making estimation less efficient.

We consider independent responses $Y_1,\ldots,Y_n$ with correctly specified means $\mu_i(\beta_0)=E[Y_i]$, where $\beta_0$ is a $p$-dimensional parameter. Write $d_i(\beta)=\partial\mu_i(\beta)/\partial\beta$, a column vector. Suppose the true variances are $\phi V_{0i}$, where $\phi>0$, and use a positive working variance function $V_{wi}(\beta)$. The focus is the effect of this choice on quasi-score estimation, not on changing the mean model.

Working-Variance Estimating Equations

Definition: For working variance values $V_{wi}(\beta)>0$, the quasi-score estimating equation is $$ U_w(\beta)=\sum_{i=1}^n \frac{d_i(\beta)\{Y_i-\mu_i(\beta)\}}{V_{wi}(\beta)}=0. $$ A solution is denoted by $\hat\beta_w$. A common multiplicative dispersion factor in every working variance does not affect the equation's root.

At the true parameter, $E[U_w(\beta_0)]=0$ for every positive working variance, because $E[Y_i-\mu_i(\beta_0)]=0$. This is the basic reason the variance choice does not change the estimating equation's target when the mean model is correct. In contrast, the derivative and variability of the equation do depend on the weights.

To state the large-sample comparison, assume the responses are independent; the mean model is differentiable and correct; the working variances are positive, smooth, and evaluated at the fitted means; and the estimator is consistent for $\beta_0$. Assume also suitable moment and Lindeberg conditions for a central limit theorem, and that the following matrix limits exist and are positive definite:

\frac{1}{n}\sum_{i=1}^n \frac{d_i(\beta_0)d_i(\beta_0)^T}{V_{wi}(\beta_0)} \longrightarrow A_w,\qquad \frac{1}{n}\sum_{i=1}^n \frac{V_{0i}\,d_i(\beta_0)d_i(\beta_0)^T}{V_{wi}(\beta_0)^2} \longrightarrow B_w.

Under these conditions, the derivative of the estimating equation contributes $A_w$, while the variance of the equation contributes $\phi B_w$. Terms from differentiating the weights contain a residual and have expectation zero at $\beta_0$. Thus the asymptotic covariance is the sandwich covariance:

Result: If the stated regularity conditions hold, then $$ \sqrt{n}(\hat\beta_w-\beta_0) \xrightarrow{d} N\!\left(0,\ \phi A_w^{-1}B_wA_w^{-1}\right). $$ The variance function determines $A_w$ and $B_w$, and therefore the precision, even though the mean-correctness argument gives the same target $\beta_0$.

Worked Example: An Intercept With Two True Variances

Suppose $n$ is even and the model is $E[Y_i]=\beta$, with half the observations having true variance $\phi$ and half having true variance $9\phi$. Compare the correct working variances $(1,9)$ with constant working variance $1$. For an intercept, every derivative $d_i=1$.

Correct working variances. The estimating equation weights the two groups by $1$ and $1/9$. Its sensitivity and variability, with the common factor $\phi$ kept outside the variability, are $$ A_w=\frac{n}{2}\left(1+\frac{1}{9}\right)=\frac{5n}{9}, \qquad B_w=\frac{n}{2}\left(1+\frac{9}{9^2}\right)=\frac{5n}{9}. $$ Consequently, the variance to first order is $\phi B_w/A_w^2=9\phi/(5n)$.

Constant working variance. Now every observation receives weight $1$. Then $$ A_w=n,\qquad B_w=\frac{n}{2}(1+9)=5n, \qquad \operatorname{Var}(\hat\beta_w)\ \text{to first order}=\frac{5\phi}{n}. $$ The ratio of the correct-working-variance variance to this variance is $(9/5)/5=9/25=0.36$. Thus the constant-variance estimator has 36% of the correct-weight estimator's precision under this scalar comparison, although both target the same intercept.

Why the True Variance Is Optimal

The scalar example suggests that using the true variance is more than a convenient choice. For independent observations with a correctly specified mean, it gives the smallest asymptotic covariance, in the positive-semidefinite matrix ordering, among the quasi-score estimators formed by alternative positive working variances. The ordering means that every linear contrast of the estimator has at least as large an asymptotic variance under an alternative weighting.

Theorem: Let $X$ be the $n$-by-$p$ matrix with row $i$ equal to $d_i(\beta_0)^T$. Let $D=\operatorname{diag}(V_{01},\ldots,V_{0n})$, with each $V_{0i}>0$, and let $W=\operatorname{diag}(1/V_{w1},\ldots,1/V_{wn})$, with each $V_{wi}>0$. Assume $X$ has full column rank. The local linearized covariance, apart from the common factor $\phi$, under working weights $W$ is $$ C_w=(X^TWX)^{-1}X^TWDWX(X^TWX)^{-1}. $$ It satisfies $$ C_w-(X^TD^{-1}X)^{-1} $$ positive semidefinite. The second term is the covariance under the true inverse-variance weights. This finite-design comparison applies to the local linearization; for a nonlinear mean model, the corresponding asymptotic comparison uses the limiting sensitivity and variability matrices.

Proof. Define the $p$-by-$n$ matrix $$ L=(X^TWX)^{-1}X^TW $$ and the true inverse-variance weighted matrix $$ L_0=(X^TD^{-1}X)^{-1}X^TD^{-1}. $$ Both satisfy $LX=I_p$ and $L_0X=I_p$. The local linearized estimation error is $L\varepsilon$, where $\varepsilon$ is the vector of mean-zero errors and $\operatorname{Cov}(\varepsilon)=\phi D$. Thus its covariance divided by $\phi$ is $LDL^T=C_w$; similarly, the true-weight covariance divided by $\phi$ is $L_0DL_0^T=(X^TD^{-1}X)^{-1}$.

Set $R=L-L_0$. Since $RX=0$, $$ RDL_0^T=RX(X^TD^{-1}X)^{-1}=0. $$ The transpose cross-product $L_0DR^T$ is also zero. Expanding $LDL^T=(L_0+R)D(L_0+R)^T$ therefore gives $$ LDL^T-L_0DL_0^T=RDR^T. $$ For any $p$-vector $a$, $a^TRDR^Ta=(R^Ta)^TD(R^Ta)\geq 0$, because $D$ is positive definite. The difference is positive semidefinite, proving the claim. $\square$

The theorem is a local information comparison, not a claim that every finite-sample nonlinear estimator has lower variance under the true working variance. Its asymptotic implication is that, under the regularity conditions above, the true inverse-variance weighting minimizes the limiting covariance among these estimating equations. Equality can occur for different working weights when they produce the same local estimating direction.

Worked Example: A Linear Mean With Three Design Points

Take the linear mean $\mu_i=\beta_0+\beta_1x_i$, with $n/3$ observations at each of $x=-1,0,1$, and true variance factors $V_0=(1,1,9)$ at those points. Compare constant working variance with the correct working variance. Write $z_i=(1,x_i)^T$, so $d_i=z_i$. All matrices below are per observation; the covariance matrices are divided by $n$ and multiplied by $\phi$.

With constant working variance, the sensitivity matrix is $$ A_w=\frac{1}{3}\sum_{x=-1,0,1}z z^T =\begin{pmatrix}1&0\\0&2/3\end{pmatrix}, $$ and the variability matrix is $$ B_w=\frac{1}{3}\sum_{x=-1,0,1}V_0zz^T =\begin{pmatrix}11/3&0\\0&10/3\end{pmatrix}. $$ Since $A_w^{-1}=\operatorname{diag}(1,3/2)$, the sandwich covariance is $$ \frac{\phi}{n}A_w^{-1}B_wA_w^{-1} =\frac{\phi}{n}\begin{pmatrix}11/3&0\\0&15/2\end{pmatrix}. $$

Under the correct working variances, the sensitivity and variability matrices agree: $$ A_0=B_0=\frac{1}{3}\sum_{x=-1,0,1}\frac{zz^T}{V_0} =\begin{pmatrix}19/27&0\\0&10/27\end{pmatrix}. $$ The covariance is therefore $$ \frac{\phi}{n}A_0^{-1} =\frac{\phi}{n}\begin{pmatrix}27/19&0\\0&27/10\end{pmatrix}. $$ Both coefficient estimates target the same mean parameters. The two diagonal entries under constant working variance are larger, as predicted by the covariance-ordering theorem.

Scale Changes and Working-Variance Choice

A working variance that is a constant multiple of the true variance is a useful check on the algebra. If $V_{wi}=cV_{0i}$ for a fixed $c>0$, then the estimating equation is the true-weight equation multiplied by $1/c$, so its root is unchanged. Its sensitivity matrix is divided by $c$, and its variability matrix by $c^2$. These factors cancel in the sandwich covariance.

Worked Example: Multiplying Every Working Variance by Four

Suppose the true working variance factors are $V_{0i}$, and replace them all by $V_{wi}=4V_{0i}$. The score becomes $$ U_w(\beta)=\sum_i\frac{d_i(Y_i-\mu_i)}{4V_{0i}} =\frac{1}{4}U_0(\beta), $$ so every root of $U_0(\beta)=0$ is a root of $U_w(\beta)=0$. Also, $A_w=A_0/4$ and $B_w=B_0/16$. Substitution gives $$ A_w^{-1}B_wA_w^{-1} =(4A_0^{-1})(B_0/16)(4A_0^{-1}) =A_0^{-1}B_0A_0^{-1}. $$ Thus a common multiplicative rescaling does not alter the estimator or its sandwich covariance. The relative weights, rather than their common scale, determine the result.

Interpretation and a Common Pitfall

The practical choice of working variance balances efficiency and reliability. If the variance function is credibly known, inverse-variance weighting can improve precision. If it is uncertain, a simpler working variance may be chosen while retaining sandwich covariance estimation for the resulting estimator. The sandwich accounts for the variance mismatch; it does not make an inefficient weighting scheme as precise as the optimal one.

Common mistake: “The mean model is correct, so any working variance gives the same estimator.” Correct mean specification makes the expected estimating equation zero at the same parameter; it does not make the equations identical. In the intercept example, the correct-weight and constant-weight equations assign different influence to the high-variance observations, and their asymptotic variances differ substantially. Conversely, “the working variance is wrong, so the target must be wrong” is also not generally valid: with a correct mean and the stated conditions, the expectation-zero property preserves the target. If the mean model is misspecified, that protection need not hold.

Key takeaway: With a correct mean model, alternative positive working variances can estimate the same parameter, but they generally have different sandwich covariances. Under independent responses and the stated regularity conditions, using the true variance gives the minimum asymptotic covariance within this class of quasi-score estimating equations.

Check Your Understanding

Use the definitions and covariance comparisons above to answer these questions.

Why does the expected quasi-score equal zero at $\beta_0$ for every positive working variance when the mean model is correct?
In the two-variance intercept example, what feature of the constant working variance causes the loss of precision?
State the positive-semidefinite covariance comparison in the theorem in words.
If every working variance is multiplied by the same positive constant, what happens to the estimating equation's root and to its sandwich covariance?
Does sandwich covariance estimation remove the efficiency cost of a poor working variance? Explain.