Efficiency of the Quasi-Score

What Does Efficiency Mean for a Quasi-Score?

A quasi-score uses a specified mean model and variance function to form an estimating equation, without requiring a complete probability distribution for the responses. Its efficiency therefore needs a precise qualification: it is optimal within a particular class of estimating equations, but the mean and variance alone do not establish optimality among every possible function of the data.

We make that distinction by first comparing linear unbiased estimating functions. We then state the additional conditions under which the quasi-score also attains the information bound for a fully specified parametric model. In both comparisons, efficiency concerns covariance: a smaller covariance matrix means greater precision in every linear combination of the estimated parameters.

Definition: Suppose $Y_1,\ldots,Y_n$ are independent responses, with $E[Y_i]=\mu_i(\beta)$ and $\operatorname{Var}(Y_i)=v_i(\beta)=\phi V(\mu_i(\beta))$, where $\beta$ is a $p$-dimensional parameter and $\phi$ is known. Let $D_i=\partial\mu_i(\beta)/\partial\beta$, a $p$-dimensional column vector. The quasi-score is $$ U(\beta)=\sum_{i=1}^n \frac{D_i}{v_i}(Y_i-\mu_i). $$ Its sensitivity matrix and covariance matrix are both $$ M(\beta)=\sum_{i=1}^n \frac{D_iD_i^T}{v_i}. $$ When $M$ is nonsingular, $M^{-1}$ is the Godambe covariance bound for the linear unbiased estimating functions defined below.

The setup assumes that the mean and variance functions are correctly specified, the responses are independent, and the relevant second moments exist. The quasi-score is unbiased because each residual has expectation zero. In calculating its sensitivity, derivatives of the weights multiply residuals whose expectations are zero; consequently, the sensitivity is $M$. Independence makes the covariance a sum of the individual covariance contributions, also giving $M$.

Optimality Among Linear Unbiased Estimating Functions

Consider the class of estimating functions $$ \psi_A(\beta)=\sum_{i=1}^n a_i(\beta)(Y_i-\mu_i), $$ where each $a_i(\beta)$ is a $p$-dimensional vector that does not depend on the responses. These functions are unbiased under the specified mean model. This is the class for which the quasi-score has a general optimality result based only on the means, variances, and independence.

Theorem: Suppose the $Y_i$ are independent, have the specified means and finite positive variances $v_i$, the coefficient functions $a_i(\beta)$ are differentiable, and the sensitivity $J_A=-E[\partial\psi_A/\partial\beta^T]$ is nonsingular. Then the Godambe covariance $J_A^{-1}K_AJ_A^{-T}$, where $K_A=\operatorname{Var}(\psi_A)$, is at least $M^{-1}$ in the positive-semidefinite order. Equality holds for the quasi-score.

Here $C_1$ is at least $C_2$ in that order when $C_1-C_2$ is positive semidefinite. This comparison concerns the covariance associated with the estimating equation; for a general nonlinear equation it is the usual local or asymptotic covariance approximation. In a linear mean model with constant coefficients, it is also the exact covariance of the resulting linear estimator.

Proof. Differentiating $\psi_A$ and taking expectations removes the terms involving derivatives of $a_i$, because $E[Y_i-\mu_i]=0$. Thus $$ J_A=\sum_{i=1}^n a_iD_i^T, \qquad K_A=\sum_{i=1}^n v_i a_i a_i^T. $$ Set $L_i=J_A^{-1}a_i$. Then $\sum_i L_iD_i^T=I_p$, where $I_p$ is the $p$-dimensional identity matrix, and $$ J_A^{-1}K_AJ_A^{-T}=\sum_{i=1}^n v_iL_iL_i^T. $$ For the quasi-score, the corresponding normalized coefficients are $L_i^*=M^{-1}D_i/v_i$. They satisfy $\sum_i L_i^*D_i^T=I_p$, and $$ \sum_{i=1}^n v_iL_i^*L_i^{*T}=M^{-1}. $$ Using $\sum_iL_iD_i^T=I_p$ and its transpose to expand the square gives $$ \begin{aligned} \sum_{i=1}^n v_i(L_i-L_i^*)(L_i-L_i^*)^T &=\sum_{i=1}^n v_iL_iL_i^T -\sum_{i=1}^n L_iD_i^TM^{-1}\\ &\quad-\sum_{i=1}^n M^{-1}D_iL_i^T+M^{-1}\\ &=J_A^{-1}K_AJ_A^{-T}-M^{-1}. \end{aligned} $$ The left side is a sum of positive-semidefinite matrices, so the difference is positive semidefinite. For the quasi-score, $a_i=D_i/v_i$, giving $J_A=K_A=M$ and covariance $M^{-1}$. This proves the result. $\square$

Worked Example: Comparing Two Linear Equations

Let $Y_1,Y_2$ be independent, with $E[Y_1]=\beta$, $E[Y_2]=2\beta$, and variances $1$ and $4$, respectively. The derivatives of the means are $D_1=1$ and $D_2=2$.

The quasi-score coefficients are $D_1/v_1=1$ and $D_2/v_2=2/4=1/2$. Therefore $$ U(\beta)=(Y_1-\beta)+\frac12(Y_2-2\beta) =Y_1+\frac12Y_2-2\beta. $$ Setting $U(\beta)=0$ gives $\hat\beta_Q=Y_1/2+Y_2/4$. Its variance is $$ \operatorname{Var}(\hat\beta_Q)=\frac14(1)+\frac1{16}(4)=\frac12. $$ The information quantity is $M=1^2/1+2^2/4=2$, so $M^{-1}=1/2$, agreeing with the direct variance calculation.

For comparison, use the unbiased equation $(Y_1-\beta)+(Y_2-2\beta)=0$. Its solution is $\hat\beta_A=(Y_1+Y_2)/3$, with variance $(1+4)/9=5/9$. Since $5/9>1/2$, this alternative linear equation is less precise.

When Does the Quasi-Score Attain the Full Information Bound?

The linear-class theorem does not assert that no nonlinear function of the responses can do better. To make a statement about the full parametric information bound, assume a regular, fully specified model with density or mass function $f_\beta$. Write its score as $s(Y;\beta)=\partial\log f_\beta(Y)/\partial\beta$ and its Fisher information as $I(\beta)=E[ss^T]$. Assume the model is differentiable in the parameter, differentiation can be interchanged with expectation for the functions below, the score has finite second moments and mean zero, and $I$ is nonsingular.

Theorem: Let $\psi(Y;\beta)$ be an unbiased, differentiable $p$-dimensional estimating function with nonsingular sensitivity $J=-E[\partial\psi/\partial\beta^T]$ and finite covariance $K$. Under the regularity conditions just stated, its Godambe covariance satisfies $$ J^{-1}KJ^{-T}\ \succeq\ I^{-1}. $$ Equality holds if and only if $J^{-1}\psi=I^{-1}s$ with probability one. Thus an estimating function attains the full-model information bound exactly when its normalized value agrees almost surely with the normalized likelihood score.

Proof. Differentiating $E_\beta[\psi(Y;\beta)]=0$ under the integral or sum gives $$ 0=E\left[\frac{\partial\psi}{\partial\beta^T}\right]+E[\psi s^T], \qquad\text{so}\qquad J=E[\psi s^T]. $$ Let $h=J^{-1}\psi$. Then $E[hs^T]=I_p$, $E[h]=0$, and $E[hh^T]=J^{-1}KJ^{-T}$. Expanding a covariance matrix, $$ \begin{aligned} E[(h-I^{-1}s)(h-I^{-1}s)^T] &=E[hh^T]-E[hs^T]I^{-1}\\ &\quad-I^{-1}E[sh^T]+I^{-1}E[ss^T]I^{-1}\\ &=J^{-1}KJ^{-T}-I^{-1}. \end{aligned} $$ The left side is positive semidefinite. It is zero exactly when $h=I^{-1}s$ with probability one, proving both the bound and its equality condition. $\square$

In a regular generalized linear model from an exponential dispersion family, with known dispersion and the specified mean and variance, the likelihood score for $\beta$ is the quasi-score. When the Fisher information is nonsingular, the quasi-score therefore attains the full-model information bound. The same conclusion applies whenever the full-model score agrees with the quasi-score, subject to the regularity conditions above. In contrast, specifying only a mean and variance does not specify a Fisher information matrix or guarantee that equality condition.

Worked Example: Poisson Intercept Model

Suppose $Y_1,\ldots,Y_4$ are independent Poisson responses with mean $\mu=e^\beta$, so $D_i=\mu$, $v_i=\mu$, and $\phi=1$. At $\mu=2$, the quasi-score is $$ U(\beta)=\sum_{i=1}^4\frac{\mu}{\mu}(Y_i-\mu) =\sum_{i=1}^4(Y_i-\mu). $$ Its sensitivity is $J=4\mu=8$. Its variance is $K=\sum_{i=1}^4\operatorname{Var}(Y_i)=4\mu=8$. Thus the Godambe covariance is $J^{-1}KJ^{-1}=8/64=1/8$.

The Poisson log-likelihood, apart from terms not involving $\beta$, is $\ell(\beta)=\sum_i(Y_i\beta-e^\beta)$. Differentiating gives $\partial\ell/\partial\beta=\sum_i(Y_i-\mu)=U(\beta)$. The likelihood score and quasi-score coincide; the Fisher information is $8$, and its inverse is $1/8$. Hence the quasi-score attains the full-model information bound here.

Worked Example: Normal Responses With a Nonlinear Mean

Let $Y_1,Y_2,Y_3$ be independent $N(\beta^2,4)$ responses, with $\beta>0$. At $\beta=2$, each mean is $4$, each derivative is $D_i=2\beta=4$, and each variance is $4$. The quasi-score coefficient for each response is $D_i/v_i=1$, so $U(\beta)=\sum_i(Y_i-\beta^2)$.

At $\beta=2$, its sensitivity is $J=\sum_i 2\beta=12$, and its variance is $K=3(4)=12$. The Godambe covariance is $J^{-1}KJ^{-1}=12/144=1/12$. The likelihood score is also $U(\beta)$: differentiating the normal log-likelihood gives $\sum_i (Y_i-\beta^2)(2\beta)/4$, which equals $\sum_i(Y_i-\beta^2)$ at $\beta=2$. Thus the quasi-score attains the likelihood information bound at this parameter value.

Scope of the Result and a Common Mistake

The theorem about linear estimating functions is useful when the model commits to a mean and variance structure but not to a complete distribution. It says that, among unbiased equations formed by weighting response residuals with coefficients that do not depend on the responses, the quasi-score uses the optimal weights. This gives a principled reason for the factors $D_i/v_i$: observations with greater variance receive less weight, while means more responsive to the parameter contribute more information.

A common mistake is to conclude that the quasi-score must be optimal among all unbiased functions of the responses. That conclusion does not follow from the linear-class theorem. A fully specified distribution may contain information expressed through nonlinear features of the responses, and its likelihood score need not be proportional to the quasi-score. The full information bound is attained only when the equality condition in the second theorem holds. If only the means and variances are specified, that condition cannot generally be checked because the full likelihood score is not specified.

The assumptions also matter. Dependence between responses changes the covariance calculation and generally requires a covariance matrix rather than separate variances. A misspecified variance changes the weights and can change the sandwich covariance. If dispersion is estimated rather than known, the statements above apply directly only when treating it as fixed; additional nuisance-parameter considerations may be needed. These qualifications do not negate quasi-score optimality, but they identify exactly what is being compared and under which information the comparison is valid.

Key takeaway: With independent responses and correctly specified means and variances, the quasi-score minimizes Godambe covariance among unbiased estimating functions linear in the response residuals. It attains the full parametric information bound only when its normalized estimating function equals the normalized likelihood score.

Check Your Understanding

Use the assumptions and distinctions in this tutorial to answer the following questions.

For an independent response with mean derivative $D_i$ and variance $v_i$, what coefficient does the quasi-score assign to its residual?
In the linear optimality proof, why do derivatives of the coefficient functions $a_i(\beta)$ make no contribution to the expected sensitivity?
What does the positive-semidefinite comparison of Godambe covariance matrices say about the precision of linear combinations of estimated parameters?
State the equality condition for attaining the full-model information bound.
Why does optimality among linear unbiased estimating functions not establish optimality among every unbiased function of the responses?

What You'll Learn