Working Weights in Quasi-Score Equations

Why Working Weights Matter

A regression quasi-score combines residuals with information about how the response mean changes with the predictors and how variable the response is at that mean. The resulting observation-specific weights are not arbitrary: they are determined by the link derivative and the specified mean–variance relationship. Understanding that construction clarifies both the score equation and the weighted least-squares calculations used to update regression estimates.

There are two related quantities to keep distinct. The coefficient multiplying an ordinary response residual in the quasi-score is \(D_i/(\phi V(\mu_i))\). The working weight is \(D_i^2/V(\mu_i)\), where \(D_i\) is the derivative of the mean with respect to the linear predictor. The working weight appears when the residual is expressed on the linear-predictor scale, and it determines the local weighted least-squares geometry.

Here \(x_i\) is a fixed \(p\)-vector of predictors and \(\beta\) is the \(p\)-vector of regression coefficients. We assume the means lie in a region where \(D_i\) is finite and nonzero. The quasi-score for \(\beta\) is

The formula shows how a response residual \(Y_i-\mu_i\) enters the score. Its coefficient depends on both the sensitivity \(D_i\) of the mean to the linear predictor and the variance \(V(\mu_i)\). To see the working weight, put the residual on the scale of the linear predictor: locally, a mean change \(Y_i-\mu_i\) corresponds to a predictor-scale change \((Y_i-\mu_i)/D_i\). Multiplying that change by \(w_i/\phi\) recovers the original score contribution.

Deriving the Weighted-Residual Form

The identity follows directly from the definitions, but it is useful to track each factor: one factor of \(D_i\) converts the response residual to the linear-predictor scale, and the other enters the working weight.

In matrix notation, let \(X\) have rows \(x_i^T\), let \(W\) be diagonal with entries \(w_i\), and let \(z\) and \(\eta\) be the vectors of working responses and linear predictors. Then

This is a weighted residual sum on the working-response scale: the residual is \(z_i-\eta_i\), not \(Y_i-\mu_i\). In contrast, writing the score directly in terms of response residuals gives coefficient \(D_i/\{\phi V(\mu_i)\}\). Calling that coefficient the working weight would lose a factor of \(D_i\) and confuse two different residual scales.

Why These Weights Govern a Scoring Update

The same weights arise from the expected sensitivity of the quasi-score to changes in \(\beta\). The result below requires differentiability of \(V\) and \(D_i\) as functions of \(\eta_i\) in the region considered, along with finite means so that \(E[Y_i-\mu_i]=0\). It identifies the Fisher-scoring information matrix; it does not say that the observed derivative of the score equals this matrix.

Proof. Write \(r_i=Y_i-\mu_i\) and \(a_i=D_i/\{\phi V(\mu_i)\}\), so the \(i\)th score contribution is \(x_i a_i r_i\). Since \(\eta_i=x_i^T\beta\), differentiation with respect to \(\beta\) gives

In this derivative, \(dr_i/d\eta_i=-d\mu_i/d\eta_i=-D_i\); the term involving \(da_i/d\eta_i\) remains multiplied by the residual. Taking expectations at the specified mean model makes that term zero because \(E[r_i]=0\). Therefore

By definition, the Fisher-scoring increment solves \(A(\beta)\delta=U(\beta)\). Substituting this expression for \(A(\beta)\), and the working-response identity for \(U(\beta)\), gives \((X^TWX/\phi)\delta=X^TW(z-\eta)/\phi\). Since \(\phi>0\), multiplying by \(\phi\) yields the stated weighted least-squares equation. If \(X^TWX\) is nonsingular, this equation has a unique solution for \(\delta\). \(\square\)

For the matrix to be positive definite, a sufficient condition is that \(X\) has full column rank and every \(w_i>0\). In fact, for any nonzero vector \(b\), \(b^TX^TWXb=\sum_i w_i(x_i^Tb)^2>0\) under those conditions. If the design is rank deficient, the update may not be unique, even though the score identity still holds.

Worked Models

The examples below calculate both the response-residual score contribution and its working-response representation. They use \(\phi\) explicitly: when it is common across observations, it scales the full score and scoring information but is not part of the definition of \(w_i\).

Interpreting and Checking the Weights

A working weight is large when a change in the linear predictor produces a substantial change in the mean relative to the response variance. Its numerator \(D_i^2\) measures mean sensitivity on the predictor scale, while its denominator \(V(\mu_i)\) accounts for the mean-dependent variance. Thus neither the link nor the variance function alone determines the weight: both matter.

The common dispersion \(\phi\) is deliberately outside \(w_i\) here. Including it in each weight as \(D_i^2/\{\phi V(\mu_i)\}\) is an alternative convention, but then the score and information formulas must be written consistently. In the convention used here, the factor \(1/\phi\) multiplies the full score and information matrix, and cancels from the Fisher-scoring increment.

• Do not confuse scales. The score coefficient for \(Y_i-\mu_i\) is \(D_i/\{\phi V(\mu_i)\}\); the working weight for \(z_i-\eta_i\) is \(D_i^2/V(\mu_i)\).
• Do not treat weights as fixed across iterations. In general, \(\mu_i\), \(D_i\), and therefore \(w_i\) change when the fitted coefficients change.
• Do not claim an exact Newton step. The weighted least-squares equation derived here uses the negative expected score derivative. The observed score derivative can contain additional residual-dependent terms.