Working Weights Near Mean Boundaries

Why Boundary Weights Matter

In iteratively reweighted least squares, each observation contributes a working weight that depends on its fitted mean. A fitted mean near the edge of its possible range can therefore alter the numerical influence of that observation: its weight may approach zero, remain moderate, or grow without bound. The direction is not determined by the variance function alone. It depends jointly on the variance and on how the inverse link changes near the boundary.

This is important when an algorithm produces fitted probabilities close to zero or one, or fitted positive means close to zero. A small weight means that an observation contributes little to the local weighted least-squares step; a very large weight can dominate that step and worsen its conditioning. These are properties of the local GLM calculation, not by themselves evidence that a fitted model is wrong.

The Working-Weight Formula

Consider independent GLM observations \(Y_i\) with means \(\mu_i\) and variances \(\operatorname{Var}(Y_i)=\phi V(\mu_i)\), where \(\phi>0\) is the dispersion parameter. Write \(\eta_i=g(\mu_i)\) for the linear predictor and \(\mu_i=h(\eta_i)=g^{-1}(\eta_i)\) for the inverse link. Assume that \(h\) is differentiable at the fitted predictor, with derivative \(q_i=d\mu_i/d\eta_i\). If observation \(i\) has a positive prior weight \(a_i\), its Fisher-scoring working weight is

W_i=\frac{a_i}{\phi}\frac{q_i^2}{V(\mu_i)} =\frac{a_i}{\phi}\frac{\{d\mu_i/d\eta_i\}^2}{V(\mu_i)}.

Some presentations absorb \(\phi\) or the prior weight into the definition of the working weight. Here they are shown explicitly. For an ordinary observation with unit prior weight, set \(a_i=1\). The formula presumes \(V(\mu_i)>0\) at interior means; boundary behavior is studied by taking a limit from the interior.

Result: For a regular GLM observation with score for the mean \((Y_i-\mu_i)/\{\phi V(\mu_i)\}\), the Fisher information for \(\eta_i\) is \(q_i^2/\{\phi V(\mu_i)\}\). A positive prior weight \(a_i\) multiplies this information by \(a_i\), giving the working weight above.

To verify the result, the chain rule gives the score for \(\eta_i\):

U_{\eta_i} =\frac{Y_i-\mu_i}{\phi V(\mu_i)}q_i.

Taking its second moment under the model, using \(E[Y_i-\mu_i]=0\) and \(\operatorname{Var}(Y_i)=\phi V(\mu_i)\), gives

E[U_{\eta_i}^2] =\frac{q_i^2}{\phi^2 V(\mu_i)^2} E[(Y_i-\mu_i)^2] =\frac{q_i^2}{\phi V(\mu_i)}.

Under the usual regularity conditions for the likelihood score, this second moment is the Fisher information. Multiplying by \(a_i\) gives the stated weighted contribution. This derivation also shows why the weight is a squared inverse-link derivative divided by the variance: both features enter the information. \(\square\)

Equivalently, a local linearization of the mean around a current predictor uses the working response \(z_i=\eta_i+(Y_i-\mu_i)/q_i\). The weighted least-squares coefficient of \((z_i-\eta_i)^2\) is proportional to \(a_iq_i^2/\{\phi V(\mu_i)\}\). Thus the same quantity controls the local step.

A General Boundary Test

Suppose the mean approaches a finite boundary \(B\), and let \(t\) denote its distance from that boundary: \(t=\mu\) for a zero boundary, or \(t=B-\mu\) for an upper boundary. Assume that as \(t\) decreases to zero,

V(\mu)\sim C t^\alpha, \qquad \left|\frac{d\mu}{d\eta}\right|\sim D t^\beta, \qquad C>0,\quad D>0.

Here \(r(t)\sim s(t)\) means \(r(t)/s(t)\to1\). The exponents \(\alpha\) and \(\beta\) describe the leading rates at which the variance and inverse-link derivative approach zero or change size. For fixed \(a_i>0\) and \(\phi>0\), these assumptions give a direct classification.

Theorem: Under the boundary assumptions above, the working weight satisfies \(W_i\sim (a_iD^2/\{\phi C\})t^{2\beta-\alpha}\). It tends to zero if \(2\beta-\alpha>0\), tends to a finite positive limit if \(2\beta-\alpha=0\), and diverges if \(2\beta-\alpha<0\).

Proof. Substitute the two asymptotic equivalences into the working-weight formula. Squaring the derivative gives \(q_i^2\sim D^2t^{2\beta}\), while the variance satisfies \(V(\mu)\sim Ct^\alpha\). Therefore

W_i =\frac{a_i}{\phi}\frac{q_i^2}{V(\mu)} \sim \frac{a_iD^2}{\phi C}t^{2\beta-\alpha}.

The leading constant is finite and strictly positive. As \(t\to0\), a positive power of \(t\) tends to zero, a zero power equals one, and a negative power diverges. These three cases prove the classification. \(\square\)

This power-rate test applies when the stated equivalences hold. If the derivative or variance has a logarithmic factor, compare the full ratio \(q_i^2/V(\mu)\) instead of using exponents alone. Also, a link can map a finite predictor to a boundary, or approach that boundary only as \(\eta_i\) tends to infinity; the weight limit is about the fitted mean approaching the boundary, regardless of which predictor sequence produces it.

Worked Examples

Worked Example: Logistic Binomial Weights Near Zero and One

For a Bernoulli response with the logit link, \(g(\mu)=\log\{\mu/(1-\mu)\}\), the inverse link is \(\mu=e^\eta/(1+e^\eta)\). Take \(\phi=1\) and unit prior weight. Differentiate the inverse link:

\frac{d\mu}{d\eta} =\frac{e^\eta}{(1+e^\eta)^2} =\mu(1-\mu).

The Bernoulli variance function is \(V(\mu)=\mu(1-\mu)\). Consequently,

W=\frac{\{\mu(1-\mu)\}^2}{\mu(1-\mu)} =\mu(1-\mu).

At \(\mu=0.01\), the weight is \(0.01(0.99)=0.0099\). At \(\mu=0.5\), it is \(0.25\), and at \(\mu=0.99\), it is again \(0.0099\). As \(\mu\to0\), \(W\sim\mu\to0\); as \(\mu\to1\), \(W\sim1-\mu\to0\). The logit derivative vanishes quickly enough at each endpoint that the weight vanishes even though the variance also vanishes there. With \(m\) binomial trials represented by a prior weight \(a_i=m\), each displayed weight is multiplied by \(m\); for fixed \(m\), it still tends to zero at both boundaries.

Worked Example: Identity-Link Binomial Weights Can Diverge

Use the binomial variance function \(V(\mu)=\mu(1-\mu)\), but take the identity link \(\eta=\mu\), restricted to means between zero and one. For a unit prior weight and \(\phi=1\), \(d\mu/d\eta=1\), so

W=\frac{1}{\mu(1-\mu)}.

At \(\mu=0.01\), \(W=1/(0.01\cdot0.99)=1/0.0099\approx101.01\). At \(\mu=0.001\), \(W=1/(0.001\cdot0.999)=1/0.000999\approx1001.00\). Near zero, \(1-\mu\to1\), so \(W\sim1/\mu\) and diverges. Near one, \(\mu\to1\), so \(W\sim1/(1-\mu)\) and also diverges. In the boundary test, the variance has exponent \(\alpha=1\) while the derivative has exponent \(\beta=0\), giving \(2\beta-\alpha=-1\). The divergence is therefore the predicted result. This link also requires the fitted predictor to remain in the admissible mean range.

Worked Example: Poisson Log-Link Weights at the Zero Boundary

For a Poisson response, \(V(\mu)=\mu\). With the log link, \(\eta=\log\mu\), the inverse link is \(\mu=e^\eta\) and \(d\mu/d\eta=\mu\). For unit prior weight,

W=\frac{\mu^2}{\mu}=\mu.

At a fitted mean of \(0.02\), the weight is \(0.02\); at a fitted mean of \(5\), it is \(5\). As \(\mu\to0\), the weight vanishes. The boundary test gives \(\alpha=1\), \(\beta=1\), and exponent \(2\beta-\alpha=1\), agreeing with the exact formula. The Poisson mean has no finite upper endpoint, so there is no finite upper-boundary limit to evaluate: as \(\mu\) grows without bound, this weight grows without bound. This is distinct from approaching a finite upper limit such as the binomial boundary \(\mu=1\).

Worked Example: A Log-Link Gamma Model Has a Bounded Weight

For a Gamma GLM, take \(V(\mu)=\mu^2\) and the log link \(\eta=\log\mu\), so \(d\mu/d\eta=\mu\). With unit prior weight,

W=\frac{\mu^2}{\phi\mu^2}=\frac{1}{\phi}.

If \(\phi=0.5\), the weight is \(2\) at \(\mu=0.01\), at \(\mu=1\), and at \(\mu=100\). In particular, as the positive mean approaches zero, the weight neither vanishes nor diverges. Here the variance and squared derivative have the same order, so their ratio stays constant. The calculation illustrates why a small variance alone does not imply a large weight: the inverse-link derivative must be considered at the same time.

Interpreting Extreme Weights

A vanishing weight means the local Fisher information in the predictor direction contributed by that observation is approaching zero. In the logistic example, observations fitted with probabilities extremely close to either endpoint have small local weights. A large weight means the local quadratic approximation assigns substantial influence to that observation; in the identity-link binomial example, the variance collapses at the endpoint while the inverse-link derivative remains one, producing a divergent ratio.

These limits are diagnostics, not universal rules to clip every weight below or above a fixed threshold. A threshold depends on the scale of the model, the prior weights, and the dispersion convention. Changing a weight artificially changes the local optimization problem. First check that the mean and inverse-link derivative are computed correctly, that the chosen link respects the mean domain, and that the fitted mean is genuinely near a boundary rather than outside the admissible range.

Write the model quantities.
Identify \(V(\mu)\), the inverse link \(\mu=h(\eta)\), and \(d\mu/d\eta\). Include the dispersion and prior weight used by the fitting procedure.

Identify the boundary and distance.
For a zero boundary use \(t=\mu\); for a finite upper boundary \(B\), use \(t=B-\mu\).

Compare rates.
Evaluate \(q^2/V(\mu)\) directly, or apply the power-rate test when the required asymptotic forms hold.

Common mistake: “The variance goes to zero, so the working weight must go to infinity.” This conclusion ignores the numerator. In the logistic binomial model both \(V(\mu)\) and \(q^2\) vanish, and their ratio tends to zero. The correction is to analyze the full expression \(q^2/V(\mu)\), not the variance in isolation. Conversely, if the derivative stays nonzero while the variance vanishes, as in the identity-link binomial calculation, divergence is indeed the result.

Key takeaway: Near a mean boundary, the working weight is governed by the competition between the squared inverse-link derivative and the variance function. Compute their ratio before deciding whether the weight vanishes, remains bounded, or becomes excessively large.

Check Your Understanding

Use the working-weight formula and boundary-rate reasoning to answer each question.

For the logistic Bernoulli model, what happens to the working weight as \(\mu\to0\), and which terms in the formula determine that limit?
A model has \(V(\mu)\sim 4\mu^2\) as \(\mu\to0\), while \(d\mu/d\eta\sim3\mu\). Does its unit-prior working weight vanish, stay finite, or diverge? Explain using the boundary exponent.
Why is it incorrect to infer the boundary behavior of a working weight from \(V(\mu)\) alone?
For the identity-link binomial model, calculate the unit-prior weight at \(\mu=0.02\) using \(V(\mu)=\mu(1-\mu)\).
In a log-link Gamma model with \(V(\mu)=\mu^2\), what is the working weight as \(\mu\to0\), and how does dispersion affect it?