Weight Leverage and Outcome Influence

Why Weight Size Is Not the Same as Influence

A weighted analysis gives some observations more influence in the estimating procedure than others, but a large weight does not by itself show that an observation changes the estimated effect substantially. Its impact also depends on its outcome relative to the weighted fit and, for a model-based estimate, on its covariate pattern and estimating-equation contribution. The useful diagnostic question is therefore not simply “Which weights are large?” but “Which observations, through their weights and outcomes, can move the estimate?”

This distinction matters in marginal structural models (MSMs), where inverse-probability weights are used to estimate marginal outcomes or model parameters. The earlier material on weight distributions and effective sample size describes concentration in the weights. Here the focus is the additional information in the outcomes and in the estimating equation. These diagnostics describe sensitivity of a fitted estimate; they do not establish that the causal assumptions or weight models are correct.

Weighted Means and Leave-One-Out Influence

Definition: For observations \(i=1,\ldots,n\), let \(W_i\geq 0\) be fixed analysis weights and \(Y_i\) their outcomes. If \(S=\sum_i W_i>0\), the weighted mean is \(\hat{\mu}_W=\sum_i W_iY_i/S\). An observation’s leave-one-out change is the estimate after deleting it minus the original estimate, provided the remaining total weight is positive.

For a single observation, the exact deletion calculation shows how weight and outcome work together. Write \(S_{-i}=S-W_i\). When \(S_{-i}>0\),

\hat{\mu}_{W,-i}-\hat{\mu}_W =\frac{S\hat{\mu}_W-W_iY_i}{S-W_i}-\hat{\mu}_W =\frac{W_i(\hat{\mu}_W-Y_i)}{S-W_i}.

The change is zero if \(W_i=0\), or if \(Y_i=\hat{\mu}_W\). It can be large when the observation has substantial weight and its outcome differs from the weighted mean, especially if deleting it leaves little weight in the denominator. The signed quantity \(W_i(Y_i-\hat{\mu}_W)\) is the observation’s unnormalised weighted residual contribution; its magnitude alone is not the deletion change because the remaining denominator also matters.

Worked Example: A Large Weight With a Nonzero Outcome Residual

Consider three observations with weights \(1,1,8\) and outcomes \(2,4,10\). The total weight is \(S=10\), and the weighted mean is

\hat{\mu}_W=\frac{1(2)+1(4)+8(10)}{10}=\frac{86}{10}=8.6.

For the third observation, the weighted residual contribution is \(8(10-8.6)=11.2\). Deleting it leaves the mean \((2+4)/2=3\), so the deletion change is \(3-8.6=-5.6\). The formula gives the same result:

\frac{8(8.6-10)}{10-8}=\frac{-11.2}{2}=-5.6.

For comparison, deleting the first observation gives the mean \((4+80)/9=9.333\ldots\), a change of about \(0.733\). The third observation’s weight and outcome together make it much more influential in this sample. Its weight is not the entire explanation: the residual and the small remaining total weight both contribute.

Worked Example: A Large Weight With No Mean Influence

Now use weights \(1,1,8\) with outcomes \(0,10,5\). The weighted mean is

\hat{\mu}_W=\frac{1(0)+1(10)+8(5)}{10}=\frac{50}{10}=5.

The observation with weight \(8\) has outcome \(5\), exactly the weighted mean. Its weighted residual is \(8(5-5)=0\), and deleting it leaves \((0+10)/2=5\). Thus, despite its large weight, its leave-one-out change is zero. The two lower-weight observations have contributions \(-5\) and \(5\), respectively; each deletion changes the mean. A weight-only ranking would miss this distinction.

Weighted Treatment Contrasts

For a binary treatment \(A_i\in\{0,1\}\), a common weighted marginal contrast is the difference between the weighted outcome means in the two treatment groups. Let \(W_i\geq0\) be the analysis weights, and suppose both groups have positive total weight. Define

\hat{\mu}_a= \frac{\sum_i I(A_i=a)W_iY_i}{S_a}, \qquad S_a=\sum_i I(A_i=a)W_i, \qquad \hat{\psi}=\hat{\mu}_1-\hat{\mu}_0.

Deleting an observation in group \(a\) changes only that group’s mean, and the exact mean formula applies within that group. The effect on the contrast has the opposite sign for a deletion from the control group, because the control mean is subtracted. Hence, diagnosing a contrast requires retaining treatment-group membership as well as weight and outcome information.

Worked Example: Deletion Effects on a Weighted Contrast

Suppose the treated observations have weights \(1,9\) and outcomes \(1,5\); the control observations have weights \(4,1\) and outcomes \(2,8\). The treated mean and control mean are

\hat{\mu}_1=\frac{1(1)+9(5)}{10}=4.6, \qquad \hat{\mu}_0=\frac{4(2)+1(8)}{5}=3.2, \qquad \hat{\psi}=4.6-3.2=1.4.

Deleting the treated observation with weight \(9\) leaves treated mean \(1\). The contrast becomes \(1-3.2=-2.2\), a change of \(-3.6\). Equivalently, its treated-mean deletion change is \(9(4.6-5)/(10-9)=-3.6\).

Deleting the control observation with weight \(4\) leaves control mean \(8\), so the contrast becomes \(4.6-8=-3.4\), a change of \(-4.8\). The control-mean deletion change is \(4(3.2-2)/(5-4)=4.8\), and subtracting the control mean makes the contrast change \(-4.8\). These are large changes in opposite group means; the direction for the contrast must be calculated with the group’s sign in mind.

Estimating-Equation Contributions

A weighted mean is a simple estimating-equation estimator. More generally, suppose an estimate \(\hat{\theta}\) solves

\sum_{i=1}^n U_i(\theta)=0,

where \(U_i(\theta)\) is the observation-level estimating function, which may include its analysis weight. A weighted regression score, for example, combines weight, outcome residual, and covariates. A large weight can matter little if the residual is near zero; a moderate residual can matter more when it is multiplied by a covariate pattern that strongly affects the parameter of interest.

Result: Let \(\hat{\theta}\) solve \(\sum_i U_i(\theta)=0\). Suppose the functions are differentiable near \(\hat{\theta}\), the matrix \(J_{-i}=\sum_{j\ne i}\partial U_j(\hat{\theta})/\partial\theta^\mathsf{T}\) is nonsingular, and deleting observation \(i\) gives a root \(\hat{\theta}_{-i}\) close enough to \(\hat{\theta}\) for a first-order Taylor approximation to be informative. Then

\hat{\theta}_{-i}-\hat{\theta} \ \approx\ J_{-i}^{-1}U_i(\hat{\theta}).

To derive this approximation, write \(\Delta_i=\hat{\theta}_{-i}-\hat{\theta}\). The deleted estimating equation is \(\sum_{j\ne i}U_j(\hat{\theta}_{-i})=0\). Taylor expansion around \(\hat{\theta}\) gives

0 \approx \sum_{j\ne i}U_j(\hat{\theta})+J_{-i}\Delta_i = -U_i(\hat{\theta})+J_{-i}\Delta_i,

because the full estimating equation sums to zero. Multiplying by \(J_{-i}^{-1}\) yields the stated approximation. This is a local approximation, not an identity: it can be inaccurate when deletion moves the estimate far enough that curvature matters. For a weighted mean, \(U_i(\mu)=W_i(Y_i-\mu)\) and \(J_{-i}=-(S-W_i)\), so the expression reduces to \(-W_i(Y_i-\hat{\mu}_W)/(S-W_i)\), exactly the deletion change derived above. \(\square\)

For a regular estimator based on independent observations, a large-sample version uses the derivative matrix of the estimating equation and the observation’s contribution. This interpretation requires the usual conditions for the estimator’s asymptotic linear expansion: a well-defined target, differentiability, a nonsingular limiting derivative, and suitable moment and law-of-large-numbers conditions. In longitudinal studies, the independent unit is commonly the participant; contributions from a participant’s repeated records should be combined before assessing participant-level influence.

Worked Example: Weighted Regression Score Contributions

Fit a weighted linear regression with intercept and slope to three points \((x_i,Y_i)=(0,0),(1,2),(2,3)\), using weights \(1,1,8\). The weighted least-squares estimating equation is \(\sum_i W_i(1,x_i)^\mathsf{T}(Y_i-\alpha-\beta x_i)=0\). The weighted sums are \(\sum W_i=10\), \(\sum W_ix_i=17\), \(\sum W_iY_i=26\), \(\sum W_ix_iY_i=50\), and \(\sum W_ix_i^2=33\). Thus,

\hat{\beta} =\frac{50-(17)(26)/10}{33-17^2/10} =\frac{5.8}{4.1} =\frac{58}{41}, \qquad \hat{\alpha}=\frac{26}{10}-\frac{17}{10}\frac{58}{41}=\frac{8}{41}.

The residuals are \(-8/41,\ 16/41,\ -1/41\). The observation with weight \(8\) has a small residual, \(-1/41\), but its score contribution for the intercept and slope is

8(1,2)^\mathsf{T}(-1/41)=(-8/41,-16/41)^\mathsf{T}.

The other two contributions are \((-8/41,0)^\mathsf{T}\) and \((16/41,16/41)^\mathsf{T}\); their sum with the high-weight contribution is zero, as required by the fitted estimating equation. This calculation illustrates why the weighted residual alone is not a complete regression diagnostic: the covariate vector determines how the observation contributes to each parameter. The score contribution is also not itself the coefficient change; the derivative or information matrix maps score changes to parameter changes.

How to Use These Diagnostics

A practical influence review combines several views rather than treating one summary as decisive. For a weighted mean or contrast, calculate exact leave-one-out changes for observations whose weights or outcomes warrant attention. For a fitted MSM, inspect weighted outcome residuals and observation-level estimating-function contributions, then assess whether the implied parameter changes are meaningful on the effect scale. Where repeated observations belong to the same participant, assess deletion at the participant level if the target diagnostic is participant influence.

These checks have different roles. Weight summaries describe the distribution of weights, and effective sample size summarizes weight concentration; neither includes whether weighted outcomes align with the fitted estimate. Outcome residuals show discrepancies from the fitted outcome, but omit the weight and, in regression, the covariate direction. Estimating-equation contributions combine these ingredients, while deletion refits show the actual change for the specified estimator. The last is more direct but can be computationally demanding.

Influence is a property of an estimate in the observed sample, not a test of causal identification. A small deletion change cannot validate exchangeability, consistency, correct model specification, or adequate support. Conversely, a large change flags sensitivity, but does not by itself identify its cause. If weights are estimated, calculations that hold them fixed measure the influence conditional on those weights. Re-estimating the weight models after deletion assesses a different, full-pipeline perturbation and may yield different results.

Common Pitfalls and Limitations

Ranking only by weight. A large weight can have little effect when the outcome is near the weighted target. Pair weight magnitude with the outcome residual or estimating-equation contribution.
Ignoring the denominator. In a weighted mean, the exact deletion change depends on \(S-W_i\), not only on \(W_i(Y_i-\hat{\mu}_W)\). If deletion leaves zero total weight, the leave-one-out mean is undefined.
Using raw contributions as parameter changes. In a multi-parameter model, the derivative matrix scales and mixes the estimating-equation contributions. Use deletion refits or the justified first-order approximation to assess coefficient impact.
Confusing a diagnostic with a remedy. Trimming weights, changing a model, or excluding an observation changes the analysis and potentially its target. Do not make such changes solely to suppress a large influence value without stating and defending the revised procedure.
Treating fixed estimated weights as known. A conditional deletion calculation does not include the effect of refitting treatment or censoring models. State which version of the analysis was perturbed.

Key takeaway: Weight leverage becomes outcome influence through the weighted residual or estimating-equation contribution, and its effect on an estimate also depends on the estimator’s derivative and the remaining information. Diagnose the estimate directly where possible; do not infer influence from weight size alone.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

For a weighted mean, what two features besides the outcome value determine the exact leave-one-out change?
A high-weight observation has outcome exactly equal to the weighted mean. What is its leave-one-out effect on that mean, assuming deletion leaves positive total weight?
Why can a small residual still produce a notable contribution in weighted regression?
In a weighted treated-minus-control contrast, why does deleting a control observation reverse the sign of its effect on the control mean when expressed as a change to the contrast?

Solutions. (1) The exact change depends on the weight \(W_i\) and the remaining total weight \(S-W_i\), as well as the difference \(Y_i-\hat{\mu}_W\); explicitly it is \(W_i(\hat{\mu}_W-Y_i)/(S-W_i)\). (2) The change is zero because the outcome residual is zero. (3) The score contribution includes the weight and covariate vector as well as the residual, and parameter change also depends on the derivative matrix. (4) The contrast is treated mean minus control mean, so a change in the control mean is subtracted: the contrast change is the negative of the control-mean change.