Influence Analysis for Extreme-Weight Observations

Why Examine Influence After Weight Diagnostics?

Weight diagnostics can identify observations with unusually large weights, but weight size alone does not establish that an observation materially affects an estimate. A large weight attached to an outcome near the weighted mean may have modest influence on that mean. A less extreme weight attached to a very different outcome may matter more. Influence analysis makes this distinction concrete by asking how much an estimate or diagnostic summary changes when an observation, or a prespecified group of observations, is removed.

In a marginal structural model (MSM), a weighted estimate is built from contributions across individuals. A deletion analysis is therefore a sensitivity diagnostic: it identifies whether the reported result depends heavily on particular contributions. It does not show that the influential observation is erroneous, nor does it establish that the estimate is unbiased. In particular, deleting a person changes the analyzed sample and can change which population the data represent.

Definition: For an estimate or diagnostic summary $T$ computed from $n$ individuals, the leave-one-out value for individual $i$, written $T_{(-i)}$, is the value recomputed after removing that individual. The case-deletion change is $\Delta_i=T_{(-i)}-T$. A grouped deletion analysis removes a specified set $G$ of individuals and compares $T_{(-G)}$ with $T$.

For a useful comparison, report the original value, the deleted value, and their difference on the same scale. When an estimate is a contrast, inspect its component means as well as the contrast: a similar contrast can sometimes conceal substantial changes in both means. For diagnostic summaries, apply the same definition and calculation before and after deletion—for example, recompute the same weighted covariate mean or balance measure.

Exact Deletion for a Weighted Mean

Let $Y_i$ be an outcome and $W_i>0$ a fixed analysis weight for individual $i$. The normalized, or Hájek, weighted mean is

\hat{\mu}=\frac{\sum_{i=1}^{n}W_iY_i}{\sum_{i=1}^{n}W_i}.

This is a ratio: removing an individual changes both the weighted outcome total and the sum of weights. Its deletion change has a simple exact form.

Result: If $\sum_{j\ne i}W_j>0$, then $$ \hat{\mu}_{(-i)}-\hat{\mu} =\frac{W_i}{\sum_{j\ne i}W_j}(\hat{\mu}-Y_i). $$ Thus, for fixed weights, case-deletion influence depends both on the individual’s weight relative to the remaining total and on the distance between that individual’s outcome and the original weighted mean.

To prove the identity, write $S_W=\sum_j W_j$ and $S_{WY}=\sum_jW_jY_j$, so that $\hat{\mu}=S_{WY}/S_W$. Then

\begin{aligned} \hat{\mu}_{(-i)}-\hat{\mu} &=\frac{S_{WY}-W_iY_i}{S_W-W_i}-\frac{S_{WY}}{S_W}\\ &=\frac{S_W(S_{WY}-W_iY_i)-S_{WY}(S_W-W_i)} {S_W(S_W-W_i)}\\ &=\frac{W_i(S_{WY}-S_WY_i)}{S_W(S_W-W_i)}\\ &=\frac{W_i}{S_W-W_i}(\hat{\mu}-Y_i). \end{aligned}

The denominator condition is essential: after deletion, there must still be positive total weight. The identity also gives a quick check on the direction of change. If $Y_i>\hat{\mu}$, deleting $i$ lowers the mean; if $Y_i<\hat{\mu}$, deleting $i$ raises it.

Worked Example: A Large Weight and a Weighted Mean

Suppose four individuals have fixed weights $1,1,1,7$ and outcomes $0,2,3,10$, respectively. Calculate the weighted mean and its leave-one-out change when the individual with weight $7$ is removed.

The total weight is $1+1+1+7=10$, and the weighted outcome total is $1(0)+1(2)+1(3)+7(10)=75$. Therefore, $\hat{\mu}=75/10=7.5$. After removing the fourth individual, the mean is $\hat{\mu}_{(-4)}=(0+2+3)/3=5/3$, or approximately $1.667$. The deletion change is $1.667-7.5=-5.833$.

The exact identity gives the same answer: \[ \frac{7}{3}(7.5-10)=\frac{7}{3}(-2.5)=-\frac{35}{6}\approx-5.833. \] The observation matters here because its weight is large relative to the remaining weight and its outcome is above the original weighted mean.

As an accompanying weight diagnostic, the effective sample size is $\operatorname{ESS}=(\sum_iW_i)^2/\sum_iW_i^2$. Before deletion it is $10^2/(1+1+1+49)=100/52\approx1.923$. After deletion it is $3^2/(1+1+1)=3$. Removing the large weight increases ESS, but this does not by itself make the deletion appropriate; it shows how concentrated the original weighted sample was.

Influence for Weighted MSM Estimates

Many weighted MSM estimates are obtained by solving an estimating equation rather than by calculating one weighted mean. Write the equation as $\sum_{i=1}^n\psi_i(\theta)=0$, where $\theta$ is the parameter and $\psi_i(\theta)$ is individual $i$'s contribution, including its analysis weight. Let $\hat{\theta}$ solve the full-sample equation and $\hat{\theta}_{(-i)}$ solve the equation with contribution $i$ removed.

Result (first-order case-deletion approximation): Suppose $\psi_i$ is differentiable near $\hat{\theta}$; a deleted-sample root $\hat{\theta}_{(-i)}$ exists near $\hat{\theta}$; $\psi_i(\hat{\theta})=O(1)$; and $$ \frac{1}{n}\sum_{j\ne i}\frac{\partial\psi_j(\theta)}{\partial\theta^\mathsf{T}} \longrightarrow -A $$ along the sequence under consideration, for a nonsingular matrix $A$. Assume also that the sum of the second-derivative remainders from Taylor expansion between $\hat{\theta}$ and $\hat{\theta}_{(-i)}$ is $O(n\|\hat{\theta}_{(-i)}-\hat{\theta}\|^2)$. Then $$ \hat{\theta}_{(-i)}-\hat{\theta} =-\frac{1}{n}A^{-1}\psi_i(\hat{\theta})+o(n^{-1}). $$

For the derivation, set $\delta_i=\hat{\theta}_{(-i)}-\hat{\theta}$. The full equation is zero, so the deleted equation evaluated at $\hat{\theta}$ equals $-\psi_i(\hat{\theta})$. Taylor expansion of the deleted equation around $\hat{\theta}$ gives

0=-\psi_i(\hat{\theta})+ \left\{\sum_{j\ne i}\frac{\partial\psi_j(\hat{\theta})}{\partial\theta^\mathsf{T}}\right\} \delta_i+R_i.

Under the stated remainder condition and the nonsingular limiting derivative, the deleted root is locally $O(n^{-1})$ from the full root: the derivative sum is of order $n$, while the removed contribution is of order one. Consequently $R_i=O(n\|\delta_i\|^2)=O(n^{-1})$. The derivative sum is $-nA+o(n)$, whose inverse is $-n^{-1}A^{-1}+o(n^{-1})$. Solving the expansion for $\delta_i$ therefore gives the stated approximation. This is a local approximation, not an exact deletion formula; it can be poor when a contribution is so large that the deleted root is not close to the full root. $\square$

The contribution $\psi_i$ contains more than the weight. Its size and direction depend on the outcome, treatment history, covariates, and the estimating equation. This is why the approximation complements, rather than replaces, direct refitting after deletion.

Worked Example: Checking the Exact Formula Against the Data

Take weights $1,2,3,4$ and outcomes $1,2,4,8$. The total weight is $10$, and the weighted outcome total is $1+4+12+32=49$, so $\hat{\mu}=4.9$. For the individual with $W_i=4$ and $Y_i=8$, the deleted mean is $(49-32)/(10-4)=17/6\approx2.833$. Thus the deletion change is $17/6-4.9=-31/15\approx-2.067$.

The exact identity yields $\{4/6\}(4.9-8)=(2/3)(-3.1)=-31/15$, confirming the calculation. This individual’s weight is not as dominant as the weight in the preceding example, but the outcome is sufficiently far above the weighted mean to produce a substantial change. The example illustrates why ranking observations only by weight can miss influential cases.

Grouped Deletion and Diagnostic Summaries

A single observation is not always the natural unit for influence analysis. Individuals may be linked by a clinic, household, site, or treatment-history pattern. A grouped deletion asks whether the analysis depends on a collection of contributions that could be influential together. Define $W_G=\sum_{i\in G}W_i$ and $S_{WY,G}=\sum_{i\in G}W_iY_i$. Provided the remaining total weight is positive, the weighted mean after deleting group $G$ is

\hat{\mu}_{(-G)} =\frac{S_{WY}-S_{WY,G}}{S_W-W_G}, \qquad \hat{\mu}_{(-G)}-\hat{\mu} =\frac{W_G}{S_W-W_G} \left(\hat{\mu}-\frac{S_{WY,G}}{W_G}\right),

when $W_G>0$. The formula follows by the same subtraction used for single-case deletion, with the group’s weighted mean replacing $Y_i$. A group can have a noticeable effect even if no individual deletion appears decisive, because the group’s contributions accumulate.

Worked Example: Deleting a Cluster From a Weighted Contrast

Consider a weighted contrast of treated and untreated means, $\hat{\tau}=\hat{\mu}_1-\hat{\mu}_0$, with three clusters. Each cluster contributes one treated and one untreated individual. The observations $(W,Y)$ are: cluster 1, treated $(1,2)$ and untreated $(1,1)$; cluster 2, treated $(2,4)$ and untreated $(2,2)$; cluster 3, treated $(7,10)$ and untreated $(1,3)$.

The treated weighted mean is $(1(2)+2(4)+7(10))/(1+2+7)=80/10=8$. The untreated weighted mean is $(1(1)+2(2)+1(3))/(1+2+1)=8/4=2$. Thus the full contrast is $8-2=6$.

After deleting cluster 3, the treated mean is $(2+8)/(1+2)=10/3$, and the untreated mean is $(1+4)/(1+2)=5/3$. The contrast becomes $10/3-5/3=5/3\approx1.667$, a change of $5/3-6=-13/3\approx-4.333$. The treated mean falls by $8-10/3=14/3$; the untreated mean falls by only $2-5/3=1/3$. The contrast change is therefore driven mainly by the treated contribution in cluster 3, not by a comparable shift in both arms.

The same deletion logic applies to diagnostics. Recalculate the chosen summary after deletion: for example, weighted covariate means, a standardized difference, or ESS. A group deletion may reveal that a seemingly acceptable balance summary depends on a small set of observations. Conversely, if the estimate moves but balance summaries barely change, the sensitivity may reflect outcome contributions rather than a large change in measured covariate balance. Neither pattern alone proves that the causal assumptions hold or fail.

How to Conduct and Interpret the Analysis

Specify the target summaries.
Choose the weighted estimate and the diagnostic summaries to inspect before examining deletion results. State whether the analysis uses fixed fitted weights or refits the treatment model.

Calculate individual deletions.
For each observation, remove its contribution, recompute the estimate and diagnostics, and record absolute changes. For a contrast, also record its component means.

Examine defensible groups.
Delete meaningful clusters or prespecified history groups and recompute the same summaries. Check that each remaining analysis has enough observations and positive total weight in every required component.

Explain the changes.
Relate large changes to the deleted observations’ weights, outcomes, and treatment groups. Report both the scale of the change and any failure of the deleted analysis to be estimable.

Holding weights fixed isolates the direct contribution of the deleted observation under the originally fitted weighting scheme. Refitting the treatment model after deletion answers a broader question: how much does removing the observation change both the model fit and the resulting weights? These are different analyses. If both are useful, label them separately; a refitted deletion does not preserve the simple exact identity above because the remaining observations’ weights may also change.

A useful comparison is with weight-only inspection. Ranking by maximum weight is quick and identifies potential concentration, but it does not incorporate outcomes or the estimating equation. Deletion analysis does incorporate their combined impact on the chosen summaries, but it is more computationally demanding and can be unstable for small groups. Bootstrap variability in weight diagnostics addresses resampling variability; case deletion instead asks how dependent a particular fitted analysis is on specified contributions. These approaches answer related but distinct questions.

Common Pitfalls and Limitations

Equating a large weight with large influence. A large weight can have little effect on a mean when its outcome is near that mean. Examine the change in the target estimate, not just the weight rank.
Treating deletion as a correction. Removing an influential individual because the estimate changes can selectively alter the target population. Investigate data quality and report the sensitivity; do not delete cases solely to obtain a preferred result.
Ignoring loss of estimability. After deletion, a treatment arm or history group may have zero total weight, or a model may no longer have a unique solution. This is a substantive warning, not a zero influence value.
Overinterpreting a stable estimate. Stability under deletion of observed cases does not establish exchangeability, positivity, correct treatment models, or absence of unmeasured confounding.
Choosing groups after seeing the answer. Extensive outcome-guided group selection can exaggerate apparent sensitivity. Define meaningful groups in advance where possible and state how they were chosen.

Influence is always relative to a chosen estimate or diagnostic. An observation can strongly affect a weighted outcome estimate but barely affect a covariate balance summary, or vice versa. Report which quantity changed, by how much, and under which deletion scheme; a single label such as “influential” is not a complete diagnosis.

Check Your Understanding

Answer each question, then compare with the solutions.

For the weighted mean in the first worked example, why does deleting the individual with outcome $10$ lower the mean?
In the exact deletion identity, what two features determine the direction and size of the case-deletion change?
Why is deleting a cluster not always equivalent to reporting the most influential single observation?
What is the difference between holding fitted weights fixed and refitting the treatment model after deletion?

Solutions. 1. The outcome $10$ is above the full weighted mean $7.5$, so its removal lowers the mean; its weight is also large relative to the remaining total, making the change substantial. 2. The factor $W_i/\sum_{j\ne i}W_j$ measures the removed weight relative to the remaining weight, while $\hat{\mu}-Y_i$ gives the direction and outcome distance. 3. A group’s contributions accumulate, and its weighted outcome mean may differ from the full-sample mean even when no one member dominates alone. 4. Fixed weights isolate the removed contribution under the original fitted scheme. Refitting can change other individuals’ weights as well, so it assesses the combined effect of case removal and model refitting.