Reporting Truncated-Weight Analyses

Why Reporting Choices Matter

A truncated-weight analysis is not fully described by saying that “extreme weights were capped.” A reader needs to know which weights were constructed, on what scale and at what level they were clipped, how many changed, and what happened to the causal estimate and its uncertainty. Otherwise, the analysis cannot be reproduced, and it is difficult to tell whether a reported result is sensitive to the clipping decision.

The central reporting principle is to make the clipped analysis a transparent modification of a clearly specified unclipped analysis. Both should target the same stated causal contrast. Their numerical difference is informative about sensitivity to the rule; it does not, by itself, show that either estimate is unbiased or that the two estimates are statistically different.

What a Complete Report Specifies

Definition: A truncated-weight analysis report specifies (1) the target causal estimand and analysis unit, (2) how the weights were constructed, (3) the exact clipping rule and scale, (4) how many weights were affected and how this was counted, and (5) the causal estimate and uncertainty for both the clipped and unclipped analyses.

For longitudinal treatment, let $A_{it}$ denote treatment for individual $i$ at time $t$, and let $L_{it}$ denote measured covariates. A common stabilized treatment weight through time $T$ has the form

W_i=\prod_{t=0}^{T} \frac{P(A_{it}\mid \bar A_{i,t-1})} {P(A_{it}\mid \bar A_{i,t-1},\bar L_{it})}.

This formula is an example, not a universal specification: reports should give the actual numerator and denominator models, their conditioning variables, the time range, and whether censoring weights or other components were also multiplied in. State whether clipping was applied to the final cumulative weight $W_i$, to each time-specific factor before multiplication, or to another quantity. Those operations generally produce different weights. If clipping occurs on a log scale, report the log-scale rule and its equivalent on the raw scale when possible.

Write a two-sided clipping rule, for example, as $W_i^{c}=\min\{u,\max(l,W_i)\}$, with lower bound $l$ and upper bound $u$. A one-sided cap is $W_i^{c}=\min(u,W_i)$. A quantile rule must identify the quantiles and state whether the cut points were calculated from the observed sample. Also say how ties at a cut point are handled if that affects the affected count. Stabilization and clipping are distinct operations, so state their order.

The phrase “percentage of weights truncated” is incomplete unless its denominator is clear. For a single final weight per person, report the number and percentage of people whose weight changed. For time-specific weights, report whether the unit is person-time records or people with at least one changed record. These are different summaries and should not be interchanged. It is also useful to report the minimum, selected quantiles, maximum, and an effective sample size before and after clipping, provided the definition of that effective sample size is stated.

Worked Example: Writing Down a Longitudinal Clipping Rule

Suppose a study follows 100 people for four treatment decisions. The analysis constructs each person’s final stabilized treatment weight as the product of the four time-specific numerator-to-denominator probability ratios. Censoring weights are not included. Investigators cap the final weights at 5 and replace weights below 0.2 by 0.2.

A reproducible description is: “We calculated each participant’s cumulative stabilized treatment weight as the product of the numerator-to-denominator conditional treatment probabilities over four decisions. We then clipped the final cumulative weight to the interval $[0.2,5]$, replacing values below 0.2 by 0.2 and values above 5 by 5. Clipping was applied after multiplication, not to the individual time-specific factors.”

Suppose 4 final weights exceed 5 and 3 fall below 0.2. Then $4+3=7$ of 100 participant-level weights changed, or $100(7/100)=7\%$. The report should give both the count and percentage and identify the unit as participants. It should not describe this as 7% of person-time records, because that is not the count being reported.

The weight description does not by itself establish causal identification. The causal interpretation still relies on consistency, sequential exchangeability given the measured history, positivity for the treatment decisions under study, and adequate specification of the models used to estimate the probabilities. A report should state the relevant assumptions and avoid implying that clipping repairs a failure of positivity or confounding control.

Report the Estimate and Its Uncertainty Together

For a marginal structural model (MSM) estimating a mean under each of two treatment regimes, report the target contrast in words and give its scale. For example, state whether the target is a difference in mean outcome under sustained treatment versus no treatment, and identify the outcome time and population. Then present the estimate under the unclipped weights and under the clipped weights, preferably side by side. Include standard errors or confidence intervals and name the uncertainty method.

A robust sandwich variance or a bootstrap can be appropriate, but the method must match the analysis. In particular, when the weights are estimated, uncertainty calculations should account for that estimation. A bootstrap should repeat the relevant analysis pipeline in each resample, including weight estimation and clipping, if those steps are part of the estimator. For clustered or repeated observations, the resampling unit or variance calculation must respect the dependence structure. A standard error computed as if estimated weights were fixed can omit a source of uncertainty.

Result: For a ratio-weighted mean with fixed weights, the influence function makes the uncertainty calculation explicit. If observations $O_i$ are independent and identically distributed, $I_i(a)$ indicates membership in treatment group $a$, and $W_i^c$ is a fixed clipped weight, define $$ \hat\mu_a^c= \frac{\sum_{i=1}^{n} I_i(a)W_i^cY_i} {\sum_{i=1}^{n} I_i(a)W_i^c}. $$ If $E[I_i(a)W_i^c]>0$ and the relevant second moments are finite, then the influence function is $$ \phi_{a,i}^c= \frac{I_i(a)W_i^c(Y_i-\mu_a^c)} {E[I_i(a)W_i^c]}, \qquad \mu_a^c= \frac{E[I_i(a)W_i^cY_i]}{E[I_i(a)W_i^c]}. $$ Under the usual independent-observation central limit conditions, $\sqrt n(\hat\mu_a^c-\mu_a^c)\xrightarrow{d}N(0,\operatorname{Var}(\phi_{a,i}^c))$.

To derive the result, write $U=E[I(a)W^cY]$ and $V=E[I(a)W^c]$, so that $\mu_a^c=U/V$. The sample estimator is the ratio $\hat U/\hat V$. A first-order Taylor expansion of $g(u,v)=u/v$ at $(U,V)$, whose gradient is $(1/V,-U/V^2)$, gives the observation-level contribution

\frac{I_i(a)W_i^cY_i-U}{V} -\frac{U}{V^2}\{I_i(a)W_i^c-V\} = \frac{I_i(a)W_i^c(Y_i-\mu_a^c)}{V}.

The central limit theorem applied to the sample averages of the numerator and denominator, followed by this Taylor expansion, gives the stated limit. In practice, $U$, $V$, and $\mu_a^c$ are replaced by sample estimates to estimate the influence values. This derivation treats the weights as fixed. With estimated weights, use a variance method that accounts for weight estimation rather than presenting this fixed-weight formula as the complete uncertainty calculation.

Worked Example: Calculating the Clipped and Unclipped Contrasts

Consider an illustrative weighted analysis with three observations in each treatment group. The final weights and outcomes are as follows; the upper clipping bound is 2.

Group	Outcome values	Unclipped weights	Weights after clipping at 2
Treatment	10, 12, 20	1, 2, 4	1, 2, 2
Control	8, 9, 15	1, 1.5, 5	1, 1.5, 2

In the treatment group, the unclipped weighted mean is $ (1(10)+2(12)+4(20))/(1+2+4)=114/7\approx16.286$. After clipping, it is $ (1(10)+2(12)+2(20))/(1+2+2)=74/5=14.8$.

In the control group, the unclipped weighted mean is $ (1(8)+1.5(9)+5(15))/(1+1.5+5)=96.5/7.5\approx12.867$. After clipping, it is $ (1(8)+1.5(9)+2(15))/(1+1.5+2)=51.5/4.5\approx11.444$. Thus, the unclipped treatment-minus-control contrast is $16.286-12.867\approx3.419$, while the clipped contrast is $14.8-11.444\approx3.356$. The change, defined as clipped minus unclipped, is approximately $-0.063$.

Two of the six weights exceed 2, so 2/6, or 33.3%, of these participant-level weights changed. This small illustration shows how to calculate and report the affected proportion and both estimates. It is not evidence that the clipping rule would have little effect in a larger study: the weighted observations, causal assumptions, and sampling uncertainty all matter.

Comparing Uncertainty Without Overclaiming

A comparison of two estimates from the same data is paired: the clipped and unclipped estimates are usually dependent. Consequently, their standard errors alone do not determine the uncertainty of their difference. The variance of the difference includes a covariance term.

Result: Suppose two regular estimators of the same stated contrast have influence functions $\phi_i^c$ and $\phi_i^u$, for clipped and unclipped analyses. If their joint first-order expansion holds, then $$ \sqrt n\{(\hat\theta^c-\hat\theta^u)-(\theta^c-\theta^u)\} \xrightarrow{d} N\!\left(0,\operatorname{Var}(\phi_i^c-\phi_i^u)\right). $$ The variance is $$ \operatorname{Var}(\phi_i^c)+\operatorname{Var}(\phi_i^u) -2\operatorname{Cov}(\phi_i^c,\phi_i^u). $$

For the derivation, subtract the two first-order expansions: $\hat\theta^c-\theta^c=n^{-1}\sum_i\phi_i^c+o_p(n^{-1/2})$ and $\hat\theta^u-\theta^u=n^{-1}\sum_i\phi_i^u+o_p(n^{-1/2})$. Their difference is $n^{-1}\sum_i(\phi_i^c-\phi_i^u)+o_p(n^{-1/2})$. The joint central limit theorem gives the stated limit, and expanding the variance of a difference gives the covariance expression. This comparison requires estimating the covariance, for example from paired influence values or a paired bootstrap that repeats both analyses in each resample. Separate confidence intervals do not provide that covariance.

Worked Example: Presenting a Sensitivity Comparison

Suppose a study reports an unclipped causal contrast of 3.42 with standard error 1.80, and a clipped contrast of 3.36 with standard error 1.25. Using a normal approximation, the unclipped 95% confidence interval is $3.42\pm1.96(1.80)=3.42\pm3.528$, or approximately $[-0.11,6.95]$. The clipped interval is $3.36\pm1.96(1.25)=3.36\pm2.45$, or $[0.91,5.81]$.

The point-estimate change is $3.36-3.42=-0.06$. The intervals describe uncertainty for each estimate; their overlap is not a test of whether the estimates differ. To illustrate the paired comparison, suppose the estimated covariance between the two estimates is 0.95. The standard error of their difference is then $\sqrt{1.80^2+1.25^2-2(0.95)}=\sqrt{2.9025}\approx1.70$. The approximate 95% interval for clipped minus unclipped is $-0.06\pm1.96(1.70)$, or about $[-3.39,3.27]$. The covariance value here is assumed for illustration; a real report must estimate it using a method appropriate to the analysis.

A clear report would distinguish the observed change in point estimate from its uncertainty, identify how the paired uncertainty was obtained, and avoid saying that clipping “did not matter” merely because separate intervals overlap.

Diagnostics, Alternatives, and Limitations

A useful report shows enough diagnostics to assess whether the chosen rule is behaving as intended. Check that the reported count of changed weights agrees with the specified rule; summarize the weight distribution before and after clipping; and look for extreme concentration of weighted contribution in a few observations. For repeated or longitudinal data, state whether diagnostics use final participant-level weights or time-specific weights. Unexpectedly unchanged weights, an affected percentage inconsistent with the reported distribution, or a sharp change in the estimate under a modest threshold adjustment warrants investigation.

The main comparison is usually between the unclipped analysis and one or more prespecified clipped analyses. The unclipped analysis retains the original estimated weights but can be unstable when weights are highly variable. Clipping can reduce the influence of extreme weights, while changing the weighted estimating equation and potentially introducing bias relative to the intended causal target. Stabilizing weights and clipping weights are not interchangeable: stabilization changes the weight construction, whereas clipping modifies values according to a bound. A report should not present one as if it were the other.

Clipping is not a substitute for checking model specification, measured confounding, or positivity. A stable-looking clipped estimate may result partly from the chosen bound, and a narrow interval after clipping does not establish that the causal assumptions hold. Thresholds selected after examining results should be identified as such; presenting only the most favorable threshold hides sensitivity. Report the rule-selection rationale and, when relevant, the results for a small set of reasonable thresholds. These comparisons are sensitivity analyses, not proof that one rule is correct.

A Practical Reporting Sequence

Name the estimand and analysis unit.
State the causal contrast, outcome, target population, and whether weights are defined per person or per time-specific record.

Define the weights and clipping operation.
Give the probability models and time range, then state the scale, bounds or quantiles, and whether clipping occurs before or after cumulative weights are formed.

Quantify what changed.
Report counts and percentages with explicit denominators, alongside useful distribution and concentration summaries.

Compare estimates and uncertainty.
Show clipped and unclipped estimates on the same scale, name the variance method, and account for estimated weights and dependence where applicable.

Common Reporting Mistakes

Giving a cutoff without the scale. “Capped at 5” is ambiguous if it is not clear whether 5 is a raw cumulative weight, a time-specific factor, or a log-weight bound.
Reporting an affected percentage without a denominator. State whether the percentage is of people, person-time records, or another analysis unit.
Comparing only point estimates. A small change can still be uncertain; a larger change may also be imprecise. Report uncertainty for both estimates.
Interpreting overlapping intervals as a test. The estimates are dependent, so a formal comparison requires the covariance or paired resampling.
Omitting how weights were estimated. Clipping thresholds alone do not make the analysis reproducible or establish a causal interpretation.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

A study clips 12 of 240 participant-level final weights. What affected percentage should it report, and what unit should it name?
An analysis uses final cumulative weights and caps values above 4. What additional details are needed for this to be a reproducible clipping description?
In one analysis, the unclipped contrast is 2.0 and the clipped contrast is 1.7. What is the change when defined as clipped minus unclipped?
Why is it not valid to conclude that clipped and unclipped estimates are equal just because their separate confidence intervals overlap?

Solutions. (1) $100(12/240)=5\%$; report 12 of 240 participant-level weights, or 5% of participants. (2) State the construction of the weights, the scale on which 4 applies, whether clipping is applied to the final cumulative weight or earlier factors, the exact replacement rule, and the number and denominator of affected weights. (3) $1.7-2.0=-0.3$. (4) Separate intervals do not directly estimate uncertainty in the difference. The estimates are paired, so that uncertainty depends on their covariance; use a paired variance calculation or resampling method to compare them.