Variance Reduction from Bounded Weights

Why a Bound Can Control Variability

A large weight can make one observation dominate a weighted estimating equation. Bounding weights limits how large that observation’s weighted contribution can be, which can make variability more manageable. But “bounded weights reduce variance” is not a universal theorem: clipping changes the contributions, and whether variance falls depends on the estimator, the outcome, and the assumptions connecting weights to outcomes.

To make the claim precise, start with independent and identically distributed observations \((W_i,Y_i)\), where \(W_i\geq 0\) is a weight and \(Y_i\) is an outcome. For a chosen weight function \(a(W)\), define the population-normalized weighted mean

assuming \(0<E[a(W)]<\infty\). This form uses the population mean weight in its denominator, rather than estimating that denominator from the sample. It is useful for isolating the variability of weighted contributions. Its target is \(E[a(W)Y]/E[a(W)]\); it equals the ordinary outcome mean \(E[Y]\) under conditions such as independence of \(W\) and \(Y\).

The upper bound on \(A_B\) is a bound on the normalized weight, not necessarily \(B\) itself. The normalization matters: even if \(W_B\leq B\), dividing by a small \(E[W_B]\) can make \(A_B\) substantially larger than \(B\).

A Variance Bound for Bounded Contributions

The first result makes the contribution-size argument explicit. It does not require independence between weights and outcomes, but it does require a bounded outcome and a positive, finite normalizing constant.

Proof. Let \(Z_i=W_{B,i}Y_i/E[W_B]\). Since \(0\leq W_{B,i}\leq B\) and \(|Y_i|\leq K\), we have \(|Z_i|\leq KB/E[W_B]\) almost surely. Independence gives \(\operatorname{Var}(\hat{\mu}_B)=\operatorname{Var}(Z_1)/n\). For any square-integrable \(Z_1\), \(\operatorname{Var}(Z_1)=E[Z_1^2]-E[Z_1]^2\leq E[Z_1^2]\). The almost-sure bound therefore implies \(E[Z_1^2]\leq K^2B^2/\{E[W_B]\}^2\), proving the stated inequality. \(\square\)

This is an absolute bound, not yet a comparison with the untruncated estimator. If outcomes are unbounded, bounded weights alone do not bound \(W_BY\): an extreme outcome can still produce an extreme contribution. A variance bound then needs additional moment assumptions, for example \(E[Y^2]<\infty\), and its value depends on the outcome distribution as well as the weight bound.

When Clipping Gives a Direct Variance Comparison

A clean comparison is possible in a special but instructive setting. Suppose \(W\geq0\), \(0<E[W]<\infty\), \(Y\) is independent of \(W\), and \(E[Y]=0\), \(E[Y^2]=\sigma^2<\infty\). Normalize either the raw or truncated weights by their population means. Independence and the zero outcome mean make both resulting estimators unbiased for zero, so their variances can be compared directly.

Proof. Independence gives \(E[a(W)Y]=E[a(W)]E[Y]=0\), so each summand \(Z_i=a(W_i)Y_i/E[a(W)]\) has mean zero. Again by independence, \(E[Z_i^2]=E[a(W)^2]E[Y^2]/\{E[a(W)]\}^2\). The summands are independent, so the variance of their average is their common variance divided by \(n\). Substituting \(E[Y^2]=\sigma^2\) gives the formula. Applying it first with \(a(W)=W_B\) and then with \(a(W)=W\) yields the comparison. \(\square\)

The theorem gives a condition for the comparison; it does not assert that the inequality holds for every distribution and every cutoff. In this setting, the normalized second moment \(E[a(W)^2]/E[a(W)]^2\) determines the variance. A cutoff is helpful when it reduces that quantity. The conditions also explain why this is not a general causal-effect result: independence and a mean-zero outcome are strong simplifications, and clipping can alter a weighted target when outcomes depend on weights.

What the Comparison Does Not Guarantee

Variance depends on the distribution of the full contribution, not on the weight alone. For example, even a pointwise reduction in some weights need not make the products \(W_iY_i\) more similar to one another. Consider two equally likely paired values: \((W,Y)=(1,10)\) and \((10,1.1)\). The raw products are \(10\) and \(11\), with variance \(0.25\). Truncating at \(B=1\) changes them to \(10\) and \(1.1\), with variance \((10-1.1)^2/4=19.8025\). This calculation concerns the distribution of the contributions, not a comparison of two unbiased estimators for one unchanged target; indeed, the association between \(W\) and \(Y\) is exactly why the simple theorem above does not apply.

There are two further distinctions to keep in view. First, a variance reduction is not automatically an improvement in overall estimation error: truncation can change the weighted target and introduce bias. The consequences for mean squared error are a separate question. Second, the estimator defined here uses the population normalizer \(E[a(W)]\). A sample-normalized ratio, such as \(\sum_i a(W_i)Y_i/\sum_i a(W_i)\), has a random denominator. Its variance is not given exactly by the displayed formula for \(\hat{\mu}_a\).

In practice, inspect more than the maximum weight. Check the largest weighted outcome contributions, the distribution of normalized weights, and whether a small number of observations account for a large share of the weighted sum. Compare results across reasonable cutoffs and consider whether the observations most affected by clipping have systematically different outcomes. These checks can reveal instability or an outcome–weight association that defeats a simple variance argument.