Two Ways to Choose a Weight Cutoff
A large inverse-probability weight can have a disproportionate influence on a weighted analysis. A cutoff rule limits how large a weight is allowed to be, but the rule used to choose the cutoff determines what “large” means. A fixed cutoff is a prespecified numerical value; a quantile cutoff is calculated from the observed weights and therefore depends on the sample.
This distinction matters when comparing studies or sample sizes. A fixed cutoff applies the same numerical cap wherever it is used, but the fraction of weights affected can vary. A fixed quantile targets approximately the same fraction of observations for clipping, but its numerical cutoff can vary across samples. Neither rule is automatically preferable: the choice should reflect whether the priority is a common numerical limit or a common rank-based rule.
The definition describes an upper cap. It is enough to compare cutoff-selection rules here; the details of lower and two-sided rules are a separate issue. The key distinction is that \(c\) is specified before seeing the sample, whereas \(\hat q_p\) is a statistic computed from it.
What Each Rule Holds Constant
| Rule | Held constant by design | Can change across samples | Useful interpretation |
|---|---|---|---|
| Fixed numerical cap \(c\) | The maximum retained weight | The fraction of weights above \(c\) | Apply the same numerical limit across studies or sample sizes |
| Empirical quantile cap \(\hat q_p\) | The rank used to locate the cutoff | The numerical cutoff | Limit clipping to approximately the upper \(1-p\) fraction |
| No cap | Nothing is clipped | The largest observed weight and its influence | Retain the observed weights without a cutoff |
With distinct weights, quantile clipping at rank \(k=\lceil np\rceil\) clips exactly the \(n-k\) observations above the cutoff. The observation at rank \(k\) is not changed, because it equals the cap. Thus the clipped proportion is \((n-\lceil np\rceil)/n\), which is close to \(1-p\), though rounding matters for small samples. Ties at the cutoff can make the proportion strictly clipped smaller.
Worked Example: Comparing the Rules on One Sample
Suppose the ten observed weights, in increasing order, are \(0.5, 0.8, 1, 1.2, 1.5, 2, 3, 8, 12, 30\). Compare a fixed cap \(c=5\) with the empirical 80th-percentile cap. Here \(n=10\), so the specified quantile rank is \(\lceil 10(0.8)\rceil=8\), and \(\hat q_{0.8}=W_{(8)}=8\).
Under the fixed cap, the clipped weights are \(0.5,0.8,1,1.2,1.5,2,3,5,5,5\). Their sum is \(10+15=25\), so their mean is \(25/10=2.5\). The weights 8, 12, and 30 are changed: \(3/10=30\%\) of the sample.
Under the quantile cap, the clipped weights are \(0.5,0.8,1,1.2,1.5,2,3,8,8,8\). Their sum is \(10+24=34\), so their mean is \(34/10=3.4\). Only 12 and 30 exceed the cutoff, so \(2/10=20\%\) are changed. The same sample therefore yields different numerical limits and different clipped summaries under the two rules.
How Quantile Cutoffs Behave as Sample Size Grows
A fixed quantile level does not mean a fixed numerical cutoff. However, under a stable distribution of weights, an empirical quantile generally settles near the corresponding population quantile as the sample grows. The following statement makes the conditions explicit. It uses an idealized independent and identically distributed sample; estimated weights in an actual analysis need not satisfy that model exactly.
The condition around \(q_p\) holds, for example, when \(F\) is continuous and strictly increasing at the quantile. It rules out a flat interval of quantile values that would make the numerical target ambiguous. The clipped expectation is finite because \(\min(W,q_p)\) is bounded above by \(q_p\).
For any sufficiently small \(\varepsilon>0\), the assumptions give \(F(q_p-\varepsilon)<p<F(q_p+\varepsilon)\). By the weak law of large numbers applied to the indicators \(1\{W_i\le q_p-\varepsilon\}\) and \(1\{W_i\le q_p+\varepsilon\}\), their sample proportions converge in probability to the corresponding distribution-function values. With probability tending to one, the first proportion is below \(p\) and the second is above \(p\). The order statistic of rank \(\lceil np\rceil\) must then lie between \(q_p-\varepsilon\) and \(q_p+\varepsilon\). This proves \(\hat q_p\xrightarrow{p}q_p\).
The variables \(\min(W_i,q_p)\) are independent and identically distributed and bounded, so the weak law gives \(\frac1n\sum_i\min(W_i,q_p)\xrightarrow{p}E[\min(W,q_p)]\). For all nonnegative \(w,a,b\), \(|\min(w,a)-\min(w,b)|\le |a-b|\). Consequently, \(\left|\frac1n\sum_i\min(W_i,\hat q_p)-\frac1n\sum_i\min(W_i,q_p)\right|\le|\hat q_p-q_p|\), which converges in probability to zero. Combining these two convergences proves the second claim. \(\square\)
The theorem describes the cutoff and the average of the clipped weights, not the behavior of a causal effect estimator. Its practical message is narrower: if the underlying weight distribution stays the same and the quantile assumptions hold, increasing \(n\) reduces the sampling variation in the numerical quantile cutoff; it does not force that cutoff to increase indefinitely. A fixed cap, by contrast, remains the same number by construction.
Worked Example: A Fixed Quantile Under a Heavy-Tailed Distribution
Let \(W\) have a Pareto distribution with minimum value 1 and tail parameter 2, so \(F(w)=1-w^{-2}\) for \(w\ge1\). Its population \(p\)-quantile solves \(1-q_p^{-2}=p\), giving \(q_p=(1-p)^{-1/2}\). At \(p=0.90\), this is \(q_{0.90}=0.10^{-1/2}=\sqrt{10}\approx3.162\). At \(p=0.99\), it is \(q_{0.99}=0.01^{-1/2}=10\).
A fixed cap of 5 stays at 5 for every sample size. Under the theorem’s conditions, the empirical 90th-percentile cutoff instead approaches approximately 3.162, while the empirical 99th-percentile cutoff approaches 10. The distribution is unbounded, yet each fixed quantile level has a finite limiting cutoff. These rules answer different questions: the fixed cap imposes a chosen numerical limit, and the quantile cap locates a chosen fraction of the weight distribution.
Worked Example: Adding One Observation Can Change the Quantile Rule
Start with weights \(1,2,3,4,100\). For \(p=0.8\), the rank is \(\lceil5(0.8)\rceil=4\), so the empirical cutoff is 4. A fixed cap of 4 and this quantile cap happen to agree; both produce \(1,2,3,4,4\), with mean \(14/5=2.8\).
Now add a weight of 5. The ordered sample is \(1,2,3,4,5,100\), and the 80th-percentile rank is \(\lceil6(0.8)\rceil=5\), giving cutoff 5. The fixed cap of 4 produces \(1,2,3,4,4,4\), with mean \(18/6=3\). The quantile cap of 5 produces \(1,2,3,4,5,5\), with mean \(20/6=10/3\). One added observation changes the sample quantile cutoff from 4 to 5, while the fixed cutoff stays at 4. This finite-sample change is not a contradiction of quantile convergence: convergence concerns repeated samples from a stable distribution as \(n\) grows, not the effect of every individual addition.
Diagnostics, Comparisons, and Limitations
A cutoff should be reported as part of the weight-processing rule, not merely as a hidden analysis setting. For a fixed cap, report its numerical value and the proportion of weights above it. For a quantile cap, report the quantile level, the empirical numerical cutoff, and the number or proportion of weights changed. Reporting both the level and the value makes it possible to see whether the numerical cutoff differs substantially across samples.
Several patterns deserve scrutiny. If the fraction above a fixed cap changes sharply across samples, the rule is not imposing a common degree of clipping. If quantile cutoffs move sharply across samples, a fixed fraction is being capped at materially different numerical values. Ties, small samples, and different software quantile conventions can also make apparently identical quantile specifications behave differently. Finally, when the weight distribution changes across studies or sample sizes, the theorem’s stable-distribution interpretation does not apply: a quantile cutoff can move because the distribution moved, not just because it was estimated more precisely.
The main alternatives have different trade-offs. A fixed cap is transparent and easy to compare numerically across analyses, but does not control the fraction affected. A quantile cap directly targets a rank-based fraction, but the resulting weight limit is sample-dependent. Leaving weights uncapped avoids altering them by a threshold rule, but does not limit extreme observed values. No cutoff choice alone establishes that a weighted analysis is reliable; it is a design and reporting choice whose consequences must be assessed in the analysis at hand.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- For six distinct ordered weights and \(p=0.75\), which order statistic is the empirical cutoff, and how many observations are strictly clipped?
- Why can two studies using the same fixed cap have different clipped proportions?
- In the Pareto example, calculate the population 95th-percentile cutoff.
- Does a fixed \(p\)-quantile rule necessarily produce a numerical cutoff that rises with sample size? Explain.
Solutions. (1) The rank is \(\lceil6(0.75)\rceil=\lceil4.5\rceil=5\), so the cutoff is the fifth ordered weight. With distinct weights, only the sixth weight is strictly above it, so one observation is clipped. (2) The studies can have different weight distributions, and therefore different numbers of weights above the same numerical value. (3) \(q_{0.95}=(1-0.95)^{-1/2}=0.05^{-1/2}=\sqrt{20}\approx4.472\). (4) No. For independent observations from a stable distribution satisfying the stated quantile condition, the empirical cutoff converges in probability to the finite population \(p\)-quantile. It can fluctuate from sample to sample, but increasing sample size does not by itself imply an increasing cutoff.