Unequal Stratum Sizes and Target-Population Weights

Why Stratum Weights Change the Question

Subclassification divides individuals into strata, often using their estimated propensity scores, and estimates a treatment contrast within each stratum. When strata contain different numbers of people, the within-stratum contrasts do not determine a single overall estimate by themselves: an aggregation rule must say how much each stratum contributes. That choice is not merely a computational detail. It determines the population average represented by the result.

For example, suppose one stratum contains most of the target population and another contains very few people. Giving the two stratum effects equal weight answers a different question from weighting them according to their population sizes. Weighting according to the treated individuals’ distribution across strata answers a third question. These rules can produce substantially different estimates when treatment effects vary across strata.

Three Common Weighting Rules

Population-share weights give each stratum a contribution proportional to its share of the target population. If \(N_s\) is the number of target-population members in stratum \(s\), and \(N=\sum_s N_s\), then \(w_s^P=N_s/N\). In a sample intended to represent that population, these shares may be estimated by sample stratum proportions. If the sample was deliberately sampled at different rates across strata, use the target-population shares rather than the unadjusted analytic-sample shares.

Treated-share weights give each stratum a contribution proportional to its share of the treated population. If \(N_{1s}\) is the number of treated individuals in stratum \(s\), and \(N_1=\sum_s N_{1s}\), then \(w_s^T=N_{1s}/N_1\). Equal-stratum weights assign \(w_s^E=1/K\) to every stratum, regardless of its size.

For an estimate, replace each \(\tau_s\) with its within-stratum estimate \(\hat{\tau}_s\). The calculation is then \(\hat{\tau}=\sum_s w_s\hat{\tau}_s\). The weights should be stated alongside the estimate: the same collection of stratum estimates can yield different overall estimates under different weighting rules.

When Do These Aggregates Have Causal Interpretations?

A causal interpretation requires more than choosing weights. Let \(S\) be the strata used for the analysis. For the stratum-specific treatment contrast to identify \(\tau_s\), assume consistency, no interference between individuals, and conditional exchangeability within strata: \(Y(a)\) is independent of \(A\) given \(S=s\), for \(a=0,1\). Also require positivity within every included stratum: \(0<P(A=1\mid S=s)<1\). Under these conditions, the treated and control groups in a stratum identify the stratum-specific average potential outcomes, and their difference identifies \(\tau_s\).

In observational analyses, exchangeability given a coarse stratum is a substantive assumption, not an automatic consequence of subclassification. Propensity-score strata can reduce differences in measured covariates without eliminating all within-stratum confounding. Balance and overlap therefore need to be examined; residual confounding within strata can bias the stratum estimates and any aggregate of them. The earlier material on residual confounding within propensity-score strata addresses this problem.

With valid stratum-specific ATEs, population-share weighting targets the ATE for the population whose stratum distribution is represented by \(N_s/N\). This follows from the law of total expectation. Treated-share weighting of those same ATEs averages stratum ATEs according to the treated population’s stratum distribution. It is not necessarily the ATT, because the ATE within a stratum need not equal the treatment effect among treated individuals in that stratum. To identify the ATT by treated-share weighting, the within-stratum effects must instead be stratum-specific effects among treated individuals.

Similarly, partitioning the treated population by stratum gives the ATT identity below. Here \(\tau_s^T=E[Y(1)-Y(0)\mid A=1,S=s]\) is the stratum-specific effect among treated individuals. Equal-stratum weighting instead averages the \(\tau_s\) values under a constructed distribution that places probability \(1/K\) on every stratum. Unless that distribution matches the scientific target, the result is not the ATE or ATT for the observed population.

Worked Examples

Choosing and Checking the Weights

Choose the weighting rule by stating the target first. If the question concerns the average effect for the source population, use its population shares with stratum ATEs. If it concerns the treated population, use treated shares with stratum-specific effects among treated. If each stratum is itself the unit of substantive interest and should count equally, equal-stratum weighting may be appropriate. Equal weighting is not a default correction for unequal sample sizes.

Before interpreting the result, check the quantities that determine the calculation:

• Confirm that the stratum counts or shares correspond to the intended target population, rather than automatically using the available sample proportions.
• Verify that the weights are nonnegative and sum to 1. For example, population shares must satisfy \(\sum_s N_s/N=1\).
• Check treatment and control representation in each stratum. A stratum with no treated or no control individuals cannot supply an ordinary within-stratum comparison.
• Review covariate balance within strata and investigate signs of residual confounding. A correctly calculated weighted average cannot repair biased stratum-specific effects.
• Report the weighting rule, the stratum weights, and the target estimand with the estimate. Large differences among weighting rules can reveal effect heterogeneity combined with different target compositions.

The main alternatives are not competing formulas for one universal quantity. They answer different questions: population-share aggregation standardizes to the population’s stratum distribution; treated-share aggregation standardizes to the treated population’s stratum distribution; equal-stratum aggregation standardizes to a uniform distribution over the strata. A separate issue is how each within-stratum contrast is estimated. Changing the aggregation weights does not resolve within-stratum confounding, and choosing a target population does not establish that the stratum effects are identified.

Common Pitfall and Limits

Wrong reasoning: “The treated-share-weighted estimate is the ATT because the weights are based on treated individuals.” This overlooks the quantity being averaged. Treated shares combined with stratum ATEs produce a treated-distribution average of ATEs; the ATT identity requires stratum-specific effects among treated. The correction is to define the within-stratum effect first, then match both the weights and those effects to the intended estimand.

Aggregation also cannot create information where a stratum lacks treatment overlap, and equal-stratum weighting can give a very small stratum as much influence as a very large one. Its interpretation may be useful for a deliberate stratum-level target, but it may be unrepresentative of any natural population. Finally, different aggregates can differ even when every calculation is correct. Such disagreement is not, by itself, evidence of an error: it may reflect treatment-effect heterogeneity and different target-population compositions.