A Complete Subclassification Analysis

From Propensity Scores to an ATE

Subclassification makes a propensity-score analysis visible in a table: divide units into groups with similar estimated probabilities of treatment, compare treated and untreated outcomes within each group, then combine those comparisons. The final combination matters. To estimate an average treatment effect (ATE) for the target population, strata must contribute in proportion to their shares of that population, not equally merely because there are several strata.

This procedure is useful when a single overall treated-versus-untreated comparison is distorted by differences in measured covariates. It is not a guarantee that confounding has disappeared. In particular, grouping a continuous score into a finite number of intervals can leave treated and untreated units different within an interval.

Definition: Let \(A_i\) be a binary treatment indicator, \(X_i\) a vector of pretreatment covariates, and \(Y_i\) the observed outcome. The propensity score is \(e(X_i)=P(A_i=1\mid X_i)\). A propensity-score stratum is a group \(s\) formed by assigning units to a specified interval of the score, usually an estimated score. Write \(n_s\) for the number of target-population units in stratum \(s\), and \(w_s=n_s/n\) for its share when the target is the observed sample.

For each stratum, calculate the treated and untreated outcome means and their difference. For a continuous outcome, the difference is measured in the outcome’s units. For a binary outcome, it is a difference in proportions. Let \(\bar{Y}_{1s}\) and \(\bar{Y}_{0s}\) denote the observed means for treated and untreated units in stratum \(s\). The subclassification estimate is

\widehat{\tau}_{\text{sub}}=\sum_{s=1}^{K} w_s(\bar{Y}_{1s}-\bar{Y}_{0s}), \qquad w_s=\frac{n_s}{n}, \qquad \sum_{s=1}^{K}w_s=1.

When the target is a population beyond the analyzed sample, \(w_s\) should instead be the target population’s proportion in stratum \(s\), if known or estimable. The weights specify whose ATE is being summarized. Equal weights target an artificial population with equal representation from every stratum, unless that happens to match the actual target distribution.

Assumptions and What the Estimate Requires

Write \(Y_i(1)\) and \(Y_i(0)\) for unit \(i\)’s potential outcomes under treatment and no treatment. The ATE in a target population is \(E[Y(1)-Y(0)]\). A causal interpretation of the subclassification calculation requires assumptions about treatment assignment and outcomes, not just a well-computed table.

Consistency and no interference: the observed outcome equals \(Y_i(A_i)\), and one unit’s treatment does not alter another unit’s outcome. The treatment must also have a sufficiently well-defined meaning.
Conditional exchangeability given measured covariates: \((Y(1),Y(0))\) is independent of \(A\) conditional on \(X\). In words, after accounting for \(X\), treatment assignment carries no further information about potential outcomes.
Positivity: for covariate profiles represented in the target, \(0<e(X)<1\). Both treatment conditions must be possible for those profiles.
Suitable score construction and target representation: strata are formed using pretreatment covariates and a score model that adequately captures treatment assignment for the analysis. To generalize beyond the analyzed sample, its stratum shares must represent the target population or be replaced by appropriate target shares.

The exchangeability condition given \(X\) does not automatically become exact exchangeability within a finite score interval. The true propensity score has a balancing property at a given score value, but a stratum usually combines many score values and covariate profiles. Thus subclassification is an approximation unless potential outcomes are exchangeable within the constructed strata (or the strata otherwise preserve the information needed for identification). Covariate balance within each stratum is important evidence about this approximation, but cannot prove that unmeasured confounding is absent.

Why the Stratum Shares Produce the ATE

The aggregation follows by partitioning the target population into strata. If \(S\) denotes stratum membership, then the ATE can be decomposed as

E[Y(1)-Y(0)] =\sum_{s=1}^{K}P(S=s)E[Y(1)-Y(0)\mid S=s].

Under consistency and exchangeability within each stratum, the conditional causal effect in stratum \(s\) is identified by the difference between its treated and untreated outcome means. Replacing the population stratum share by its sample estimate \(n_s/n\), and replacing those conditional means by observed sample means, gives the subclassification estimator. In a finite sample these are estimates, not exact identities: sampling variation and residual within-stratum confounding remain.

The same decomposition can be expressed by standardizing both treatment conditions to the target stratum distribution. Define the standardized treated and untreated means as \(\sum_s w_s\bar{Y}_{1s}\) and \(\sum_s w_s\bar{Y}_{0s}\). Their difference equals the weighted average of the stratum-specific differences because the weights are identical in both sums.

Worked Example: A Three-Stratum ATE Calculation

Suppose a hypothetical sample has 100 people, divided into low-, middle-, and high-score strata using an estimated propensity score. The outcome is a symptom score, with higher values indicating more symptoms. The table gives the number of people and the outcome total in each treatment group. Assume the treatment and outcome definitions satisfy consistency, and that the covariates are measured before treatment.

Score stratum	Total \(n_s\)	Treated \(n_{1s}\)	Treated outcome total	Untreated \(n_{0s}\)	Untreated outcome total
Low	40	10	120	30	240
Middle	40	20	300	20	220
High	20	15	270	5	75

First calculate the treated and untreated means and differences within each stratum:

\begin{aligned} \widehat{\tau}_{\text{low}}&=\frac{120}{10}-\frac{240}{30}=12-8=4,\\ \widehat{\tau}_{\text{middle}}&=\frac{300}{20}-\frac{220}{20}=15-11=4,\\ \widehat{\tau}_{\text{high}}&=\frac{270}{15}-\frac{75}{5}=18-15=3. \end{aligned}

The sample stratum shares are \(40/100=0.40\), \(40/100=0.40\), and \(20/100=0.20\). Therefore,

\widehat{\tau}_{\text{sub}} =0.40(4)+0.40(4)+0.20(3) =1.60+1.60+0.60 =3.80

The estimated ATE for the population represented by this sample is a 3.80-point increase in the symptom score under treatment, relative to no treatment, subject to the identification assumptions and the adequacy of the strata. The same answer follows by standardizing the means: the treated mean is \(0.40(12)+0.40(15)+0.20(18)=14.4\), and the untreated mean is \(0.40(8)+0.40(11)+0.20(15)=10.6\); their difference is \(14.4-10.6=3.80\) points.

The crude comparison would not make the same adjustment. Across the whole sample, the treated mean is \(690/45=15.33\) and the untreated mean is \(535/55=9.727\), giving a crude difference of \(15.33-9.727=5.606\) points when calculated from the unrounded means. That difference compares groups with different stratum compositions, so it is not the subclassification estimate.

For a causal interpretation, one would still inspect covariate balance within each stratum and confirm there are treated and untreated units in every stratum. The counts provide both treatment groups in all three strata, but counts alone do not establish balance or exchangeability.

Unequal Strata and the Target Population

Stratum-specific effects may differ, and the target population may contain many more people in one stratum than another. The weights preserve that composition. The following hypothetical calculation shows why the choice cannot be replaced by a simple average over strata.

Worked Example: Why Equal Stratum Weights Change the Target

Suppose a target sample contains 100 people. In a large stratum of 80 people, the treated mean outcome is 10 and the untreated mean is 8, so the effect estimate is 2 units. In a smaller stratum of 20 people, the means are 20 and 12, so the estimated effect is 8 units. Both treatment groups are observed in both strata, and assume within-stratum exchangeability is plausible for this illustration.

The ATE estimate for the population represented by these 100 people is

\widehat{\tau}_{\text{sub}} =\frac{80}{100}(2)+\frac{20}{100}(8) =1.6+1.6 =3.2\text{ units}.

Equivalently, the standardized treated mean is \(0.80(10)+0.20(20)=12\), and the standardized untreated mean is \(0.80(8)+0.20(12)=8.8\), for a difference of \(12-8.8=3.2\) units. Giving each stratum weight \(1/2\) instead would yield \((2+8)/2=5\) units. That is not the ATE for this target: it overrepresents the smaller, higher-effect stratum. It would describe an equal-mixture target population, if that were the intended target.

Diagnostics, Failure Modes, and Alternatives

A completed calculation is not itself a diagnostic. Before interpreting it, check the ingredients that make within-stratum comparisons credible.

Check covariate balance within strata. Compare important pretreatment covariates between treated and untreated units within each stratum. A good overall balance summary can conceal poor balance in a particular stratum.
Inspect score distributions and cell counts. Very small treatment cells give unstable outcome means. A stratum with no treated or no untreated units has no within-stratum contrast.
Look for residual score variation and covariate patterns. If score values or key covariates differ substantially between treatment groups inside a stratum, the bin may be too broad for the proposed adjustment.
Check sensitivity to reasonable alternative cutpoints. Large changes in the estimate can indicate that the result depends heavily on how the score was grouped. This is a warning, not a rule that a particular cutpoint is correct.

If a stratum has no observations from one treatment group, its effect cannot be calculated from the observed data without additional assumptions or modeling. Omitting that stratum and renormalizing the remaining weights changes the target population; it does not recover the original ATE automatically. Sparse cells are less absolute but still matter: a calculable difference based on very few observations may be highly variable.

Subclassification is one approach among several. Inverse probability weighting uses propensity-score-based weights rather than a finite set of bins and can preserve more score information, but extreme weights can make an estimate unstable. Matching constructs treated and untreated comparisons locally, often changing which units contribute directly. Outcome standardization models outcomes conditional on covariates and averages predicted outcomes over a target distribution. Each approach has different modeling and stability demands; none removes the need to justify treatment exchangeability, support, and the target population.

Worked Example: A Stratum Without a Comparison Group

Consider a hypothetical target sample of 50 people. In the low-score stratum, 30 people include 10 treated and 20 untreated participants; their outcome means are 7 and 5, respectively. In the high-score stratum, 20 people are all treated, with mean outcome 14; there are no untreated participants.

The low-stratum contrast is calculable: \(7-5=2\) outcome units. Its target share is \(30/50=0.60\), so its contribution to the ATE decomposition is \(0.60(2)=1.2\) units. The high-stratum share is \(20/50=0.40\), but the high-stratum untreated mean is unobserved. If its causal effect is denoted by \(\tau_{\text{high}}\), the target ATE would have the form

\widehat{\tau}_{\text{target}}=0.60(2)+0.40\tau_{\text{high}}.

The data do not identify \(\tau_{\text{high}}\), so they do not provide a subclassification ATE for the full 50-person target without extra assumptions. Reporting \(1.2\) as the full ATE would silently treat the high-stratum contribution as zero. Dropping the high stratum and dividing the remaining weight by 0.60 would instead report the low-stratum effect, 2 units, for a restricted target. A different model might extrapolate a high-stratum untreated mean, but that would rely on assumptions beyond the observed within-stratum comparison.

Common Mistake

Wrong reasoning: “There are three strata, so each effect should receive weight one-third.” This is correct only for a target population with equal shares in all three strata. In general, it changes the target and can overemphasize small strata. Correction: write down the target population first, calculate or specify its stratum shares, and use those shares consistently to combine the within-stratum estimates. Also keep the causal claim separate from the arithmetic: correct weights do not by themselves establish exchangeability within the bins.

Key takeaway: A subclassification ATE is a target-share-weighted average of treated–untreated contrasts within propensity-score strata. Its causal interpretation depends on adequate within-stratum comparisons and credible control of confounding, not merely on dividing the score into bins.

Check Your Understanding

Answer each question, then compare your work with the solutions below.

A target population has stratum shares 0.25 and 0.75, with estimated within-stratum effects of 6 and 2 minutes. Calculate the ATE estimate and explain why an equal average is not the target estimate.
In a stratum, the treated outcome total is 96 among 12 people and the untreated total is 140 among 20 people. Find the within-stratum difference in means.
State one reason why exchangeability given all measured covariates does not guarantee exact exchangeability within a finite propensity-score bin.
A stratum has a target share of 0.30 but no untreated units. What can be concluded about the full-target subclassification ATE from the observed within-stratum comparisons alone?

Solutions. (1) The estimate is \(0.25(6)+0.75(2)=1.5+1.5=3.0\) minutes. An equal average gives 4 minutes and corresponds to equal representation of the two strata, not the stated target shares. (2) The means are \(96/12=8\) and \(140/20=7\), so the treated-minus-untreated difference is \(8-7=1\) outcome unit. (3) A finite bin combines units with potentially different score values and covariate profiles; residual treatment–covariate differences can remain inside it, so the score’s balancing property at a score value need not hold exactly for the whole interval. (4) Its within-stratum contrast is not identified from those observed outcomes because the untreated mean is missing. The full-target ATE cannot be obtained from the observed within-stratum comparisons alone without additional assumptions or modeling; dropping the stratum changes the target.