Subclassification on the Propensity Score

Why Subclassify on the Propensity Score?

In an observational study, treated and untreated units may differ in their baseline covariates. A comparison of their raw outcome means can therefore mix treatment effects with pre-existing differences. Subclassification addresses this by grouping units with similar propensity scores and comparing treated and untreated outcomes within each group. The central idea is to make comparisons among units whose estimated chances of receiving treatment are similar, rather than comparing all treated units with all untreated units at once.

Let \(A\) be a binary treatment indicator, with \(A=1\) for treated and \(A=0\) for untreated; let \(X\) denote measured pretreatment covariates; and let \(Y\) be the observed outcome. The propensity score is the conditional probability of treatment given those covariates. A subclass is a set of units whose propensity scores fall in a specified interval. The intervals are chosen before making outcome comparisons, and every unit is assigned to the interval containing its score.

Assumptions and the Balancing-Score Justification

Subclassifying on a propensity score does not by itself guarantee a causal interpretation. The usual causal conditions are needed. First, consistency requires that a unit’s observed outcome equals its potential outcome under the treatment actually received: \(Y=Y(1)\) if \(A=1\), and \(Y=Y(0)\) if \(A=0\). The treatment and outcome must also be well-defined, with no interference between units assumed for this analysis. Second, conditional exchangeability requires \((Y(1),Y(0))\mathbin{\perp} A\mid X\): after conditioning on measured pretreatment covariates, treatment assignment is independent of the potential outcomes. This rules out unmeasured confounding, an assumption that cannot generally be verified from the observed outcomes alone.

Third, positivity requires \(0<e(X)<1\) for covariate patterns in the target population. Both treatment states must be possible at those patterns. A stratum containing only treated units or only untreated units provides no within-stratum comparison. Finally, the propensity score used to create strata must adequately represent treatment assignment given the measured covariates. In practice it is estimated, so model misspecification or unstable score estimates can undermine the intended grouping.

The theoretical motivation is that the exact propensity score is a balancing score: conditional on its value, the distribution of measured covariates is the same in the treated and untreated groups. To see this, consider a covariate value \(X=x\) for which \(e(x)=e\). Within the set of units with score \(e\), Bayes’ rule gives the treated covariate distribution in proportion to \(e(x)f(x\mid e)\), while the untreated distribution is in proportion to \((1-e(x))f(x\mid e)\). Because \(e(x)=e\) throughout that set, these are constant multiples of the same distribution \(f(x\mid e)\). After normalization, the distributions coincide.

Under conditional exchangeability given \(X\), this balancing result also supports exchangeability of the potential outcomes across treatment groups conditional on the exact score. In outline, the conditional distribution of \(X\) is the same for treated and untreated units at a fixed score, and, given \(X\), treatment assignment carries no further information about \(Y(a)\). Integrating over that common covariate distribution therefore gives the same distribution of \(Y(a)\) in the two treatment groups at that score. This is the theoretical basis for comparing outcomes at similar propensity scores.

Actual subclassification usually uses score ranges, not exact score values. Units in one range can still have meaningfully different scores and covariates. Thus the exact balancing argument does not imply that a finite partition perfectly balances covariates or eliminates confounding. Subclassification is an approximation whose adequacy must be checked.

Assigning Units and Calculating a Within-Stratum Contrast

Choose \(K\) score intervals that cover the relevant score range without overlapping, using a consistent endpoint convention so that every unit belongs to exactly one stratum. For each stratum, record the number of treated and untreated units, summarize their observed outcomes separately, and calculate the difference in their means. This is a descriptive calculation first; a causal interpretation depends on the assumptions above and on adequate comparability within the stratum.

Here \(S_i=k\) means unit \(i\) is assigned to stratum \(k\), and \(n_{1k}\) and \(n_{0k}\) are the treated and untreated counts in that stratum. Both counts must be positive for this contrast to be calculated. The result is expressed in the outcome’s units. It estimates a treatment contrast for the units represented by that stratum only to the extent that the assumptions and within-stratum comparability are credible.

Why Comparing Within Strata Can Matter

A pooled comparison can be misleading when treatment groups have different distributions across the score range. The following hypothetical example shows why within-stratum comparisons are informative. Suppose symptom improvement is measured in points.

Diagnostics, Alternatives, and Limitations

After forming strata, check whether the proposed comparison is working rather than relying on the partition alone. Useful diagnostics include:

• Treatment representation: verify that each stratum has treated and untreated units. A missing treatment group means the within-stratum contrast cannot be estimated from the observed data.
• Score overlap: inspect whether both groups have scores in the ranges used for comparisons. Severe separation or sparse regions signal weak empirical support.
• Covariate balance: compare the distributions of important pretreatment covariates between treated and untreated units within strata. Similar propensity scores do not guarantee that every covariate is balanced in a finite sample.
• Within-stratum score spread: a broad range can contain units whose treatment probabilities, and potentially covariate profiles, differ substantially. Persistent imbalance suggests that the grouping is too coarse for the data.
• Counts and precision: very small treatment groups make stratum means unstable. A numerically calculable contrast can still be too imprecise to be useful.

Subclassification offers a transparent alternative to weighting, which uses propensity-score-based weights, and to propensity-score matching, which constructs comparisons between selected units. It also differs from outcome regression, which models the outcome conditional on covariates. These approaches have different sensitivities to model specification, overlap, and the units retained or emphasized. Subclassification makes the comparison structure visible, but it does not automatically solve poor overlap, unmeasured confounding, or misspecification of the propensity-score model.

A common mistake is to say, “The propensity scores are in the same stratum, so treated and untreated units are exchangeable and the difference is causal.” That reasoning confuses an exact score with a finite interval. The balancing result applies to the exact propensity score under conditional exchangeability; a range groups together potentially distinct scores. The correction is to treat stratum membership as a design device, then assess covariate balance and treatment representation within each stratum and interpret contrasts causally only under defensible assumptions.

Another limitation is that an estimate within one stratum describes a local comparison, not necessarily the effect for the full study population. The target population determines how stratum-specific information should contribute to an overall summary. Different targets, including the average treatment effect and the effect among treated units, need not assign equal importance to each stratum. Report which target is intended rather than treating a simple average of stratum contrasts as universally appropriate.