Why the Number of Strata Matters
Subclassification groups units with similar propensity scores, then compares treated and untreated outcomes within each group. The number of groups sets a practical compromise. Wider strata retain more observations in each comparison, but treated and untreated units can differ more in their propensity scores and covariates. Narrower strata can limit that mismatch, but may leave few observations of one treatment status in some strata. The resulting estimate can then be imprecise or unstable.
There is no universally optimal number of strata. The consequences of a choice depend on the score distribution, the relationship between covariates and outcomes, the treatment allocation, the target population, and the amount of treatment-effect heterogeneity. The purpose of this tutorial is to make those consequences explicit, not to prescribe a fixed count.
For an average treatment effect (ATE), the intended weights represent the target population's distribution across strata. When explaining the effect of changing \(K\), it helps to keep that target and its weighting rule fixed. Otherwise a difference between estimates could reflect a changed target population as well as a changed partition. The relevant issue here is how the partition affects within-stratum comparisons and their precision.
Conditions and the Residual-Confounding Trade-Off
The causal interpretation begins with the usual conditions: consistency (the observed outcome equals the potential outcome under the treatment received), conditional exchangeability \(Y(1),Y(0)\perp A\mid X\), and positivity for treatment groups in the target population. The true propensity score is a balancing score: conditional on its exact value, treatment is independent of the measured covariates \(X\). But putting a range of scores into one stratum is not conditioning on an exact value. Treated and untreated units in that interval can still have different score distributions and, consequently, different covariate and potential-outcome distributions.
A smaller maximum stratum width can restrict how far apart scores within a stratum can be. This gives a useful bound on imbalance in the untreated outcome mean as a function of the score. It is a bound, not a guarantee that confounding is negligible: it requires a smooth outcome relationship, and it addresses this particular baseline-outcome component rather than all possible sources of bias.
A Bound for Baseline-Outcome Imbalance
Write \(m_0(e)=E[Y(0)\mid e(X)=e]\), and suppose it is Lipschitz on a stratum: for some finite \(L\), \(|m_0(e)-m_0(e')|\leq L|e-e'|\) for any scores in that interval. Let \(E_1\) and \(E_0\) denote the score distributions among treated and untreated units in the stratum. Couple these distributions by pairing scores \(E_1\) and \(E_0\) so that each paired value remains in the same interval. The difference in their mean untreated outcomes obeys:
Here \(h\) is the stratum's score width: both paired scores lie in an interval of width \(h\), so \(|E_1-E_0|\leq h\). The argument does not require \(m_0\) to be linear or the groups to have the same mean score. In fact, just comparing mean scores would not in general bound the difference between average values of a nonlinear \(m_0\). For a partition of a score range of width \(R\) into equal-width intervals, \(h=R/K\); for quantile strata or other partitions, widths can differ, and the largest width is more relevant to a worst-case bound.
This derivation concerns one component of a within-stratum contrast. If treatment effects vary with the score, treated and untreated units can also represent different distributions of treatment effects. The interpretation of the combined estimate then depends on its target weights. A narrower partition can reduce within-stratum score mismatch, but does not by itself resolve a target-weighting problem or guarantee an unbiased estimate.
Worked Example: How Width Changes a Bound
Suppose the score range in a hypothetical analysis is 0.10 to 0.90, and the untreated outcome mean is known to satisfy the Lipschitz condition with \(L=12\) outcome units per score unit. Compare three equal-width strata with six equal-width strata.
The range width is \(R=0.90-0.10=0.80\). With \(K=3\), each interval has width \(h=0.80/3=0.2667\). The bound is \(Lh=12(0.2667)=3.200\) outcome units. With \(K=6\), \(h=0.80/6=0.1333\), and the bound is \(12(0.1333)=1.600\) outcome units.
Thus the worst-case bound from this assumption is halved when the number of equal-width strata doubles. It is not a claim that the actual residual confounding is 3.200 or 1.600 outcome units; the actual difference may be smaller, and this calculation does not bound other sources of bias.
Why Finer Strata Can Lose Precision
Each stratum needs observations from both treatment groups to estimate a within-stratum contrast. If the sample standard deviations of the outcome in stratum \(s\) are \(s_{1s}\) and \(s_{0s}\), and the group sample sizes are \(n_{1s}\) and \(n_{0s}\), the usual independent-sample variance estimate for the difference in means is \(\widehat{\mathrm{Var}}(\hat{\tau}_s)=s_{1s}^2/n_{1s}+s_{0s}^2/n_{0s}\). For disjoint strata and fixed weights, the corresponding approximate variance of the combined estimate is \(\sum_s w_s^2\widehat{\mathrm{Var}}(\hat{\tau}_s)\). These formulas describe sampling variability under the stated sample-variance approximation; they do not account for every complication of an observational analysis.
More strata do not automatically increase the standard error. If the total sample sizes, treatment allocation, outcome variances, and target weights are favorable, splitting observations can leave precision nearly unchanged. But fine partitioning can reveal or create sparse cells: some strata may have very few treated or untreated units. A small denominator in the variance formula then makes the estimate noisy, and one unusual observation can have a large effect. If a stratum has no units in one treatment group, its contrast is not estimable from the observed data.
Worked Example: Precision Under Balanced and Imbalanced Cells
Consider a hypothetical sample with six strata and 60 treated and 60 untreated units in total. Assume, for this calculation, that each group's outcome standard deviation is 10 units in every stratum and that the combined estimate gives each stratum weight \(1/6\). First suppose every stratum has 10 treated and 10 untreated units.
In one stratum, the estimated variance is \(10^2/10+10^2/10=20\) square outcome units. Since the six stratum estimates are independent under this illustrative calculation, the combined variance is \(\sum_{s=1}^6(1/6)^2(20)=6(1/36)(20)=3.333\) square outcome units. The standard error is \(\sqrt{3.333}=1.826\) outcome units.
Now keep the same total sample sizes but suppose three strata have counts \(18\) treated and \(2\) untreated, while the other three have \(2\) treated and \(18\) untreated. Each stratum's variance estimate is \(10^2/18+10^2/2=100(1/18+1/2)=55.56\) square outcome units. The combined variance is \(6(1/36)(55.56)=9.259\), giving a standard error of \(\sqrt{9.259}=3.043\) outcome units. The total sample size has not changed, but the imbalance across treatment groups within strata has substantially reduced precision for this equally weighted estimate.
This comparison isolates the effect of the cell counts under the stated variance assumptions. In practice, stratum weights and outcome variances can also differ, so inspect the actual contribution of each stratum rather than assuming that a given \(K\) has a predictable standard error.
Worked Comparison of Coarse and Fine Partitions
Worked Example: Subclassification With Two Versus Four Strata
Consider a hypothetical data set with outcomes generated around \(E[Y(0)\mid e]=100+20e\) and a constant treatment effect of 5 outcome units. Four narrow score bins are \([0.1,0.3)\), \([0.3,0.5)\), \([0.5,0.7)\), and \([0.7,0.9]\). For each narrow bin, suppose the treated and untreated mean scores are 0.02 apart, and the within-bin counts are the same as those described for the first two bins below. The last two bins repeat the first two bins' score-and-count pattern shifted upward by 0.4.
For the first narrow bin, the treated mean score is 0.19 and the untreated mean score is 0.21. The corresponding outcome means are \(100+20(0.19)+5=108.8\) for treated units and \(100+20(0.21)=104.2\) for untreated units. The within-bin contrast is \(108.8-104.2=4.600\) outcome units. In the second narrow bin, scores are 0.39 and 0.41; the means are 112.8 and 108.2, so the contrast is also \(112.8-108.2=4.600\). The other two narrow-bin contrasts are likewise 4.600. Giving the four bins equal target weights yields \((4.600+4.600+4.600+4.600)/4=4.600\) outcome units.
Now combine the first two bins into one wider stratum. Suppose there are 2 treated and 18 untreated units in the first narrow bin, and 18 treated and 2 untreated units in the second. The treated mean outcome across the wider stratum is \([2(108.8)+18(112.8)]/20=112.4\). The untreated mean is \([18(104.2)+2(108.2)]/20=104.6\). The wide-stratum contrast is \(112.4-104.6=7.800\) outcome units. The second wide stratum has the same contrast, so the equally weighted two-stratum estimate is also 7.800 outcome units.
In this constructed example, the finer partition reduces the score-distribution mismatch within each comparison and moves the estimate from 7.800 toward the known treatment effect of 5. The finer estimate is still 0.400 outcome units below 5 because the treated and untreated mean scores differ by 0.02 within each narrow bin, producing a baseline-outcome difference of \(20(0.02)=0.400\). The example also has only two observations in some treatment-by-bin cells. Consequently, its fine-stratum estimate may be unstable in a sample; the example illustrates the bias–precision trade-off rather than a rule that four strata are preferable.
Diagnostics, Alternatives, and Limits
A practical choice of \(K\) should be guided by diagnostics as well as by the estimate. Examine the score distributions within each proposed stratum, check that both treatment groups are represented, and inspect covariate balance within strata. Summarize the smallest treatment-group cell counts and note strata where one or two observations dominate an outcome mean. Compare estimates across a modest range of defensible partitions while keeping the target estimand and aggregation rule fixed. Large changes can signal residual confounding, sparse-cell instability, or both; stability across partitions is reassuring but does not prove that confounding is absent.
Subclassification is one way to use a propensity score. Continuous adjustment for the score avoids imposing the same within-bin comparison, but depends on how that adjustment is specified. Matching targets comparisons between selected units, while inverse probability weighting uses weights to construct a target pseudo-population; each approach has its own overlap, modeling, and precision concerns. None makes an unsuitable propensity-score model, inadequate measured confounding control, or lack of positivity disappear.
The score itself is generally estimated in applications, and finite-sample diagnostics can differ from population properties. This tutorial's calculations treat the score values and outcome variances as given to clarify the role of partitioning; they do not account for uncertainty from estimating the scores. Also, narrower bins do not fix unmeasured confounding. They can make measured-score comparisons closer while simultaneously producing a less stable estimate.
A Practical Decision Process
Specify the estimand and the rule for combining stratum-specific contrasts before comparing partitions.
For each candidate partition, identify empty or sparse treatment-by-stratum cells and inspect the width of each interval.
Examine covariate balance within strata and calculate or estimate the uncertainty contributed by each stratum.
Look for material changes in the estimate or its uncertainty across partitions. Report the chosen partition and the diagnostics that support it.
Check Your Understanding
Answer each question, then compare with the solutions below.
- A score range has width 0.72. What is the equal-bin width with six strata, and what is the Lipschitz bound if \(L=8\) outcome units per score unit?
- Why does a smaller difference between treated and untreated mean scores not, by itself, establish a small baseline-outcome difference when the outcome function is nonlinear?
- With four equally weighted, independent stratum contrasts, each having estimated variance 16, what is the estimated variance and standard error of their average?
- What should be concluded if a proposed stratum contains treated units but no untreated units?
- Why is a stable estimate across several values of \(K\) useful but not proof that the analysis is unbiased?
Solutions. (1) The width is \(0.72/6=0.120\), so the bound is \(8(0.120)=0.960\) outcome units. (2) For nonlinear \(m_0\), the average of \(m_0(e)\) need not equal \(m_0\) evaluated at the average score; the distributions of scores, not only their means, matter. (3) The variance is \(4(1/4)^2(16)=4.000\), and the standard error is \(\sqrt{4}=2.000\) outcome units. (4) The within-stratum contrast is not estimable from the observed data because there is no untreated comparison group. (5) Stability reduces concern that the result depends strongly on the chosen partition, but all partitions may share unmeasured confounding, poor score modeling, or other sources of bias.