Estimating Propensity Scores for Subclassification

Why Estimate Propensity Scores for Subclassification?

Subclassification compares treated and untreated units within groups formed from their estimated probabilities of receiving treatment. Those probabilities summarize measured pretreatment covariates: units with similar scores have similar modeled chances of treatment. To use the method, analysts first need a treatment-assignment model that produces scores for every unit, then must check that the scores support comparisons within the chosen strata.

A fitted score is not itself an effect estimate, and a good-fitting treatment model does not guarantee that confounding has been removed. Its role is to guide the formation of groups in which treatment and control units can be compared. The validity of a causal interpretation still depends on the assumptions below and on adequate covariate balance within strata.

Assumptions and Model Setup

The propensity-score model predicts treatment assignment, not the outcome. In a common specification, the conditional probability of treatment follows a logistic regression:

Here, \(X_{i1},\ldots,X_{ip}\) are recorded before treatment, and the coefficients describe how those covariates relate to the log odds of treatment. Once the coefficients are estimated, the predicted score is

For a causal comparison, the model is only one part of the design. The key assumptions include consistency (the observed outcome under the received treatment equals the corresponding potential outcome), conditional exchangeability (given measured \(X_i\), treatment assignment is independent of potential outcomes), and positivity (each covariate profile of interest has a nonzero probability of either treatment). These assumptions are not established by fitting a logistic regression. In particular, exchangeability requires that relevant confounders have been measured and included appropriately.

The model also needs a useful functional form. A linear predictor in the original covariates may be inadequate if treatment assignment depends on nonlinear terms or interactions. Researchers should choose covariates based on subject-matter knowledge and their relevance to treatment assignment and outcomes, rather than selecting them only to maximize prediction accuracy. Include pretreatment covariates; conditioning on variables affected by treatment can create bias rather than control confounding.

Fitting the Model and Obtaining Scores

For independent units with binary treatment indicators, logistic regression estimates the coefficients by maximum likelihood. For a proposed coefficient vector \(\beta\), the likelihood is the probability of the observed treatment assignments under the model:

The fitted coefficients maximize this likelihood; equivalently, they maximize its logarithm. For the logistic model, differentiating the log-likelihood with respect to each coefficient gives the score equations. For example, the intercept equation is \(\sum_i\{A_i-\hat e_i\}=0\), and a coefficient for covariate \(X_{ij}\) has equation \(\sum_iX_{ij}\{A_i-\hat e_i\}=0\). These equations express model-based agreement between observed and predicted treatment totals, including covariate-weighted totals. They do not, by themselves, prove that covariates are balanced within subclassification strata.

The balancing-score property helps explain why the true propensity score is useful. Conditional on the true score, treatment is independent of the measured covariates in distribution. For any fixed score value \(e\), the probability of treatment given covariates is \(e\) for every covariate profile with that score. Thus, among units at that score, treated and control units have the same covariate distribution. In practice, the score is estimated and observations are grouped into ranges, so this exact property does not automatically hold within empirical strata.

From Predicted Scores to Within-Stratum Contrasts

After calculating \(\hat e_i\), assign units to strata according to their scores, often using score quantiles as described in the earlier material on forming propensity-score strata by quantiles. The estimated score should be calculated for both treated and control units using the same fitted model. Within a stratum, compare the observed outcomes of its treated and control units. For a continuous outcome, the unadjusted stratum contrast is the treated mean minus the control mean:

This contrast is descriptive unless the causal assumptions are credible and the stratum contains comparable treated and control units. A stratum with no observations in one treatment group has no within-stratum contrast. Even when both groups are present, substantial remaining covariate differences can leave residual confounding.

Diagnostics, Alternatives, and Limitations

Diagnostics should evaluate whether the fitted scores and the resulting strata support the intended comparisons, not merely whether the regression converged. Inspect the score distributions separately for treated and control units, and check whether both groups are represented across the score range relevant to the target population. Within each stratum, compare the distributions of important covariates between treatment groups. A model can predict treatment well yet produce poor covariate balance for causal analysis; conversely, a modest prediction model may be adequate if the resulting strata balance relevant covariates.

• Limited overlap: if a stratum contains only treated units or only controls, a within-stratum contrast cannot be calculated. Very sparse groups can also make contrasts unstable.
• Residual imbalance: differences in covariate means or distributions within strata indicate that grouping has not sufficiently controlled those measured covariates. Consider revising the model or stratification design and reassessing balance.
• Separation or extreme fitted scores: when covariate patterns nearly determine treatment, logistic estimates may be unstable or scores may approach zero or one. This often signals weak empirical support for comparisons, not a problem that can be fixed by interpreting predictions as certainty.
• Model misspecification: omitted nonlinear terms, interactions, or important confounders can yield scores that do not balance relevant covariates. Check balance directly rather than relying only on coefficient significance or a global fit statistic.

Subclassification is one of several ways to use propensity scores. Matching pairs units with similar scores; weighting assigns contributions according to score-based weights; regression adjustment models the outcome conditional on covariates. Each approach has different implementation and sensitivity properties. Subclassification offers transparent within-group comparisons, but grouping discards some score detail, and too few broad strata can leave residual imbalance while excessive subdivision can create sparse groups. No approach removes bias from unmeasured confounding simply by using a fitted propensity score.