Quantile Treatment Effects Under Unconfoundedness

Why Marginal Distributions Matter

A treatment can change more than an average outcome. It may shift the lower tail, the median, or the upper tail by different amounts. A quantile treatment effect describes a contrast between quantiles of the two marginal potential-outcome distributions. The word “marginal” matters: each distribution describes outcomes under one treatment condition for the same target population, rather than for the subset that happened to receive that treatment.

When treatment is not randomly assigned, the treated and untreated groups can differ in prognostic characteristics. Comparing their observed outcome quantiles directly may therefore mix treatment differences with differences in group composition. Conditional exchangeability and overlap provide conditions under which the covariates can be used to identify the population distributions of potential outcomes.

Potential Outcomes and the Target Effect

Let \(D\in\{0,1\}\) denote treatment status, let \(X\) be a vector of observed pre-treatment covariates, and let \(Y\) be the observed outcome. For each unit, \(Y(1)\) is the outcome that would be observed under treatment and \(Y(0)\) is the outcome that would be observed without treatment. The observed outcome is \(Y=DY(1)+(1-D)Y(0)\): only one potential outcome is observed for each unit.

For example, \(\tau_{0.5}\) is the difference between the median outcome under treatment and the median outcome without treatment. It is not, in general, the median of the individual treatment effects \(Y(1)-Y(0)\). The two potential outcomes for a person are never jointly observed, and a difference in marginal quantiles does not identify the distribution of individual-level effects.

Conditions for Identification

Write \(e(X)=P(D=1\mid X)\) for the propensity score, or conditional probability of treatment. Identification from observed data relies on three conditions. These are assumptions about the assignment and measurement process, not conclusions that can be established merely by calculating an estimator.

• Consistency: the observed outcome equals the potential outcome corresponding to the treatment actually received: \(Y=DY(1)+(1-D)Y(0)\).
• Conditional exchangeability: \((Y(0),Y(1))\) is independent of \(D\) conditional on \(X\). Once the observed covariates are held fixed, treatment status carries no additional information about either potential outcome.
• Overlap: \(0<e(X)<1\) for covariate values in the target population. Each such covariate profile has a positive chance of appearing in either treatment group.

Exchangeability requires that the covariates adequately account for confounding relevant to the potential outcomes. It does not follow from having a large sample, and it cannot rule out unmeasured confounding by itself. Overlap is also substantive: if some profiles are essentially always treated or never treated, the data contain little or no information about the missing treatment condition for those profiles.

Identification of the Marginal CDFs

For either treatment condition, first condition on \(X\), then use exchangeability and consistency to link the potential outcome to outcomes observed among units with that treatment. The resulting conditional distribution can be averaged over the covariate distribution of the target population.

The expectation is over the population distribution of \(X\), not the covariate distribution within treatment arm \(d\). This distinction is central: standardization uses the conditional outcome distribution for each arm but averages both over one common target population.

The same identified CDF can be written using inverse-probability weights. Let \(p_1(X)=e(X)\) and \(p_0(X)=1-e(X)\). For either \(d\), the indicator \(I(D=d)\) selects observed outcomes in arm \(d\), and weighting by \(1/p_d(X)\) corrects for the probability of entering that arm.

To see why this expression works, condition on \(X\). For a unit with covariates \(X\), exchangeability and consistency imply that the conditional outcome distribution among units with \(D=d\) equals the conditional distribution of \(Y(d)\). The treatment indicator has conditional expectation \(p_d(X)\), so the inverse weight cancels the chance of being observed in that arm:

Taking expectations over \(X\) gives \(F_d(y)\). This argument identifies a distribution at every threshold \(y\); the quantile treatment effect follows by taking the generalized inverse of each identified CDF and subtracting.

Estimating the CDFs by Propensity-Score Weighting

In a sample of \(n\) units, estimate the propensity score \(e(X_i)\), or use its known value if treatment probabilities are known by design. The inverse-probability-weighted empirical CDF for treatment arm \(d\) is a weighted share of observed outcomes at or below \(y\). The normalized version below divides by the sum of the weights in that arm.

The normalized weighted CDF is a finite-sample estimate of the marginal distribution for the target population. Estimate \(Q_d(q)\) as the generalized inverse of \(\widehat F_d^{\,IPW}\), then calculate \(\widehat\tau_q=\widehat Q_1(q)-\widehat Q_0(q)\). Since an empirical CDF is a step function, its quantile is found at the smallest observed outcome where the estimated CDF reaches or exceeds \(q\).

Estimation by Covariate Standardization

An alternative is to estimate the conditional CDF \(m_d(y,x)=P(Y\le y\mid D=d,X=x)\) and average its predictions over the observed covariate distribution. For each threshold \(y\), the standardized estimator is

This procedure predicts the outcome distribution under arm \(d\) for each unit’s covariates, then averages those predictions over the target sample. The conditional CDF may be estimated using a distribution regression model—one binary-response model for each threshold—or another suitable conditional-distribution method. To calculate quantiles, the estimated marginal CDF must be nondecreasing in \(y\) and take values between zero and one; fitted predictions should be checked for these basic CDF properties before inversion.

Choosing and Interpreting an Estimator

Weighting and standardization implement the same identification result, but they estimate its ingredients differently. Weighting uses an estimated treatment probability to reweight observed outcomes. Standardization models the outcome distribution within treatment arm and predicts it across the target covariate distribution. Either approach can be sensitive to model misspecification, limited data, or poor overlap. The later material on doubly robust estimation considers combining these kinds of models; the procedures here do not acquire that property simply by being used together.

Overlap deserves particular attention for distributional effects. If \(\widehat e(X_i)\) is close to zero, a treated unit with those covariates can receive a very large weight; if it is close to one, the same problem can arise for an untreated unit. A few large weights can make an estimated CDF—and therefore its quantiles—highly sensitive to a small number of observations. Inspect propensity-score distributions and weight sizes, and state clearly what population the analysis targets if observations are trimmed or the target population is otherwise changed.

Common Mistake: Comparing the Observed Group Quantiles

A tempting calculation is to subtract the untreated group’s observed median from the treated group’s observed median. That comparison generally contrasts \(Y\) among people who received treatment with \(Y\) among a different set of people who did not. If the groups have different covariate distributions, the result need not equal the difference between marginal potential-outcome quantiles for a common population. Conditional exchangeability does not make the raw group quantiles comparable without adjustment; it permits identification after appropriately standardizing or weighting.