Why Estimate Distributions Before Quantiles?
A quantile treatment effect compares a quantile of the outcome distribution under treatment with the corresponding quantile under control. In observational data, estimating those distributions requires adjusting for differences in pre-treatment covariates. Inverse-probability weighting makes that adjustment by reweighting observed outcomes; outcome regression instead models the conditional outcome distribution. An augmented estimator combines both approaches and, under standard identification conditions, remains consistent if either one of the two nuisance models is correctly specified.
The target here is the marginal distribution of each potential outcome, not a conditional distribution at a particular covariate value. Once the two marginal CDFs have been estimated, their quantiles are obtained by inversion. This distinction matters: generally, a quantile of a marginal distribution is not the average of conditional quantiles.
Setup and Identification
For each unit, let \(D\in\{0,1\}\) indicate treatment, let \(X\) be a vector of observed pre-treatment covariates, and let \(Y(1)\) and \(Y(0)\) denote the potential outcomes under treatment and control. The observed outcome is \(Y=D Y(1)+(1-D)Y(0)\). For \(d\in\{0,1\}\), define the potential-outcome CDF \(F_d(y)=P(Y(d)\leq y)\). Its quantile at level \(\tau\in(0,1)\) is \(Q_d(\tau)=\inf\{y:F_d(y)\geq\tau\}\); the quantile treatment effect is \(Q_1(\tau)-Q_0(\tau)\).
Write \(e(X)=P(D=1\mid X)\) for the propensity score and \(p_1(X)=e(X)\), \(p_0(X)=1-e(X)\). Define the conditional potential-outcome CDF \(m_d(y,X)=P(Y\leq y\mid D=d,X)\). Under consistency and conditional unconfoundedness, this observed conditional CDF equals \(P(Y(d)\leq y\mid X)\). Therefore, \(F_d(y)=E[m_d(y,X)]\): the target distribution averages the conditional distribution over the covariate distribution of the target population.
The Augmented Estimator
Let \(\widehat e(X_i)\) estimate the propensity score and let \(\widehat m_d(y,X_i)\) estimate the conditional CDF. Put \(\widehat p_1(X_i)=\widehat e(X_i)\) and \(\widehat p_0(X_i)=1-\widehat e(X_i)\). For each threshold \(y\), use the outcome indicator \(Z_i(y)=\mathbf{1}\{Y_i\leq y\}\). The augmented inverse-probability estimator is
The first term predicts the conditional probability of an outcome at or below \(y\) for every unit and averages those predictions. The second term corrects the prediction using observed outcomes in treatment group \(d\), weighted by the inverse estimated probability of receiving that treatment. It is an augmentation because it adds a weighted prediction error to the outcome-regression average. With a known propensity score, this correction has conditional expectation zero when the conditional CDF is correct. With a correct propensity score, it corrects the average even if the conditional CDF is misspecified.
Worked Example: Computing an Augmented CDF Estimate
Consider four hypothetical observational units and estimate the treated-outcome CDF at \(y=10\). Suppose each estimated propensity score is \(0.5\). The fitted treated conditional CDF values and observed treatment indicators are shown below; the outcome indicator is needed only for treated units.
| Unit | \(D_i\) | \(Y_i\) | \(\widehat m_1(10,X_i)\) | \(\mathbf{1}\{Y_i\leq10\}\), if treated |
|---|---|---|---|---|
| 1 | 1 | 8 | 0.25 | 1 |
| 2 | 0 | 12 | 0.40 | — |
| 3 | 1 | 14 | 0.60 | 0 |
| 4 | 0 | 9 | 0.75 | — |
For treated units the inverse-probability multiplier is \(1/0.5=2\). The four contributions inside the sample average are \(0.25+2(1-0.25)=1.75\), \(0.40\), \(0.60+2(0-0.60)=-0.60\), and \(0.75\). Thus
The estimate says that the augmented estimate of the treated potential-outcome probability of an outcome no greater than 10 is 0.575. It is a sample estimate, not an observed proportion among the four units under treatment: two units were controls, and their predicted treated CDF values also contribute. An individual contribution can be below zero or above one even though a genuine CDF probability cannot; the average estimator is not automatically a valid CDF in finite samples.
Why the Estimator Is Doubly Robust
The key property is an expectation identity. Fix a threshold \(y\) and treatment level \(d\). Consider arbitrary working functions \(q_d(y,X)\) and \(\pi_d(X)>0\) in place of the fitted conditional CDF and treatment probability. For \(Z(y)=\mathbf{1}\{Y\leq y\}\), the corresponding population expression is
To obtain the equality, condition on \(X\): the conditional expectation of \(\mathbf{1}\{D=d\}\{Z(y)-q_d(y,X)\}\) is \(p_d(X)\{m_d(y,X)-q_d(y,X)\}\). The equality uses the definition of \(m_d\) among units with \(D=d\), which is well-defined under overlap.
If the propensity model is correct, \(\pi_d(X)=p_d(X)\), the right-hand side reduces to \(m_d(y,X)\), regardless of \(q_d\). If the outcome model is correct, \(q_d(y,X)=m_d(y,X)\), the correction term is zero, regardless of \(\pi_d\) (provided its denominator is positive). Averaging over \(X\) gives \(F_d(y)\) in either case. This is the estimator’s double-robustness property.
Worked Example: Checking Both Routes to Double Robustness
Suppose the covariate \(X\) takes two values, A and B, each with probability \(0.5\). At a fixed threshold \(y\), suppose the true treated probabilities are \(m_1(y,A)=0.25\) and \(m_1(y,B)=0.75\), while the true propensity scores are \(p_1(A)=0.8\) and \(p_1(B)=0.4\). The target treated CDF is
First, use a misspecified outcome model \(q_1(y,A)=q_1(y,B)=0.50\), but the correct propensity model. For A, the conditional expectation of the augmented expression is \(0.50+(0.8/0.8)(0.25-0.50)=0.25\). For B, it is \(0.50+(0.4/0.4)(0.75-0.50)=0.75\). Averaging gives \(0.50\), the target.
Now use a misspecified propensity model with \(\pi_1(A)=\pi_1(B)=0.50\), but the correct conditional CDF model. For A, the expression is \(0.25+(0.8/0.50)(0.25-0.25)=0.25\); for B, it is \(0.75+(0.4/0.50)(0.75-0.75)=0.75\). Again the average is \(0.50\). The first calculation relies on the correct propensity model; the second relies on the correct conditional CDF model. Neither calculation says the estimator is protected when both models are wrong.
From Estimated Distributions to Quantile Effects
After estimating \(\widehat F_1(y)\) and \(\widehat F_0(y)\), obtain an estimated quantile by inversion: \(\widehat Q_d(\tau)=\inf\{y:\widehat F_d(y)\geq\tau\}\). The estimated quantile treatment effect is \(\widehat Q_1(\tau)-\widehat Q_0(\tau)\). The sign and units follow the subtraction: a positive estimate means the treated quantile is higher, and the difference is measured in the outcome’s units.
Worked Example: Inverting Two Estimated CDFs
Suppose hypothetical augmented estimates, after enforcing monotonicity, give the following CDF values on an outcome scale measured in dollars per day.
| Threshold \(y\) | \(\widehat F_1(y)\) | \(\widehat F_0(y)\) |
|---|---|---|
| 0 | 0.10 | 0.10 |
| 10 | 0.35 | 0.60 |
| 20 | 0.65 | 0.85 |
| 30 | 0.90 | 0.98 |
At \(\tau=0.50\), the treated CDF is below 0.50 at 10 and reaches or exceeds 0.50 at 20, so the first listed threshold meeting the quantile definition is \(\widehat Q_1(0.50)=20\). The control CDF reaches 0.50 at 10, so \(\widehat Q_0(0.50)=10\). The estimated median treatment effect is therefore \(20-10=10\) dollars per day. This inversion uses the stated grid and the infimum definition; interpolating between grid points would require a separate interpolation convention.
Implementation and Common Pitfalls
In practice, the propensity score and conditional outcome CDF are estimated from data. Flexible models can be useful, but extreme fitted treatment probabilities can create very large inverse weights and unstable corrections. Analysts often constrain estimated probabilities away from zero and one or investigate the covariate regions generating extreme weights. Such safeguards can improve numerical stability, but they do not repair a failure of overlap in the target population.
When nuisance models are flexible, sample splitting or cross-fitting can help limit the impact of using the same observations both to fit nuisance functions and to evaluate the augmented score. It does not by itself ensure correct specification, overlap, or consistency. In addition, the pointwise augmented estimates across thresholds need not be nondecreasing or remain between zero and one in a finite sample. Since quantile inversion presumes a CDF-like curve, a nonmonotone estimate should be diagnosed and, when appropriate, replaced by a monotone adjustment before inversion. The adjustment and its effect on results should be reported.
- Mistake: calling the method robust to any misspecification. The estimator is consistent if at least one nuisance model is correctly specified, subject to the identification assumptions and suitable estimation conditions. If both models are wrong, double robustness offers no general guarantee.
- Mistake: interpreting a conditional quantile as the target quantile. The target is obtained by averaging conditional distributions and then inverting. In general, averaging conditional quantiles does not give the quantile of the marginal distribution.
- Mistake: treating each augmented contribution as a probability. Contributions can lie outside \([0,1]\); it is the estimated curve that is intended to approximate a CDF, and even that curve may need monotonicity correction in finite samples.
- Mistake: reading a positive quantile difference as proof of causation. The causal interpretation depends on consistency, conditional unconfoundedness, overlap, and the estimation conditions—not on the sign of the calculated difference alone.
Check Your Understanding
Use the definitions and conditions above to answer the following questions.
- What does the conditional unconfoundedness assumption imply about identifying \(F_d(y)\) from observed outcomes and covariates?
- At a fixed threshold, explain why a correct propensity model makes the conditional expectation of the augmented expression equal to \(m_d(y,X)\), even if the working outcome model is wrong.
- Why does a correct outcome CDF model make the expected residual correction zero, even when the propensity model is wrong?
- If the estimated treated median is 24 units and the estimated control median is 19 units, what is the estimated median treatment effect, and what are its units?
- Why can an augmented CDF estimate be negative at a particular threshold, and why is that a concern before quantile inversion?