Inference For Quantile Treatment Effects

Why Quantile Treatment Effects Need Their Own Uncertainty Calculation

A quantile treatment effect (QTE) compares points in two potential-outcome distributions. For example, it can describe how the median outcome would differ under treatment and control, or how the difference varies at lower and upper quantiles. Its uncertainty is not obtained by treating the estimated quantiles as directly observed: each quantile is recovered by inverting an estimated distribution function. The uncertainty in that CDF, and the local slope of the true CDF at the quantile, both matter.

This tutorial develops inference for a fixed quantile and then explains how to make statements over a range of quantiles. The derivation uses the doubly robust CDF estimator from the earlier material on distributional treatment effects. The same quantile-inversion argument applies to other CDF estimators when they have an appropriate asymptotic representation.

From an Estimated CDF to an Estimated Quantile

Suppose observations \((Y_i,D_i,X_i)\), \(i=1,\ldots,n\), are independent and identically distributed. Identification by unconfoundedness requires consistency, \(Y=Y(D)\), conditional exchangeability, \((Y(0),Y(1))\) independent of \(D\) given \(X\), and positivity: the treatment probability \(e(X)=P(D=1\mid X)\) is bounded away from zero and one. Under these conditions, the potential-outcome CDF can be identified from observed outcomes and covariates.

Write \(p_1(X)=e(X)\), \(p_0(X)=1-e(X)\), and \(m_d(y,X)=P(Y\le y\mid D=d,X)\). One doubly robust estimator is \(\widehat F_d(y)=\mathbb P_n[\widehat m_d(y,X)+\mathbf 1(D=d)\{\mathbf 1(Y\le y)-\widehat m_d(y,X)\}/\widehat p_d(X)]\), where \(\mathbb P_n\) denotes the sample average. Under suitable nuisance-estimation and empirical-process conditions, its influence function at a fixed \(y\) is \(\phi_d(y)=m_d(y,X)-F_d(y)+\mathbf 1(D=d)\{\mathbf 1(Y\le y)-m_d(y,X)\}/p_d(X)\). It has mean zero. For the arguments below, assume the estimator also has a uniform asymptotic linear representation in a neighborhood of the target quantile: \[ \sup_{y\in\mathcal Y}\left|\widehat F_d(y)-F_d(y)-\mathbb P_n\phi_d(y)\right|=o_p(n^{-1/2}). \]

This representation is an assumption about the estimator and its nuisance estimates, not a consequence of quantile inversion. For example, it can be justified with cross-fitting, overlap, suitable moment bounds, and nuisance-estimation remainders that are uniformly \(o_p(n^{-1/2})\). We also require local stochastic equicontinuity: for any shrinking neighborhoods of \(Q_d(\tau)\), the centered CDF-estimation error changes by \(o_p(n^{-1/2})\) across that neighborhood. This condition lets us evaluate the CDF error at a random estimated quantile after first establishing that the quantile is close to its target.

Here is the order of the argument; establishing the rate before claiming a root-\(n\) Taylor remainder is essential.

In particular, the Taylor remainder is first \(o_p(|\widehat Q_d-Q_d|)\); it becomes \(o_p(n^{-1/2})\) only after the rate is proved. Similarly, replacing the fixed argument \(Q_d\) by the random \(\widehat Q_d\) in the CDF-estimation error relies on local stochastic equicontinuity, not just on a pointwise CDF expansion.

Inference for a Difference of Quantiles

For the QTE, subtract the two quantile expansions. Both potential-outcome quantiles are estimated from the same observations, so their influence functions can be correlated. Let \(f_d=f_d(Q_d(\tau))\). The influence function for \(\widehat\Delta(\tau)=\widehat Q_1(\tau)-\widehat Q_0(\tau)\) is \[ \psi_\Delta(\tau)=-\frac{\phi_1(Q_1(\tau))}{f_1}+\frac{\phi_0(Q_0(\tau))}{f_0}. \] If \(\widehat\psi_{\Delta,i}\) estimates this influence function for observation \(i\), a standard error estimator is the sample standard deviation of these estimated contributions divided by \(\sqrt n\). A pointwise large-sample confidence interval is \(\widehat\Delta(\tau)\pm z_{1-\alpha/2}\widehat{SE}\), where \(z_{1-\alpha/2}\) is the standard normal quantile.

The covariance term is important: omitting it treats two estimates from a shared sample as independent. In practice, estimating the variance from the observation-level QTE influence contributions automatically includes their covariance. The density estimates also matter. A small density at a quantile means that a modest CDF estimation error can correspond to a comparatively large quantile error.

Pointwise Inference and Simultaneous Bands

A pointwise interval targets one specified \(\tau\). If intervals are computed at many quantile levels, each has its stated coverage at its own level, but the collection does not thereby have the same coverage jointly. In particular, a set of 95% pointwise intervals need not contain the entire QTE curve with probability 95%.

For inference over a range of quantiles, one option is a simultaneous band based on the estimated influence-function process. On a prespecified range \(\mathcal T\), a multiplier bootstrap can approximate the distribution of the supremum of the standardized QTE process. A critical value is chosen from that supremum distribution and used across all \(\tau\in\mathcal T\); this accounts for dependence across quantiles. Such a band requires a valid uniform approximation, consistent density estimates over the range, and suitable control of the process near its endpoints.

A different route starts with simultaneous bands for the two CDFs. Suppose bands \([L_d(y),U_d(y)]\) cover \(F_d(y)\) jointly over the outcome range and both treatment states with probability approximately \(1-\alpha\). Inverting each band gives quantile bounds \[ Q_d^{L}(\tau)=\inf\{y:U_d(y)\ge\tau\}, \qquad Q_d^{U}(\tau)=\inf\{y:L_d(y)\ge\tau\}. \] When the bands are valid and the CDFs are monotone, the corresponding QTE bounds are \[ [\,Q_1^{L}(\tau)-Q_0^{U}(\tau),\quad Q_1^{U}(\tau)-Q_0^{L}(\tau)\,]. \] The reversed endpoints for the control quantile follow because increasing \(Q_0\) decreases \(Q_1-Q_0\). If the CDF bands cover jointly over the full outcome range and both arms, the inverted bounds cover the QTE curve over the associated quantile range, subject to the stated band validity. Inversion can be conservative, especially on a coarse outcome grid.

Common Mistakes and Practical Checks

A common derivation error is to write a first-order expansion with an \(o_p(n^{-1/2})\) remainder before proving that the estimated quantile is root-\(n\) consistent. Differentiability initially gives only \(o_p(|\widehat Q_d-Q_d|)\). The correction is to establish consistency, derive the rate from the local positive density and uniform CDF error, and only then convert the Taylor remainder to \(o_p(n^{-1/2})\). One must also justify evaluating the CDF-estimation error at the random estimated quantile, typically with local stochastic equicontinuity or a uniform asymptotic linear representation.

• Ignoring dependence across arms. The two potential-outcome CDF estimates use the same sample. Estimate QTE uncertainty from the difference of influence functions, including their covariance.
• Ignoring the density. CDF uncertainty alone is not quantile uncertainty. Divide by the density at the target quantile; weak density makes quantile inference less precise.
• Calling many pointwise intervals a simultaneous band. Pointwise coverage at each chosen level does not guarantee coverage of the full curve. Use a procedure calibrated for the supremum or invert genuinely simultaneous CDF bands.
• Overstating the causal interpretation. A QTE compares marginal potential-outcome quantiles. It is not generally the quantile of individual treatment effects, and its causal interpretation depends on the identification assumptions.