Quantile-Based Estimation of Conditional Distributions

From Quantile Curves to Conditional Distributions

A conditional distribution describes more than a conditional mean: it records how the entire outcome distribution changes with predictors. One way to estimate it is to fit a conditional quantile curve at many probability levels. Those curves are useful summaries, but they must fit together: a higher probability level cannot have a lower quantile than a lower probability level. Once the curves are ordered, their values can be used to reconstruct a conditional cumulative distribution function (CDF).

Let $X$ be a predictor and $Y$ a real-valued outcome. For a fixed predictor value $x$, write $F_x(y)=P(Y\le y\mid X=x)$ for a version of the conditional CDF. At predictor values where a conditional law is only defined up to null sets, statements about $F_x$ are understood for almost every $x$ under the distribution of $X$.

Definition: For $0<u<1$, the generalized conditional quantile is $$ Q_x(u)=\inf\{y\in\mathbb{R}:F_x(y)\ge u\}. $$ It is nondecreasing in $u$. If $F_x$ has jumps, this generalized inverse still selects a quantile, but the set of values minimizing expected quantile loss can contain more than this one selected value.

Quantile Loss and Its Minimizers

Quantile regression estimates $Q_x(u)$ by minimizing the pinball, or check, loss. For residual $r$, define $\rho_u(r)=r(u-\mathbf{1}\{r<0\})$. Equivalently, $\rho_u(r)=ur$ when $r\ge0$, and $\rho_u(r)=(u-1)r$ when $r<0$. The loss weights under-prediction and over-prediction differently according to $u$.

Theorem: Suppose $E[|Y|\mid X=x]<\infty$, and let $F_x$ be the conditional CDF at the fixed $x$. The minimizers over $q\in\mathbb{R}$ of $E[\rho_u(Y-q)\mid X=x]$ are exactly the values satisfying $$ F_x(q-)\le u\le F_x(q), $$ where $F_x(q-)=P(Y<q\mid X=x)$. In particular, the generalized inverse $Q_x(u)$ is one such minimizer; it need not be the only one.

To prove the characterization, set $R(q)=E[\rho_u(Y-q)\mid X=x]$. This is a finite convex function of $q$: finiteness follows from the conditional first-moment assumption, and convexity follows because the loss is convex in $q$. Its left and right derivatives are $$ R'_-(q)=F_x(q-)-u,\qquad R'_+(q)=F_x(q)-u. $$ Indeed, for an outcome below $q$, the loss has slope $1-u$ with respect to $q$; for an outcome above $q$, it has slope $-u$. At $Y=q$, the left slope is $-u$ and the right slope is $1-u$. Taking the corresponding probabilities gives the displayed derivatives. For a finite convex function, $q$ is a minimizer exactly when its left derivative is nonpositive and its right derivative is nonnegative. Thus $F_x(q-)\le u\le F_x(q)$, as claimed. The generalized inverse satisfies this inequality by the definition of the CDF and its generalized inverse. $\square$

Worked Example: A Quantile Loss with Multiple Minimizers

Suppose, conditional on a particular $x$, $P(Y=0\mid X=x)=P(Y=2\mid X=x)=0.5$. Take $u=0.5$. For any $q\in[0,2]$, the expected pinball loss is $$ E[\rho_{0.5}(Y-q)\mid X=x] =0.5\left(\frac{q}{2}\right)+0.5\left(\frac{2-q}{2}\right)=0.5. $$ For $q<0$, both outcomes are at least $q$, so the expected loss is $0.5(1-q)>0.5$. For $q>2$, both outcomes are below $q$, giving expected loss $0.5(q-1)>0.5$. Hence every $q\in[0,2]$ is a minimizer. This also follows from $F_x(q-)\le0.5\le F_x(q)$ throughout that interval. The generalized inverse is $Q_x(0.5)=0$, one minimizer rather than the whole set.

Estimating Curves and Enforcing Their Order

Given observations $(X_i,Y_i)$, a quantile regression procedure at level $u$ estimates a function $q_u(x)$ by minimizing an empirical or regularized version of the pinball objective. For example, an empirical objective at a fixed $x$ can weight nearby observations: $$ \sum_{i=1}^{n} w_i(x)\rho_u(Y_i-q), $$ where $w_i(x)\ge0$ are specified weights. The modeling method can vary; the essential point here is that fitting each level separately does not automatically guarantee a coherent distribution.

The population quantiles are nondecreasing in $u$, but finite-sample estimates may cross. A standard repair on a finite probability grid $u_1<\cdots<u_J$ is weighted isotonic projection. Given raw estimates $z_j$ and positive weights $a_j$, choose the nondecreasing sequence $\tilde z_1\le\cdots\le\tilde z_J$ minimizing $\sum_{j=1}^J a_j(\tilde z_j-z_j)^2$. The pool-adjacent-violators algorithm (PAVA) computes this projection by merging adjacent blocks whose weighted means violate the required order. This is a post-processing step: it corrects order, but does not by itself establish statistical accuracy or remove model bias.

Worked Example: Projecting Crossed Quantile Estimates

At a fixed $x$, suppose the estimates at levels $0.25,0.50,0.75$ are $8,6,10$, with equal projection weights. The first two violate monotonicity. Pooling them gives their mean, $(8+6)/2=7$. The projected sequence is $(7,7,10)$, which is nondecreasing. For the first two entries, the unconstrained fitted values violate the order constraint $z_1\le z_2$, so their constrained optimum lies on the boundary where both fitted values equal $t$. Minimizing $(t-8)^2+(t-6)^2$ gives $t=7$. The final value 10 already respects the order. The correction changes the first two estimates, so it should be reported as a coherence adjustment, not as new independent evidence about either quantile.

Reconstructing a CDF from Quantiles

A quantile function is a representation of a distribution. If $U$ is uniform on $(0,1)$, then $Q_x(U)$ has conditional CDF $F_x$. In particular, for any real $y$, $$ F_x(y)=\int_0^1 \mathbf{1}\{Q_x(u)\le y\}\,du. $$ The integrand is nonincreasing in $u$, because $Q_x$ is nondecreasing. This identity makes reconstruction direct: the CDF at $y$ is the fraction of probability levels whose quantiles do not exceed $y$.

Worked Example: Turning a Quantile Grid into a CDF

Suppose an estimated quantile function is constant on four equal-width probability intervals, with values $0,1,1,3$, respectively. This is nondecreasing. At $y=0.5$, only the first interval has a quantile no greater than $y$, so the reconstructed CDF is $1/4$. At $y=1$, the first three intervals qualify, so it is $3/4$. At $y=2$, the same three intervals qualify, and the CDF remains $3/4$. Thus the implied distribution puts mass $1/4$ at 0, mass $1/2$ at 1, and mass $1/4$ at 3. The repeated quantile value correctly represents an atom; a quantile curve need not be strictly increasing.

Tail and Grid Error

In practice, quantiles are often fitted only on a central range $[a,1-a]$, and only at finitely many levels. Both limitations matter. The central range omits tail information; a finite grid approximates the integral over probability levels. The following bound makes the contributions explicit. Suppose the true conditional distribution has a density bounded by $M$, and a nondecreasing fitted curve $\tilde Q_x$ is defined on $[a,1-a]$, where $0<a<1/2$. Assume $$ \sup_{u\in[a,1-a]}|\tilde Q_x(u)-Q_x(u)|\le\delta. $$ Define the central-range CDF approximation $$ \tilde F_{x,a}(y)=a+\int_a^{1-a}\mathbf{1}\{\tilde Q_x(u)\le y\}\,du. $$ The added $a$ assigns the omitted lower tail below the evaluation point; the omitted upper tail is assigned above it. This is a central approximation, not necessarily a proper CDF on the whole real line.

Result: Under these conditions, $$ \sup_y|F_x(y)-\tilde F_{x,a}(y)|\le a+2M\delta. $$ If the central integral is replaced by a midpoint sum over a partition of $[a,1-a]$ whose largest cell width is $h$, the bound becomes $a+2M\delta+h$.

For the proof, let $A(y)=\int_0^a\mathbf{1}\{Q_x(u)\le y\}\,du$ and $B(y)=\int_{1-a}^1\mathbf{1}\{Q_x(u)\le y\}\,du$. Each lies in $[0,a]$, and the quantile representation gives $F_x(y)=A(y)+\int_a^{1-a}\mathbf{1}\{Q_x(u)\le y\}\,du+B(y)$. Replacing the two tails by $a$ and 0 changes their combined contribution by $A(y)+B(y)-a$, whose absolute value is at most $a$.

For the central integral, if the indicators based on $Q_x(u)$ and $\tilde Q_x(u)$ differ, the uniform error bound implies $|Q_x(u)-y|\le\delta$. Since $Q_x(U)$ has CDF $F_x$, the measure of such $u$ is at most $P(y-\delta\le Y\le y+\delta\mid X=x)\le2M\delta$. This proves the first bound. For the midpoint sum, the function $u\mapsto\mathbf{1}\{\tilde Q_x(u)\le y\}$ is nonincreasing and takes values in $[0,1]$. On each cell its integral differs from the cell width times its midpoint value by at most the cell width times the drop of the function on that cell. Summing these bounds gives at most $h$, because the total drop is at most 1. Adding this quadrature error proves the second bound. $\square$

If the conditional distribution has no bounded density, the probability in the proof still gives a valid error expression: use $P(y-\delta\le Y\le y+\delta\mid X=x)$ instead of $2M\delta$. This distinction matters near atoms, where a small quantile error can produce a substantial CDF discrepancy at a jump. Tail modeling or tail quantile estimates are also needed when a proper full-range distribution estimate is required; central quantiles alone do not determine the tails.

Practical Interpretation and a Common Mistake

Quantile-based estimation is attractive when conditional spread, skewness, or tail behavior changes with $x$. It avoids forcing every conditional distribution into a single parametric shape and provides interpretable summaries at selected probability levels. To obtain a CDF, however, one needs a coherent curve over levels, a strategy for the unobserved tails, and an account of finite-grid approximation. The error bound separates these concerns rather than treating a smooth-looking plot as evidence that all three have been solved.

Common mistake: treating the generalized inverse as the only loss minimizer. At a jump or across a gap in the support, several values can satisfy $F_x(q-)\le u\le F_x(q)$. The generalized inverse chooses one of them; the loss characterization describes the full set.
Common mistake: interpreting independently fitted crossing curves as a valid quantile function. A conditional distribution's quantiles must be nondecreasing in probability level. Apply an explicit monotonicity correction or use a fitting method that guarantees the constraint.
Common mistake: assuming a central quantile grid fully identifies the tails. The omitted probability levels have nonzero mass. State the tail convention or model the tails, and report grid resolution when reconstructing the CDF.

Key takeaway: Conditional quantile curves can encode an entire conditional distribution, but only when their ordering, probability-level coverage, and numerical reconstruction are handled explicitly.

Check Your Understanding

Try these questions using the definitions and results above.

For a conditional distribution with an atom at $q$, what condition characterizes every minimizer of expected pinball loss at level $u$?
Why does a fitted sequence of quantiles need to be nondecreasing in its probability levels?
What does a flat segment of a quantile function imply about the corresponding distribution?
In the stated reconstruction bound, which terms account for omitted tails, quantile error, and grid discretization?
Why might the density-based part of the bound fail to be informative for a distribution with a large atom?