Quantile Loss and Conditional Quantiles

Why Quantile Loss Is Useful

A prediction is not always intended to estimate a conditional mean or median. If underpredicting a response is more costly than overpredicting it, or if a decision is designed around a particular tail of a distribution, a conditional quantile can be the relevant target. Pinball loss encodes that asymmetry: its penalty for an error in one direction differs from its penalty for an error of the same size in the other direction.

The key question is what prediction minimizes expected pinball loss. For a smooth distribution, the answer is a quantile at the chosen level. With atoms or gaps in the distribution, however, the minimizer may need to be described as a set rather than a single number. The conditional distribution function gives an exact characterization in both cases.

Pinball Loss and Conditional Risk

Definition: For quantile level $\tau\in(0,1)$, the pinball loss for residual $u$ is $$ \rho_\tau(u)=u\bigl(\tau-\mathbf{1}\{u<0\}\bigr) = \begin{cases} \tau u, & u\geq 0,\\ (1-\tau)(-u), & u<0. \end{cases} $$ For a response $Y$ and prediction $a$, the loss is $\rho_\tau(Y-a)$. The conditional risk given covariates $X$ is $R_\tau(a\mid X)=E[\rho_\tau(Y-a)\mid X]$.

When $Y>a$, the prediction is below the response and the loss is $\tau(Y-a)$. When $Y<a$, the prediction is above the response and the loss is $(1-\tau)(a-Y)$. Thus, if $\tau$ is large, underprediction receives the larger weight. This direction follows from defining the residual as $Y-a$; reversing the residual without changing the loss formula would reverse the interpretation.

For the results below, take a real-valued random variable with $E|Y|<\infty$. In the conditional setting, assume $E[|Y|\mid X]<\infty$ almost surely. These conditions ensure the risks under discussion are finite for finite predictions. Write $F(y)=P(Y\leq y)$ for the distribution function and $F(y-)=P(Y<y)$ for its left limit. For conditional risk, the same notation refers to the conditional distribution function $F_X(y)=P(Y\leq y\mid X)$, with $F_X(y-)=P(Y<y\mid X)$, using a regular conditional distribution.

Worked Example: Reading the Direction of the Penalty

Let $\tau=0.8$, let the response be $Y=10$, and compare predictions $a=7$ and $a=13$. For $a=7$, the residual is $10-7=3$, so the loss is $0.8(3)=2.4$. For $a=13$, the residual is $10-13=-3$, so the loss is $(1-0.8)(3)=0.6$. Under this convention, underpredicting by three units costs four times as much as overpredicting by three units.

Deriving the Minimizers From the CDF

Let $r(a)=E[\rho_\tau(Y-a)]$ be the unconditional risk at prediction $a$. The loss as a function of $a$ is piecewise linear. For a fixed response value $y$, its slope is $1-\tau$ when $a>y$, and $-\tau$ when $a<y$. At $a=y$, the slope changes from $-\tau$ on the left to $1-\tau$ on the right.

Theorem: If $E|Y|<\infty$, the left and right derivatives of the pinball risk exist and are $$ r'_-(a)=F(a-)-\tau, \qquad r'_+(a)=F(a)-\tau. $$ Consequently, the complete set of risk minimizers is $$ \operatorname*{arg\,min}_{a\in\mathbb{R}} r(a) = \{a\in\mathbb{R}: F(a-)\leq \tau\leq F(a)\}. $$

Proof. First, the loss is convex in the prediction: for each fixed $y$, it is linear on either side of $y$, with left slope $-\tau$ and right slope $1-\tau$, and the slope increases at $y$. Hence $r$, the expectation of these convex functions, is convex.

For a positive increment $h$, the difference quotient $[\rho_\tau(Y-a-h)-\rho_\tau(Y-a)]/h$ is bounded in absolute value by $\max(\tau,1-\tau)$, because the loss is Lipschitz in its prediction with that constant. For each fixed $Y$, as $h$ decreases to zero, this quotient tends to $1-\tau$ when $Y\leq a$, and to $-\tau$ when $Y>a$. The bound is integrable, so the dominated convergence theorem allows the limit to pass through the expectation. Therefore $$ r'_+(a)=(1-\tau)P(Y\leq a)-\tau P(Y>a) =(1-\tau)F(a)-\tau\{1-F(a)\} =F(a)-\tau. $$ For the left derivative, use $[r(a)-r(a-h)]/h$. The corresponding pointwise slopes are $1-\tau$ when $Y<a$, and $-\tau$ when $Y\geq a$. The same Lipschitz bound and dominated convergence give $$ r'_-(a)=(1-\tau)F(a-)-\tau\{1-F(a-)\} =F(a-)-\tau. $$ For a convex function, $a$ minimizes the function exactly when its left derivative is nonpositive and its right derivative is nonnegative. Applying that criterion here yields $F(a-)\leq\tau\leq F(a)$, as claimed. $\square$

The two inequalities are both needed. At a point of positive probability, $F(a)$ can exceed $F(a-)$; the target level $\tau$ may lie anywhere between them. The inequality involving $F(a-)$ also captures flat portions of the CDF: when the CDF equals $\tau$ across a gap in the support, predictions throughout that gap can minimize risk.

Worked Example: A Quantile at an Atom

Suppose $Y$ has probabilities $P(Y=-1)=0.2$, $P(Y=2)=0.5$, and $P(Y=5)=0.3$. Find the minimizer for $\tau=0.6$. At $a=2$, $F(2-)=0.2$ and $F(2)=0.7$, so $$ 0.2\leq 0.6\leq 0.7. $$ The theorem says $a=2$ minimizes risk. Because the inequalities are strict, no other prediction satisfies the condition: below 2, the CDF values are at most $0.2$, and above 2 the left limit is at least $0.7$. Thus the minimizer is unique.

The risk there can be checked directly. The response $-1$ gives residual $-3$, with loss $0.4(3)=1.2$; the response $2$ gives zero loss; and the response $5$ gives residual $3$, with loss $0.6(3)=1.8$. Weighting by the probabilities gives $$ r(2)=0.2(1.2)+0.5(0)+0.3(1.8)=0.24+0.54=0.78. $$

The Full Quantile Interval

A common quantile convention reports the lower quantile $q_\tau=\inf\{a:F(a)\geq\tau\}$. This is always a minimizer for $\tau\in(0,1)$, but it need not be the only one. The full set of minimizers can be written as a closed interval: $$ \left[\inf\{a:F(a)\geq\tau\},\ \sup\{a:F(a-)\leq\tau\}\right]. $$ For a real-valued distribution and $0<\tau<1$, both endpoints are finite. Monotonicity of $F$ shows that every point between these endpoints satisfies $F(a-)\leq\tau\leq F(a)$; points outside cannot satisfy both inequalities. In particular, if the CDF jumps across $\tau$ at one point, the interval collapses to that point. If the CDF stays at $\tau$ over a gap, the interval extends across the gap.

Worked Example: A Whole Interval of Minimizers

Let $P(Y=0)=0.25$, $P(Y=4)=0.50$, and $P(Y=10)=0.25$, and take $\tau=0.25$. For any $a$ with $0\leq a\leq4$, $F(a-)=0.25$ when $a>0$ and $F(a)=0.25$ when $a<4$; at the endpoints the corresponding one-sided inequalities still hold. Thus every $a\in[0,4]$ satisfies $F(a-)\leq0.25\leq F(a)$.

The constant risk can also be verified directly. Across this interval, the response $0$ is at or below the prediction and contributes $0.25(0.75a)$. The response $4$ contributes $0.50(0.25(4-a))$, and the response $10$ contributes $0.25(0.25(10-a))$. Therefore $$ r(a)=0.1875a+0.125(4-a)+0.0625(10-a) =0.1875a+0.5-0.125a+0.625-0.0625a =1.125. $$ Every prediction from 0 to 4 has the same minimal risk. Choosing just the lower quantile, 0, is valid, but it does not describe all optimal predictions.

Conditional Quantiles as Bayes Predictions

For prediction with covariates $X$, apply the same result to the conditional distribution at each covariate value. At almost every $X$ for which $E[|Y|\mid X]$ is finite, a prediction $a$ minimizes $R_\tau(a\mid X)$ exactly when $$ F_X(a-)\leq\tau\leq F_X(a). $$ Consequently, a measurable prediction rule $g(X)$ is conditionally optimal almost surely if it satisfies this condition at almost every $X$. There may be multiple such rules when conditional distributions have atoms or flat regions.

Worked Example: Conditional Quantile Predictions

Let $X$ take two values, A and B, and choose $\tau=0.8$. Given A, suppose $Y=1$ and $Y=3$ each have probability $0.5$. The CDF crosses $0.8$ at 3, so the unique conditional minimizer is 3. Its conditional risk is $0.5\{0.2(2)\}+0.5(0)=0.2$.

Given B, suppose $P(Y=0\mid X=\mathrm{B})=0.2$, $P(Y=2\mid X=\mathrm{B})=0.6$, and $P(Y=8\mid X=\mathrm{B})=0.2$. Here $F_X(2)=0.8$, and $F_X(8-)=0.8$, so every prediction from 2 to 8 is conditionally optimal. For any such prediction $a$, the three contributions to conditional risk are $$ 0.2\{0.2a\}+0.6\{0.2(a-2)\}+0.2\{0.8(8-a)\} =0.04a+0.12a-0.24+1.28-0.16a =1.04. $$ In particular, choosing 2 or 8 gives the same risk. The optimal prediction is unique at A but not at B, despite using the same quantile level.

Common Mistake and Why the Characterization Matters

A common mistake is to demand $F(a)=\tau$ and conclude that there is no minimizer when the CDF jumps from below $\tau$ to above it. For example, in the first worked example $F(a)$ never equals $0.6$, yet $a=2$ is the unique minimizer. The correction is to check both one-sided CDF values: the correct condition is $F(a-)\leq\tau\leq F(a)$.

Another mistake is to report the lower quantile as the only minimizer. At a level where the CDF is flat, the risk can remain constant over an interval, as the second example demonstrates. This distinction matters when reporting fitted quantiles, comparing predictions, or interpreting nonuniqueness: different optimal predictions may have exactly the same conditional risk, rather than one prediction being an approximation to a uniquely determined target.

Key takeaway: Pinball loss at level $\tau$ is minimized by exactly those predictions $a$ for which the conditional distribution places the level $\tau$ between $F_X(a-)$ and $F_X(a)$. Jumps and flat regions determine whether that set is a point or an interval.

Check Your Understanding

Use the CDF condition and the stated loss convention to answer each question.

For $\tau=0.7$, a distribution has $F(2-)=0.4$ and $F(2)=0.8$. Is 2 a risk minimizer? What additional information would be needed to decide whether it is the unique minimizer?
For a response of 6 and prediction of 8, write the pinball loss at $\tau=0.25$.
If $F(a-)=F(a)=\tau$ throughout an interval, what does the minimizer characterization imply for predictions in that interval?
Explain why checking only $F(a)=\tau$ can miss a valid minimizer when the distribution has an atom.
For a conditional prediction rule $g(X)$, state the condition it must satisfy almost surely to minimize conditional pinball risk.