Estimating The Density at A Conditional Quantile

Why Density Estimation Enters Quantile Inference

Quantile regression estimates how a specified part of an outcome distribution varies with covariates. Its sandwich variance depends not only on the quantile score but also on how much probability density lies near the fitted conditional quantile. When that density is low, a small change in probability level can correspond to a large change in the outcome, and the fitted quantile is less precisely determined.

The density is usually unknown, so estimating the sandwich covariance requires estimating it as well. Two approaches are useful: smoothing the fitted residuals with a kernel, or measuring how fitted quantiles change across nearby probability levels. Their bandwidths have different units and control different approximations. This tutorial compares these approaches and shows how their choices affect estimated uncertainty.

Model and Sandwich Covariance

Let \(Y_i\) be a scalar outcome and \(X_i\) a \(k\)-dimensional vector of regressors, including an intercept when appropriate. For a fixed quantile index \(\tau\), where \(0<\tau<1\), suppose the conditional quantile is correctly specified as \(Q_\tau(Y_i\mid X_i)=X_i^\top\beta_\tau\). Write \(f_i\) for the conditional density of \(Y_i\) given \(X_i\), evaluated at that quantile: \(f_i=f_{Y\mid X}(X_i^\top\beta_\tau\mid X_i)\).

For the standard quantile-regression sandwich result, assume the observations are independent and identically distributed, the conditional quantile model is valid near \(\tau\), and the conditional density is positive and sufficiently smooth at the quantile. Also assume suitable finite moments for \(X_i\) and that the matrices below are nonsingular. These conditions exclude a density that vanishes at the target quantile; atoms and ties require separate care.

Define the two population matrices

The asymptotic covariance matrix for \(\sqrt{n}(\hat{\beta}_\tau-\beta_\tau)\) is \(A_\tau^{-1}B_\tau A_\tau^{-1}\). Consequently, the covariance matrix for \(\hat{\beta}_\tau\) is estimated by \(n^{-1}\hat A_\tau^{-1}\hat B_\tau\hat A_\tau^{-1}\). A common plug-in estimate of \(B_\tau\) is \(\hat B_\tau=\tau(1-\tau)n^{-1}\sum_{i=1}^n X_iX_i^\top\). The density estimates considered below supply \(\hat A_\tau\), often called the bread of the sandwich.

Kernel Estimation From Fitted Residuals

Fit the quantile regression at \(\tau\), and define the fitted residual \(\hat r_i=Y_i-X_i^\top\hat\beta_\tau\). A kernel estimate of the bread matrix is

Here \(K\) is a nonnegative kernel that integrates to one, and \(h>0\) is a bandwidth measured in units of \(Y\). The factor \(1/h\) in this expression is implicit if the kernel is defined as \(K_h(r)=K(r/h)/h\); the displayed formula includes that scaling by writing the full expression as \(K_h(\hat r_i)\). Equivalently, and explicitly,

The kernel weights observations according to how close their fitted outcomes are to the fitted quantile. This matrix estimate targets the density-weighted regressor moment \(E[f_iX_iX_i^\top]\). It does not, by itself, report a separate conditional density for every possible covariate value. Estimating density separately at a particular \(X=x\) generally requires additional smoothing or modeling of \(X\).

A simple choice is the rectangular kernel, \(K(u)=1/2\) for \(|u|\leq 1\) and zero otherwise. With this choice, only residuals within \(h\) of zero contribute. A very small \(h\) can leave few contributing observations, making the bread estimate unstable or singular. A large \(h\) includes observations farther from the quantile and can blur changes in density around it.

Difference Quotients Across Quantile Levels

A second approach uses nearby conditional quantiles. If the conditional distribution is smooth and has a positive density at its \(\tau\)-quantile, the derivative of the conditional quantile with respect to its probability index is the reciprocal of the conditional density:

This identity follows by differentiating the defining relationship \(F_{Y\mid X}(Q_\tau(Y\mid X=x)\mid x)=\tau\) with respect to \(\tau\). The chain rule gives \(f_{Y\mid X}(Q_\tau\mid x)\,\partial Q_\tau/\partial\tau=1\), and dividing by the positive density gives the displayed result. A symmetric finite difference therefore estimates the derivative, and its reciprocal estimates the density.

Fit quantile regressions at \(\tau-\delta\) and \(\tau+\delta\), where \(0<\delta<\min(\tau,1-\tau)\). Under the linear conditional quantile model, the estimated quantile difference at \(X_i\) is \(X_i^\top(\hat\beta_{\tau+\delta}-\hat\beta_{\tau-\delta})\). This gives the observation-specific density estimate

The estimate requires a positive, nonzero fitted quantile difference. A nonpositive value can signal sampling instability or quantile crossing and does not yield a valid positive density estimate. When the estimates are usable, the corresponding bread estimate is \(n^{-1}\sum_i\hat f_{i,DQ}X_iX_i^\top\). Unlike the residual-kernel estimate, this approach smooths across quantile indices rather than across residual distances. Its bandwidth \(\delta\) is measured in probability units, not outcome units.

Bandwidth Choices and Their Consequences

For the kernel estimator, \(h\) controls how far in outcome units residuals can be from zero and still contribute. If \(h\) is too small, the estimate is driven by few residuals and can vary sharply across samples. If \(h\) is too large, the estimate averages density over a wider outcome range, potentially obscuring local curvature or changes in density. Under smoothness conditions, reducing \(h\) reduces smoothing bias, while increasing \(h\) generally reduces sampling variability. This is a bias–variance trade-off, not a rule that one direction always improves the standard error.

For the difference quotient, \(\delta\) controls how far apart the two quantile fits are in probability units. A small \(\delta\) approximates the local derivative more closely, but the difference between two estimated quantiles can be noisy relative to its size. A larger \(\delta\) can stabilize that difference while estimating an average slope over a wider interval, rather than the derivative precisely at \(\tau\). The target also restricts the choice: both \(\tau-\delta\) and \(\tau+\delta\) must lie between zero and one.

The estimated bread enters the covariance through its inverse on both sides. In an intercept-only model, \(\widehat{\operatorname{Var}}(\hat\beta_\tau)\) is proportional to \(1/\hat f^2\). Thus, in that setting, a smaller estimated density gives a larger standard error. With multiple regressors, the result also depends on the matrix structure and its inverse; a change in one density weight need not move every coefficient’s standard error in the same direction.

Practical Checks and Common Pitfalls

The two methods estimate the same local density under their respective smoothness and model conditions, but they need not agree in a finite sample. The kernel method is sensitive to residual-window choices; the difference quotient is sensitive to the spacing and stability of the nearby quantile fits. It is informative to check whether plausible bandwidth choices yield materially different covariance estimates and whether the fitted bread matrix is positive and well-conditioned.

• Confusing bandwidth units. The kernel bandwidth \(h\) is in outcome units; the difference-quotient bandwidth \(\delta\) is in probability units. They are not interchangeable.
• Ignoring the reciprocal. The quantile difference estimates the derivative of the quantile with respect to probability. The density is the reciprocal of that derivative, not the derivative itself.
• Assuming a larger bandwidth always increases the standard error. The covariance depends on the estimated density and, with multiple regressors, on matrix inversion. The direction is not universal.
• Treating density as a causal effect. The conditional density controls local statistical precision; it is not a causal effect of a regressor on the outcome.