Inference For Distribution Regression Curves

Why Inference for a Distribution-Regression Curve Is Different

Distribution regression models the conditional cumulative distribution function by fitting a binary-response regression separately at each outcome threshold. For a threshold \(y\), the response is \(1\{Y_i\le y\}\), and the fitted probability at covariates \(x\) estimates \(F_{Y\mid X}(y\mid x)=P(Y\le y\mid X=x)\). Inference for one such probability is pointwise. Inference for a curve asks a stronger question: how uncertain are the predictions across many thresholds, covariate profiles, or both?

The distinction matters because estimates at different thresholds are not independent. The binary responses are nested: if an observation is at or below a smaller threshold, it is also at or below a larger one. A collection of ordinary 95% intervals therefore does not automatically provide 95% coverage for the entire collection of predictions.

Pointwise Uncertainty: From Coefficients to a Predicted Probability

At a fixed threshold \(y\), write \(Z_i(y)=1\{Y_i\le y\}\) and \(p_i(y)=\Lambda(X_i'\beta(y))\). The logistic score equation is $$ \frac{1}{n}\sum_{i=1}^{n}X_i\{Z_i(y)-\Lambda(X_i'\hat\beta(y))\}=0. $$ For the usual large-sample approximation, observations are independent and identically distributed, the conditional probability model is correctly specified, the relevant moments are finite, and the information matrix is nonsingular. The threshold is treated as fixed. Under these regularity conditions, \(\hat\beta(y)\) is asymptotically normal. A sandwich covariance estimator can also be used to allow for a more general variance of the score.

Let \(\widehat V(y)\) estimate the asymptotic covariance matrix of \(\sqrt{n}\{\hat\beta(y)-\beta(y)\}\). Thus, the estimated covariance matrix of \(\hat\beta(y)\) itself is \(\widehat V(y)/n\). For a fixed profile \(x\), the derivative of the predicted probability with respect to \(\beta(y)\) is \(\Lambda(x'\beta)\{1-\Lambda(x'\beta)\}x\). Applying the delta method gives the standard error of the predicted probability:

Equivalently, if \(\widehat\Sigma(y)=\widehat V(y)/n\) estimates the covariance matrix of \(\hat\beta(y)\), the variance estimate is \(g'\widehat\Sigma(y)g\), where \(g=\hat p(1-\hat p)x\). A pointwise Wald interval is \(\hat p\pm 1.96\widehat{SE}\{\hat p\}\) for approximate 95% coverage at this particular \(y\) and \(x\). It may extend outside \([0,1]\). A link-scale interval can instead be formed for \(x'\beta(y)\) and transformed through \(\Lambda\), producing endpoints inside \([0,1]\); that transformation does not turn pointwise coverage into simultaneous coverage.

Contrasts and Covariance at a Fixed Threshold

Often the target is a difference between predictions at two profiles, not either probability alone. Since both predictions use the same estimated coefficient vector at threshold \(y\), they are correlated. Inference for their difference must use the gradient of the difference and the full coefficient covariance matrix; adding two standard errors as if the predictions were independent is generally wrong.

Why Predictions Across Thresholds Are Dependent

At different thresholds \(y\), distribution regression estimates different coefficient vectors. Their sampling errors are nevertheless linked because they use the same observations and nested binary responses. To see the source of the dependence, define \(A(y)=E[p_i(y)\{1-p_i(y)\}X_iX_i']\). Under the stated model and regularity conditions, an influence function for the coefficient estimate at threshold \(y\) is $$ \psi_i(y)=A(y)^{-1}X_i\{Z_i(y)-p_i(y)\}. $$ In particular, \(\sqrt n\{\hat\beta(y)-\beta(y)\}\) is approximated by \(n^{-1/2}\sum_i\psi_i(y)\). This representation gives both the pointwise sandwich variance and the cross-threshold covariance.

For \(y\le y'\), the nesting \(Z_i(y)\le Z_i(y')\) implies, conditional on \(X_i\), that the covariance of the two binary residuals is \(F(y\mid X_i)\{1-F(y'\mid X_i)\}\) when the conditional distribution model is correct. More generally, a robust estimate of the cross-threshold covariance can be calculated from the paired estimated influence functions. For predictions at covariate profiles, apply the corresponding prediction gradients to those coefficient influence functions. The resulting joint covariance accounts for dependence across both thresholds and profiles.

Simultaneous Bands Over a Grid

A simultaneous band aims to cover all predictions in a prespecified collection with probability approximately 95% in repeated samples. For a finite grid of threshold-profile pairs \((y_j,x_j)\), let \(\hat p_j=\hat F(y_j\mid x_j)\) and let \(\hat s_j\) be its pointwise standard error. A simultaneous band has the form \(\hat p_j\pm c_{0.95}\hat s_j\), where \(c_{0.95}\) is chosen from the distribution of the maximum absolute standardized error across the whole grid. Because the maximum is usually larger than a single standard normal absolute value, \(c_{0.95}\) is generally larger than 1.96.

A multiplier bootstrap estimates that critical value while preserving the estimated dependence. First compute estimated influence contributions \(\hat\phi_{ij}\) for each prediction \(j\), scaled so their sum approximates \(\sqrt n(\hat p_j-p_j)\). Draw independent mean-zero, variance-one multipliers \(\xi_i\), such as standard normal variables, and calculate $$ T^*=\max_j\left| \frac{n^{-1/2}\sum_{i=1}^{n}\xi_i\hat\phi_{ij}}{\hat s_j} \right|. $$ Repeat this calculation many times. The empirical 95th percentile of the bootstrap values of \(T^*\) is \(c_{0.95}\). Since each draw uses the same multiplier \(\xi_i\) across all grid points, it retains the estimated cross-threshold and cross-profile dependence. The method requires suitable regularity conditions and a sufficiently accurate estimate of the influence functions; the grid and range of thresholds should be chosen with the inferential target in mind.

Pointwise Versus Simultaneous Interpretation

A finite-grid band is not automatically a uniform band over a continuum. Extending coverage claims to a continuous range requires appropriate uniform approximations and control of behavior between grid points. In practice, it is important to state whether the inferential target is a handful of prespecified predictions, a dense grid, or an entire range of the distribution-regression curve.

Common Pitfall

A common mistake is to calculate a 95% interval at every point on a curve and then describe the collection as a 95% band. Each interval has approximate 95% coverage at its own fixed point, but the probability that all intervals cover simultaneously can be substantially lower. The correction is to calibrate a maximum statistic over the target grid, using the joint dependence of the estimates. A second frequent error is to treat predictions at two profiles as independent when calculating their difference; the correct calculation uses the gradient of the difference and the full covariance matrix.