Distribution Regression For Conditional CDFs

Why Model a Conditional Distribution?

A regression for the conditional mean describes one feature of an outcome distribution. It may not show how covariates relate to the probability of falling below a particular value. For example, a researcher studying wages may want to compare the probability of earning at most $50,000 for people with different education levels, conditional on other characteristics. Distribution regression addresses such questions by modeling the conditional cumulative distribution function (CDF) directly.

The basic construction is simple: choose an outcome threshold, mark each observation as below or above it, and fit a binary-response model to that indicator. Repeat this procedure across thresholds. Each fit describes the conditional probability of being at or below its own threshold; the coefficients are therefore allowed to change with the threshold.

Fitting a Binary-Response Model at Each Threshold

Suppose the data consist of observations \((Y_i,X_i)\), for \(i=1,\ldots,n\), with the outcome and covariates observed for each unit. At a chosen threshold \(y\), form \(D_i(y)=1\{Y_i\leq y\}\). The model asserts that, conditional on \(X_i\), this indicator has success probability \(G(X_i^\top\beta(y))\). A logit specification uses \(G(t)=\Lambda(t)=e^t/(1+e^t)\); a probit specification uses the standard normal CDF.

The coefficient is estimated separately at each threshold by maximizing the binary-response likelihood. For example, under logit, write \(p_i(y)=\Lambda(X_i^\top\beta(y))\). The log-likelihood at threshold \(y\) is

The maximizer, when it exists, is the threshold-specific maximum-likelihood estimate. The same observations and covariates can be used at every threshold, but the binary response changes when the threshold changes. Consequently, \(\widehat{\beta}(y)\) is a function of the threshold, not one common coefficient vector for the entire outcome distribution.

Why the Logit Score Has This Form

For logit, one observation contributes \(D_i(y)\eta_i-\log(1+e^{\eta_i})\) to the log-likelihood, where \(\eta_i=X_i^\top\beta(y)\). Since \(\partial\eta_i/\partial\beta=X_i\) and \(\partial\log(1+e^{\eta_i})/\partial\eta_i=\Lambda(\eta_i)=p_i(y)\), differentiating and summing gives

Thus, at an interior maximum, the fitted conditional probabilities balance the observed threshold indicators in the score equations. This is a justification for estimating a binary-response model at each threshold: its response is precisely the event whose conditional probability defines the CDF. It does not imply that a fitted probability for every covariate value equals the corresponding observed proportion; the model imposes a structured relationship across covariate values.

Interpreting Threshold-Specific Coefficients

A coefficient in a logit distribution regression is a change in the log odds of the event \(Y\leq y\) for a one-unit change in the associated covariate, holding the other included covariates fixed. For a continuous covariate \(x_j\), the coefficient \(\beta_j(y)\) is on the log-odds scale. Its probability-scale effect depends on both the threshold and the covariate values:

where \(g\) is the density corresponding to the link CDF \(G\). The derivative is a local change in the conditional probability per unit of \(x_j\), evaluated at a particular \(x\) and threshold \(y\). For a binary covariate, a finite difference between its two values is usually the clearer probability-scale interpretation.

The sign must be read in relation to the event being modeled. A positive coefficient raises the modeled probability of being at or below the chosen threshold, conditional on the other covariates. A negative coefficient lowers that probability. Neither sign alone describes the covariate’s relationship with every part of the outcome distribution: coefficients can have different signs at different thresholds.

A Small-Sample Illustration of Estimation

When a model includes only an intercept and a binary group indicator, its fitted probability in each group is the group’s observed proportion of threshold indicators equal to one. This special case makes the threshold-specific fitting step visible. In richer models, maximum likelihood instead estimates a shared parametric relationship across covariate patterns.

Assumptions and Limits of the Fitted Curve

Distribution regression describes the conditional CDF only to the extent that the binary-response specifications are suitable at the thresholds of interest. A typical analysis assumes the observed outcomes and covariates are correctly recorded, the observations are independent under the sampling model, and the chosen link and covariate specification adequately describe \(P(Y\leq y\mid X=x)\) for each fitted threshold. If observations are clustered or sampled under a complex design, that structure cannot simply be ignored when conducting inference.

The link guarantees that each fitted value lies between zero and one. However, estimating thresholds separately does not automatically guarantee that fitted probabilities are nondecreasing in \(y\). A CDF must be nondecreasing: if \(y_1<y_2\), then \(F_{Y\mid X}(y_1\mid x)\leq F_{Y\mid X}(y_2\mid x)\). Separate estimates can violate this ordering in finite samples or under misspecification. Such a fitted sequence is not a valid CDF at that covariate value without further attention.

The indicator uses \(Y\leq y\), including outcomes equal to the threshold. This definition works for continuous and discrete outcomes; it does not require a density or rule out ties. But if outcomes are censored, missing, or observed only for a selected subset, the indicator may not be observed as defined for every unit. Applying the ordinary binary likelihood to only the observed indicators can then fail to represent the target conditional CDF.

Why Distribution Regression Matters

A conditional CDF provides a direct way to ask threshold-focused questions across the outcome range. Fitting at several thresholds can reveal that a covariate is associated with a higher probability of low outcomes at one part of the distribution and a different pattern elsewhere. This flexibility is the central contrast with imposing one common coefficient vector on all thresholds. It also means that the interpretation must remain threshold-specific: each coefficient pertains to the event \(Y\leq y\) at the particular \(y\) used in that fit.

A frequent mistake is to interpret \(\beta_j(y)\) as a change in the outcome measured in its original units, or as a uniform shift of the entire distribution. In a logit model, it is a change in log odds per covariate unit for being at or below \(y\); the probability change depends on the index, and both the coefficient and its probability-scale effect can vary with \(y\). A second mistake is to treat any collection of fitted probabilities as a valid CDF merely because each lies between zero and one. The fitted values must also respect threshold ordering to form a CDF.