Why Least Absolute Deviations Targets a Conditional Median

Why the Target of Absolute Error Matters

Least-absolute-deviations (LAD) regression chooses coefficients to minimize the sum of absolute residuals. That choice is not just a different computational way to fit a line: it changes the quantity the fitted line is designed to describe. Squared-error fitting is associated with a conditional mean; absolute-error fitting is associated with a conditional median. The reason comes from the shape of the loss function and can be derived directly.

This distinction matters when the conditional distribution of the response is skewed or has unusual values. The mean and median can then differ, so two fitting criteria can give different answers without either calculation being erroneous. The key is to interpret the fitted quantity correctly.

Expected Absolute Error at a Fixed Predictor Value

Fix \(X=x\), and consider predicting \(Y\) with a single number \(a\). The prediction incurs absolute error \(|Y-a|\). Its conditional expected loss is \(L_x(a)=E(|Y-a|\mid X=x)\), assumed finite for the candidate values under consideration. The question is: which \(a\) minimizes this expectation?

Let \(F_x(a)=P(Y\leq a\mid X=x)\) be the conditional distribution function, and let \(F_x(a-)=P(Y<a\mid X=x)\). The one-sided derivatives of the expected absolute loss are

These expressions also apply when the distribution has point masses. Intuitively, moving the prediction a small amount to the right reduces the error for responses above the prediction and increases it for responses below. The balance depends on how much probability lies on either side, not on how far away the responses are.

Deriving the Median Condition

For a convex function, a point minimizes the function precisely when its left derivative is at most zero and its right derivative is at least zero. The expected absolute loss is convex in \(a\), since \(|y-a|\) is convex in \(a\) for every fixed \(y\), and an expectation preserves convexity. Applying this minimum condition gives

The right-hand condition is exactly the condition that \(a\) be a conditional median. Thus every conditional median minimizes expected absolute error, and every minimizer of that expected loss is a conditional median.

For a continuous conditional distribution, there is no probability mass at \(a\), so \(F_x(a-)=F_x(a)\). In that case, an interior minimum satisfies \(F_x(a)=0.5\). With a discrete distribution, the distribution function can jump over 0.5, and any value at the jump satisfying the median condition minimizes the loss. A flat interval of minimizers is also possible.

Worked Example: A Discrete Conditional Distribution

From a Conditional Target to LAD Coefficients

In a regression, the prediction is restricted to have a particular form. For a predictor vector \(X\) and coefficient vector \(\beta\), write the prediction as \(X^\mathsf{T}\beta\). The population absolute-error criterion is

If predictions could be chosen separately at every predictor value, the result just derived says to choose a conditional median at each value. Regression imposes a restriction: the predictions must all come from one coefficient vector. If the conditional median has the linear form

then \(\beta_0\) gives a linear model for that conditional median. Under this correct specification, and when the predictors identify the coefficients, minimizing expected absolute error targets \(\beta_0\). In a sample, LAD regression minimizes the corresponding sum of absolute residuals,

The sample criterion is an estimate of the population criterion. It does not guarantee that the fitted coefficients equal the population coefficients in a particular sample; sampling variation remains. Nor does the median-target result require conditional errors to be symmetric around zero. What matters for the interpretation is that the conditional median is represented by the chosen regression form.

Worked Example: A Skewed Conditional Distribution

What the Linear Assumption Does—and Does Not—Say

A linear LAD fit is most directly interpreted as a model for the conditional median when the conditional median is in fact linear in the predictors. If that form is wrong, the LAD coefficients still minimize the chosen absolute-error criterion within the restricted family of linear predictions. They should then be understood as the best-fitting linear approximation under that criterion, not automatically as the coefficients of the true conditional median at every predictor value.

In an ordinary linear model with an intercept, a fitted LAD solution has residuals balanced around zero in the sense that the optimization accounts for their signs and magnitudes through absolute loss. This is not a claim that exactly half of the residuals are positive and half negative in every sample, especially when there are ties or multiple optimal solutions. The population interpretation is about conditional medians, not a mechanical count in a particular data set.

Common Mistake: Calling the LAD Fit a Mean Model

A common mistake is to say, “The LAD line predicts the average response because it is a regression line.” That reasoning confuses the regression format with the target of its loss function. The correction is to identify the criterion first: squared error targets a conditional mean under the corresponding unrestricted pointwise argument, whereas absolute error targets a conditional median. Then check whether the regression’s linear form is a suitable model for the conditional quantity being discussed.

A second mistake is to assume that skewness or asymmetry invalidates the median result. It does not. The derivation uses the conditional distribution function and does not require symmetry. In fact, asymmetry is a reason the conditional mean and median may differ, making it especially important to name the target accurately.