Point Predictions and the Unobserved Error

What Does a Regression Point Prediction Predict?

A regression model can give a predicted outcome for a specified set of predictor values. The key question is what that number represents. If the goal is to predict one new person’s outcome, it is tempting to treat the fitted value as the outcome that person will have. But the fitted value targets the conditional mean: the average outcome among cases with those predictor values, according to the model. A particular new outcome can differ from that mean.

There are two distinct reasons for uncertainty. The regression mean is estimated from data, so another sample could produce a different estimated mean. In addition, even if the conditional mean were known exactly, a new individual outcome would still vary around it. This tutorial focuses on why the fitted mean is a point prediction rather than a complete description of an individual future outcome.

The Model Has a Mean Part and an Error Part

A useful way to describe the regression setting is to write each outcome as its conditional mean plus an error:

Here \(x_i\) denotes the predictor values for case \(i\), \(\mu(x_i)\) is the model’s conditional mean at those values, and \(\varepsilon_i\) is the case’s deviation from that mean. For a standard regression model, the errors have conditional mean zero: \(E(\varepsilon_i\mid x_i)=0\). This means the mean function describes the center of the outcome distribution at each predictor profile; it does not say every outcome equals that center.

For the usual regression interpretation, observations are independent, the mean function is correctly specified for the predictor profiles of interest, and the error variation is stable across those profiles. A normal-error assumption is not needed simply to calculate or interpret a point prediction. These assumptions support the model’s description of the conditional mean and its errors; they do not make an individual error vanish.

In practice, \(\mu(x_0)\) is unknown, so the fitted model estimates it with \(\hat{\mu}(x_0)\). Because the data used to fit the model could have been different, \(\hat{\mu}(x_0)\) is itself uncertain. But a future case also has its own \(\varepsilon_{\text{new}}\), which is not known when the point prediction is made. These are separate ideas: uncertainty about where the mean is, and variation of individual outcomes around that mean.

Why the Conditional Mean Is a Natural Point Prediction

The conditional mean is the best single-number prediction when prediction errors are judged by squared error. This result explains why regression commonly reports a fitted mean as its point prediction. It does not imply that this number will be close to every individual outcome.

To see this, write \(Y-a=(Y-\mu(x_0))+(\mu(x_0)-a)\), square both sides, and take the conditional expectation. The cross-term has expectation zero because \(E[Y-\mu(x_0)\mid X=x_0]=0\). The first term becomes the conditional variance, leaving the squared distance between the proposed prediction and the mean. The variance does not depend on \(a\), while the squared distance is minimized at \(a=\mu(x_0)\). Thus the conditional mean minimizes expected squared error. Since the true mean is generally unknown, the fitted mean is used as its estimate.

The variance term in this decomposition remains even when \(a\) equals the true mean. In other words, predicting the mean perfectly would still leave individual outcomes dispersed around it. Estimating the mean introduces another source of uncertainty; it does not replace or remove the new case’s outcome variation.

Mean-Estimate Uncertainty and New-Outcome Variation

The fitted mean changes from sample to sample. A standard error for a fitted mean describes the sampling uncertainty in \(\hat{\mu}(x_0)\): it summarizes how much the estimated mean would typically vary across repeated samples under the model. It is not the spread of individual outcomes at \(x_0\).

The error \(\varepsilon_{\text{new}}\), by contrast, belongs to the new case. It represents the difference between that case’s outcome and the conditional mean at its predictor profile. Before the outcome is observed, this error is unobserved. A model may have a small standard error for the fitted mean and still have substantial individual variation, or the reverse. The two quantities answer different questions and should not be described interchangeably.

Observed Residuals Are Not the Future Error

For a case already in the data, its fitted value is \(\hat{\mu}(x_i)\), and its residual is \(e_i=y_i-\hat{\mu}(x_i)\). A residual is calculated after observing that case’s outcome. It records how far the observed outcome is from the fitted value. The future error \(\varepsilon_{\text{new}}\) is different: it is the deviation of an as-yet-unobserved outcome from its conditional mean.

Residuals can help assess whether the model’s assumptions are plausible and can provide information about the scale of unexplained variation. But a particular training-case residual is not the new case’s error. Knowing the residuals from earlier cases does not reveal exactly how far the next outcome will lie from its mean.

Why the Distinction Matters

A point prediction is useful because it gives a concise, interpretable summary of the model’s estimated center at a predictor profile. But reporting it as though it were a guaranteed individual outcome overstates what regression has established. For questions about one future case, a complete account must address both the uncertainty in the estimated mean and the outcome’s variation around that mean. The construction and calibration of prediction intervals are separate tasks; the point prediction alone is not an interval.

A common mistake is to say, “The fitted value is 70, so the next student will score 70,” or to add a residual from an earlier observation and call the result the new student’s prediction. The correction is to say, “The fitted value estimates the conditional mean at this profile; an individual outcome may differ because its error is unobserved.” This wording is both precise and appropriately cautious.