What Makes a Question a Prediction Question?
A regression question is a prediction question when the goal is to use information about a case whose outcome is not yet known to produce a forecast of that outcome. The case might be a person, a trip, a day, or another observational unit. “How much fuel will this particular trip use?” is a prediction question if the trip’s outcome has not yet been observed and the available trip information is used to forecast it.
A useful prediction task is specific about three things: the outcome to predict, the predictor information available at prediction time, and the cases for which predictions will be made. Without these details, a fitted regression rule may be mathematically defined but not answer the practical question being asked.
Here \(Y\) denotes the outcome variable and \(X\) denotes the predictor information, which may consist of one or several predictor variables. For an individual case, \(x\) is the information recorded for that case. A prediction task does not, by itself, claim that changing a predictor would cause the outcome to change. Its aim is to forecast outcomes, not to establish a causal effect.
Individual Outcomes and Average Relationships
Two targets can use the same outcome and predictors but answer different questions. A descriptive regression target such as \(E(Y\mid X=x)\) concerns the average outcome among cases with predictor information \(x\). An individual prediction concerns the outcome \(Y_*\) for a particular case with information \(x_*\). The first describes an average relationship; the second is intended to give a useful forecast for one case.
A common point prediction for an individual is the estimated conditional mean at that individual’s predictor values. These are related, but they are not identical targets. The conditional mean is an average, so it need not equal the outcome for any particular individual. A point prediction can still be used as a forecast for that person; it should not be interpreted as a promise that the person’s actual outcome will equal the forecast.
Worked Example: Forecasting Fuel Use for One Trip
Suppose a hypothetical transport company uses a simple prediction rule for fuel consumed on a trip. Let \(d\) be trip distance in kilometres, and let the rule return the estimated average fuel use, in litres, for trips of that distance:
A driver is about to make a 120 km trip. The intended outcome is fuel used by this one trip, in litres; the available predictor is its planned distance, in kilometres; and the future case is this particular trip. Applying the given rule:
The point prediction is 9.9 litres for this trip. The rule’s value at 120 km also represents its estimated average fuel use for trips with that distance. Those interpretations are connected, but they answer different questions: one is a forecast for the driver’s trip, and the other describes an average across comparable trips. The actual trip could use more or less than 9.9 litres.
This calculation assumes the stated rule is the one to use and that planned distance is measured in the same way as distance in the data used to form the rule. The calculation does not establish that distance alone captures every source of variation in fuel use; it only applies the supplied rule to the available predictor value.
Choosing the Outcome and Available Information
The outcome must be defined in a way that matches the decision or question. “Predict delivery performance” is too vague to guide an analysis. A more useful specification might be “predict the number of minutes a package arrives after its scheduled time.” It states what is recorded and gives the outcome a unit. The time at which the outcome is measured also matters: a forecast of delay at the scheduled arrival time is different from a forecast of delay at some later follow-up.
Predictor information must be available when the forecast is needed. A predictor can be strongly associated with the outcome and still be unusable for a particular prediction task if its value is only known afterward. For example, actual arrival time cannot be used to forecast a package’s delay before delivery. Using it would make the calculation depend on information that is unavailable at the moment of prediction.
The case must also be described at the level for which an outcome will be predicted. If a model produces one forecast per trip, then a future case is a trip, not a driver in the abstract. If a model forecasts a daily total, a future case is a day. Naming the unit prevents a mismatch between the data used to define the outcome and the unit receiving the prediction.
Worked Example: Predicting a Delivery Delay With Timely Predictors
Consider a hypothetical rule for predicting how many minutes late a package will arrive. Let \(d\) be the delivery route’s distance in kilometres, recorded before dispatch, and let \(L=1\) if dispatch is scheduled after 5 p.m. and \(L=0\) otherwise. Suppose the rule is:
A package is assigned to a 50 km route scheduled to depart after 5 p.m. The case is this package’s delivery; the outcome is lateness in minutes; and both predictors are known before dispatch. Substituting the values:
The forecast is 10 minutes late for this package. It is a point prediction, not a guarantee. Also, actual arrival time is not among the predictors: it would only become known after the event the rule is meant to forecast. Adding actual arrival time would change the task into a calculation made after delivery rather than a useful pre-dispatch prediction.
The example assumes the given rule is appropriate for these predictor definitions and that distance is measured in kilometres. It also assumes the scheduled departure indicator is recorded consistently as defined. If, in practice, either predictor were unavailable or recorded differently at dispatch, this calculation would not implement the specified task.
Why the Conditional Mean Is a Natural Point Forecast
When the goal is to choose one numerical forecast and errors are judged by squared error, the conditional mean has a precise justification. Fix predictor information \(X=x\). Assume the outcome has a finite conditional mean and variance, and write \(m(x)=E(Y\mid X=x)\). Among all fixed numerical forecasts \(a\) at this \(x\), the conditional mean minimizes the expected squared difference between the outcome and the forecast.
To see why, add and subtract \(m(x)\) inside the squared error. The cross-product has conditional expectation zero because \(E(Y-m(x)\mid X=x)=0\):
The first term does not depend on the chosen forecast \(a\), and the second term is nonnegative. It equals zero when \(a=m(x)\), so no other fixed forecast has a smaller conditional expected squared error. This establishes why a regression estimate of the conditional mean is often used as a point prediction under squared-error scoring. It does not say that every individual outcome equals that mean, or that squared error is the right criterion for every practical decision.
A point prediction deliberately compresses uncertainty into one number. The spread of possible individual outcomes around that number remains important, especially when decisions have consequences. Describing or quantifying that spread is a separate task from identifying what outcome and cases the prediction is intended to address.
Worked Example: Separating a Student’s Forecast From a Group Average
Suppose a hypothetical model predicts a quiz score from the number of practice sets completed. Let \(x\) be the number of sets and let the model’s estimated average score, in points out of 100, be:
A student who completed four practice sets is about to take the quiz. If the question is “What score should be forecast for this student?”, substitute \(x=4\):
The individual student’s point prediction is 70 points. If instead the question is “What average score does the model associate with four practice sets?”, the answer is also 70 points, but it describes the estimated average for cases with \(x=4\), not a single student’s outcome. The arithmetic is the same because the point forecast uses the estimated conditional mean; the target and interpretation differ.
This use of the rule assumes the student’s practice-set count is measured as defined and that four is a predictor value for which the rule is intended to be used. The forecast does not imply that completing four practice sets causes a score of 70, nor that the student’s actual score must be 70.
Common Mistake: Treating the Average as an Exact Individual Outcome
A tempting but incorrect argument is: “The regression gives 70 points at four practice sets, so every student who completes four sets will score 70.” The conclusion confuses an estimated average with each individual outcome. Even when two cases have the same recorded predictors, their outcomes can differ because those predictors do not capture every relevant difference or source of variation.
The correction is to state the target precisely. The model gives an estimated average score of 70 points for cases with four practice sets and can use 70 points as a squared-error point forecast for one such student. It does not claim that the student’s exact score is known. The reverse mistake is also possible: dismissing a conditional-mean prediction for an individual because the mean is “only about a group.” It can still be a sensible individual point forecast under the stated loss criterion, while remaining uncertain as a forecast of that person’s outcome.
Before using a regression prediction, check that the outcome, units, case, and predictor information match the real question. These checks make clear whether the result is intended for one unobserved outcome or for describing an average relationship, and whether the information needed to calculate it would actually be available.
Check Your Understanding
For each question, identify the prediction target or explain the distinction being tested.
- A model forecasts repair cost in dollars for a newly received appliance using its age and model type. Name the outcome, predictors, and future case.
- A rule gives an estimated average blood pressure for people with specified predictor values. Explain why that value need not equal the blood pressure of a particular person with those values.
- A clinic wants to predict tomorrow’s appointment duration but proposes using the appointment’s actual end time as a predictor. What is the problem?
- Under squared-error loss, why does \(E(Y\mid X=x)\) minimize expected squared error among fixed forecasts at \(X=x\)?
- A regression rule returns 42 minutes for one future commute. Give one correct individual-prediction interpretation and one interpretation that would be too strong.