From a Fitted Model to an Individual Prediction
A regression model can produce a predicted outcome for a person or other case with specified predictor values. But an individual outcome will usually differ from the model’s predicted average at those values. A prediction interval represents that extra uncertainty: it is an interval intended to cover one new outcome at a stated predictor profile, under the model’s assumptions.
A useful applied analysis therefore does more than report an interval from software. It identifies the target case, describes how the model was fitted, checks whether the model’s residual patterns are compatible with its assumptions, carries out the calculation, and states what the result does and does not support. A numerical interval cannot compensate for a poorly specified mean model or a residual pattern that conflicts with the assumed error behavior.
The Model and the Prediction-Interval Formula
For a linear regression with \(p\) fitted coefficients, write the model as \(Y=X\beta+\varepsilon\). Here \(Y\) is the vector of observed outcomes, \(X\) is the design matrix containing the intercept and predictor values, \(\beta\) is the vector of unknown coefficients, and \(\varepsilon\) is the vector of errors. For the usual exact \(t\)-based prediction interval, assume that, conditional on the predictor values, the errors are independent and normally distributed with mean zero and a common variance \(\sigma^2\). The predictor profile for the new case is treated as fixed.
Let \(\hat{y}_0\) be the fitted prediction at profile \(x_0\), \(s\) the residual standard error, and \(h_0=x_0^\mathsf{T}(X^\mathsf{T}X)^{-1}x_0\) the prediction’s leverage. The residual degrees of freedom are \(df=n-p\). A two-sided prediction interval is:
The added 1 in the square root accounts for the random error of the new individual outcome. By comparison, the standard error for estimating the mean response at the same profile is \(s\sqrt{h_0}\). Thus an individual-outcome interval is wider than an interval for the mean response.
A Short Justification for the Extra Uncertainty
At profile \(x_0\), the new outcome is \(Y_{\mathrm{new},0}=x_0^\mathsf{T}\beta+\varepsilon_{\mathrm{new},0}\), while its fitted prediction is \(\hat{y}_0=x_0^\mathsf{T}\hat{\beta}\). Their difference consists of the new individual’s error plus the estimation error in the fitted mean:
Under the stated model, the new error is independent of the training errors used to estimate the coefficients. Its variance is \(\sigma^2\), and the variance of the estimated mean at the profile is \(\sigma^2h_0\). Adding these independent variances gives \(\sigma^2(1+h_0)\). Replacing \(\sigma\) with \(s\) and using the \(t\) critical value with \(n-p\) degrees of freedom gives the interval formula above.
A Practical Prediction Workflow
State the outcome, its units, and the predictor values for the one new case being considered.
Record the fitted prediction, residual standard error, residual degrees of freedom, and leverage. Examine residuals for patterns inconsistent with the model assumptions.
Use the prediction standard error \(s\sqrt{1+h_0}\) and the appropriate \(t\) critical value. Keep full precision during calculation, then round the reported results sensibly.
Interpret the interval for one new outcome at the specified profile, conditional on the model assumptions. Do not describe it as a range for the mean or as a guarantee for a particular case.
Worked Example: Predicting One Quiz Score
Worked Example: Predicting One Quiz Score
Suppose a hypothetical instructor records quiz scores for eight students in two study groups. Let \(x=0\) denote the first group and \(x=1\) the second. The scores are 65, 67, 65.5, and 66.5 for the first group, and 69, 71, 69.5, and 70.5 for the second. The intended prediction is for one new student in the first group. Fit the linear model and calculate a 95% prediction interval.
Fit and residual check. The group means are 66 and 70 points, so the fitted line is \(\hat{y}=66+4x\). The prediction for the first group, \(x_0=0\), is \(\hat{y}_0=66\) points. Within each group, the residuals are \(-1,1,-0.5,0.5\), whose sum is zero and whose squared values sum to \(2.5\). Across both groups, \(SSE=5\). There are \(n=8\) observations and \(p=2\) fitted coefficients, so \(df=8-2=6\). The residual standard error is:
The residuals show no difference in their pattern or spread between the two groups in this small example. However, eight observations provide little information about normality or independence; the diagnostic check cannot establish those assumptions.
Calculate the leverage and interval. There are four observations at each of \(x=0\) and \(x=1\), so \(\bar{x}=0.5\) and \(S_{xx}=\sum(x_i-\bar{x})^2=2\). For a simple regression with an intercept, \(h_0=1/n+(x_0-\bar{x})^2/S_{xx}\). At \(x_0=0\), \(h_0=1/8+0.25/2=0.25\). The 95% critical value for \(df=6\) is approximately 2.447. Therefore:
Under the stated assumptions, the interval procedure has 95% repeated-sampling coverage for the score of one new student in the first group. It does not mean there is a 95% probability that this fixed interval contains the student’s score, nor does it describe uncertainty about the group’s mean score.
For comparison, the estimated mean-response standard error at this profile is \(s\sqrt{h_0}=\sqrt{5/6}\sqrt{0.25}\approx0.4564\) points. Its 95% interval would be narrower because it concerns the group mean, not a new student’s score.
Worked Example: A Prediction From Regression Output
Worked Example: A Prediction From Regression Output
In a hypothetical dataset, a linear model predicts recovery time in days from a numerical baseline measurement. For a new case with baseline value \(x_0=8\), the fitted prediction is 18.4 days. The model was fitted to \(n=29\) cases with \(p=2\) coefficients; software reports \(s=3.2\) days and \(h_0=0.12\). Residual-versus-fitted and residual-versus-predictor plots show no pronounced curvature or change in spread, and a normal quantile plot has no strong departure from a straight-line pattern. These checks are compatible with, but do not prove, the working assumptions.
The degrees of freedom are \(df=29-2=27\). For a 95% interval, \(t^*\approx2.052\). The prediction standard error is:
Using the unrounded standard error and critical value, the margin is approximately \(2.0518(3.3866)=6.949\) days. Thus the interval is:
The interpretation is that the fitted procedure gives a 95% prediction interval from about 11.45 to 25.35 days for one new case with baseline value 8, conditional on the stated model assumptions. The broad interval reflects individual-outcome variability as well as uncertainty in the fitted mean. It is not a range guaranteed to contain every future recovery time.
Worked Example: A Diagnostic Pattern Changes the Conclusion
Worked Example: A Diagnostic Pattern Changes the Conclusion
A hypothetical dataset contains 20 measurements of a continuous outcome \(Y\), modeled using one numerical predictor \(x\). The fitted simple regression is \(\hat{y}=40+5x\), with \(s=4\) outcome units. A software report gives \(h_0=0.15\) for a proposed new case at \(x_0=6\). Residual plots show a clear curved pattern: residuals tend to be positive at low and high fitted values and negative in the middle. Calculate the nominal 95% interval, then decide whether to present it as a reliable prediction interval.
The fitted value is \(40+5(6)=70\) outcome units. With \(n=20\) and \(p=2\), \(df=18\), and the two-sided 95% critical value is approximately 2.101. The nominal standard error and interval are:
The arithmetic is consistent, but the residual curvature suggests that the conditional mean may not be linear in \(x\). If so, the fitted prediction can be systematically wrong at some predictor values, and the nominal interval’s claimed coverage is not justified by the model. The analyst should not present this interval as a dependable prediction without addressing the misspecification—for example, by considering a scientifically appropriate mean model and checking its residuals. A formula calculated correctly is not enough when a key model assumption is contradicted by the data.
Communicating the Result Responsibly
A useful report identifies the profile and outcome explicitly: “For one new case with baseline value 8, the model predicts a recovery time of 18.4 days; the 95% prediction interval is 11.45 to 25.35 days, conditional on the fitted linear model and its error assumptions.” This wording distinguishes the individual target, supplies units, and avoids treating the interval as an unconditional guarantee.
A prediction interval applies to the stated profile and model. It may not describe cases from a different setting, cases whose predictors are measured differently, or a population with a changed outcome process. Even when residual checks look acceptable, they provide evidence rather than proof. A careful conclusion reports the interval together with relevant diagnostic concerns and limits its interpretation accordingly.
Common Mistake: Reporting an Interval Without Naming Its Target
Wrong reasoning: “The 95% interval is from 11.45 to 25.35 days, so the average recovery time for cases like this is probably in that range.” This confuses an interval for one new outcome with an interval for a mean response. Another mistake is to treat 95% as the probability that this particular fixed interval contains the future outcome.
Correction: State that the interval targets one new outcome at the specified predictor profile. Under the model assumptions, the interval-construction procedure covers such outcomes in 95% of repeated samples. The particular future outcome may fall outside the interval, and the interval is not an estimate of the mean response.
Check Your Understanding
Answer each question using the model assumptions and interval target carefully.
- What is the difference between the target of a prediction interval and the target of an interval for the mean response?
- Why does the prediction standard error contain \(1+h_0\), rather than \(h_0\) alone?
- A fitted model has \(n=25\) observations and \(p=3\) coefficients. What are its residual degrees of freedom?
- A residual plot shows a clear increase in residual spread as fitted values increase. Why should that pattern affect how you communicate a nominal prediction interval?
- Write one sentence interpreting a 95% prediction interval for one new patient’s waiting time at a specified predictor profile. Include units and a model-assumption qualification.