Treating New Predictor Values as Fixed

What Does a Prediction Interval Condition On?

A regression prediction interval answers a question about a future outcome, but its meaning depends on what is treated as known. In the standard calculation, the new case’s predictor values are specified and treated as fixed. The interval allows for uncertainty in the fitted regression and for the future outcome’s error, but not for uncertainty about which predictor value the case actually has.

This distinction matters when a predictor is measured with error, or when a future predictor value is itself uncertain. For example, a prediction at a stated medication dose differs from a prediction when the delivered dose may vary around that stated value. In the second question, variation in dose can create additional variation in the predicted outcome.

Definition: A fixed-predictor prediction interval is an interval for a future response, calculated conditional on specified predictor values that are treated as known. In a regression with predictor vector $x_0$, the interval concerns a future response $Y_{\mathrm{new}}$ for a case whose predictor vector is $x_0$.

For a linear regression with $p$ fitted coefficients, let $\hat{y}_0=x_0^\mathsf{T}\hat{\beta}$ be the fitted response at $x_0$, let $s$ be the residual standard error, and let $h_0=x_0^\mathsf{T}(X^\mathsf{T}X)^{-1}x_0$. Under the usual normal, constant-variance, independent-error model, the standard two-sided prediction interval is

\hat{y}_0 \pm t_{1-\alpha/2,\,n-p}\,s\sqrt{1+h_0}.

Here $n$ is the number of training cases, $df=n-p$, and $t_{1-\alpha/2,\,n-p}$ is the corresponding $t$ critical value. The $1$ under the square root accounts for the new case’s outcome error; $h_0$ accounts for estimation uncertainty in the fitted mean at $x_0$. Neither term represents uncertainty that the future predictor value differs from $x_0$.

Worked Example: A Prediction at a Specified Predictor Value

Suppose a hypothetical regression based on $n=20$ cases has $p=2$ coefficients, residual standard error $s=4$ minutes, and leverage $h_0=0.05$ at the specified predictor value. The fitted completion time there is 52 minutes. Calculate a 95% prediction interval, treating the new case’s predictor value as known.

The model assumptions for using the standard interval are that the response is linear in the predictors, errors are independent and normally distributed with common variance, and the fitted regression is appropriate. Here we also assume the specified predictor value is measured accurately and is within a range where the model is suitable. With $df=n-p=18$, the 95% critical value is approximately $t_{0.975,18}=2.101$.

The prediction standard error is

SE_{\mathrm{pred}}=s\sqrt{1+h_0} =4\sqrt{1+0.05} =4\sqrt{1.05} \approx 4.099\text{ minutes}.

Thus the margin of error is $2.101(4.099)\approx 8.61$ minutes, and the interval is

52\pm 8.61=(43.39,\ 60.61)\text{ minutes}.

Under the stated model, this is a 95% prediction interval for a future completion time at the specified predictor value. It does not account for that case having a different, uncertain predictor value.

When the Future Predictor Is Uncertain

To see the change in the prediction target, consider simple linear regression. Write the outcome for a future case as

Y_{\mathrm{new}}=\beta_0+\beta_1X_{\mathrm{new}}+\varepsilon_{\mathrm{new}}, \qquad \varepsilon_{\mathrm{new}}\sim N(0,\sigma^2).

A fixed-predictor interval sets $X_{\mathrm{new}}=x_0$. If instead the case’s actual predictor is $X_{\mathrm{new}}=x_0+U$, where $U$ has mean zero and variance $\tau^2$, the target is no longer the outcome at exactly $x_0$. It is the outcome for a case whose predictor varies around $x_0$. Assume for this calculation that $U$ is independent of the outcome error and of the training data.

With the coefficients known, the expected outcome remains $\beta_0+\beta_1x_0$, but the variance now includes the effect of predictor variation. The extra component is the slope squared times the predictor variance.

Result: If $X_{\mathrm{new}}=x_0+U$, with $E(U)=0$, $\operatorname{Var}(U)=\tau^2$, and $U$ independent of $\varepsilon_{\mathrm{new}}$, then $$ E(Y_{\mathrm{new}})=\beta_0+\beta_1x_0,\qquad \operatorname{Var}(Y_{\mathrm{new}})=\sigma^2+\beta_1^2\tau^2. $$ This result concerns uncertainty in the future predictor in addition to the outcome error; it does not by itself include uncertainty from estimating the regression coefficients.

In practice, the coefficients and error variance are estimated. A useful first-order approximation is to add the estimated predictor-uncertainty component to the usual fixed-predictor prediction variance:

\widehat{\operatorname{Var}}(Y_{\mathrm{new}}) \approx s^2(1+h_0)+\hat{\beta}_1^2\tau^2.

This is a plug-in approximation, not the standard exact $t$-based prediction interval. It treats $\tau^2$ as known and uses the estimated slope to quantify how strongly predictor variation changes the outcome. Its accuracy depends on the assumptions and on whether uncertainty in the slope and predictor distribution is small enough to be neglected. A full analysis may need to account for those uncertainties as well.

Why the Extra Variance Appears

The additional term follows by writing the outcome as the sum of its value at $x_0$, the change caused by $U$, and the outcome error:

Y_{\mathrm{new}} =\beta_0+\beta_1x_0+\beta_1U+\varepsilon_{\mathrm{new}}.

Since $U$ and $\varepsilon_{\mathrm{new}}$ are independent, their covariance is zero. Therefore the variance of their weighted sum is the sum of their variances:

\begin{aligned} \operatorname{Var}(Y_{\mathrm{new}}) &=\operatorname{Var}(\beta_1U+\varepsilon_{\mathrm{new}})\\ &=\beta_1^2\operatorname{Var}(U)+\operatorname{Var}(\varepsilon_{\mathrm{new}})\\ &=\beta_1^2\tau^2+\sigma^2. \end{aligned}

This derivation isolates the contribution of predictor uncertainty when the regression coefficients are known. When they are estimated from data, the fixed-predictor interval already includes uncertainty in the fitted mean at $x_0$, through $h_0$. Adding a predictor-uncertainty term is a practical approximation for the changed target, not a claim that every source of uncertainty has been handled exactly.

Worked Example: Allowing a Future Predictor to Vary

Suppose a hypothetical regression predicts a test score from study time. At $x_0=8$ hours, the fitted score is 70 points, the estimated slope is $\hat{\beta}_1=2.5$ points per hour, the residual standard error is $s=3$ points, and $h_0=0.04$. For a future student, suppose actual study time varies around 8 hours with standard deviation $\tau=1.2$ hours. Use the first-order approximation with $df=18$ and a 95% critical value of 2.101.

First calculate the fixed-predictor prediction variance:

s^2(1+h_0)=3^2(1.04)=9.36\text{ points}^2.

The predictor-uncertainty variance is

\hat{\beta}_1^2\tau^2=(2.5)^2(1.2)^2 =6.25(1.44)=9.00\text{ points}^2.

Adding the components gives an approximate prediction variance of $18.36$ points squared, so

SE_{\mathrm{pred,approx}}=\sqrt{18.36}\approx 4.285\text{ points}.

The approximate margin is $2.101(4.285)\approx 9.00$ points. Centering at the fitted value of 70 gives

70\pm 9.00=(61.00,\ 79.00)\text{ points}.

The ordinary fixed-predictor standard error would be $3\sqrt{1.04}\approx3.059$ points, giving a narrower interval. The wider approximate interval reflects variation in study time as well as response noise and estimated-mean uncertainty. It is approximate because it uses estimated quantities in an added variance component.

Measurement Uncertainty Can Change the Center Too

Not every uncertain predictor is best described as a known value plus mean-zero variation. Sometimes a future predictor is measured imperfectly, and the goal is to predict the outcome given the recorded measurement. If the recorded value is $W$ and the underlying predictor is $X$, the relevant target is then the distribution of $Y$ given $W$. The center may shift if the best estimate of $X$ given $W$ is not equal to $W$.

For a simple illustration, suppose a calibration process gives $X\mid W=10$ a mean of 9.6 and variance 0.64. Assume a regression fitted using accurately known predictor values has $\beta_0=5$, $\beta_1=1.5$, and outcome-error variance $\sigma^2=4$. Assume the outcome error is independent of $X$ given $W$, and treat the coefficients and calibration quantities as known for this calculation. Then the predicted outcome conditional on the recorded value is centered at $5+1.5(9.6)$, not at $5+1.5(10)$.

Worked Example: Predicting Given an Imperfect Predictor Measurement

Using the calibration information above, find an approximate 95% prediction interval for the future outcome conditional on the recorded predictor value $W=10$. The predictor’s conditional distribution and the regression coefficients are treated as known, and normality is assumed for the resulting prediction distribution.

The conditional mean is

E(Y\mid W=10)=5+1.5(9.6)=19.4.

The conditional variance includes both outcome-error variance and variation in the underlying predictor:

\operatorname{Var}(Y\mid W=10) =4+(1.5)^2(0.64) =4+1.44 =5.44.

The resulting standard deviation is $\sqrt{5.44}\approx2.332$. Under the stated normal model, an approximate 95% interval is

19.4\pm1.96(2.332) \approx19.4\pm4.57 =(14.83,\ 23.97).

This interval is conditional on the recorded measurement $W=10$, not on the underlying predictor being exactly 10. It uses a calibration distribution to represent what is known about that underlying value. In a real analysis, uncertainty in the calibration and fitted regression would also need consideration.

Be Clear About the Prediction Target

Before calculating an interval, state what is being predicted and what information is being conditioned on. These questions distinguish three situations that can otherwise be confused:

Outcome at a specified, accurately known predictor value: use the standard fixed-predictor prediction interval.
Outcome for a future case whose predictor varies around a nominal value: predictor variation contributes additional outcome variation, as in the first-order calculation above.
Outcome given a noisy measurement of an underlying predictor: use information about the underlying predictor conditional on that measurement; both the prediction center and spread can differ from a calculation that substitutes the measurement directly.

These distinctions concern the prediction question, not merely the arithmetic. An interval for outcomes at exactly 8 hours of study and an interval for outcomes among students whose study times vary around 8 hours have different targets. Neither is automatically the right one: the appropriate target depends on how the predictor is set, generated, or measured in the application.

Common Mistake

Wrong reasoning: “A standard prediction interval already includes the uncertainty of a future case, so it must include uncertainty in that case’s predictor value.”

Correction: The standard interval includes uncertainty in the future response error and in the estimated regression mean, conditional on the stated predictor values. If the actual future predictor differs from those values, that additional uncertainty is not included automatically. State whether the predictor is fixed and known, varies around a target, or is measured with error; then use a prediction target and calculation that match that situation.

There is a further caution when the training predictors were also measured with error. The formulas here assume the regression was fitted using the predictor values required by its model. Measurement error in training predictors can affect the fitted regression itself, not just the width of a future prediction interval. Simply adding a future-predictor variance term does not correct that problem.

Key takeaway: A standard regression prediction interval is conditional on specified predictor values treated as known. Uncertainty in a future predictor changes the prediction distribution; depending on the information available, it can increase the spread, change the center, or require a different model.

Check Your Understanding

Answer each question by identifying the prediction target and the uncertainty included.

In the standard prediction interval, what does the term $1$ represent, and what does $h_0$ represent?
A future predictor varies independently around $x_0$ with variance $\tau^2$ in a simple linear model. What additional outcome-variance component results, and what model quantity determines its size?
Why is an interval for an outcome at exactly $x_0$ not necessarily appropriate when a future case’s actual predictor varies around $x_0$?
If a recorded predictor value is noisy, why might substituting it directly into the regression give the wrong prediction center?
Does adding a term for uncertainty in a future predictor correct measurement error in the training predictors? Explain briefly.