How Predictor Location Affects Interval Width

Why Predictor Location Matters

Two estimated responses from the same regression can have different levels of precision even when they use the same sample and the same confidence level. The key is where the predictor value lies relative to the observed predictors. A value near their center generally gives a more precise estimated mean response than a value far from that center. A prediction interval for one future response also widens with distance, while retaining additional uncertainty from the future observation itself.

This tutorial quantifies that location effect using the leverage term for a specified predictor value. The focus is on interval width, not on whether a particular fitted line is appropriate outside the observed range.

Model and Location-Dependent Widths

Assume the simple linear regression model with fixed predictor values \(x_1,\ldots,x_n\):

Y_i=\beta_0+\beta_1x_i+\varepsilon_i,\qquad i=1,\ldots,n.

For exact t intervals, assume the errors are independent and normally distributed with mean zero and common variance \(\sigma^2\). Let \(\bar{x}\) be the sample mean of the predictor values, let \(S_{xx}=\sum_{i=1}^n(x_i-\bar{x})^2\), and let \(s=\sqrt{\mathrm{MSE}}\) be the residual standard error. The residual degrees of freedom are \(n-2\). For a specified predictor value \(x_0\), define

h_0=\frac{1}{n}+\frac{(x_0-\bar{x})^2}{S_{xx}}.

Here \(h_0\) is the leverage term for estimating the mean response at \(x_0\). It is smallest at \(x_0=\bar{x}\), and increases as \(x_0\) moves farther from \(\bar{x}\).

Theorem: For a pointwise confidence level \(1-\alpha\), the full width of the confidence interval for the mean response at \(x_0\) is \(2t_{1-\alpha/2,n-2}s\sqrt{h_0}\). The full width of the prediction interval for one future response at \(x_0\) is \(2t_{1-\alpha/2,n-2}s\sqrt{1+h_0}\). The interval center in each case is the fitted response \(\hat{y}(x_0)\).

The full width is twice the margin of error. Therefore, comparing widths at different predictor locations does not require knowing the fitted response: the center changes with the fitted line, but the width is controlled by \(n\), \(S_{xx}\), \(s\), the confidence level, and the distance \(x_0-\bar{x}\).

Why the Leverage Term Has This Form

The mean-response estimate at \(x_0\) can be written as \(\hat{y}(x_0)=\bar{Y}+\hat{\beta}_1(x_0-\bar{x})\), where \(\hat{\beta}_1\) is the fitted slope. Under the stated fixed-predictor model, \(\operatorname{Var}(\bar{Y})=\sigma^2/n\) and \(\operatorname{Var}(\hat{\beta}_1)=\sigma^2/S_{xx}\). The covariance of \(\bar{Y}\) and \(\hat{\beta}_1\) is zero: the slope depends on predictor-centered responses, and the centered predictor values sum to zero. Thus the variance of the estimated mean is

\begin{aligned} \operatorname{Var}\bigl(\hat{y}(x_0)\bigr) &=\operatorname{Var}(\bar{Y})+(x_0-\bar{x})^2\operatorname{Var}(\hat{\beta}_1)\\ &=\sigma^2\left(\frac{1}{n}+\frac{(x_0-\bar{x})^2}{S_{xx}}\right) =\sigma^2h_0. \end{aligned}

Replacing \(\sigma\) with \(s\) and using the t critical value gives the mean-response interval margin. A future response has an independent error with variance \(\sigma^2\), so its prediction error variance is \(\sigma^2+\sigma^2h_0=\sigma^2(1+h_0)\). This explains the additional 1 in the prediction-interval formula. The exact t intervals require the stated normal-error model; without normality, exact finite-sample t coverage is not guaranteed.

Since \((x_0-\bar{x})^2\) is nonnegative, both widths attain their minimum at \(x_0=\bar{x}\). They are also symmetric around the sample mean: two predictor values equally far from \(\bar{x}\) have the same width, even though their fitted response centers may differ. The prediction interval’s extra 1 does not change with location; it reflects the uncertainty of an individual future response.

Worked Width Calculations

Worked Example: Comparing Center and Nearby Predictor Values

Assume a hypothetical study follows the stated normal-error regression model. It has \(n=21\), \(\bar{x}=50\), \(S_{xx}=2{,}000\), and \(s=6\). The fitted line is \(\hat{y}(x)=42+0.8x\). For 95% intervals, use \(t_{0.975,19}=2.093\). Calculate the mean-response and prediction interval widths at \(x_0=50\) and \(x_0=60\), and give the interval endpoints.

Method. Calculate \(h_0\) at each location, then multiply the relevant square root by \(2t s\) for the full width. The fitted responses are \(\hat{y}(50)=42+0.8(50)=82\) and \(\hat{y}(60)=42+0.8(60)=90\).

At \(x_0=50=\bar{x}\), \(h_0=1/21=0.047619\). The mean-response width is \(2(2.093)(6)\sqrt{1/21}=5.48\), rounded to two decimals. Its half-width is \(5.48/2=2.74\), so the interval is \(82\pm2.74=[79.26,84.74]\).

At \(x_0=60\), \(h_0=1/21+(60-50)^2/2{,}000=0.047619+0.05=0.097619\). The mean-response width is \(2(2.093)(6)\sqrt{0.097619}=7.85\). Its half-width is \(3.92\), giving \(90\pm3.92=[86.08,93.92]\).

For prediction at \(x_0=50\), use \(1+h_0=1.047619\). The prediction width is \(2(2.093)(6)\sqrt{1.047619}=25.71\), with half-width \(12.85\); the interval is \(82\pm12.85=[69.15,94.85]\). At \(x_0=60\), \(1+h_0=1.097619\), so the width is \(2(2.093)(6)\sqrt{1.097619}=26.31\). The half-width is \(13.16\), giving \(90\pm13.16=[76.84,103.16]\). Endpoints are rounded to two decimals.

Check. Both kinds of interval are wider at 60 than at the sample mean 50. The prediction intervals are much wider than the corresponding mean-response intervals because they include the variation of an individual future response.

Worked Example: Equal Distances Give Equal Widths

Assume the stated model conditions hold for a hypothetical sample with \(n=16\), \(\bar{x}=8\), \(S_{xx}=80\), and \(s=4\). Use \(t_{0.975,14}=2.145\). Compare the 95% mean-response and prediction interval widths at \(x_0=8\), \(x_0=10\), and \(x_0=12\).

At the center, \(h_0=1/16=0.0625\). The mean-response width is \(2(2.145)(4)\sqrt{0.0625}=8.58(0.25)=2.15\). The prediction width is \(8.58\sqrt{1.0625}=8.58(1.03078)=8.85\).

At \(x_0=10\), the distance from the center is 2, so \(h_0=0.0625+2^2/80=0.1125\). The mean-response width is \(8.58\sqrt{0.1125}=8.58(0.33541)=2.88\). The prediction width is \(8.58\sqrt{1.1125}=8.58(1.05475)=9.05\).

At \(x_0=12\), the distance is 4, so \(h_0=0.0625+4^2/80=0.2625\). The mean-response width is \(8.58\sqrt{0.2625}=8.58(0.51235)=4.40\). The prediction width is \(8.58\sqrt{1.2625}=8.58(1.12361)=9.64\). Widths are rounded to two decimals.

Check. The locations 10 and 12 are not equally distant from the sample mean, so their widths differ. The results increase as the distance from 8 increases, for both interval types. The location effect is determined by squared distance, not by whether \(x_0\) is above or below the center.

Worked Example: Endpoint Versus Center in the Observed Predictor Range

A hypothetical regression study satisfies the stated model conditions and has \(n=10\), \(\bar{x}=30\), \(S_{xx}=900\), and \(s=5\). The observed predictor values range from 10 to 50. Using \(t_{0.975,8}=2.306\), calculate the 95% mean-response and prediction interval widths at the center \(x_0=30\) and at the observed endpoint \(x_0=50\).

At \(x_0=30\), \(h_0=1/10=0.1\). The mean-response width is \(2(2.306)(5)\sqrt{0.1}=23.06(0.31623)=7.29\). The prediction width is \(23.06\sqrt{1.1}=23.06(1.04881)=24.19\).

At \(x_0=50\), the distance from the center is 20, so \(h_0=1/10+20^2/900=0.1+400/900=0.544444\). The mean-response width is \(23.06\sqrt{0.544444}=23.06(0.73787)=17.02\). The prediction width is \(23.06\sqrt{1.544444}=23.06(1.24276)=28.67\). All widths are rounded to two decimals.

Check. The endpoint is still within the observed predictor range, yet its intervals are wider than those at the center. This illustrates a precision difference within the data’s range, without making a claim about the reliability of extrapolation beyond it.

Common Mistake: Using the Uncentered Predictor

A frequent error is to substitute \(x_0^2/S_{xx}\) for \((x_0-\bar{x})^2/S_{xx}\). That treats zero as the predictor center, which is generally wrong. For example, in the first worked example, using \(50^2/2{,}000\) at \(x_0=50\) would give \(h_0=1/21+2{,}500/2{,}000=1.297619\), rather than the correct \(1/21\). The resulting mean-response width would be \(2(2.093)(6)\sqrt{1.297619}=28.57\), rather than \(5.48\). Always center the predictor value at \(\bar{x}\) before squaring the distance.

Another mistake is to add the extra 1 to the mean-response formula, or omit it from the prediction formula. The mean interval concerns uncertainty in the estimated mean; the prediction interval concerns one future response and must include that response’s additional error variance.

Key takeaway: In simple linear regression, leverage at \(x_0\) is \(1/n+(x_0-\bar{x})^2/S_{xx}\). Both interval widths are minimized at the sample mean and increase with distance from it. A prediction interval has the additional \(1\) under its square root because it covers an individual future response.

Check Your Understanding

Use the stated fixed-predictor regression formulas. Unless noted otherwise, assume normal errors and calculate widths to two decimals.

A study has \(n=9\), \(\bar{x}=4\), \(S_{xx}=72\), \(s=3\), and \(t_{0.975,7}=2.365\). Find the 95% mean-response interval widths at \(x_0=4\) and \(x_0=10\). (Answer: \(4.73\) and \(11.09\), respectively.)
A study has \(n=12\), \(\bar{x}=20\), \(S_{xx}=110\), \(s=2.5\), and \(t_{0.975,10}=2.228\). Find the 95% mean-response width at \(x_0=25\). (Answer: \(6.21\).)
For \(n=20\) and \(S_{xx}=500\), compare prediction interval widths at \(x_0=\bar{x}-5\) and \(x_0=\bar{x}+5\), using the same \(s\) and t critical value at both locations. What is their ratio? (Answer: \(1.0000\); the widths are equal because the distances from \(\bar{x}\) are equal.)
A study has \(n=15\), \(\bar{x}=12\), \(S_{xx}=280\), \(s=4\), and \(t_{0.975,13}=2.160\). Find the 95% prediction interval width at \(x_0=19\). (Answer: \(19.25\).)
At \(x_0=\bar{x}\), what are the leverage term and the prediction-interval square-root factor, expressed in terms of \(n\)? (Answer: \(h_0=1/n\), and the prediction factor is \(\sqrt{1+1/n}\).)

What You'll Learn