Prediction Requires Both Calculation and Diagnosis
A regression interval is a model-based statement, not a guarantee that a future outcome will fall within its endpoints. To use an interval responsibly, first distinguish the quantity being estimated: the average response at a predictor value or one future response. Then check whether the fitted line and its error assumptions are plausible in the data range of interest. This integrated view is aligned with the SOA VEE Mathematical Statistics syllabus.
The case studies below combine these tasks. The calculations use simple linear regression; the diagnostic discussion asks whether the assumptions behind those calculations appear reasonable. A diagnostic plot can reveal warning signs, but it cannot prove that a model is correct.
Write \(\bar{x}\) for the sample mean of the predictor, \(S_{xx}=\sum_{i=1}^n(x_i-\bar{x})^2\), and \(s=\sqrt{\mathrm{SSE}/(n-2)}\) for the residual standard error, where \(\mathrm{SSE}=\sum_{i=1}^n e_i^2\). At a selected predictor value \(x_0\), let \(\hat{y}_0=b_0+b_1x_0\). The estimated standard error for the fitted mean response is \(s\sqrt{1/n+(x_0-\bar{x})^2/S_{xx}}\). For a single future response, the corresponding standard error is \(s\sqrt{1+1/n+(x_0-\bar{x})^2/S_{xx}}\).
Worked Case: Two Kinds of Interval at Two Predictor Values
Worked Example: Estimating Scores at Typical and High Study Hours
A hypothetical study relates a practice-test score \(Y\), in points, to weekly study hours \(x\). A simple linear regression summary for 9 students gives \(b_0=10\), \(b_1=1.5\), \(\bar{x}=5\), \(S_{xx}=60\), and \(\mathrm{SSE}=28\). Assume the regression model’s independence, linearity, constant-variance, and normal-error conditions are reasonable. Find 95% confidence and prediction intervals at \(x_0=5\) and \(x_0=9\). Use \(t_{0.975,7}=2.365\).
Method and residual standard error. There are \(n-2=7\) residual degrees of freedom, so \(s=\sqrt{28/7}=\sqrt{4}=2\) points. The fitted line is \(\hat{y}=10+1.5x\), and the relevant multiplier for all four intervals is 2.365.
At \(x_0=5\). The fitted score is \(\hat{y}_0=10+1.5(5)=17.5\). Since this is \(\bar{x}\), the predictor-location term is zero. The standard error for the mean is \(2\sqrt{1/9}=2/3=0.6667\), giving a margin \(2.365(0.6667)=1.5767\). The 95% confidence interval for the mean score is \(17.5\pm1.5767=(15.9233,19.0767)\), or \((15.92,19.08)\) points to two decimals.
For one future score, the standard error is \(2\sqrt{1+1/9}=2\sqrt{10/9}=2.1082\). The margin is \(2.365(2.1082)=4.9869\), so the 95% prediction interval is \(17.5\pm4.9869=(12.5131,22.4869)\), or \((12.51,22.49)\) points to two decimals. It is wider because it includes individual response variation.
At \(x_0=9\). This value is within the observed predictor range of 1 to 9, though it is away from the sample mean. The fitted score is \(10+1.5(9)=23.5\). The location term is \(1/9+(9-5)^2/60=1/9+16/60=17/45\). The mean-response standard error is \(2\sqrt{17/45}=1.2293\), giving margin \(2.365(1.2293)=2.9073\). The confidence interval is \(23.5\pm2.9073=(20.5927,26.4073)\), or \((20.59,26.41)\) points to two decimals.
For a future score, the standard error is \(2\sqrt{1+17/45}=2\sqrt{62/45}=2.3476\). The margin is \(2.365(2.3476)=5.5526\), giving the prediction interval \((17.9474,29.0526)\), or \((17.95,29.05)\) points to two decimals.
Check. At each predictor value, the prediction interval is wider than the confidence interval. Both intervals are wider at \(x_0=9\) than at \(x_0=5\), because 9 is farther from \(\bar{x}\). The calculation avoids the common mistake of treating an individual future score as though it were the estimated mean.
Why the Two Standard Errors Differ
The width formulas follow from the variance of the fitted mean and the variance of a new response. Under the model, the least-squares fitted mean at \(x_0\) is \(b_0+b_1x_0\), whose sampling variance is \(\sigma^2[1/n+(x_0-\bar{x})^2/S_{xx}]\). Estimating \(\sigma\) with \(s\) gives its estimated standard error. A future response can be written as its mean plus a new error, \(Y_{\mathrm{new}}=\beta_0+\beta_1x_0+\varepsilon_{\mathrm{new}}\). That new error is independent of the fitted mean estimate and has variance \(\sigma^2\). Therefore the variance for prediction adds \(\sigma^2\) to the variance for estimating the mean:
Replacing \(\sigma\) with \(s\) and using the \(t\) critical value with \(n-2\) degrees of freedom produces the stated prediction interval. This derivation also shows why predictor location matters: the squared distance from \(\bar{x}\) increases uncertainty in the estimated mean, while the extra 1 accounts for future response variation at every \(x_0\).
Reading Diagnostics Alongside the Intervals
Residuals \(e_i=y_i-\hat{y}_i\) are the observed vertical differences from the fitted line. A residual-versus-fitted plot or residual-versus-predictor plot should be inspected for systematic structure. A roughly patternless band around zero is compatible with a linear mean function. Curvature suggests the mean relationship may not be linear; a funnel-shaped spread suggests nonconstant error variance; and isolated unusually large residuals merit investigation. A normal Q-Q plot assesses whether residuals depart substantially from a normal pattern, especially in the tails.
Leverage measures how unusual an observation’s predictor value is relative to the other predictor values. In simple linear regression, the leverage for observation \(i\) is \(h_{ii}=1/n+(x_i-\bar{x})^2/S_{xx}\). A high-leverage observation is not automatically an outlier or influential point: it has an unusual predictor value, but its response may still agree with the overall trend. Cook’s distance combines residual size and leverage to describe how much the fitted regression would change if an observation were omitted. With two fitted coefficients, one common formula is \(D_i=[e_i^2/(2s^2)] [h_{ii}/(1-h_{ii})^2]\). It is a diagnostic measure, not a hypothesis test with a universal cutoff.
Worked Example: Checking a High-Leverage Observation
In a hypothetical calibration study, 10 sensors are tested at settings with \(\bar{x}=5\) and \(S_{xx}=30\). The fitted regression has \(s=2\). One sensor was tested at \(x_i=8\), and its residual is \(e_i=3\). Calculate its leverage and Cook’s distance, then interpret the results.
Method. First calculate leverage from the predictor location. Here \(h_{ii}=1/10+(8-5)^2/30=0.1+9/30=0.4\). This is above the average leverage \(2/n=0.2\), so the observation has higher-than-average leverage.
Next, \(s^2=4\), and there are \(p=2\) fitted coefficients. Cook’s distance is \(D_i=[3^2/(2(4))][0.4/(1-0.4)^2]\). The first factor is \(9/8=1.125\). The second is \(0.4/0.36=1.1111\). Thus \(D_i=1.125(1.1111)=1.25\), to two decimals.
The point has both a nonzero residual and higher-than-average leverage, and the resulting Cook’s distance flags it for investigation as potentially influential. This calculation does not establish that the observation is erroneous or that it must be removed. Check the measurement and study context, and compare fitted results with and without the point if that comparison is useful. The mistake to avoid is calling a point influential from leverage alone: influence depends on how the response and predictor position work together.
Integrated Case: A Numerical Interval Can Be Fragile
Worked Example: Predicting Equipment Output With Diagnostic Warnings
A hypothetical engineering team regresses hourly output \(Y\), in units, on operating temperature \(x\), in degrees. For 12 machines, the summary is \(b_0=4\), \(b_1=0.8\), \(\bar{x}=6\), \(S_{xx}=110\), and \(\mathrm{SSE}=40\). Consider a machine operating at \(x_0=9\). Assume initially that the standard regression conditions hold, and use \(t_{0.975,10}=2.228\). The diagnostic plots also show a slight curved residual pattern and greater residual spread at high fitted values.
Compute the nominal intervals. The residual standard error is \(s=\sqrt{40/(12-2)}=\sqrt{4}=2\). The fitted output is \(\hat{y}_0=4+0.8(9)=11.2\). The location term is \(1/12+(9-6)^2/110=1/12+9/110=0.16515\).
The standard error for the mean response is \(2\sqrt{0.16515}=0.8128\). Its margin is \(2.228(0.8128)=1.811\), so the nominal 95% confidence interval is \(11.2\pm1.811=(9.39,13.01)\) units to two decimals.
The prediction standard error is \(2\sqrt{1+0.16515}=2.1598\). Its margin is \(2.228(2.1598)=4.811\), so the nominal 95% prediction interval is \(11.2\pm4.811=(6.39,16.01)\) units to two decimals. Since \(x_0=9\) is within the observed temperature range, this is not an extrapolation beyond the data range.
Interpret the diagnostics. A curved pattern questions the assumed straight-line mean response. Increasing residual spread questions the constant-variance assumption. Those warnings matter because the nominal t intervals depend on the model assumptions; the arithmetic can be correct while the stated coverage is not well supported. The team should investigate the pattern and consider whether a more appropriate model or data collection plan is needed before relying on the intervals. Do not claim that the interval is invalid merely because a plot is imperfect, but do not report it as dependable without qualification.
State the limits of the conclusion. The confidence interval estimates a mean response at \(x_0=9\), while the prediction interval concerns one machine’s future output under the model. Neither establishes that temperature causes output to change, and neither guarantees the value for a particular machine. The model summary alone also does not establish independence, valid measurement, or representativeness of these machines. The correct conclusion is conditional: if the stated regression assumptions and sampling context are adequate, these are the model-based intervals; diagnostic warnings limit how confidently they can be used.
A Practical Integrated Review
Decide whether the question concerns the mean response at \(x_0\) or a single future response. Use the corresponding standard error.
Compare \(x_0\) with \(\bar{x}\) and the observed predictor range. Distance from the mean increases interval width; a value outside the observed range requires particular caution.
Look for curvature, changing spread, unusual residuals, and substantial departures from normality. These are warnings about assumptions, not automatic verdicts.
Use leverage and Cook’s distance together with the residual and the observation’s context. Do not delete a valid observation solely to improve a diagnostic plot.
Report what the interval estimates and state important assumptions or diagnostic concerns. Distinguish association and model-based prediction from causal claims.
A frequent exam trap is to calculate a precise endpoint and then state it without qualification. Precision in arithmetic does not repair a poor model. The opposite error is to treat every imperfect plot as proof that no useful inference is possible. Instead, identify which assumption is in question, how it could affect the interval, and what additional investigation would clarify the issue.
Check Your Understanding
For each problem, identify the target quantity and show how interval calculations or diagnostics affect the conclusion.
- A regression has \(n=16\), \(\bar{x}=10\), \(S_{xx}=80\), and \(s=3\). At \(x_0=10\), calculate the standard errors for the estimated mean response and a future response. (Answer: Mean SE \(=3\sqrt{1/16}=0.75\); future-response SE \(=3\sqrt{17/16}=3.0923\).)
- For a fitted line \(\hat{y}=6+2x\), a 95% interval is requested at \(x_0=4\). Explain which interval is wider and why, without calculating endpoints. (Answer: The prediction interval is wider because its standard error includes the additional 1 for individual response variation.)
- A simple regression has \(n=8\), \(\bar{x}=4\), and \(S_{xx}=28\). Find the leverage of an observation at \(x_i=7\), and compare it with average leverage. (Answer: \(h_{ii}=1/8+9/28=23/56=0.4107\), greater than the average \(2/8=0.25\).)
- A residual-versus-fitted plot has a clear U-shaped pattern, while the Q-Q plot is roughly straight. Which concern is most directly indicated, and does a near-straight Q-Q plot resolve it? (Answer: The U shape suggests the linear mean relationship may be inadequate; a near-straight Q-Q plot does not resolve that concern.)
- At a predictor value inside the observed range, a nominal prediction interval is narrow, but residual spread increases with fitted values and one observation has a large Cook’s distance. Give a defensible conclusion. (Answer: Report that the numerical interval is conditional on the regression assumptions, but the changing spread and potentially influential point require investigation before relying on its usual coverage.)