Why Plot Residuals Against Fitted Values?
A regression line or plane summarizes a mean relationship using a linear function of the predictors. That summary can miss a curved or otherwise systematic pattern in the mean response. A residual-versus-fitted plot provides a compact diagnostic: it places each observation’s fitted response on the horizontal axis and its residual on the vertical axis. The main question is whether the residuals fluctuate around zero without a systematic shape, or instead trace a pattern suggesting that a linear mean relationship is inadequate.
A residual plot is evidence for investigation, not a proof that a model is correct or incorrect. Random scatter can occur even when the mean structure is wrong, especially in a small sample. Conversely, a few irregular points can appear by chance. Interpret the overall pattern in context, and do not infer a particular predictor’s role from this plot alone.
Definitions and the Regression Model
For \(n\) observations and \(p\) predictor variables, consider the linear mean model \(Y_i=\beta_0+\beta_1x_{i1}+\cdots+\beta_px_{ip}+\varepsilon_i\). The key assumption for assessing linearity is that the conditional mean has this form: \(E[Y_i\mid X_i]=\beta_0+\beta_1x_{i1}+\cdots+\beta_px_{ip}\). Here \(X_i\) denotes the predictor values for observation \(i\). The errors are assumed to have conditional mean zero. The model may also make assumptions about error variance and dependence for inference, but those assumptions are not what a residual-versus-fitted plot primarily assesses.
Ordinary least squares (OLS) chooses the coefficient estimates to minimize the sum of squared residuals. The resulting fitted values are not the true conditional means; they are estimates based on the observed sample. In an adequate mean model, residuals should generally be centered around zero across the range of fitted values, without a persistent curve or other systematic shape.
How to Read the Plot
Start with the zero line. A point above it has \(e_i>0\), so the observed response exceeds its fitted value; a point below it has \(e_i<0\), so the fitted value exceeds the observation. Next, look across the horizontal axis for a sustained pattern rather than judging isolated points one at a time.
- Curvature: a smooth U-shape, inverted U-shape, or other curved trend can indicate that a straight-line mean function does not capture the relationship adequately.
- Other systematic structure: bands, waves, or distinct clusters of positive and negative residuals may also indicate a mean pattern the model has not captured. The plot alone does not identify its cause.
- No clear pattern: residuals scattered around zero without a visible systematic shape are consistent with the specified linear mean structure, but do not establish that it is correct.
- Changing vertical spread: a fan-shaped spread is worth noting, but it concerns variability as well as the mean. Do not treat it as proof of curvature.
The horizontal coordinate is a fitted response, not necessarily a predictor. In a model with several predictors, observations with similar fitted values can have quite different predictor combinations. A pattern in this plot can suggest that some aspect of the mean model needs attention, but it generally cannot tell which predictor is responsible. Avoid assigning a cause without examining the predictors and the context.
Worked Example: A U-Shape in a Simple Regression
A hypothetical analyst regresses a claim-processing time \(y\), in hours, on a workload score \(x\), fitting a line with an intercept. The observations are \((x,y)=(0,2),(1,2),(2,4),(3,8),(4,14)\). Calculate the fitted values and residuals, then interpret the residual-versus-fitted pattern.
Method. For simple OLS regression with an intercept, the slope is \(b_1=S_{xy}/S_{xx}\), where \(S_{xy}=\sum_i(x_i-\bar{x})(y_i-\bar{y})\) and \(S_{xx}=\sum_i(x_i-\bar{x})^2\). The intercept is \(b_0=\bar{y}-b_1\bar{x}\).
First calculate the means: \(\bar{x}=(0+1+2+3+4)/5=2\) and \(\bar{y}=(2+2+4+8+14)/5=6\). The cross-products for \(S_{xy}\) are \( (-2)(-4)=8\), \((-1)(-4)=4\), \(0(-2)=0\), \(1(2)=2\), and \(2(8)=16\). Thus \(S_{xy}=30\). Also, \(S_{xx}=(-2)^2+(-1)^2+0^2+1^2+2^2=10\). Therefore \(b_1=30/10=3\), and \(b_0=6-3(2)=0\). The fitted line is \(\hat{y}=3x\).
The fitted values at \(x=0,1,2,3,4\) are \(0,3,6,9,12\). Subtract each fitted value from its observed response: the residuals are \(2,-1,-2,-1,2\). The plotted points are therefore \((0,2),(3,-1),(6,-2),(9,-1),(12,2)\).
At the low and high fitted values, residuals are positive; in the middle, they are negative. That U-shaped pattern suggests that the linear mean relationship may be inadequate. It is consistent with a curved mean response, but the plot by itself does not establish the correct alternative model.
Check. The residuals sum to \(2-1-2-1+2=0\), as expected for an OLS fit with an intercept. The common mistake to avoid is calling the plot pattern random simply because the residuals sum to zero; that total does not rule out curvature.
Why a Linear Residual Trend Is Not the Diagnostic
With an intercept, OLS residuals have zero sum and are orthogonal to every fitted predictor column. In particular, they are orthogonal to the fitted values. As a result, an ordinary linear association between fitted values and residuals is forced to be zero (apart from numerical rounding). A straight-line trend test on this plot is therefore not the key check. The diagnostic value comes from looking for nonlinear or otherwise systematic structure.
With a design matrix \(X\) that includes an intercept, the OLS estimates minimize \(\sum_i e_i^2\). Differentiating this sum with respect to the coefficient vector gives \(X^\mathsf{T}e=0\).
The vector of fitted values is \(\hat{y}=X\hat{\beta}\), so \(\hat{y}^\mathsf{T}e=(X\hat{\beta})^\mathsf{T}e=\hat{\beta}^\mathsf{T}X^\mathsf{T}e=0\).
Because the intercept column is in \(X\), \(\sum_i e_i=0\); because \(\hat{y}\) is a linear combination of columns of \(X\), the residuals have zero linear association with fitted values in this inner-product sense. This algebra does not prevent a curved pattern such as positive residuals at both ends and negative residuals in the middle.
This derivation explains why the plot is judged visually for shape rather than by looking for a nonzero straight-line slope. The zero-sum and orthogonality results apply to the OLS fit on the same data with an intercept. They do not imply that residuals are independent, normally distributed, or free of nonlinear structure.
Worked Example: Scatter Without a Clear Curved Pattern
A hypothetical team models review time \(y\), in minutes, using a workload index \(x\). The data are \((1,5),(2,7),(3,8),(4,11),(5,12)\). Compute the fitted line and residuals, then decide what the plot does and does not support.
The predictor mean is \(\bar{x}=3\), and the response mean is \(\bar{y}=(5+7+8+11+12)/5=8.6\). The cross-products for \(S_{xy}\) are \((-2)(-3.6)=7.2\), \((-1)(-1.6)=1.6\), \(0(-0.6)=0\), \(1(2.4)=2.4\), and \(2(3.4)=6.8\), giving \(S_{xy}=18\). The predictor sum of squares is \(S_{xx}=4+1+0+1+4=10\). Hence \(b_1=18/10=1.8\) and \(b_0=8.6-1.8(3)=3.2\). The fitted line is \(\hat{y}=3.2+1.8x\).
For \(x=1,2,3,4,5\), the fitted values are \(5.0,6.8,8.6,10.4,12.2\). Subtracting them from the observed responses gives residuals \(0,0.2,-0.6,0.6,-0.2\). These points alternate around zero without a clear smooth curve.
The plot offers no obvious evidence of curvature in this small sample. That is a limited conclusion: it does not prove the linear mean model is correct, and it does not certify other regression assumptions. A different sample could reveal a pattern that these five observations do not show.
Check. The residuals sum to \(0+0.2-0.6+0.6-0.2=0\). Their fitted-value-weighted sum is \(5(0)+6.8(0.2)+8.6(-0.6)+10.4(0.6)+12.2(-0.2)=0\). The common mistake is to interpret absence of a visible curve as confirmation that every aspect of the regression model is adequate.
Worked Example: A Structured Residual Pattern Despite Orthogonality
For a hypothetical regression of settlement time \(y\) on a complexity score \(x\), suppose the data are \((1,8),(2,6),(3,19),(4,12),(5,20)\). Fit the OLS line and explain why its residual pattern still merits attention.
The means are \(\bar{x}=3\) and \(\bar{y}=(8+6+19+12+20)/5=13\). The products for \(S_{xy}\) are \((-2)(-5)=10\), \((-1)(-7)=7\), \(0(6)=0\), \(1(-1)=-1\), and \(2(7)=14\). Thus \(S_{xy}=30\). Since \(S_{xx}=4+1+0+1+4=10\), the slope is \(b_1=3\), and the intercept is \(b_0=13-3(3)=4\). The fitted line is \(\hat{y}=4+3x\).
The fitted values are \(7,10,13,16,19\). The residuals are \(8-7=1\), \(6-10=-4\), \(19-13=6\), \(12-16=-4\), and \(20-19=1\). In fitted-value order, the residuals are positive, negative, positive, negative, positive. The central positive residual and the alternating signs form structured variation rather than an obvious random cloud; the plot warrants investigating whether the mean function captures the response pattern.
The check is \(1-4+6-4+1=0\). The fitted-value-weighted sum is \(7(1)+10(-4)+13(6)+16(-4)+19(1)=7-40+78-64+19=0\). Thus the pattern is compatible with OLS orthogonality: zero linear association does not eliminate systematic nonlinear structure. The common mistake is to dismiss the pattern because the fitted values and residuals have zero inner product.
Practical Interpretation and Common Traps
A residual-versus-fitted plot is most useful when interpreted as a focused check of the mean structure. If a smooth pattern is visible, ask whether the fitted response might need a different functional form or whether an important feature of the modeling context has been omitted. The plot identifies a reason to investigate; it does not prescribe a specific remedy.
- Do not reverse the residual sign. Under the definition \(e_i=y_i-\hat{y}_i\), positive residuals mean the observed response is above the fitted value. Using \(\hat{y}_i-y_i\) silently flips every sign.
- Do not equate zero average residual with a good mean model. OLS with an intercept makes residuals sum to zero even when they form a pronounced curve.
- Do not search only for a straight-line trend. OLS orthogonality removes that linear association by construction; smooth curvature and other shapes are the relevant visual clues.
- Do not claim the plot identifies a predictor. Fitted values combine predictor information, especially in multiple regression. A pattern suggests a model concern, not its unique source.
- Do not overstate a clean-looking plot. A lack of visible structure is not proof of correctness, particularly with few observations or a narrow range of fitted values.
Check Your Understanding
For each problem, use the residual definition and distinguish what the plot shows from what it cannot establish.
- A model gives a fitted response of 26 days for a file whose observed completion time is 22 days. What is its residual, and is the point above or below the zero line? (Answer: \(e=-4\) days; below zero.)
- For a fitted model with an intercept, five residuals in fitted-value order are \(3,-1,-4,-1,3\). What shape do they suggest, and what does their sum tell you? (Answer: A U-shaped pattern suggesting possible curvature; the sum is zero, as expected for OLS with an intercept, but this does not rule out the pattern.)
- An analyst reports that the residuals have zero linear association with fitted values, so the mean model must be adequate. What is the error in this conclusion? (Answer: OLS orthogonality with an intercept forces zero linear association; it does not prevent curvature or other systematic residual structure.)
- For workload scores \(x=1,2,3\), a fitted line gives fitted values \(10,12,14\), while observed responses are \(11,9,16\). Calculate the residuals and describe the pattern in fitted-value order. (Answer: Residuals are \(1,-3,2\); they change sign across the fitted values but three points do not establish a sustained curve.)
- A multiple regression’s residual-versus-fitted plot shows a smooth wave. Can the analyst conclude from this plot alone which predictor should be transformed? (Answer: No; the plot suggests systematic mean structure but does not identify which predictor, or which specific change, explains it.)