Checking Linearity and Constant Variance With Residual Plots

Why Inspect Residuals?

A multiple regression equation can produce predictions that look reasonable on average while missing an important feature of the data. For example, the relationship between a predictor and the response might bend rather than follow a straight line, or prediction errors might become more variable as fitted responses increase. A coefficient table or a single summary of model fit may not make either problem obvious.

Residual plots make these patterns easier to see. They do not prove that a model is correct or incorrect. Instead, they help identify evidence that the model's description of the conditional mean or its assumption of constant error variance may be inadequate.

The Model, Fitted Values, and Residuals

For observation \(i\), a multiple linear regression model with \(k\) predictors can be written as

Y_i = \beta_0 + \beta_1 x_{i1} + \cdots + \beta_k x_{ik} + \varepsilon_i

Here \(Y_i\) is the response, \(x_{ij}\) is the value of predictor \(j\) for observation \(i\), and \(\varepsilon_i\) is the error. The model assumes that the conditional mean response is linear in the included predictors, and that the errors have mean zero and constant variance given the predictors. In the usual regression framework, observations are also assumed independent. These plots focus on the first two conditions; they do not establish independence.

Definition: The fitted value for observation \(i\) is its prediction from the estimated regression equation, written \(\hat{y}_i\). The residual is the observed response minus its fitted value: \(e_i = y_i - \hat{y}_i\). A positive residual means the model underpredicted that observation; a negative residual means it overpredicted. Residuals have the same units as the response.

A residual-versus-fitted plot places \(\hat{y}_i\) on the horizontal axis and \(e_i\) on the vertical axis. A residual-versus-predictor plot places one predictor's values on the horizontal axis and the residuals on the vertical axis. In either plot, a horizontal reference line at zero helps show whether residuals tend to lie above or below zero.

What Patterns to Look For

If the conditional mean model is adequate, residuals should generally scatter around zero without a systematic curve. A smooth curve or a clear bend in a residual plot suggests that the fitted mean may miss structure. A common case is curvature with respect to a quantitative predictor: the model's straight-line contribution may not represent the response pattern well. A residual-versus-fitted plot can reveal an overall mean-pattern problem, while a residual-versus-predictor plot can help show which predictor's values are associated with it.

Constant variance means that the error variance is the same at different combinations of predictor values. In a residual plot, this is assessed by comparing the vertical spread across the horizontal axis. A roughly even band is consistent with constant variance. A fan, funnel, or other systematic widening or narrowing suggests that the spread changes with fitted values or a predictor. The direction matters: increasing spread is not the only possible violation.

Residual plots can also show clusters or other structure that does not fit either simple pattern. Such features are reasons to investigate the model and data context, not automatic diagnoses of a specific cause. In particular, a residual-versus-fitted plot combines information from all predictors. It can signal a problem without identifying which term needs reconsideration.

Worked Example: Curvature Across a Predictor

Suppose a hypothetical model predicts a laboratory measurement \(y\), in units, from two predictors: \(\hat{y} = 10 + 2x_1 + 3x_2\). The following six observations are selected from a larger data set. Calculate each fitted value and residual, then interpret the residual-versus-\(x_1\) pattern.

\(x_1\)	\(x_2\)	Observed \(y\)	Fitted \(\hat{y}\)	Residual \(e=y-\hat{y}\)
1	0	13	12	1
2	1	16	17	-1
3	0	14	16	-2
4	1	19	21	-2
5	0	19	20	-1
6	1	27	25	2

For the first observation, \(\hat{y}=10+2(1)+3(0)=12\), so \(e=13-12=1\). For the second, \(\hat{y}=10+2(2)+3(1)=17\), so \(e=16-17=-1\). Applying the same calculations gives the remaining fitted values and residuals in the table.

Against \(x_1\), the residuals are positive at the low end, mostly negative in the middle, and positive again at the high end. That U-shaped pattern is evidence that a straight-line contribution for \(x_1\) may not capture the conditional mean pattern. It does not prove that curvature in \(x_1\) is the cause: the points are only a small selected set, and the full model and data context must be considered.

Worked Example: Increasing Residual Spread

A hypothetical model predicts a response in milligrams. To compare residual spread at lower and higher fitted values, consider four selected observations in each range. Calculate their residuals and compare the root mean square residual within each range.

Fitted value \(\hat{y}\) (mg)	Residual \(e\) (mg)
10	-1
12	1
14	-1
16	1
30	-5
32	5
34	-5
36	5

For example, an observation with \(\hat{y}=10\) mg and \(e=-1\) mg has observed response \(y=\hat{y}+e=10-1=9\) mg. In the lower fitted-value range, the root mean square residual is

\sqrt{\frac{(-1)^2+1^2+(-1)^2+1^2}{4}} = \sqrt{\frac{4}{4}} = 1\text{ mg}.

In the higher range, it is

\sqrt{\frac{(-5)^2+5^2+(-5)^2+5^2}{4}} = \sqrt{\frac{100}{4}} = 5\text{ mg}.

The residuals are much more spread out at higher fitted values. A plot would show a widening band, which is evidence against constant variance. These selected values illustrate the comparison; in an actual diagnostic, assess the overall pattern rather than treating these small groups as formal estimates of variance.

Why the Plots Work

The basic justification follows from the model. If the conditional mean is correctly described by the fitted linear expression, then after accounting for predictor values there should not be a systematic mean pattern left in the errors. Residuals approximate those unobserved errors, so a persistent curve in residuals is a warning that the fitted mean has left some structure unexplained.

Similarly, if the model has constant error variance, then the vertical spread of errors should be comparable across predictor settings. In notation, the constant-variance condition is

\operatorname{Var}(\varepsilon_i \mid x_{i1},\ldots,x_{ik}) = \sigma^2

for all observations. Since \(e_i=y_i-\hat{y}_i\) is the observable stand-in for the unobserved error, changing residual spread can warn that this condition is not plausible. Residuals are not the errors themselves, however, so plots provide diagnostic evidence rather than a direct view of the model errors.

There is also an important limitation: when ordinary least squares fits a model containing an intercept, its residuals sum to zero. In matrix notation, the least-squares equations imply

\sum_{i=1}^{n} e_i = 0.

Thus, the residuals' overall average being zero is expected by construction. It does not show that the model's conditional mean is correct. What matters in a diagnostic plot is whether residuals have a pattern across fitted values or predictors, not whether their grand average is zero.

Worked Example: A Roughly Even Residual Band

A hypothetical regression predicts an outcome in seconds. Six selected observations have the following fitted values and residuals. Consider whether the displayed residuals give a clear warning about curvature or changing spread.

Fitted value \(\hat{y}\) (s)	Residual \(e\) (s)	Observed \(y=\hat{y}+e\) (s)
8	-1.2	6.8
10	0.6	10.6
12	1.0	13.0
14	-0.8	13.2
16	0.4	16.4
18	0.0	18.0

For the first row, \(y=8+(-1.2)=6.8\) seconds; for the second, \(y=10+0.6=10.6\) seconds. Across the fitted-value range, the residuals lie on both sides of zero and their magnitudes do not show a clear increasing or decreasing spread. These points do not display an obvious curve or fan, so they offer no clear warning of those two problems.

That is not proof of linearity or constant variance. Six observations may miss a real pattern, and a fuller plot might show structure not apparent in this small selection. The appropriate conclusion is limited: this set of residuals does not make a systematic pattern obvious.

Using Both Plot Types

A practical diagnostic begins with a residual-versus-fitted plot, then examines residuals against each quantitative predictor. For a categorical predictor, residuals can be compared across its groups to look for systematic shifts or different spreads. Use the plot that matches the question: fitted values summarize the model's combined predictions; individual predictor plots help locate patterns associated with particular predictor values.

Check the center.
Look for residuals scattered around the zero reference line, rather than a sustained band above or below it over part of the horizontal axis.

Check the shape.
Look for curves, bends, or other systematic changes in the residual center. A smooth curve can help reveal broad structure but should not be treated as decisive evidence by itself.

Check the spread.
Compare the vertical scatter across the plot. A fan or narrowing band suggests that residual variance may change with fitted values or predictor values.

Compare plots and context.
Use the predictor-specific plots to investigate where a fitted-versus-residual pattern may arise. Consider the study and measurement process before deciding what model change, if any, is justified.

A residual-versus-predictor plot in a multiple regression model is not the same as a simple plot of response against that predictor: the residuals have already been calculated after accounting for the model's other included terms. Correlations among predictors and the form of the full model can affect the appearance of the plot. Treat it as a diagnostic view, not as a stand-alone estimate of the predictor's relationship with the response.

Common Mistakes and Limits

Calling any scattered plot proof that assumptions hold. A plot with no obvious pattern is reassuring, but limited data can conceal departures.
Assuming zero average residuals mean the mean model is correct. With an intercept, the overall residual sum is zero by the fitting procedure. Examine systematic patterns across the axes instead.
Calling a widening band curvature. Curvature concerns the residual center; changing variance concerns the residual spread. A plot can show one or both, so inspect center and spread separately.
Inferring a specific cause from one pattern. A curve points to unexplained structure, but it does not by itself identify which predictor, transformation, or other model feature is responsible.
Using these plots to assess every regression condition. They are useful for linearity and variance patterns, but they do not establish independence or residual normality. A lack of curvature or a fan does not answer those other questions.

Key takeaway: In residual plots, look for systematic changes in the residual center to assess linearity and systematic changes in vertical spread to assess constant variance. A pattern is a reason to investigate the model, not a complete diagnosis on its own.

Check Your Understanding

Try these questions without looking back.

An observation has observed response 42 units and fitted value 46 units. What is its residual, and what does its sign mean?
A residual-versus-fitted plot bends from positive residuals at low fitted values to negative residuals in the middle and positive residuals at high fitted values. What model feature should be investigated?
A residual plot has a roughly level center but its vertical spread grows across the horizontal axis. Which condition is in question?
Why does a residual sum of zero not demonstrate that a regression model is adequate?
What can residual-versus-predictor plots add to a residual-versus-fitted plot in multiple regression, and what limitation should be kept in mind?