Systematic Offsets in Residual Plots

Why the Sign of a Residual Matters

A regression model can make predictions that are close on average yet consistently miss in a particular range. For example, it might tend to predict too low for observations with small fitted values and too high for observations with large fitted values. Looking at a residual plot helps reveal this kind of systematic offset: a sustained region of positive residuals or negative residuals is different from residuals that fluctuate around zero without a visible pattern.

A residual plot is a diagnostic, not a verdict. A visible offset suggests that prediction errors may depend on where an observation lies in the fitted-value range. It does not, by itself, identify the cause or prove that a particular change to the model is appropriate.

A residual-versus-fitted plot places \(\hat{y}_i\) on the horizontal axis and \(e_i\) on the vertical axis. The horizontal line \(e_i=0\) marks exact agreement between observed and predicted responses. A point above this line represents underprediction; a point below it represents overprediction. A sustained offset is a run or region of nearby fitted values where residuals tend to have the same sign, rather than alternating around zero without an apparent structure.

What a Sustained Offset Suggests

The main question is not whether every residual is exactly zero. That is rarely expected. Instead, look for a region in which the residuals are mostly on one side of zero, or cluster at values noticeably above or below zero. The pattern may indicate that predictions are systematically too low or too high for observations in that range. The residual magnitudes matter too: a handful of tiny residuals on one side of zero is weaker evidence of a practically important offset than a sustained group of larger residuals.

A residual plot does not tell you why a region is offset. It may prompt a closer look at the model, the data, or the prediction task, but a plot alone cannot establish which explanation is correct. Nor does a visible offset automatically show that the model is wrong in every respect: a model can be useful for some purposes and still have limitations in a particular range.

The interpretation also depends on what data the predictions come from. On a training sample, the model was fitted using the same outcomes whose residuals are plotted. If predictions are evaluated on a separate validation sample, the residuals are calculated for data not used to fit the model. A run of same-sign residuals in either plot is worth noticing, but the samples serve different purposes and their residual patterns should be described accordingly.

A Justification for Reading Residuals This Way

A simple calculation explains why residual sign has this interpretation. Suppose the response is \(Y_i\), and its conditional mean under the candidate model is \(m_i\). Write \(Y_i=m_i+\varepsilon_i\), where \(\varepsilon_i\) is the deviation from that mean. For a residual plot to support reasoning about conditional means given the predictors used in the fit, a relevant assumption is \(E(\varepsilon\mid X)=0\), where \(X\) denotes the full predictor information used in the model. This is stronger than assuming only \(E(\varepsilon_i\mid x_i)=0\) for a single predictor value when other predictor information is also used. Under the full-information assumption, \(E(Y_i\mid X)=m_i\).

For a particular observation, the residual is the observed response minus its fitted value. Taking its conditional expectation, under that stated assumption, gives:

This identity says that the conditional mean residual compares the conditional mean response with the conditional mean fitted prediction. A group of residuals that tends to be positive is therefore evidence that responses in that region tend to exceed the fitted predictions. The calculation does not make every finite-sample pattern conclusive: residuals vary from observation to observation, and a plot is used to inspect the pattern in the observed data.

For a correctly specified linear mean model, this idea has a useful special case. Let \(Y\) be the vector of responses, \(X\) the full design matrix, and suppose \(E(Y\mid X)=X\beta\). For least squares with an intercept and a full-rank design matrix, the fitted vector is \(HY\), where \(H=X(X^\mathsf{T}X)^{-1}X^\mathsf{T}\). The residual vector is \(e=(I-H)Y\). Since \(HX=X\), its conditional expectation under the specified mean model is:

This result describes the expectation under the model; it does not mean that observed residuals must be scattered perfectly at random, or that an apparent pattern proves the model’s conditional mean is incorrect. It explains why a sustained offset is a diagnostic signal to examine rather than a pattern to ignore.

Why the Overall Average Can Hide an Offset

For least-squares regression with an intercept, the residuals in the fitting sample sum to zero. This is a property of the fitted model, not evidence that predictions are equally good throughout the fitted-value range. Positive residuals in one region can be balanced by negative residuals elsewhere. A plot can reveal this cancellation when a single overall average cannot.

Reading the Plot Carefully

When inspecting a residual-versus-fitted plot for offsets, describe what is visible before proposing an explanation. It is useful to state the response units, the range of fitted values involved, which side of zero the residuals occupy, and whether the pattern is sustained or based on only a few points. For example: “Between fitted values of 30 and 40 units, most residuals are positive, suggesting underprediction in that range.” This is more informative than saying only that “the residual plot looks bad.”

A residual plot is not a formal test with a universal cutoff for how many same-sign points count as an offset. The visual assessment should account for the number and spread of observations: a region supported by many points is generally more informative than a region represented by one or two. Numerical summaries, such as mean residuals in clearly described ranges, can help communicate what the plot shows, but they do not replace the plot or automatically provide a formal test.

Keep the conclusion appropriately narrow. An offset indicates a pattern in prediction errors; it does not establish causation, identify a missing variable, or determine which model should replace the current one. The pattern is a reason to investigate further, using the study context and other diagnostics.

Common Mistake

Wrong reasoning: “The residuals average to zero, so there cannot be systematic prediction errors.”

Correction: In least-squares regression with an intercept, an overall residual average of zero can result from cancellation. Positive residuals in one fitted-value region and negative residuals in another may sum to zero while showing clear regional offsets. Examine the residual plot across the range, not just the overall average.