Checking Residual Dependence by Data Order

Why the Order of Observations Can Matter

Residual plots are often examined against fitted values or predictor values, but the order in which observations were collected can also carry information. Measurements made day after day, patients recruited one after another, or parts produced consecutively may share conditions that change gradually or repeat. If nearby observations tend to have related errors, a model that describes the average relationship may still miss a feature of how the data were generated.

An ordered residual plot places each residual at its observation number or collection time, preserving the original sequence. Random-looking variation around zero is consistent with independent errors. Long runs on one side of zero, a gradual rise or fall, or a repeating cycle are possible signs that successive errors are related. Such patterns are reasons to investigate the model and data collection process, not proof by themselves that dependence exists.

Model and Diagnostic

For a simple linear regression, write the response for observation \(i\) as

Here \(\beta_0+\beta_1x_i\) is the model's mean response at predictor value \(x_i\), and \(\varepsilon_i\) is the error: the deviation of the response from that mean. For the usual regression model, errors have mean zero, a common variance \(\sigma^2\), and are independent across observations. The independence condition concerns the errors, which are not directly observed. Residuals \(e_i=y_i-\hat{y}_i\) are estimates of those errors, calculated after fitting the regression line.

To check for possible dependence, plot \(e_i\) vertically against the actual collection time or observation sequence number horizontally. Keep the sequence as it was recorded. A substantive order is important: if the rows were sorted by response or rearranged arbitrarily, the resulting plot would not diagnose the original process. When the order has no meaningful interpretation, a pattern in an arbitrary row order generally has no such diagnostic meaning.

Look for departures from an unstructured scatter centered near zero. A run is a sequence of consecutive residuals on the same side of zero. A trend is a sustained upward or downward movement. A cycle is a recurring rise-and-fall pattern. These visual clues may reflect neighboring errors being positively related, an omitted time-related change, a repeated process, or another feature that the fitted model does not capture.

This check asks a different question from residuals versus fitted values or predictor values. A curved pattern against a predictor can indicate that a straight-line mean relationship is inadequate; a fan shape can indicate unequal spread. An ordered pattern instead raises a question about how errors relate from one observation to the next. More than one issue can occur at the same time.

Recognizing Runs and Cycles

A long run can be informative even if residuals do not move steadily. If many consecutive values are positive, the model has repeatedly underpredicted the response; if they are negative, it has repeatedly overpredicted. A cycle differs from a one-way trend because residuals repeatedly move above and below zero. Either kind of structure can arise when consecutive errors share influences that are absent from the model.

Not every uneven-looking sequence is persuasive evidence of dependence. Short runs and isolated reversals can occur by chance. Interpretation should consider the length and strength of the pattern, the number of observations, and what is known about how data were collected. A plot is a diagnostic tool for spotting structure; it is not a formal test with a stated probability of false alarm.

Why Dependence Matters

Dependence matters because the amount of variation in a collection of errors depends not only on the variation of each error, but also on how errors co-vary. Let \(S\) be the sum of four errors. The variance of a sum is the sum of the individual variances plus twice each distinct pairwise covariance:

If the four errors are independent and each has variance \(\sigma^2\), all pairwise covariances are zero, so \(\operatorname{Var}(S)=4\sigma^2\). If only neighboring pairs have a common positive covariance \(c\), there are three such pairs, giving

The variance is then larger than \(4\sigma^2\) when \(c>0\). This derivation shows why treating related errors as independent can misrepresent uncertainty: the independence-based calculation leaves out the covariance terms. An ordered residual pattern is a practical warning that those terms may matter, not a calculation of them.

The order must be interpreted in context. If observations are days, inspect calendar time; if they are production items, inspect production sequence. A visible trend might point to a shift in conditions, while a cycle might correspond to a repeating operating schedule. These are possibilities to investigate, not conclusions that can be read off the plot alone. Rearranging the observations before plotting can hide the pattern; sorting by residual size can create an apparent order pattern that was never present.

Common Mistakes

• Calling a pattern proof of dependence. A trend or cycle is evidence that the independence assumption may be questionable, not proof of a specific cause. Describe the pattern and consider how the data were collected.
• Assuming a messy scatter proves independence. A plot with no obvious structure does not guarantee that errors are independent, particularly with few observations. State that no clear pattern was seen.
• Plotting in an arbitrary or changed order. Row order is useful only if it preserves a meaningful sequence. Use collection time or the original observation order, and do not sort the residuals first.
• Confusing dependence with changing spread or curvature. An ordered pattern concerns the relationship among errors across observations. Check the shape and spread of residuals in their appropriate plots as separate questions.
• Interpreting residuals as the true errors. Residuals are calculated from a fitted model. They help reveal possible structure in the errors, but they are not the unobserved errors themselves.