Residual Drift Across Data Collection Order

Why Plot Residuals in Collection Order?

A regression diagnostic is often made by plotting residuals against fitted values or against predictors. But the sequence in which measurements were collected can matter too. A measuring device may gradually change its calibration, a procedure may be updated partway through data collection, or observations from one production batch may share a common feature. These changes can leave a recognizable pattern when residuals are plotted in collection order, even if a plot against a predictor does not make the pattern obvious.

Collection order is not automatically a predictor, and a pattern against order does not by itself identify its cause. It is a prompt to investigate whether the data-generating process changed, whether a relevant time- or batch-related factor is missing from the model, or whether the apparent pattern is simply irregular variation. The diagnostic is most useful when the order represents a meaningful sequence, such as measurement time, enrollment order, or production run.

Definition: For observation \(i\), the residual is \(e_i=y_i-\hat{y}_i\), where \(y_i\) is the observed response and \(\hat{y}_i\) is its fitted value from the regression model. A residual-versus-order plot places the residuals \(e_i\) on the vertical axis and the observations’ collection positions, times, or recorded sequence numbers on the horizontal axis. The horizontal reference line is at residual zero: points above it have observed responses greater than their fitted values, and points below it have observed responses smaller than their fitted values.

Model Conditions and What the Plot Checks

Suppose a regression model represents the response as a specified mean function of its predictors plus an error. For the usual interpretation of residual patterns, the model’s conditional mean should be correctly represented: after accounting for the included predictors, the expected error should be zero. The standard regression framework also commonly assumes constant error variance and independent errors; normality is an additional condition used for some forms of inference, not a requirement for drawing this diagnostic.

For an order plot to be informative, the order must be recorded accurately and should reflect the sequence relevant to the question. If the data have been sorted by response or by a predictor after collection, that sorted order should not be mistaken for collection order. When there is no systematic change in the process and the mean model is adequate, residuals should fluctuate around zero without a sustained order-related structure. They need not alternate perfectly, and a few large residuals do not establish a trend.

A gradual rise or fall suggests that residuals change systematically across the sequence. A sudden step suggests a possible change point, such as a protocol adjustment or a new batch. Repeated offsets that align with batch boundaries suggest a batch-related difference. These are visual clues, not diagnoses: order may be associated with an omitted predictor, a process change, dependence among nearby observations, or a chance pattern.

How to Construct and Read the Plot

Preserve the actual order.
Use the original collection sequence or time variable, not an order created by sorting on a response or predictor.

Calculate the fitted values and residuals.
For each observation, compute \(e_i=y_i-\hat{y}_i\) using the same fitted regression model.

Plot against order with a zero line.
Keep the horizontal scale in collection order and mark residual zero so above- and below-fit observations are easy to distinguish.

Describe the structure before explaining it.
Note whether the points show a sustained rise or fall, a step, recurring batch offsets, or no clear organization. Then compare the pattern with collection records and the predictors used in the model.

Connecting points in order can make sustained changes easier to see, but the line between points is only a visual aid; it does not imply that the response was observed continuously between measurements. Label meaningful batch boundaries or known procedural changes where possible. Avoid treating every short run of positive or negative residuals as a meaningful signal, especially in a small sample.

Worked Example: A Gradual Increase Across the Sequence

Consider eight hypothetical observations collected in order. An intercept-only regression model gives a fitted value of 50 response units for every observation. The observed responses are 46, 47, 48, 49, 51, 52, 53, and 54 response units. Calculate the residuals and assess the order pattern.

For each observation, subtract its fitted value from its observed response. Thus the residual sequence is \((-4,-3,-2,-1,1,2,3,4)\) response units. For example, the first residual is \(46-50=-4\), and the eighth is \(54-50=4\). The sequence moves from negative residuals to positive residuals, which is a clear gradual rise in this hypothetical data set.

The residuals sum to zero: \(-4-3-2-1+1+2+3+4=0\). That balance does not remove the visible pattern. To summarize its linear direction, regress the residuals on collection position \(i\). The mean order is \(\bar{i}=4.5\), and the slope is

\frac{\sum_{i=1}^{8}(i-\bar{i})e_i}{\sum_{i=1}^{8}(i-\bar{i})^2} = \frac{49}{42} \approx 1.167 \quad\text{response units per position}.

The numerator is \(14+7+3+0.5+0.5+3+7+14=49\); the denominator is \(12.25+6.25+2.25+0.25+0.25+2.25+6.25+12.25=42\). This descriptive slope confirms the upward direction. It is not, by itself, a formal test or proof that the process changed. The pattern warrants checking whether collection order corresponds to a changing condition or a predictor not represented in the model.

A Short Justification: Zero-Sum Residuals Can Still Drift

For ordinary least squares with an intercept, the residuals sum to zero. This fact is sometimes mistakenly taken to mean that residuals cannot show a systematic pattern. The zero-sum property constrains their overall average, not their order.

To see why, write the residual sum of squares as \(\sum_{i=1}^{n}(y_i-\hat{y}_i)^2\). In an intercept model, changing the fitted intercept shifts every fitted value by the same amount. At the least-squares minimum, the derivative of the residual sum of squares with respect to the intercept is zero:

\frac{\partial}{\partial \hat{\beta}_0} \sum_{i=1}^{n}(y_i-\hat{y}_i)^2 = -2\sum_{i=1}^{n}(y_i-\hat{y}_i) = -2\sum_{i=1}^{n}e_i = 0.

Therefore, \(\sum_{i=1}^{n}e_i=0\). But this equation says nothing about whether negative residuals occur early and positive residuals late. In the preceding example, the early negative values balance the later positive values exactly. Their order still makes the sequence unmistakably upward. Always inspect the arrangement of residuals, not just their total.

Worked Example: An Abrupt Shift After the First Four Measurements

Suppose eight hypothetical measurements are collected in order and an intercept-only model fits 100 response units to each. The residuals are \((-3,-2,-1,-2,2,1,2,3)\) response units. The observed responses are therefore 97, 98, 99, 98, 102, 101, 102, and 103 response units. The residuals sum to zero, so 100 is also the mean response and is the fitted value from this intercept-only model.

The mean residual for positions 1 through 4 is \( (-3-2-1-2)/4=-2 \) response units. The mean for positions 5 through 8 is \( (2+1+2+3)/4=2 \) response units. The change between these two segment means is \(2-(-2)=4\) response units. The plot would show residuals below zero in the first segment and above zero in the second, consistent with an abrupt upward shift near the boundary.

That description does not tell us why the shift occurred. If a new instrument, operator, protocol, or source of observations began after measurement four, the residual change may reflect that process change. If the observations also changed in a predictor that the model omitted, the pattern may instead point to an unrepresented predictor relationship. Check the collection log and the predictor values before choosing between those explanations.

Worked Example: Repeated Offsets Across Three Batches

A hypothetical process produces nine observations in three consecutive batches of three. An intercept-only model fits 100 response units. The ordered residuals are \((-3,-2,-1,-1,0,1,1,2,3)\), so the observed responses are 97, 98, 99, 99, 100, 101, 101, 102, and 103 response units. Each batch has residuals that sum to zero across all three batches combined.

The first batch mean residual is \((-3-2-1)/3=-2\) response units. The second is \((-1+0+1)/3=0\), and the third is \((1+2+3)/3=2\). Thus the batch means move from \(-2\) to 0 to \(2\) response units. The plot shows a sequence of batch-level offsets, along with variation within each batch.

Because the batches occurred in order, these data alone cannot separate a batch effect from a gradual process change that happened to progress across the same sequence. Batch labels and process records matter. If similar observations from the same batch tend to share an offset, batch membership may be relevant to the model. If records instead show a continuous calibration change, the explanation is different. In either case, the plot identifies a question for investigation; it does not establish causation.

Interpreting the Evidence Carefully

The key distinction is between a pattern in residuals and a cause of that pattern. Residuals in collection order can suggest a changing mean or dependence over the sequence. They cannot establish that time itself caused the response to change, nor can they identify an omitted variable without additional evidence.

Consider the context before revising the model. Ask whether collection order is related to an included predictor, whether any equipment or protocol changes occurred, whether observations were grouped into batches, and whether the recorded order reflects the order in which measurements were actually made. A candidate time or batch variable may help explain the pattern, but it should have a defensible substantive interpretation. Adding a term solely because it makes the plot look flatter can conceal an important process problem or produce a misleading model.

Also compare the order plot with other residual displays and the known data-collection design. A curved residual pattern against a predictor and a trend over collection order suggest different questions, even though they can occur together. A few isolated large residuals are not the same as a sustained run, and a changing vertical spread is not the same as a changing average. Describe the feature you see before deciding what analysis is appropriate.

Common Mistakes

Assuming a zero residual sum rules out drift. The sum constrains the overall balance, not the sequence. Early negative residuals can balance later positive residuals.
Calling collection order a predictor without justification. Order is initially a diagnostic axis. A relationship with time or batch should be interpreted in light of the data-collection process.
Treating a plot as proof of a process change. A visual pattern is evidence worth investigating, not a causal conclusion or formal test.
Using an artificial order. Sorting observations by response or predictor can create a pattern that was not present in collection order.
Confusing shifts in average with shifts in spread. A run of residuals above or below zero concerns their location; increasing scatter concerns their variability. Describe these features separately.

Key takeaway: Plot residuals against their genuine collection order to look for gradual drift, abrupt shifts, or batch offsets. Treat any structure as a diagnostic clue, then use context and the other predictors to investigate whether it reflects a changing data process, a missing mean relationship, or another feature of the data.

Check Your Understanding

Answer each question using the definitions and reasoning above.

An observation has response 28 units and fitted value 31 units. What is its residual, and how should it be described relative to the fitted value?
Why does an ordinary least-squares model with an intercept have residuals that sum to zero, and why does that not rule out a trend in collection order?
Residual means are \(-1.5\) response units in the first half of a sequence and \(1.5\) in the second half. State what the pattern suggests and name one additional source of information to check.
Residual means differ across three consecutive batches. Give two distinct explanations that may be consistent with this pattern.
Why should observations not be sorted by their response before making a residual-versus-collection-order plot?