Residual Autocorrelation Within Individuals

Why Residual Order Matters

A residual plot can look acceptable when points are viewed without their order, yet show a persistent sequence when the observations are connected in time. For example, several positive residuals followed by several negative residuals may indicate that the model misses a time-dependent feature, or that nearby observations share error. In a nonlinear mixed-effects model (NLME), this question is naturally asked within each individual: do residuals from nearby observations tend to move together after accounting for the model?

The diagnostic is about dependence in the residual sequence, not simply whether residuals have a nonzero average or an incorrect spread. A residual autocorrelation summary can help identify a pattern worth investigating, but it does not by itself establish its cause or prove that a particular error model is correct.

Definition: Let $r_{ij}$ denote the chosen residual for individual $i$ at the $j$th observation in time order, where $j=1,\ldots,n_i$. A lag-one residual association compares successive residuals $r_{ij}$ and $r_{i,j+1}$ within the same individual. A common descriptive summary is the centered lag-one sample autocorrelation $$ \widehat{\rho}_{i}(1)= \frac{\displaystyle\sum_{j=1}^{n_i-1}(r_{ij}-\bar r_i)(r_{i,j+1}-\bar r_i)} {\displaystyle\sum_{j=1}^{n_i}(r_{ij}-\bar r_i)^2}, \qquad \bar r_i=\frac{1}{n_i}\sum_{j=1}^{n_i}r_{ij}. $$

This ratio is a descriptive statistic for one ordered sequence. Positive values indicate that successive residuals tend to be on the same side of their individual mean; negative values indicate a tendency to alternate. Values near zero indicate little measured lag-one association in that sequence, not proof of independence. For short sequences, the value can be highly variable.

The residual type must be stated and kept consistent. For example, a diagnostic can use a chosen conditional or standardized residual, but it should not silently combine different residual definitions. Standardization can make residual magnitudes more comparable when their prediction variances differ, but it does not guarantee independence. Residuals also inherit features of parameter estimation and individual fitting.

Working Assumptions and a Centering Detail

A simple reference case is a sequence of independent residuals with mean zero and common variance $\sigma^2$. This is a working null model for understanding the summary, not an assertion that fitted NLME residuals satisfy it exactly. In this reference case, the expected product of two distinct raw residuals is zero. However, the centered products in the displayed sample autocorrelation do not have expectation zero because the same sample mean is subtracted from every observation.

Important distinction: Under independent residuals with $E(r_{ij})=0$ and $\operatorname{Var}(r_{ij})=\sigma^2$, for distinct observations $j\ne k$, $$ E(r_{ij}r_{ik})=0, \qquad E\big[(r_{ij}-\bar r_i)(r_{ik}-\bar r_i)\big]=-\frac{\sigma^2}{n_i}. $$ Thus, a negative expected centered cross-product is induced by estimating and subtracting the individual sample mean. It is not, by itself, evidence of negative serial dependence.

To see the second result, suppress the individual index and write $n=n_i$. Independence and the common variance imply $E(r_jr_k)=0$ for $j\ne k$, $E(r_j\bar r)=\sigma^2/n$, and $E(\bar r^2)=\sigma^2/n$. Expanding the centered product gives

\begin{aligned} E[(r_j-\bar r)(r_k-\bar r)] &=E(r_jr_k)-E(r_j\bar r)-E(r_k\bar r)+E(\bar r^2)\\ &=0-\frac{\sigma^2}{n}-\frac{\sigma^2}{n}+\frac{\sigma^2}{n}\\ &=-\frac{\sigma^2}{n},\qquad j\ne k. \end{aligned}

Consequently, neither a centered adjacent-product sum nor the ratio built from it has an exact zero-centered null distribution just because the raw residuals are independent. The expectation of a ratio is not generally the ratio of its expectations, either. For short records, do not apply a large-sample rule of thumb mechanically or treat a small negative sample autocorrelation as an automatic model failure. If formal calibration is needed, use a method that reflects the fitted model, residual construction, sampling schedule, and parameter estimation, such as simulation under the fitted model.

Worked Example: Calculating a Within-Individual Summary

Suppose a hypothetical individual has four time-ordered standardized residuals $-1.0,-0.5,0.5,1.0$. The mean is $\bar r_i=0$. The three adjacent products are $(-1.0)(-0.5)=0.50$, $(-0.5)(0.5)=-0.25$, and $(0.5)(1.0)=0.50$, which sum to $0.75$.

The denominator is the sum of squared deviations from the mean: $1.00+0.25+0.25+1.00=2.50$. Therefore, $$ \widehat{\rho}_{i}(1)=\frac{0.75}{2.50}=0.300. $$

This sequence has a positive descriptive lag-one summary, but only four residuals are available. The result is a prompt to inspect the ordered values and the model context, not a precise estimate of a population correlation or a stand-alone significance test. The raw adjacent products also sum to $0.75$ here because the sample mean happens to be zero; in general, raw and centered products differ.

Inspect Sequences Before Summarizing Them

A time-ordered residual plot is an essential partner to the numerical summary. Plot each individual's residuals against observation order or actual time, retaining subject boundaries. Look for sustained runs, alternating signs, or smooth waves. A single summary compresses these patterns: distinct sequences can have similar lag-one values, and one unusual pair can matter greatly when there are very few observations.

Where there are enough observations, calculate summaries separately by individual and examine their distribution, rather than treating all observations as one long sequence. The number of adjacent pairs is $n_i-1$, so an individual with many observations contributes more pairs than one with few observations. Report how summaries are combined if an across-individual summary is used: an unweighted average gives each individual equal weight, whereas pooling adjacent pairs gives more weight to individuals with longer records. Neither choice is universally correct; they answer different descriptive questions.

Worked Example: Why a Pooled Sequence Can Mislead

Consider two hypothetical individuals. Individual A has residuals $1,1,-1,-1$, and individual B has residuals $-1,-1,1,1$. Each individual has mean zero and denominator $4$. For A, the adjacent products sum to $1-1+1=1$, so $\widehat{\rho}_{A}(1)=1/4=0.250$. For B, they also sum to $1-1+1=1$, so $\widehat{\rho}_{B}(1)=0.250$. The equal-weight average of these two within-individual summaries is $0.250$.

If the values are concatenated into a single sequence, the result is $1,1,-1,-1,-1,-1,1,1$. Its seven adjacent products sum to $1-1+1+1+1-1+1=3$, and its sum of squared deviations from the pooled mean zero is $8$. The pooled ratio is $3/8=0.375$.

The pooled calculation includes the product between the final residual of A and the first residual of B. Those observations are not successive measurements on one individual. This boundary contributes a spurious adjacent pair and changes the result. In real data, differences in individual means or residual scales can further distort a pooled summary. Preserve subject identity and calculate within-individual association before deciding how to summarize across individuals.

Unequal Time Gaps and Interpretation

Lag one means “next observation,” not necessarily “one fixed unit of time later.” If measurements are taken at irregular intervals, the adjacent pairs can represent very different elapsed times. A positive association over a one-hour gap and the same association over a three-day gap need not be comparable. Always retain actual times alongside observation order.

Worked Example: Adjacent Records With Unequal Gaps

Suppose one hypothetical individual has observations at times $0,1,4,5$ hours and residuals $1.0,0.8,0.5,0.4$. Their mean is $\bar r_i=0.675$. The centered values are $0.325,0.125,-0.175,-0.275$. The adjacent centered products sum to $0.325(0.125)+0.125(-0.175)+(-0.175)(-0.275)=0.066875$. The denominator is $0.325^2+0.125^2+(-0.175)^2+(-0.275)^2=0.2275$. Thus the lag-one summary is $0.066875/0.2275=0.294$ (approximately).

This single number treats each of the three adjacent pairs equally, although their time gaps are $1,3,$ and $1$ hours. To illustrate why elapsed time may matter, suppose a hypothetical continuous-time correlation pattern were $\exp(-0.2\Delta t)$, where $\Delta t$ is the gap in hours. The corresponding correlations would be $\exp(-0.2)=0.819$ for a one-hour gap and $\exp(-0.6)=0.549$ for a three-hour gap. These are values from the illustrative pattern, not estimates from the four residuals. A lag-one summary alone cannot assess that time-decay pattern.

What Autocorrelation Can and Cannot Diagnose

A persistent positive sequence can arise from serially correlated observation error, but it can also reflect a misspecified time course, an omitted time-varying predictor, an incorrect event time, or a residual construction affected by fitting. Negative or alternating patterns can also arise from model structure, sampling, or short-sequence variability. Autocorrelation is therefore a signal to investigate, not a unique explanation.

If correlated residuals are treated as independent, the diagnostic may overstate how much independent information the sequence contains. With positive correlation, neighboring observations tend to repeat information, so calculations that assume independence can understate uncertainty for some summaries. The effect is not invariably an inflation: negative dependence and the particular statistic matter. In a diagnostic setting, the central concern is that reference judgments based on independent residuals may be poorly calibrated.

Use the residual definition appropriate to the question, check its time-ordered plot, inspect within-individual summaries, and compare the pattern with the design and model predictions. A residual autocorrelation summary is not a substitute for evaluating residual spread, time trends, or population and individual fit. Earlier material on residual runs and time-ordered patterns describes another way to notice order-related structure; autocorrelation adds a numerical measure of adjacent association, with the centering and sampling caveats described here.

Approach	What it contributes	Main caution
Time-ordered plot within individual	Shows runs, waves, alternation, and unusual transitions.	Visual judgment is difficult with many individuals or very short records.
Within-individual lag-one summary	Compresses adjacent residual association into a number.	Short sequences, centering, and unequal gaps affect interpretation.
One pooled sequence across individuals	May appear simple to calculate.	Creates false across-subject adjacency and can mix different subject patterns.
Model-based simulation reference	Can calibrate a diagnostic under a specified fitted model and design.	Conclusions depend on the model and simulation reproducing the fitting process.

A useful diagnostic report states which residual was used, how observations were ordered, whether time gaps were equal, how individual summaries were combined, and whether any formal reference came from simulation. The limitation is fundamental: no single lag-one statistic describes all forms of dependence, and a value near zero does not rule out longer-lag or nonlinear patterns.

Common mistake: “The residuals are independent with mean zero, so the expected centered adjacent products are zero.” This is incorrect when the same sample mean is subtracted from each residual. For distinct observations, the expected centered product is $-\sigma^2/n_i$; the raw product has expectation zero under the stated independence assumptions. Interpret the centered summary with its finite-sample behavior in mind.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

Under independent, zero-mean residuals with common variance $\sigma^2$, what is $E(r_{ij}r_{ik})$ for distinct observations, and what is $E[(r_{ij}-\bar r_i)(r_{ik}-\bar r_i)]$?
An individual has residuals $-1,0,1$. Calculate the centered lag-one summary using the definition in this tutorial.
Why should residual sequences from different individuals not be concatenated before calculating a within-individual lag-one summary?
What does “lag one” mean when observation times are irregular, and what should be checked in that setting?
Does a positive residual autocorrelation prove that the observation-error model is wrong? Explain.

Solutions. (1) Independence and zero means give $E(r_{ij}r_{ik})=0$. Subtracting the common sample mean induces $E[(r_{ij}-\bar r_i)(r_{ik}-\bar r_i)]=-\sigma^2/n_i$ for distinct $j,k$, as shown by expansion above. (2) The mean is zero. The adjacent products are $(-1)(0)=0$ and $0(1)=0$, so the numerator is zero; the denominator is $1+0+1=2$. The summary is $0/2=0$. (3) Concatenation treats the last observation of one person and the first of another as adjacent measurements, although they are not a within-person pair. That artificial boundary can change the summary. (4) It means the next recorded observation, regardless of elapsed time. Check actual gaps; unequal gaps mean the adjacent pairs do not represent a common time interval. (5) No. Positive association is a diagnostic signal that may reflect correlated error, model or predictor misspecification, timing, or residual construction. Further checks are needed to distinguish explanations.