Residual Runs And Time-Ordered Patterns

Why the order of residuals matters

Residual summaries can look acceptable even when their order contains a warning. A subject may have residuals of both signs and an average close to zero, yet the residuals may be mostly positive early and mostly negative later. That sequence suggests a period of directional mismatch, not merely a few isolated deviations. A run count gives a simple summary of how often the sign changes as observations proceed in time.

The analysis is within subject: sort each subject’s observations by the time relevant to the model, then examine the signs of one consistently defined residual type. A residual is positive when the observation exceeds its prediction under the chosen residual convention, and negative when it falls below. Run patterns are diagnostic clues. They do not, by themselves, identify the cause of a mismatch or establish that a model component is missing.

Definition: A run is a maximal consecutive sequence of the same sign in an ordered series of positive and negative residuals. For example, \(++---++-\) has four runs: \(++\), \(---\), \(++\), and \(-\). A low run count means relatively few sign changes; a high run count means relatively frequent sign changes.

Setting up a within-subject run analysis

For each subject, order the observations by sampling time, using the model-relevant time scale consistently. If multiple observations have the same time, apply a prespecified tie rule or treat the ordering as ambiguous rather than inventing a temporal order. Choose one residual definition and keep it fixed for the sequence. Mixing residual types can change the sign pattern and make the result hard to interpret.

A common reference is the runs test conditional on the number of positive and negative signs. Under its null model, the observed signs are exchangeable over the ordered positions once those counts are fixed. Informally, conditional on seeing \(n_+\) positive and \(n_-\) negative signs, every arrangement of those signs is treated as equally likely. This is a reference model for assessing ordering, not a claim that fitted NLME residuals are exactly independent: parameter estimation, random effects, and the observation process can affect their behavior.

Assumptions and conventions: The usual conditional runs reference requires a meaningful within-subject order, a fixed sign sequence, and exchangeability of the signs under the null. Let \(N=n_+ + n_-\) be the number of nonzero residuals and \(R\) the observed number of runs. Exact conditional inference is available by enumerating sign arrangements. A normal approximation is generally used only when the positive and negative counts are not small. Zero residuals are omitted in the standard test, with the resulting sequence closed across the omitted positions; report how many were omitted. If zeros are frequent, this convention can conceal information, so show the original residual-time plot as well.

The usual run count concerns the number of transitions between adjacent signs. Its conditional expectation and variance are

E(R)=1+\frac{2n_+n_-}{N}, \qquad \operatorname{Var}(R)= \frac{2n_+n_-\left(2n_+n_- - N\right)} {N^2(N-1)}.

These formulas require both signs to occur and \(N>1\). The expected number is often not an integer. For a two-sided check, a run count far below or far above its expectation can be unusual under the conditional reference model.

Why the expected run count has this form

Write \(I_i=1\) when positions \(i\) and \(i+1\) have different signs, and \(I_i=0\) otherwise, for \(i=1,\ldots,N-1\). Each sign change starts a new run, so \(R=1+\sum_{i=1}^{N-1} I_i\). Under uniform allocation of the fixed signs, the chance of a positive followed by a negative is \((n_+/N)(n_-/(N-1))\). The chance of a negative followed by a positive is the same, so

E(I_i)=\frac{2n_+n_-}{N(N-1)} \quad\Longrightarrow\quad E(R)=1+(N-1)E(I_i) =1+\frac{2n_+n_-}{N}.

The variance formula accounts for the fact that adjacent transition indicators can share a sign position, so they are not all independent. It is the variance of \(R\) under the same fixed-count, uniformly permuted sign reference. The derivation above justifies the expectation; it does not make independence of fitted residual signs a guaranteed property of an NLME model.

Calculating and interpreting run counts

When both sign counts are adequate for a normal approximation, a continuity-corrected statistic for unusually few runs is

Z_{\mathrm{cc}}= \frac{R-E(R)+0.5}{\sqrt{\operatorname{Var}(R)}}.

For unusually many runs, use \(R-E(R)-0.5\) in the numerator. For a two-sided test, the continuity correction moves the observed count toward the expectation before standardizing, and the approximate p-value is \(2\Phi(-|Z_{\mathrm{cc}}|)\), where \(\Phi\) is the standard normal cumulative distribution function. Use an exact conditional calculation instead when the sequence is short or the approximation is doubtful.

Worked Example: A Long Positive Block Followed by a Negative Block

Suppose a hypothetical subject has 40 nonzero residuals in time order: the first 20 are positive and the next 20 are negative. Thus \(n_+=20\), \(n_-=20\), \(N=40\), and the sequence has \(R=2\) runs. Assess whether this is unusually few under the conditional runs reference, using a normal approximation.

The expected count is

E(R)=1+\frac{2(20)(20)}{40}=21.

The variance and standard deviation are

\operatorname{Var}(R)= \frac{2(20)(20)\left(2(20)(20)-40\right)} {40^2(39)} = \frac{608000}{62400} \approx 9.7436, \qquad SD(R)\approx \sqrt{9.7436}=3.1215.

Because \(R=2\) is below its expectation, use the low-count continuity correction:

Z_{\mathrm{cc}}= \frac{2-21+0.5}{\sqrt{608000/62400}} = \frac{-18.5}{3.1215} \approx -5.927.

The two-sided normal approximation is \(p=2\Phi(-|Z_{\mathrm{cc}}|)\approx 3.1\times10^{-9}\). This is strong evidence against exchangeable ordering of these signs under the stated reference model. In context, the two long blocks are a conspicuous pattern of sustained directional residuals. The result does not prove why they occurred: inspect the residual-time display and consider whether the model or data have a time-related feature not represented in the analysis.

Worked Example: An Exact Test for a Short Sequence

A second hypothetical subject has the sign sequence \(++++----\). There are four positive and four negative signs, with \(N=8\) and \(R=2\). Since the sequence is short, use an exact conditional test rather than relying on a normal approximation.

There are \(\binom{8}{4}=70\) equally likely arrangements of four positive and four negative signs under the conditional null. Here \(E(R)=1+2(4)(4)/8=5\). For a two-sided test based on distance from that expectation, counts at least as far away as \(R=2\) satisfy \(|R-5|\geq 3\), which means \(R\leq2\) or \(R\geq8\).

Exactly two arrangements have \(R=2\): all four positives followed by all four negatives, or the reverse. Exactly two arrangements have \(R=8\), the two alternating sequences. Thus

p=\frac{2+2}{70}=\frac{4}{70}\approx 0.0571.

At a 0.05 significance level, this exact test does not reject the conditional reference model. That result is not proof that the sequence is random or that the model is adequate. The plot still shows a clear block pattern, and the test has limited resolution with only eight observations.

Compare subjects before making a population claim

The runs test described here is defined for an ordered sequence with fixed sign counts. In an NLME analysis, the natural unit for this diagnostic is usually the subject: each person has their own observation times, residual count, and possible time pattern. A pooled sequence formed by concatenating subjects is generally not a valid within-subject sequence. The transition between the last observation of one person and the first of another does not represent elapsed time for either person.

Worked Example: Distinct Patterns Across Three Subjects

Consider three hypothetical subjects, each with six residual signs and three signs of each kind:

Subject A: \(+++---\), so \(R_A=2\).
Subject B: \(-+-+-+\), so \(R_B=6\).
Subject C: \(+--++-\), so \(R_C=4\).

For each subject, the fixed-count expectation is \(1+2(3)(3)/6=4\). Subject A has relatively few runs, Subject B has frequent sign changes, and Subject C has exactly the expected count. These counts are descriptive here; six observations per person are too few to make a useful large-sample normal approximation, and the examples do not establish a population-level pattern.

If these three sequences are concatenated in the order shown, the boundary from A to B joins two negative signs, and the boundary from B to C joins two positive signs. Both boundaries merge runs, giving \(R_{\mathrm{pooled}}=2+6+4-2=10\). That pooled count treats artificial subject boundaries as if they were adjacent sampling times. It therefore answers a different, generally meaningless question. Examine within-subject sequences and summarize their patterns with the sampling design in view.

Reading patterns and choosing follow-up checks

Few runs can reflect a sustained positive or negative period, a transition in the direction of bias, or another temporal feature that the fitted predictions do not capture. Many runs can indicate frequent alternation of signs. Neither pattern identifies a mechanism. A few runs may be driven by one long block, while a test statistic alone does not show when the block occurred or whether it aligns with a dose, a change in sampling interval, or a prediction-time feature.

Use the time-ordered residual display alongside the count. Check that observations were sorted on the appropriate time scale, that the residual definition is consistent, and that zeros and tied times were handled transparently. Look for a pattern that repeats across subjects, while remembering that a few highly sampled subjects can dominate a pooled summary. A runs test is a compact supplement to visual inspection, not a substitute for it.

Approach	What it contributes	Main limitation
Time-ordered residual plot	Shows where signs cluster and when a directional change occurs.	Visual interpretation can be subjective, especially with many observations.
Conditional runs test	Summarizes whether sign changes are unusually few or frequent, given sign counts.	Discards residual magnitudes and most information about timing between observations.
Residual summaries without order	Describe overall location or spread of residuals.	Can miss a sequence of positive residuals followed by negative residuals.

A flagged sequence motivates targeted checks for omitted temporal structure, such as a time-dependent prediction bias or an unrepresented change in the observation process. It is not itself evidence for a particular pharmacokinetic mechanism, nor does it replace assessment of other aspects of residual behavior. Repeated testing of many subjects also increases the chance of finding at least one unusual result by chance; treat isolated flags as prompts for review, and seek coherent patterns rather than selecting a model change from a single p-value.

Common mistakes and limitations

Ignoring order. Counting positive and negative residuals alone cannot reveal runs. Sort within each subject before counting.
Concatenating subjects. A boundary between subjects is not a temporal transition. Do not treat it as one in a within-subject runs test.
Calling a non-significant test proof of adequacy. A large p-value means the sequence is not unusually ordered by this test at the chosen threshold; it does not establish exchangeability or model correctness.
Interpreting a small p-value as a diagnosis. The test flags a departure from its reference assumptions; it cannot tell whether the cause is a missing time effect, data issue, or another form of misspecification.
Overlooking the reference assumptions. Fitted residual signs need not be perfectly exchangeable. Sparse data, parameter estimation, and unequal observation schedules can make the conditional test a rough diagnostic reference rather than a calibrated model-based test.

The main value of residual runs is practical: they help detect persistent stretches of directional error that an unordered summary could hide. The strength of the conclusion should match the evidence: a run pattern is a reason to inspect and investigate, not a causal explanation.

Check Your Understanding

Answer each question, then compare with the solutions below.

How many runs are in the sequence \(++-+---\), and what does a low run count generally indicate?
A sequence has \(n_+=5\), \(n_-=3\), and \(N=8\). Calculate \(E(R)\).
Why is an exact conditional test preferable to a normal approximation for a very short sign sequence?
What is wrong with joining the last residual sign for one subject directly to the first sign for another?

Solutions. (1) The runs are \(++\), \(-\), \(+\), and \(---\), so \(R=4\). A low count relative to its reference expectation indicates fewer sign changes, often a clustered or sustained sign pattern. (2) \(E(R)=1+2(5)(3)/8=1+30/8=4.75\) runs. (3) With few observations, the conditional distribution is discrete and may not be well approximated by a normal distribution; enumerating arrangements gives the exact conditional tail probability. (4) Those observations belong to different people, so the apparent boundary is not an adjacent point in either person’s time series and creates an artificial transition or run merge.