From a Diagnostic Pattern to a Testable Explanation
A residual pattern is evidence that a model may be inadequate, not a diagnosis of which component is wrong. A wave over time, for example, could reflect an incorrect absorption shape, a distribution process, time-varying clearance, or a poorly chosen residual-error model. The practical task is to separate these explanations using predictions they make, rather than adding model complexity until a plot looks smoother.
A disciplined workflow has four linked parts: describe the discrepancy precisely, rank explanations before refitting, change one component at a time, and check whether the original discrepancy resolves. The workflow is not a mechanical search for the model with the smallest objective function. It is a way to make each model change answer a specific diagnostic question.
Let \(Y_{ij}\) be the observed concentration for individual \(i\) at observation \(j\), and let \(\hat Y_{ij}\) be the corresponding model prediction. The prediction may be a population prediction or an individual prediction; these answer different diagnostic questions, so the choice must be stated and kept consistent during a comparison. A standardized residual can be written as
Here, positive residuals indicate observations above the selected prediction, and negative residuals indicate observations below it. Standardization accounts for the model's assumed observation variability, but it does not make residuals independent or guarantee that their distribution is well behaved. Residuals are most useful when their pattern is examined against scientifically relevant features: time since dose, predicted concentration, dose level, occasion, covariates, and individual.
Set Up the Comparison Before Changing the Model
Before fitting an alternative, verify that the pattern is not explained by data handling: check dose times, sample times, units, missing or below-quantification-limit records, and the prediction used to calculate the residuals. Then summarize the signal in a form that can be compared. Useful summaries include the direction of residuals by time region, the time of peak concentration, the terminal slope, or a prespecified measure of concentration-dependent spread.
The pattern should be described without embedding a preferred explanation. “Residuals are mostly positive before the peak and mostly negative around it” is more informative than “absorption is wrong,” because it records what the data show while leaving competing causes open. Consider whether the pattern is widespread or driven by a few individuals. A population-wide pattern suggests a different issue from a pattern confined to one occasion or a small subgroup.
Parameter estimates add clues, not verdicts. A parameter at a boundary, a very large uncertainty, an implausible estimate, or a strong change in correlated parameters can indicate that the model is asking its components to compensate for one another. But a parameter can be imprecise because the study design contains little information about it. Sparse sampling and shrinkage can also make individual-level diagnostics look deceptively mild. An estimate should therefore be interpreted alongside the sampling design and residual pattern.
Before fitting alternatives, write down a short list of plausible explanations and the feature each predicts. For example, an absorption explanation might predict a change in the early rising limb or peak timing; a missing distribution component might predict a systematic terminal-phase discrepancy; and a residual-error explanation might predict changing residual spread with concentration rather than a consistent time-ordered shift in residual signs. These predictions are discriminating only to the extent that the data include observations capable of showing them.
Worked Example: Ranking Explanations From a Residual Pattern
Hypothetical data. For one oral-dose analysis, standardized residuals from the same diagnostic calculation were summarized in four time regions. The regional means were \(+1.10\) over 0–1 hour (\(n=4\)), \(-0.90\) over 1–3 hours (\(n=5\)), \(+0.10\) over 3–8 hours (\(n=6\)), and \(+0.95\) over 8–24 hours (\(n=5\)). Assume the records and dose times have been checked and no single individual accounts for the pattern.
The first region's residual sum is \(4(1.10)=4.40\), so its mean is \(4.40/4=+1.10\). The corresponding sums in the other regions are \(5(-0.90)=-4.50\), \(6(0.10)=0.60\), and \(5(0.95)=4.75\). The pattern is therefore not a uniform offset: observations tend to be above prediction early, below prediction around the peak, and above prediction again late.
A reasonable initial ranking is (1) a structural explanation that changes the shape or timing of the profile, and (2) a late-phase explanation that could account for the positive terminal residuals. A residual-error change is less directly suggested by these summaries because the notable feature is a sequence of signed means, not an established change in residual spread. This ranking is provisional: the early and peak regions should be examined more closely, and the late positive residuals could be a separate problem rather than part of one mechanism.
The next test should be targeted. For example, compare the observed peak timing and the early-to-peak residual contrast with predictions under an alternative absorption structure, while retaining the rest of the model. If that change improves the early and peak pattern but leaves the late residuals, the evidence supports separating the discrepancies rather than claiming a single fix.
Change One Component at a Time
A controlled comparison begins with a defined base model and a single prespecified change. Keep the data, records, covariates, estimation settings, and diagnostic definitions fixed. Re-estimate the models under comparable conditions, and check that both runs converged adequately. If several features are changed at once, an improved fit cannot be attributed to a particular change; parameter compensation can also hide deterioration in one part of the profile.
Record where it occurs, its direction, and whether it concerns residual location, residual spread, an individual or subgroup, or a profile feature such as peak timing.
For each candidate, state which observations or diagnostic summaries should change if that explanation is relevant. Note which alternatives make similar predictions.
Alter one structural, random-effect, covariate, or residual-error component while keeping the rest of the analysis fixed. Avoid broadening the model without a specific diagnostic prediction.
Compare the prespecified summaries, parameter estimates, uncertainty, convergence, and other important diagnostics. A resolved pattern is encouraging only if it is not replaced by a new one.
Simulate data under a candidate model using the study design, calculate the same diagnostic summaries, and ask whether the observed feature is plausible under that model.
If the candidate models are nested, a likelihood-ratio comparison may be useful when its regularity conditions hold: the simpler model must be a special case of the more complex one, the models must be fitted to the same data, and the relevant parameter values must not create a boundary or other nonstandard situation that invalidates the usual reference distribution. In regular cases, twice the log-likelihood difference is compared with a chi-squared distribution whose degrees of freedom equal the difference in the number of free parameters. A numerical improvement alone does not establish that the added component is scientifically justified. For non-nested alternatives or nonstandard comparisons, do not use that likelihood-ratio reference mechanically.
Worked Example: Interpreting a One-Component Change
Hypothetical data. A base oral model estimates an absorption rate constant of \(0.42\ \text{h}^{-1}\). A single prespecified change to the absorption structure produces an estimate of \(0.18\ \text{h}^{-1}\), with an estimated standard error of \(0.07\ \text{h}^{-1}\), and reduces the objective function by 7.8 units. The competing models use the same observations and estimation settings.
The estimate-to-standard-error ratio for the alternative absorption parameter is \(0.18/0.07=2.57\). This is a rough indication that the estimate is not negligible relative to its reported uncertainty; it is not, by itself, a model-selection test. The objective-function difference is 7.8. If this is a regular nested comparison with one additional free parameter, the likelihood-ratio statistic is \(7.8\) on \(1\) degree of freedom, giving an approximate \(p\)-value of \(0.0052\). That calculation is not valid if the parameter is on a boundary or the regularity conditions otherwise fail.
The workflow does not end at the objective function. Check whether the change improves the early and peak residual pattern that motivated it, whether the late residuals remain, and whether estimates are stable with reasonable uncertainty. If it only improves the objective function while leaving the targeted pattern intact, it has not isolated the proposed fault. If it resolves the early pattern but not the late pattern, retain the possibility of a second discrepancy rather than adding several components simultaneously.
Use Targeted Simulations to Challenge a Candidate
A targeted simulation asks a narrow question: if the candidate model were adequate, and the observed sampling design were repeated, how often would a diagnostic feature at least as marked as the observed one appear? Specify the feature before simulating. It might be the maximum residual contrast between two time windows or the difference between observed and predicted median peak time. Simulating the full study design matters because sparse or uneven sampling can itself create unusual-looking summaries.
The simulation must generate observations under the model being assessed, including the relevant residual variability and, when appropriate, between-individual variability. If parameter uncertainty is substantial, account for it or state that the assessment is conditional on the fitted parameter estimates. Recalculate the same summary for every simulated data set. This yields a model-checking reference distribution, not automatic proof that the model is true; an alternative can also produce a similar feature.
Worked Example: A Simulation Check for Peak Timing
Hypothetical data. In an oral study, the observed median time to peak concentration is \(2.4\) hours. Under a candidate model, 500 simulated data sets are generated with the study's sampling schedule. For each, calculate the median peak time. Suppose 9 simulations have a median peak time at least as far from the model's simulated median as the observed value, using the same prespecified two-sided distance rule.
The observed fraction of simulations meeting or exceeding the discrepancy is \(9/500=0.0180\), or \(1.80\%\). A small-sample adjusted estimate is \((9+1)/(500+1)=10/501=0.01996\), approximately \(2.00\%\). Thus, under this simulation setup, a discrepancy at least this large occurred in about 2% of the simulated data sets. This is evidence that the candidate model may not reproduce the observed peak-timing feature well; it is not the probability that the model is false.
Before treating the result as informative, confirm that the simulation used the actual observation schedule, the intended definition of peak time, and the same handling of residual variability as the fitted model. If peak time is poorly resolved by the sampling schedule, the statistic may be noisy and its interpretation limited. Also examine the simulated and observed profiles: a single tail fraction can conceal whether the mismatch is systematic or caused by a few unusual profiles.
Diagnostics, Alternatives, and Limits
A convincing candidate should improve the feature it was designed to address, not merely a global fit statistic. Inspect residuals across time and concentration, individual profiles, parameter uncertainty and correlations, and convergence behavior. Warning signs include estimates at boundaries, large changes in unrelated parameters, unstable estimates across reasonable starting values, a new residual pattern, or an apparent improvement confined to a few observations. A model that fails to converge reliably cannot provide a stable basis for ranking explanations.
There are alternatives to one-at-a-time fault isolation. A planned comparison of several prespecified candidate models can be efficient when each encodes a distinct scientific hypothesis and all are fitted comparably. Broad automated searches may find improved fit, but they increase the risk of overfitting and make the apparent winner dependent on the search. A global fit criterion can summarize fit-complexity tradeoffs, but cannot tell whether a specific residual signature has been explained. Targeted simulations are valuable for checking a predicted feature under the study design, but their conclusions depend on the simulated model and parameter assumptions. These approaches complement rather than replace diagnostic reasoning.
Fault isolation has limits. Two mechanisms may make nearly indistinguishable predictions in the available data; no workflow can identify between them without informative sampling or other evidence. Sparse observations can miss the peak or terminal phase, while shrinkage can obscure individual-level deviations. Correlated parameters may trade off, and a correctly specified mean profile does not guarantee an adequate variability model. In such cases the defensible conclusion is that the data do not distinguish the explanations, not that the most convenient candidate has been proven correct.
The practical value of this workflow is restraint as much as detection. It makes the reasoning behind model development auditable, distinguishes a targeted improvement from indiscriminate complexity, and clarifies when a study design cannot resolve the cause of a pattern.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- Residuals are positive early, negative near the peak, and positive late. Why is “add a more flexible residual-error model” not yet a well-supported diagnosis?
- A candidate model reduces the objective function by 5.0 units relative to a base model. What must be checked before interpreting this as a likelihood-ratio result?
- In a targeted simulation, 4 of 200 replicates have a prespecified discrepancy at least as large as the observed one. Calculate the unadjusted fraction and explain its meaning.
- A change resolves the peak-time pattern but leaves a terminal residual trend. What is the appropriate next interpretation?
Solutions. (1) The described feature is a sequence of signed residual means, which suggests a location or profile-shape issue; an error model is more directly motivated by a change in residual spread. Check the spread pattern and rank alternatives before changing the error model. (2) Confirm that both models use the same data, that the models are nested, and that regularity conditions hold, including no problematic boundary parameter; the degrees of freedom must equal the difference in free parameters. (3) The fraction is \(4/200=0.0200\), or \(2.00\%\). Under the specified simulation setup, 2% of replicates showed a discrepancy at least that large; it is not the probability that the model is false. (4) The change may address the peak-time feature while the terminal trend remains a separate unresolved discrepancy. Recheck the profile and investigate a distinct, targeted explanation rather than assuming the first change fixed the whole model.