Residual Diagnostics for Log-Scale Crossover Models

Why Examine Residuals?

A fitted crossover model summarizes log-transformed pharmacokinetic outcomes through effects such as treatment, period, and sequence, while accounting for repeated observations within subjects. The model’s treatment estimate is meaningful only insofar as the model represents the data adequately. Residual plots provide a practical check: they show what remains after the fitted model has explained the systematic part of the observations.

Two plots are especially useful. A residual-versus-fitted plot can reveal changing residual spread or a pattern in the mean that the model has not captured. A normal quantile plot can show whether the residual distribution is reasonably compatible with the model’s normal-error assumption. Neither plot proves that a model is correct or incorrect. They are diagnostic evidence to interpret alongside the crossover design, the fitted covariance structure, and the scientific context.

Model and Residual Definitions

Let \(Y_i\) be the vector of log-transformed outcomes for subject \(i\), observed across that subject’s crossover periods. A general mixed-model representation is

Here, \(X_i\beta\) represents the fixed effects in the specified model, \(b_i\) represents subject-level random effects, and \(\varepsilon_i\) represents the within-subject error. The model’s covariance assumptions specify the distributions and dependence of these terms. For a simple homoscedastic normal-error model, the errors are independent of the random effects and have mean zero, a common variance \(\sigma^2\), and a normal distribution. More generally, the model may specify a nontrivial covariance structure; diagnostics must be interpreted relative to that structure.

For a conditional fit, write the residual for subject \(i\), period \(j\), as \(e_{ij}=Y_{ij}-\widehat{Y}_{ij}\). A residual-versus-fitted plot places \(e_{ij}\) vertically against \(\widehat{Y}_{ij}\) horizontally. A normal quantile plot orders the residuals and compares them with the quantiles expected from a normal distribution. Standardized residuals, which express residuals in units of an estimated standard deviation, are often used so that their scale is easier to read.

The usual visual interpretations assume that the residuals being plotted are appropriate for the fitted model. If a model includes subject-level random effects or correlated within-subject errors, its residuals are not automatically independent observations with identical variances. Some software provides conditional, marginal, or covariance-adjusted residuals. Read the output definition, and do not interpret a plot as if all residuals were independent when the model says otherwise.

Residuals Versus Fitted Values

If the model’s mean and variance assumptions are adequate, residuals should generally be scattered around zero without a persistent shape. Under a constant-variance assumption, their vertical spread should be broadly similar over the range of fitted values. In practice, random fluctuation is expected; the diagnostic concern is a clear, sustained pattern rather than a few points on one side of zero.

A widening or narrowing band can indicate that residual variability changes with the fitted mean. A curve or other systematic shape can indicate that the mean structure is incomplete. Groups or bands may reflect features of the design, such as distinct treatment or period combinations, or may arise because the fitted values themselves are concentrated in groups. Such features are prompts to investigate the model and data, not automatic instructions to add a term.

Normal Quantile Plots

A normal quantile plot sorts the residuals from smallest to largest and plots them against corresponding theoretical quantiles from a normal distribution. If the residual distribution is approximately normal, the points should lie roughly along a straight line. Exact alignment is not expected, especially in a small crossover study. Smooth curvature, marked asymmetry, or unusually distant tail points can signal departures worth investigating.

The plot evaluates the shape of the residual distribution, not whether the treatment effect is significant or whether two formulations are equivalent. It also does not establish normality: with limited observations, substantial departures may be hard to see, and isolated deviations can occur by chance. Look for an overall pattern, and consider whether the residuals were calculated in a way that accounts for the model’s covariance.

Looking for Systematic Structure

A residual-versus-fitted plot can reveal systematic structure as well as changing spread. For crossover data, fitted values may cluster by period, treatment, or sequence. A trend in the residuals across such groups can suggest that the model has not accounted for some structure, but it must be interpreted in light of the design. For example, a sequence of positive residuals in one period and negative residuals in another is a reason to inspect period-related fit; it is not, by itself, proof of a particular omitted effect.

Why the Plots Need Careful Interpretation

Even under a correct model, observed residuals do not behave exactly like independent errors. In the simpler fixed-effects linear model \(Y=X\beta+\varepsilon\), ordinary least squares gives residual vector \(e=(I-H)\varepsilon\), where \(H\) is the projection matrix onto the columns of \(X\). If the errors have covariance \(\sigma^2 I\), then

Thus, the variance of residual \(e_i\) is \(\sigma^2(1-h_{ii})\), where \(h_{ii}\) is the corresponding diagonal element of \(H\). Residual variances can differ because of leverage even when the underlying error variance is constant. Mixed models have their own fitted-residual covariance behavior, so this simple identity should not be applied as if it described every mixed-model diagnostic. It illustrates the broader point: visible residual spread is affected by both the errors and the fitting process.

A practical review uses plots as complementary tools. The residual-versus-fitted plot is especially useful for spread and mean structure. The normal quantile plot is aimed at distributional shape. A grouped display against period or treatment can make design-related structure easier to see. These views do not replace one another, and none supplies a definitive pass-or-fail test.

Common Pitfalls and Limitations

• Treating a funnel as proof. A wider band may suggest changing variance, but small samples, group structure, or leverage can affect the appearance. Seek a persistent pattern and interpret it in context.
• Calling a nearly straight quantile plot proof of normality. A plot can fail to reveal departures, particularly with few residuals. It is evidence, not confirmation.
• Ignoring the residual definition. Conditional and marginal residuals answer different questions. Correlation or unequal residual spread may also reflect a covariance structure that the plotted residuals have not removed.
• Overreacting to individual points. An unusual residual deserves investigation, but a plot alone does not show that an observation is erroneous or should be excluded.
• Assuming a visual pattern names its cause. A period pattern suggests a model-fit issue to investigate; it does not establish which effect is responsible.

Residual diagnostics are limited by the number and arrangement of subjects and periods. A small dataset may provide little power to expose nonnormality or changing spread. Conversely, a plot may look irregular simply because there are few points. Interpret diagnostics together with model assumptions and design information rather than as a mechanical selection rule.