Why Diagnose a Reference-Scaled Model?
Reference-scaled bioequivalence analysis depends on estimates of treatment effects and within-subject reference variability. A model can produce those estimates even when its assumptions are poor: an omitted period pattern, an unusual observation, or a variance structure that does not fit the replicate data may distort the estimates and their uncertainty. Diagnostics help identify such problems before interpreting a scaled decision. They do not replace the prespecified analysis or establish that bioequivalence holds.
The focus here is model fit, especially whether residual behavior and replicate-reference variability are compatible with the working model. A diagnostic flag calls for investigation. It is not, by itself, proof that a record is erroneous or permission to exclude it.
A Working Model and Its Assumptions
Let \(Y_{ip}\) denote the log-transformed endpoint for subject \(i\) in period \(p\). For a replicate design, a useful diagnostic representation is
Sequence, period, and treatment are fixed effects in this representation; \(b_i\) is a subject effect, often modeled as \(b_i \sim N(0,\sigma_b^2)\). The residual \(\varepsilon_{ip}\) represents within-subject variation not explained by those effects. A simple model assumes independent residuals with a common variance. A more specific model may allow the residual variance to depend on treatment; in that case, the reference residual variance is denoted \(\sigma_{WR}^2\). The chosen diagnostic model should match the analysis model and the design.
The usual residual checks are most interpretable when the model accounts for the design: treatment and period are correctly coded, sequence is represented, subject effects are handled appropriately, and missing observations are not silently treated as observed. Normality and constant variance are working distributional assumptions, not conditions that can be proven from a plot. Independence is also conditional on the modeled effects; residual dependence may signal an omitted structure.
Residual Plots and What They Show
Plot residuals against fitted values and against design variables such as treatment, period, sequence, and subject order. A well-fitting model should not leave a systematic pattern in these plots. A curved trend can indicate an unmodeled relationship; a shift by period or treatment can indicate an omitted or incorrectly coded effect; a widening band can suggest nonconstant variance. A single large residual merits checking against the source record and the design, but it is not automatically invalid.
A normal quantile plot compares ordered residuals with the quantiles expected under a normal distribution. Approximate linearity is compatible with normal residuals. Strong skewness, heavy tails, or isolated extremes are warnings, especially when conclusions depend on model-based uncertainty. In small samples, the plot has limited ability to distinguish a genuine departure from ordinary sampling variation.
Residuals may be examined on the original log scale and in standardized form. A standardized residual divides a residual by an estimate of its standard deviation; exact scaling depends on the model and software. Because fitted effects and variance parameters are estimated from the same data, a standardized residual is a diagnostic summary, not a test statistic with a universal cutoff. Always examine its context and neighboring observations.
Worked Example: Reading a Large Standardized Residual
Suppose a hypothetical subject has an observed log endpoint of 1.42 and a fitted value of 1.18. The model-based residual standard deviation for this observation is estimated as 0.12. Calculate the residual and its standardized value, then interpret the result.
The conditional residual is
The standardized residual is
This observation is two estimated residual standard deviations above its fitted value, so it is worth checking in residual plots and against the source data. The result does not show that the observation is erroneous, nor does it justify removing it. A large residual can be a genuine subject response, a data issue, or evidence that the model does not describe a relevant feature of the observations.
Checking the Reference Variance Component
In a replicate design, repeated reference measurements provide information about within-subject reference variability. If two reference measurements for a subject have been adjusted for modeled period effects, write their difference as \(D_i\). Under independent residual errors with common reference variance, the shared subject effect cancels, and
Thus the variance of adjusted replicate differences is approximately twice the reference within-subject variance. If the period effects are not adjusted when they differ, a difference can contain both period and residual variation. For an unbalanced design or a model with additional covariance terms, the simple contrast calculation is a diagnostic aid rather than a replacement for the specified variance-component estimator.
Worked Example: A Replicate-Reference Variance Check
In a hypothetical full-replicate study, six subjects have period-adjusted differences between their two log reference measurements of \(-0.20,\ 0.10,\ 0.05,\ -0.15,\ 0.25,\ -0.05\). Use the sample variance of the differences as a rough check on the reference variance component.
The differences sum to zero, so their sample mean is \(0\). The sum of squared deviations is
The sample variance of the six differences is
Under the independent, equal-variance replicate assumption, the corresponding rough estimate is
on the log scale. This is a small-sample diagnostic calculation, not a substitute for the model-based estimator specified for an analysis. Compare it with the fitted reference variance component and inspect the individual contrasts. A disagreement may reflect period adjustment, an unusual subject, missing data, or a model structure that the simple calculation does not represent.
Variance-component checks should include the estimate itself, its uncertainty, and the pattern of data that supports it. In a full-replicate design, each subject’s reference pair contributes a within-subject contrast. In a partial-replicate design, fewer subjects provide that direct information, so the variance estimate depends more heavily on the model and the available replication. A near-zero estimate, a very large estimate, convergence warnings, or marked sensitivity to a few contrasts deserves attention. None alone proves that the model is wrong.
Worked Example: Comparing Treatment Residual Variability
Suppose a hypothetical balanced replicate dataset yields the following residuals after fitting treatment, period, sequence, and subject effects. Four reference residuals are \(0.10,-0.10,0.20,-0.20\); four test residuals are \(0.05,-0.05,0.10,-0.10\). Calculate their within-group sample variances as a simple diagnostic comparison.
Both groups have mean residual \(0\). For reference, the squared residuals sum to \(0.10\), giving
For test, the squared residuals sum to \(0.025\), giving
The ratio is \(0.0333/0.00833 \approx 4.00\). In this hypothetical sample the reference residuals are more variable. That pattern is a reason to check whether a common residual-variance assumption is appropriate, not a formal test that the variances differ: these group summaries are based on only four residuals each and do not account fully for fitted-model degrees of freedom. The analysis should follow its prespecified variance structure, and any alternative should be justified rather than selected solely to obtain a preferred result.
How to Organize the Diagnostic Review
Check that the endpoint transformation, treatment and period coding, sequence structure, subject effects, and handling of missing periods correspond to the design and intended analysis.
Examine residuals against fitted values and design factors, and inspect a normal quantile plot. Look for structure, changing spread, skewness, and isolated extremes.
Compare reference replicate contrasts with the fitted reference variance component. Check whether unusual contrasts or missing replicates account for much of the available information.
Describe the diagnostic finding, its possible explanations, and any justified sensitivity analysis. Keep the primary analysis identifiable and do not treat a diagnostic plot as a bioequivalence test.
A useful comparison is between the model-based estimate and a simple calculation from replicate contrasts, when that calculation is supported by the design. Agreement is reassuring but does not prove the model assumptions; disagreement is a prompt to investigate adjustment, data completeness, covariance assumptions, and influential patterns. For incomplete or unbalanced data, compare like with like: crude variances can differ from model-based estimates simply because they use different observations or adjustments.
Failure Modes, Alternatives, and Limitations
A common mistake is to treat every residual as if it were an independent observation from an identical distribution. Repeated measurements on a subject share subject-level features, and the model accounts for that dependence through its effects or covariance structure. Diagnostics should therefore be based on residuals defined by the fitted model, not on raw deviations from a grand mean.
Another mistake is to call a reference replicate difference a pure residual difference without accounting for period. If one reference measurement occurs in a period with a systematic shift, the unadjusted difference includes that shift. The correction is to use period-adjusted contrasts or a model-based diagnostic that represents period effects.
Residual plots are descriptive; formal goodness-of-fit tests are an alternative, but they can have low power in small samples and may flag negligible departures in large samples. A model-based variance-component estimate is generally preferable to a crude variance when the design is unbalanced or includes missing periods. Conversely, crude contrasts can be a transparent cross-check when their assumptions apply. No single plot, statistic, or alternative model can verify all assumptions.
The main limitations are limited replication, missing observations, and dependence of diagnostics on the fitted model. A small study may not reveal nonnormality or unequal variances, while a few atypical subjects may dominate a variance estimate. Diagnostics also do not determine whether a regulatory criterion has been met; the decision procedure and its assumptions remain separate. Record the checks and findings, investigate data-quality concerns, and avoid changing the primary model after looking at results without a defensible reason.
Check Your Understanding
Answer each question, then compare with the short solutions below.
- Why can a subject effect cancel from a difference between two reference measurements from the same subject?
- A residual plot shows a steadily increasing spread as fitted values increase. What assumption may be questionable, and what should be done next?
- Why should period effects be considered before interpreting a raw reference replicate difference as within-subject variability?
- A single standardized residual is large. Does that alone justify excluding the observation?
- What does a discrepancy between a simple replicate-contrast estimate and a model-based reference variance estimate tell the analyst?
Solutions. (1) If the subject contributes the same additive effect to both measurements, subtraction removes it: \((b_i+\varepsilon_{iR1})-(b_i+\varepsilon_{iR2})=\varepsilon_{iR1}-\varepsilon_{iR2}\). (2) Constant residual variance may be questionable; check treatment and design patterns, the model structure, and whether a variance specification supported by the design is needed. (3) The two reference measurements may occur in periods with different fixed effects, so an unadjusted difference can include a period shift. (4) No. Check the record and surrounding diagnostics; a large residual is a warning, not proof of error. (5) It calls for checking adjustments, missingness, the observations supporting each calculation, and covariance assumptions; it does not by itself show which estimate is correct.