Why Incomplete Replicate Records Need Careful Evaluation
Replicate crossover studies collect repeated observations under at least one formulation so that within-subject variability can be estimated. Missing data can weaken both parts of the analysis: the treatment contrast and the estimate of within-subject reference variability. The impact depends on which period and formulation are missing, how the missingness occurred, and which model is used. A subject with one observed reference period may still contribute information about the overall treatment contrast, but cannot supply a direct within-subject reference difference.
The first task is therefore not to decide whether a record is “usable” in general. It is to map the observed data to the estimand and variance component being estimated. This tutorial considers a single log-transformed exposure endpoint at a time. AUC and Cmax require their own endpoint-specific analyses; no joint decision is made here.
Let \(Y_{ij}\) be the log-transformed endpoint for subject \(i\) in period \(j\), and let \(F_{ij}\) indicate the formulation administered in that period. A treatment contrast, \(\delta\), is the model-based mean difference between test and reference on the log scale. Its exponentiation, \(\exp(\delta)\), is the corresponding geometric mean ratio. In a replicate design, the reference within-subject variance, commonly denoted \(\sigma_{WR}^{2}\), describes variation among repeated reference observations within a subject after the model’s fixed effects have been accounted for.
What the Missing Pattern Leaves Identifiable
A complete subject record can contribute to the treatment contrast and, if the design includes repeated reference periods, to the reference variability estimate. In contrast, a subject with observations under both test and reference can contribute a within-subject treatment comparison even if another planned period is absent. A subject with only one observed formulation can still contribute to a model-based analysis, but that contribution depends more strongly on the model, the study design, and assumptions about missingness.
For a direct estimate of reference within-subject variability, consider two reference observations \(R_{i1}\) and \(R_{i2}\) from the same subject. If their residual errors are independent, have mean zero, and have common variance \(\sigma_{WR}^{2}\), then the within-subject difference \(D_i=R_{i1}-R_{i2}\) has variance \(2\sigma_{WR}^{2}\). Thus the sample variance of the paired differences estimates twice the reference variance. A subject with only one reference observation does not provide such a direct difference.
This identity relies on the specified error structure. Period effects must be handled appropriately, and any correlation between repeated residuals changes the variance of the difference. In practice, a mixed-effects model can represent period and sequence effects and estimate formulation-specific residual variances by restricted maximum likelihood (REML). The earlier material on mixed-effects models for NTI replicate crossover studies gives the general model framework; here the concern is what the observed pattern can support.
Worked Examples
Worked Example: A Missing Test Period and the Treatment Contrast
Suppose a hypothetical four-period replicate study records log-endpoint values under test and reference. Four subjects have both a test and a reference observation, giving within-subject differences \(d_i=Y_{iT}-Y_{iR}\) of \(0.02,-0.01,0.04,\) and \(0.01\). A fifth subject has a reference observation but no test observation.
The four directly paired differences have mean
Their sample variance and standard error are
Using only the four directly paired subjects, the estimated log-scale contrast is \(0.015\), corresponding to a ratio of \(\exp(0.015)=1.015\), or about 1.5% higher test exposure. The fifth subject cannot contribute a direct paired difference because no test value was observed. The estimate above is a complete-pair estimate, not automatically the preferred analysis: a suitable mixed model may retain the fifth subject’s observed reference value while estimating the treatment contrast from all available records. Whether that is defensible depends on the design, the model, and the missingness assumptions.
Worked Example: Missing a Repeat Reference Observation
Suppose four hypothetical subjects have both planned reference observations, and their reference differences \(D_i=R_{i1}-R_{i2}\) are \(0.10,-0.06,0.04,\) and \(-0.08\) log units. A fifth subject has only one reference observation.
The differences average to zero. Their sample variance is
Under the independent, equal-variance residual assumptions, the direct estimate is
The fifth subject’s single reference value cannot form a within-subject reference difference, so it does not add a direct paired difference to this calculation. A model-based estimate may use more of the available pattern, but it cannot manufacture the missing repeat. If the data contain too few repeated-reference subjects or too few informative treatment-period patterns, the reference variance may be imprecise or not separately identifiable.
Worked Example: Interpreting an Available-Data Model Fit
Consider hypothetical REML output from a prespecified mixed model fitted to an incomplete replicate dataset. The model adjusts for sequence and period, includes formulation as a fixed effect, and uses a covariance structure appropriate to the replicate design. Suppose the estimated log-scale test-minus-reference contrast is \(0.018\), its standard error is \(0.012\), and the approximate degrees of freedom are \(18.6\). For illustration, use a two-sided 95% interval with critical value \(2.10\).
The interval on the log scale is
Exponentiating the endpoints gives a geometric mean ratio interval of approximately \([0.9928,\ 1.044]\), or 99.28% to 104.4%. The point estimate is \(\exp(0.018)=1.018\), or 101.8%. This summarizes the model’s estimated treatment contrast and uncertainty; it does not establish that the model is adequate, that missingness is ignorable, or that any particular bioequivalence criterion is met. Those judgments require the applicable analysis plan and diagnostics.
Justification: Why Available-Data Likelihood Can Help
A mixed-model likelihood can use an observed subset of a subject’s planned outcomes without imputing the missing values. Let \(Y_{i,\mathrm{obs}}\) denote the vector of outcomes actually observed for subject \(i\), and let the model specify its mean vector \(X_{i,\mathrm{obs}}\beta\) and covariance matrix \(V_{i,\mathrm{obs}}\). Under a multivariate normal model, the subject’s observed-data contribution to the log likelihood is
where \(m_i\) is the number of observed outcomes for subject \(i\). The likelihood is formed from the observed components, not from fabricated values for absent periods. If missingness is MAR conditional on modeled observed information, and the mean and covariance models are correctly specified, maximizing this observed-data likelihood can give valid estimates while retaining incomplete records.
This result does not mean that any incomplete pattern is informative about every parameter. A subject with one observation may contribute information about fixed effects and overall variability, yet provide little or no subject-specific information about a within-subject contrast. If no observed subjects have repeated reference measurements, a reference-specific residual variance generally cannot be separated from other sources of variation without additional identifying assumptions or information. The design matrix and covariance structure must remain identifiable after the missing observations are removed.
Diagnostics, Alternatives, and Sensitivity Checks
Before fitting a model, tabulate missingness by sequence, period, formulation, endpoint, and reason. Show how many subjects have each observed pattern, especially how many contribute paired test-reference data and repeated reference data. A total completion percentage alone can hide a serious loss of information: missingness concentrated in one sequence or one reference repeat can damage variance estimation more than the same number of missing observations distributed across patterns.
Then inspect whether model estimates are supported by the observed design. Warning signs include convergence problems, very large standard errors, a variance estimate near a boundary, strong dependence of \(\widehat{\sigma}_{WR}^{2}\) on a few subjects, or substantial changes after removing one incomplete pattern. Check residuals and influential observations under the fitted model. These diagnostics assess model fit and leverage; they cannot prove MAR, because the outcomes that are missing are not observed.
| Approach | Potential strength | Main caution |
|---|---|---|
| Complete-pair analysis | Uses directly observed within-subject treatment comparisons; calculations and data requirements are transparent. | Discards subjects without the required pair and can be biased if completion is related to outcomes or treatment response. |
| Available-data mixed model | Can retain observed outcomes from incomplete subjects and represent period, sequence, and within-subject covariance. | Depends on correct model specification, identifiability, and an ignorable missingness assumption for standard likelihood inference. |
| Sensitivity analysis | Shows whether conclusions depend on plausible alternative assumptions or record-selection rules. | Cannot recover information absent from the data; results depend on the scenarios examined. |
A defensible sensitivity plan can compare the prespecified available-data model with a complete-pair analysis, and examine the impact of excluding influential subjects or using a plausible alternative covariance specification. If missingness may depend on unobserved outcomes—for example, withdrawal associated with an unrecorded extreme response—consider a scientifically justified MNAR sensitivity analysis where feasible. Report the assumptions and the range of results; do not present one sensitivity model as having identified the unobserved values.
The analysis should also distinguish an observation made unavailable by a technical assay failure from one missing after a subject discontinued because of an adverse event. The causes may relate differently to the endpoint. Excluding all such records without explanation can introduce selection bias, while automatically treating all causes as ignorable can also be unjustified. Document the pattern, reason, model, and sensitivity checks.
Common Pitfall and Practical Limits
Incomplete data reduce precision even when the analysis assumptions are reasonable. They can also alter which subjects and treatment-period contrasts drive the estimate. A complete-pair estimate is easy to interpret but may be inefficient or selected; a mixed-model estimate can be more efficient under MAR but more dependent on assumptions. Neither approach is automatically superior in every dataset. The defensible choice follows the estimand, planned model, observed pattern, and sensitivity of results to plausible alternatives.
Count observed patterns by sequence, period, and formulation; separate missing periods from missing endpoint values and record reasons.
Confirm that observed subjects provide the treatment and repeated-reference patterns needed for the parameters of interest.
Review convergence, uncertainty, covariance estimates, residuals, and influential patterns.
Use a transparent complete-pair comparison and targeted sensitivity checks; explain differences rather than selecting the most favorable result.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- A subject has two reference observations but no test observation. Can the subject contribute a direct repeated-reference difference? Can the subject contribute a direct paired treatment difference?
- Why does one reference observation from a subject not provide the direct difference used to estimate reference within-subject variance?
- State the missingness assumption commonly needed for valid available-data likelihood inference, and explain one limitation of that assumption.
- In a dataset, nearly all missing observations occur in one sequence during the final period. Name two checks or sensitivity analyses that should be considered.
Solutions. (1) Yes: the two reference observations form a within-subject difference and may inform reference variability, subject to the model assumptions. No: without a test observation, there is no direct test-minus-reference pair for that subject. (2) A single value cannot be differenced against a second within-subject reference value, so it provides no direct paired contrast; model-based estimation may use it through other parts of the likelihood but relies on the fitted covariance structure. (3) MAR conditional on observed information in the model is commonly required, along with correct model specification. MAR cannot be verified from observed data alone and does not cover dependence on an unobserved endpoint after conditioning. (4) Tabulate the pattern and reasons by sequence and period, then check whether the model remains identifiable and whether estimates are influential or unstable. Compare the prespecified mixed model with a complete-pair analysis and a plausible alternative model or missingness sensitivity scenario.