Why Incomplete Replication Needs Careful Analysis
In a replicate crossover, participants receive the test formulation and reference formulation more than once according to an assigned sequence. Repeated reference observations can inform within-subject reference variability, which is useful for the planned analysis of some highly variable drugs. But participants may discontinue, miss a period, or have an unusable exposure measurement. The resulting data are unbalanced: some participants contribute repeated measurements for both formulations, while others contribute only one or contribute fewer periods than planned.
Unbalanced data do not automatically invalidate a study. A model can use the observed measurements without pretending that every participant completed every period. The key questions are whether the treatment contrast and any variance component needed for the planned decision are estimable, whether the missingness and model assumptions are credible, and whether the remaining data support adequate precision. A numerical answer from software is not, by itself, evidence that these conditions hold.
Model and Conditions for Interpretation
Suppose the endpoint is a positive exposure measure such as AUC, analyzed after logarithmic transformation. For a two-sequence, four-period replicate design, the planned sequences might be TRTR and RTRT, where T denotes test and R reference. A mixed model for an observed log endpoint is
Here \(Y_{snj}\) is the log endpoint for the applicable participant and period; sequence, period, and treatment are fixed effects; and \(b_{sn}\) is a random participant effect for participant \(n\) nested within sequence \(s\). A common formulation assumes \(b_{ij}\sim N(0,\sigma_b^2)\). To allow different within-subject variances by formulation, assume \(\varepsilon_{ijk}\sim N(0,\sigma^2_{w,k})\), independently conditional on the participant effect and model terms. Thus observations from the same participant are correlated through \(b_{ij}\), while the residual variance can differ between T and R. The treatment contrast is the model-adjusted difference in mean log exposure, T minus R.
This model and its estimates are interpretable only under important conditions. The design must provide enough overlap between formulations and periods to distinguish treatment from period and sequence effects. The fixed-effects model must reflect the planned design, and the covariance and residual-variance assumptions must be reasonable. In particular, assuming independent residual errors after accounting for the participant effect is a modeling choice, not a consequence of having repeated measurements.
Likelihood-based mixed-model estimation can use participants with incomplete records under a missing-at-random (MAR) assumption: conditional on observed outcomes and relevant observed covariates, the probability that an outcome is missing does not depend on its unobserved value. For likelihood-based ignorability, the missingness and outcome-model parameters must also be distinct. It is not guaranteed by fitting a mixed model. If, for example, participants discontinue because of an unobserved poor response or a treatment-related problem not captured in the model, missingness may be informative. A sensitivity assessment or a qualified interpretation may then be needed.
How Repeated Reference Measurements Inform Variance
The basic justification for using repeated reference observations is easiest to see in a simplified model. After adjusting for period and other fixed effects, suppose a participant’s two reference residuals are \(e_{R1}=b+\varepsilon_{R1}\) and \(e_{R2}=b+\varepsilon_{R2}\), with independent residuals of variance \(\sigma^2_{w,R}\). Their difference removes the shared participant effect:
Consequently, the sample variance of properly adjusted within-participant reference differences estimates twice the within-subject reference variance; dividing by two estimates \(\sigma^2_{w,R}\). In the full mixed model, estimation accounts for the design and available observations rather than necessarily using this simple calculation directly. The derivation explains why participants with only one reference observation do not directly supply a reference difference, and why few paired references can make the variance estimate unstable.
For a log-scale within-subject variance estimate \(\hat{\sigma}^2_{w,R}\), the corresponding within-subject coefficient of variation is often expressed as
This is a relative measure, reported as a proportion or multiplied by 100 to give a percentage. It is not the standard deviation of the original-scale exposure, and uncertainty in the variance estimate should not be hidden by reporting only a point estimate.
Worked Example: Counting What Dropout Leaves
Consider a hypothetical TRTR/RTRT replicate study in which 24 participants are randomized, 12 to each sequence. Of these, 21 provide at least one usable post-dose endpoint. Among the 21, sixteen provide all four periods, three provide three periods, and two provide only two periods. The incomplete records are distributed across sequences, and all observed periods have both treatments represented.
What remains usable? All 21 participants can contribute observed log endpoints to a mixed-model analysis, subject to the model and missingness assumptions. The 16 complete participants contribute two observations under each formulation. A participant with three periods may contribute two observations for one formulation and one for the other; a participant with two periods may contribute one under each, depending on the periods completed.
What supports reference variance? Suppose inspection of the actual period records shows that 17 participants have two usable reference observations, while four have only one. The four single-reference participants may inform the treatment contrast but do not yield an individual reference difference. The repeated-reference variance is therefore informed directly by 17 participants, not by all 21 participants or by the total count of observed endpoints.
Is the design automatically interpretable? No. The counts are encouraging only if sequence, period, and treatment remain distinguishable in the observed data, and if the reason for missing periods does not make the observed outcomes systematically unrepresentative. The analyst should tabulate completion by sequence and period, examine dropout reasons, and verify that the planned contrast and variance component are estimable.
Worked Model Output: Variance and Treatment Contrast
A fitted model may report the adjusted treatment contrast, its standard error, degrees of freedom, and a variance-component estimate. For average bioequivalence, the treatment contrast on the log scale is exponentiated to obtain the test-to-reference geometric mean ratio. A 90% confidence interval is compared with the prespecified limits, commonly 80.00% to 125.00% when those limits apply. The inference depends on the planned analysis and applicable decision rule; dropout does not justify changing that rule after seeing the results.
Worked Example: Interpreting a Model-Based Treatment Contrast
Suppose a hypothetical mixed-model fit to the observed log AUC data gives a T-minus-R contrast of \(-0.0250\), standard error \(0.0310\), and \(df=24\). For a two-sided 90% interval, use \(t_{0.95,24}=1.711\).
Compute the log-scale interval.
Transform to the ratio scale. The point estimate is \(\exp(-0.0250)=0.9753\), or 97.53%. The interval endpoints are \(\exp(-0.07804)\approx 0.9249\) and \(\exp(0.02804)=1.0284\), giving 92.49% to 102.84%.
Interpretation. Under the model and its assumptions, the 90% confidence interval for the test-to-reference geometric mean ratio is 92.49% to 102.84%. It lies within 80.00% to 125.00%, so this contrast meets those limits. This conclusion concerns the treatment contrast and the specified limits; it does not establish that every variance estimate is reliable or that missingness assumptions are satisfied.
Worked Example: Converting a Within-Reference Variance Estimate
Suppose a hypothetical model estimates the within-subject reference log variance as \(\hat{\sigma}^2_{w,R}=0.0105\). Convert this estimate to a within-subject coefficient of variation.
Substitute and calculate.
Thus the estimated within-subject reference CV is 10.27%. The percentage summarizes estimated relative within-subject variation on the original scale. It should be considered alongside the amount of replication and uncertainty in the variance estimate: the same point estimate based on a few paired observations may be much less stable than one supported by many participants.
Diagnostics, Alternatives, and Limits
Before interpreting output, inspect the observed-data structure rather than relying only on the model summary. Tabulate usable periods, treatments, and completion patterns by sequence. Count participants with repeated usable reference measurements, and check whether missing periods disproportionately remove a treatment or period. Confirm that the model matrix supports separate treatment and period effects; if treatment occurs only in particular observed periods after dropout, those effects may be confounded. Inspect residuals and influential participants, and look for implausible variance estimates, boundary estimates, or large changes when a small number of subjects are removed.
A mixed model using all observed data is often preferable to an analysis restricted to complete cases because it can retain information from incomplete participants under its assumptions. A complete-case analysis is simpler, but it discards partial records and can be biased if completion is related to outcomes. A paired-reference calculation is useful for understanding the source of within-reference information, but by itself it does not estimate the full treatment contrast adjusted for sequence and period. These approaches answer related but nonidentical questions; they are not interchangeable simply because they use the same study.
Variance estimates deserve particular scrutiny. A small number of repeated references can produce substantial uncertainty, and the estimate may be sensitive to a few large within-subject differences. Report the estimator and uncertainty available from the planned method, not only a CV rounded to a percentage. If the decision procedure requires a variance estimate or a particular replicate-data structure, verify that the actual data support that requirement. Do not infer that a scaled decision is appropriate merely because the design included repeated reference dosing.
The treatment confidence interval also needs context. A narrow interval can coexist with a fragile variance estimate if different parts of the design inform the two quantities. Conversely, an interval that crosses a BE limit is inconclusive under the specified rule; it is not evidence that the formulations are equivalent. The planned analysis should be retained unless a prespecified rule says otherwise, and any sensitivity analyses should clarify how conclusions depend on plausible missingness or modeling assumptions.
Assessing Whether the Planned Decision Holds
List usable observations by participant, sequence, period, and treatment. Identify complete and partial records and the number of repeated reference measurements.
Verify treatment-period overlap and model rank. Review dropout timing and reasons, and assess whether conditional missing-at-random and covariance assumptions are credible.
Evaluate the treatment contrast and its interval, then separately evaluate the reference variance estimate, its uncertainty, and the number of participants contributing repeated references.
Apply the prespecified BE rule only if the required estimates are supported. If severe imbalance, informative dropout, or inadequate replication undermines interpretation, state that limitation rather than treating model output as decisive.
The central limitation is that no model can recover information that the study did not collect without additional assumptions. Mixed-model estimation makes principled use of incomplete records, but it does not recreate missing periods, guarantee that dropout is ignorable, or make a poorly represented variance component precise. A defensible analysis therefore reports both the result and the observed structure that supports it.
Check Your Understanding
Answer each question, then compare with the brief solutions.
- A participant has one usable T observation and one usable R observation, but no repeated R measurement. Can this participant contribute to the treatment contrast? Do they provide an individual within-reference difference?
- In the simplified model, adjusted repeated-reference differences have sample variance \(0.0400\) on the log scale. What is the corresponding estimate of \(\sigma^2_{w,R}\)?
- Why does a mixed model’s use of incomplete participants not automatically resolve bias from dropout?
- A treatment-ratio 90% confidence interval is inside the prespecified BE limits, but only three participants have two usable reference observations. What additional concern should be reported?
Solutions. (1) Yes, the two observed treatments can contribute to the treatment contrast under the model; no, a single R observation cannot form a within-participant reference difference. (2) Divide by two: \(\hat{\sigma}^2_{w,R}=0.0400/2=0.0200\). (3) The likelihood-based analysis relies on assumptions about the relationship between missingness and unobserved outcomes; informative dropout can violate them. (4) The reference variance estimate may be imprecise or fragile because very few participants directly supply repeated-reference information, so the planned decision’s support should be assessed rather than inferred from the treatment interval alone.