Interpreting Sequence Effects Without Overstating Them

What a Sequence Effect Compares

In a two-period crossover, subjects are assigned to a treatment sequence: one group receives test then reference (TR), and the other receives reference then test (RT). A sequence effect compares the average response of subjects assigned to TR with the average response of subjects assigned to RT. It is therefore a contrast between two groups of subjects, not a direct comparison of the test and reference formulations within the same subject.

This distinction matters because the crossover’s formulation comparison uses each subject’s observations under both formulations. By contrast, each subject belongs to only one sequence group. A sequence effect can describe a difference between the groups’ average responses, but it does not by itself say that one formulation produces greater exposure than the other.

Definition: In a complete, balanced two-period crossover, the sequence effect is the difference between the mean subject-level responses for the TR and RT groups, with each subject-level response formed by averaging that subject’s two period responses. Its sign depends on the chosen order of subtraction; here, the contrast is TR minus RT.

Model and Assumptions

Let \(Y_{ik}\) denote the log-transformed pharmacokinetic response for a subject in sequence \(i\) during period \(k\), where \(i\) is TR or RT and \(k\) is 1 or 2. The response might be log AUC or log Cmax. A standard additive crossover model can be written as

Y_{ijk}=\mu+S_i+U_{j(i)}+P_k+F_{d(i,k)}+\varepsilon_{ijk}.

Here, \(\mu\) is an overall mean, \(S_i\) is the fixed sequence effect, \(U_{j(i)}\) is the random subject effect for subject \(j\) nested within sequence \(i\), \(P_k\) is the period effect, and \(F_{d(i,k)}\) is the effect of the formulation administered in that sequence and period. The remaining variation is \(\varepsilon_{ijk}\). Usual identifying constraints make the fixed effects estimable, for example by setting the sequence, period, and formulation effects to sum to zero within their respective factors.

For the simple interpretation below, assume subjects are independently randomized to sequence; both periods are observed for every subject; the same formulations and periods occur in both sequences; and the model is additive, with no sequence-specific treatment or period effects. The random subject effects have mean zero and are independent across subjects. Residuals have mean zero and are independent of subject effects. A normal, constant-variance residual model supports the usual small-sample ANOVA test, though the expectation-level cancellation below does not require normality. If repeated-measure covariance is modeled more generally, the estimated sequence contrast still has a between-subject interpretation, but its standard error must reflect that covariance structure.

Why Treatment and Period Effects Cancel

Define each subject’s response averaged across the two periods. In the TR group, that average includes one test response in period 1 and one reference response in period 2. In the RT group, it includes one reference response in period 1 and one test response in period 2. Under the additive model, the expected group means are therefore

\begin{aligned} E(\bar{Y}_{TR})&=\mu+S_{TR}+\frac{P_1+P_2}{2}+\frac{F_T+F_R}{2},\\ E(\bar{Y}_{RT})&=\mu+S_{RT}+\frac{P_1+P_2}{2}+\frac{F_R+F_T}{2}. \end{aligned}

Subtracting the second expected mean from the first removes the overall mean, the average period contribution, and the average contribution from the two formulations:

E(\bar{Y}_{TR}-\bar{Y}_{RT})=S_{TR}-S_{RT}.

This is the key justification for interpreting the contrast as a sequence effect in the balanced additive model. The two groups receive the same pair of formulations and cover the same two periods; sequence is the remaining group-level fixed-effect contrast. However, the cancellation does not turn the contrast into a formulation comparison. It compares different subjects’ averaged responses.

Worked Example: A Sequence Contrast From Subject Averages

Suppose a hypothetical complete crossover has four subjects per sequence. Their log-AUC averages across periods are 5.04, 5.00, 5.02, and 4.98 for TR, and 4.99, 5.03, 4.97, and 5.01 for RT. Calculate the sequence contrast.

The TR mean is \((5.04+5.00+5.02+4.98)/4=20.04/4=5.01\). The RT mean is \((4.99+5.03+4.97+5.01)/4=20.00/4=5.00\). Thus the estimated sequence contrast, defined as TR minus RT, is \(5.01-5.00=0.0100\) log-AUC units.

The estimate says that the average subject-level log-AUC was 0.0100 units higher in the TR group than in the RT group. It does not estimate a 0.0100-unit test-versus-reference effect: each of these group averages contains both formulations. Interpreting the number as a formulation advantage would confuse the between-subject sequence contrast with the within-subject treatment contrast.

Sequence Effects Are Between-Subject Effects

The sequence groups consist of different subjects, so their comparison includes between-subject variation. In the model, that variation is represented by subjects nested within sequence. This explains why sequence is not estimated with the same information as the formulation contrast: the formulation contrast is informed by comparisons within subjects, whereas sequence is informed by differences between subject groups.

For a balanced complete design with \(n_{TR}\) and \(n_{RT}\) subjects, let \(\bar{Y}_{j\cdot}\) be the average of subject \(j\)’s two period responses. The estimated sequence contrast is

\hat{\delta}_S= \frac{1}{n_{TR}}\sum_{j=1}^{n_{TR}}\bar{Y}_{j\cdot,TR} - \frac{1}{n_{RT}}\sum_{j=1}^{n_{RT}}\bar{Y}_{j\cdot,RT}.

The standard sequence-effect test in a classical balanced crossover ANOVA uses subjects nested within sequence as the error term. Under the usual normal, independent, equal-variance assumptions for those subject-level averages, it tests \(H_0:S_{TR}-S_{RT}=0\) against the alternative that the contrast is not zero. Its uncertainty comes from variation among subjects within the sequence groups. The treatment test instead uses the model’s appropriate within-subject error term; these error terms are not interchangeable.

Worked Example: A Sequence Contrast Is Not a Formulation Contrast

Consider a hypothetical additive model in log-AUC units with overall mean 5.00, period effects \(P_1=0\) and \(P_2=0.10\), and formulation effects \(F_T=0.20\) and \(F_R=0\). Let both sequence effects be zero and consider group expectations, ignoring random subject and residual deviations.

For TR, the period-1 test expectation is \(5.00+0+0+0.20=5.20\), and the period-2 reference expectation is \(5.00+0+0.10+0=5.10\). The expected subject average is \((5.20+5.10)/2=5.15\).

For RT, the period-1 reference expectation is \(5.00+0+0+0=5.00\), and the period-2 test expectation is \(5.00+0+0.10+0.20=5.30\). The expected subject average is \((5.00+5.30)/2=5.15\). The sequence contrast is therefore \(5.15-5.15=0\), even though the model’s test-minus-reference formulation contrast is \(0.20-0=0.20\) log units.

This example shows the purpose of the averaging argument: both groups contain each formulation and each period once. The zero sequence contrast does not mean that the formulations have equal expected responses.

Testing and Interpreting the Contrast

A test of the sequence effect addresses whether the observed between-group contrast is unusual under a model in which the population sequence contrast is zero. A small p-value is evidence against that model for the sequence contrast. It is not evidence, by itself, that one formulation performs better, that carryover occurred, or that bioequivalence fails. A large p-value likewise does not establish that the sequence groups are identical; it can reflect limited precision.

The following hypothetical calculation uses the subject averages from a balanced design. Suppose five TR averages are 5.00, 5.02, 4.98, 5.01, and 4.99, and five RT averages are 4.98, 4.99, 5.00, 4.97, and 4.96. Assume independent subject averages, normal errors, and a common variance across sequences. These assumptions justify a pooled two-sample t test of the sequence contrast.

Worked Example: Uncertainty in a Sequence Contrast

The means are \(\bar{Y}_{TR}=5.000\) and \(\bar{Y}_{RT}=4.980\), so \(\hat{\delta}_S=0.0200\) log-AUC units. In each group, the squared deviations from its mean sum to 0.00100, giving sample variance \(s^2=0.00100/(5-1)=0.000250\) and sample standard deviation \(s=0.0158\) log-AUC units.

The pooled variance is \(s_p^2=\{(5-1)(0.000250)+(5-1)(0.000250)\}/(5+5-2)=0.000250\). Thus \(s_p=0.0158\), and the standard error for the difference is \(SE=s_p\sqrt{1/5+1/5}=0.0158\sqrt{0.4}=0.0100\) log-AUC units.

The test statistic is \(t=0.0200/0.0100=2.00\), with \(df=5+5-2=8\). The two-sided p-value is approximately 0.0805. At \(\alpha=0.05\), this result does not provide sufficient evidence against a zero sequence contrast. It does not prove the contrast is zero. A 95% confidence interval is \(0.0200\pm2.306(0.0100)=(-0.00306,0.0431)\) log-AUC units. Under the stated model, this procedure has 95% long-run coverage for the sequence contrast; it does not assign a 95% probability to this particular interval containing the parameter.

The estimate and interval describe a difference between sequence-group averages. Neither is a confidence interval for the test-to-reference formulation ratio or a bioequivalence conclusion.

Diagnostics, Alternatives, and Limits

Before interpreting a sequence estimate, check whether the design and analysis support the intended contrast. Confirm that each sequence has subjects and that the response being analyzed is defined on the intended scale. In a complete balanced design, verify that each subject contributes both periods and each sequence contributes the same formulation-period pattern. Examine the number and pattern of missing observations: if one sequence disproportionately lacks a period, its observed averages may no longer contain the same mix of treatments and periods as the other sequence. A mixed model can use incomplete records under its modeling assumptions, but its adjusted sequence estimate then depends on the specified model and missing-data assumptions.

Also check the basis for the standard error and test. The classical subject-within-sequence test presumes independent subjects and an appropriate common-variance, approximately normal error model for subject-level averages. Marked group variance differences or influential subject averages can make that test’s uncertainty unreliable. A mixed-effects model can represent subject-level variation and repeated-measure covariance more explicitly, especially with incomplete data, but it does not change what the sequence contrast means. Model choice affects estimation and uncertainty; it cannot make sequence a within-subject formulation effect.

The additive model is another important limitation. The cancellation derivation relies on a common formulation and period contribution across sequences. If formulation response depends on period or sequence, the two sequence averages need not differ solely by a sequence main effect. That possibility requires separate assessment in the relevant model; a sequence-effect test alone cannot diagnose its cause.

Sequence effects may also reflect chance baseline imbalance between randomized groups, differences in subject characteristics, recruitment or site patterns, or other group-level influences. Randomization supports an unbiased comparison on average across repetitions of the design, but it does not guarantee identical groups in one realized study. A sequence contrast is therefore best reported as a modeled between-subject contrast with its uncertainty, not assigned a causal explanation without additional evidence.

Common pitfall: “The sequence test is not significant, so the test and reference formulations have equal performance.” This is incorrect because the sequence test compares different subjects’ averages, not the within-subject test-versus-reference contrast. Report the sequence result as a sequence-group comparison; evaluate formulation performance using the treatment contrast and its appropriate inferential procedure.

Compared with a direct treatment contrast, a sequence contrast answers a different question and usually carries more between-subject variability. Compared with a period contrast, which compares observations across the two periods, sequence compares groups assigned to different orders. Neither contrast substitutes for the treatment estimate. The sequence result can help describe the crossover model and its group structure, but its interpretation should remain limited to the contrast actually estimated.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions below.

In a complete balanced TR/RT crossover, what two groups does the sequence effect compare?
Under the additive model, why do formulation and period effects cancel from the expected sequence contrast?
A sequence contrast is estimated as 0.03 log-AUC units with a confidence interval that includes zero. Does this establish equal formulation performance? Explain.
Why can missing periods complicate the interpretation of a simple difference between sequence-group averages?
Which source of variation is associated with the classical sequence-effect test: within-subject formulation differences or subject variation within sequence?

Solutions. (1) It compares the mean subject-level averages for subjects assigned to TR and subjects assigned to RT. (2) Each sequence includes one observation from each formulation and one from each period, so averaging gives both groups the same average formulation and period contributions; subtraction removes those common terms. (3) No. The interval concerns the between-subject sequence contrast, not the within-subject formulation contrast, and inclusion of zero does not prove the sequence contrast is exactly zero. (4) Unequal missingness can cause the observed groups to contain different mixes of periods or formulations, so a raw group-average difference may no longer isolate sequence. (5) It uses subject variation within sequence as the error term; the treatment comparison uses within-subject information.