Paired t-Test and ANOVA Treatment Estimates

Why Compare the Paired Estimate With the ANOVA Estimate?

In a two-period crossover, each subject receives both treatments, so the data naturally support a within-subject comparison. A paired t-test summarizes those comparisons by averaging each subject’s log-scale response under treatment T minus that under treatment R. A crossover ANOVA instead estimates a treatment contrast while accounting for period and sequence. In a balanced design, these procedures give the same treatment point estimate. The reason is algebraic: the period effects enter the two sequence-specific comparisons with opposite signs and cancel when those comparisons are averaged equally.

This equality is useful as a check on calculations and as a way to understand the relationship between a familiar paired analysis and the crossover model. It does not mean the procedures always produce the same standard error, degrees of freedom, confidence interval, or test result. Those quantities depend on how period and sequence are modeled and on the assumptions used for inference.

Definition: For subject \(i\), define the signed within-subject log difference as \(D_i = Y_{iT}-Y_{iR}\), where \(Y_{iT}\) is the subject’s log response under T and \(Y_{iR}\) is the subject’s log response under R. The paired treatment estimate is \(\bar{D} = \frac{1}{n}\sum_{i=1}^{n}D_i\). Its exponent, \(\exp(\bar{D})\), is the estimated ratio of geometric means on the original response scale.

Model and Conditions for the Equality

Consider a complete two-period, two-treatment crossover with two sequences: TR (T in period 1 and R in period 2) and RT (R in period 1 and T in period 2). Let \(\tau\) denote the treatment contrast on the log scale, defined as T minus R, and let \(\pi_1\) and \(\pi_2\) denote the period effects. For the algebra below, assume an additive model: the treatment and period effects combine additively, and there is no treatment-by-period interaction or carryover effect in the response being analyzed.

Subjects may have different underlying response levels. These subject-level contributions disappear when each subject’s T response is subtracted from their R response. The remaining subject-specific errors are assumed to have mean zero for the treatment estimate to target \(\tau\). Inference additionally requires an appropriate error model. For a conventional paired t-test, the subject-level differences are treated as independent observations from a normal distribution with a common mean and variance. This is an assumption about the differences, not a consequence of subtracting paired observations.

The balance condition is that the same number of complete subjects is observed in each sequence. Both sequences must be represented. With unequal sequence counts, the unadjusted mean of all subject-level differences generally does not give the same estimate as the ANOVA contrast that adjusts for period.

\begin{aligned} \text{TR sequence:}\quad \operatorname{E}(D_i) &= \tau + (\pi_1-\pi_2),\\ \text{RT sequence:}\quad \operatorname{E}(D_i) &= \tau + (\pi_2-\pi_1). \end{aligned}

The sequence-specific period contributions have opposite signs because the treatment order is reversed. Averaging the two expectations with equal weight eliminates the period contrast and leaves \(\tau\). This is the core of the result.

Derivation: The Paired Mean Is the Balanced ANOVA Contrast

Let \(n_{TR}\) and \(n_{RT}\) be the numbers of complete subjects in the two sequences, and let \(\bar{D}_{TR}\) and \(\bar{D}_{RT}\) be their respective mean signed differences. The paired mean across all subjects is a sample-size-weighted average of these sequence means. If the design is balanced, \(n_{TR}=n_{RT}=m\), so that weighted average gives each sequence equal weight.

\begin{aligned} \bar{D} &= \frac{n_{TR}\bar{D}_{TR}+n_{RT}\bar{D}_{RT}}{n_{TR}+n_{RT}}\\ &= \frac{m\bar{D}_{TR}+m\bar{D}_{RT}}{2m}\\ &= \frac{\bar{D}_{TR}+\bar{D}_{RT}}{2}. \end{aligned}

Under the additive crossover model, the expected sequence means are \(\tau+(\pi_1-\pi_2)\) and \(\tau+(\pi_2-\pi_1)\). Their equally weighted average is

\frac{\{\tau+(\pi_1-\pi_2)\}+\{\tau+(\pi_2-\pi_1)\}}{2} = \frac{2\tau}{2} = \tau.

In the balanced crossover ANOVA, the treatment contrast is the estimate of the common T-minus-R effect after accounting for the period contrast. The same two sequence-specific differences enter that contrast with equal weight. Therefore, its treatment estimate is \((\bar{D}_{TR}+\bar{D}_{RT})/2\), exactly the pooled paired mean in a balanced design. This proves equality of the point estimates under the stated design and model structure. It does not prove that the two analyses estimate uncertainty in the same way.

Worked Example: Calculating the Estimate in a Balanced Crossover

Suppose a hypothetical balanced crossover has three subjects in each sequence. All responses below are log-scale values, and each difference is defined as log T minus log R.

Sequence	Log response under T	Log response under R	\(D_i\)
TR	0.50	0.42	0.08
TR	0.72	0.63	0.09
TR	0.63	0.53	0.10
RT	0.61	0.53	0.08
RT	0.44	0.35	0.09
RT	0.80	0.70	0.10

The sequence means are \(\bar{D}_{TR}=(0.08+0.09+0.10)/3=0.0900\) and \(\bar{D}_{RT}=(0.08+0.09+0.10)/3=0.0900\). Thus the balanced ANOVA treatment contrast is

\hat{\tau} = \frac{0.0900+0.0900}{2} = 0.0900.

The paired estimate calculated directly from all six subjects is

\bar{D} = \frac{0.08+0.09+0.10+0.08+0.09+0.10}{6} = \frac{0.54}{6} = 0.0900.

Both methods therefore estimate a log-scale T-minus-R contrast of 0.0900. Exponentiating gives \(\exp(0.0900)=1.094\), an estimated geometric-mean ratio of about 1.094 for T relative to R. This is an estimate, not by itself a conclusion about equivalence.

What the Equality Does—and Does Not—Say About the Paired t-Test

A paired t-test uses the differences \(D_i\). To test a null mean difference \(\mu_D=0\), its statistic is \(t=\bar{D}/(s_D/\sqrt{n})\), where \(s_D\) is the sample standard deviation of the differences; under the usual paired t-test assumptions, the reference distribution has \(n-1\) degrees of freedom. Its estimated treatment effect is \(\bar{D}\), so the balanced-design result above identifies the same point estimate as the ANOVA treatment contrast.

However, if the period effect is nonzero, the distribution of \(D_i\) can have different means in the TR and RT sequences. In a balanced design the pooled mean still cancels that period effect, but the pooled sample variance can reflect both within-sequence variation and the difference between sequence means. An ANOVA that models period explicitly estimates residual variation after accounting for the sequence-specific period pattern. Consequently, the paired t-test and ANOVA need not have the same standard error, degrees of freedom, or p-value.

When the period effect is absent and the analyses use compatible assumptions and error estimates, their results may align more closely. Even then, analysts should verify the model and degrees of freedom rather than infer equality of the entire analyses from equality of the treatment estimates alone. In particular, a paired t-test is not a substitute for specifying the crossover model when period or sequence structure matters.

Worked Example: Seeing Period Cancellation in Sequence Means

Consider a second hypothetical balanced crossover with two subjects per sequence. Suppose the TR signed differences are 0.10 and 0.12, while the RT signed differences are 0.02 and 0.04. These values are consistent with a common treatment contrast of 0.07 and a period contrast of 0.04: the TR differences add 0.04, while the RT differences subtract 0.04.

The sequence means are \(\bar{D}_{TR}=(0.10+0.12)/2=0.11\) and \(\bar{D}_{RT}=(0.02+0.04)/2=0.03\). The ANOVA treatment contrast is the equally weighted mean:

\hat{\tau} = \frac{0.11+0.03}{2} = 0.07.

The paired mean across all four subjects gives the same result:

\bar{D} = \frac{0.10+0.12+0.02+0.04}{4} = \frac{0.28}{4} = 0.07.

Here, simply pooling the signed differences gives the correct balanced treatment estimate despite a clear difference between the sequence means. But the differences are not distributed around one common sequence-specific mean in the way a simple paired t-test assumes. The point-estimate equality does not make the paired t-test’s uncertainty calculation automatically appropriate.

Balance, Diagnostics, and Alternatives

The equality depends on the weighting of the sequences. If there are more TR subjects than RT subjects, the pooled paired mean gives more weight to the TR sequence. A treatment contrast adjusted for period in the crossover model instead averages the two sequence-specific estimates equally under this model. The estimates can then differ, as the next example shows.

Worked Example: Why Sequence Imbalance Can Change the Estimate

Suppose a hypothetical incomplete allocation has two TR subjects with signed differences 0.12 and 0.08, and one RT subject with difference 0.02. The sequence means are \(\bar{D}_{TR}=(0.12+0.08)/2=0.10\) and \(\bar{D}_{RT}=0.02\).

The unadjusted paired mean weights each subject equally:

\bar{D} = \frac{0.12+0.08+0.02}{3} = \frac{0.22}{3} = 0.07333.

The period-adjusted treatment contrast under the additive model averages the two sequence means equally:

\hat{\tau}_{ANOVA} = \frac{0.10+0.02}{2} = 0.06000.

The estimates differ because the paired mean gives two-thirds of its weight to TR, whereas the adjusted contrast gives half the weight to each sequence. With unequal sequence counts, the adjusted contrast may also have greater uncertainty, particularly if one sequence has few subjects. This example illustrates the weighting difference; it is not a reason to discard subjects or alter the analysis after seeing results.

Practical checks help identify when the simple relationship is a poor guide:

Check sequence counts and completeness. Confirm how many complete subjects contribute to each sequence. Equal counts are needed for the pooled paired mean to give equal sequence weight.
Keep the difference direction consistent. Define every difference as T minus R. Reversing the subtraction for one sequence changes the estimand and prevents period cancellation.
Inspect sequence-specific differences. Markedly different means may be consistent with a period effect; they also warn that a single pooled-difference distribution may not describe the data well.
Check the model structure. Treatment-by-period interaction, carryover, missing periods, or unequal residual variances can undermine the simple additive argument or require a different model.
Check the paired-test assumptions. Differences should be independent across subjects, and normality of differences should be plausible for the small-sample t inference. Residual and influence diagnostics can reveal departures that matter.

The paired analysis is simple and transparent when complete paired observations are available and its assumptions are suitable. A crossover ANOVA represents treatment, period, and sequence structure directly and is often preferable when that structure is central to the estimand or when data are incomplete or unbalanced. More flexible mixed models can address additional covariance or missing-data structures, but their estimates depend on the specified model. None should be chosen merely because its numerical result is preferred.

Common Mistake

Common mistake: “The paired mean equals the ANOVA treatment estimate in a balanced crossover, so the paired t-test and ANOVA must have the same p-value.” The first clause concerns the point estimate only. The period term can affect the variability of pooled paired differences, while an ANOVA may account for period when estimating residual variation. Compare the model assumptions, standard errors, and degrees of freedom separately before comparing tests.

Key takeaway: In a complete, balanced two-period crossover with a consistent T-minus-R definition and an additive treatment-and-period model, the paired mean log difference equals the period-adjusted ANOVA treatment estimate. Equality of estimates does not establish equality of inferential procedures.

Check Your Understanding

Answer each question, then compare with the solutions below.

In a balanced crossover, the TR and RT sequence means of \(D_i\) are 0.06 and 0.10. What is the balanced ANOVA treatment estimate?
Why does the period effect cancel when the sequence-specific differences are averaged equally?
With three TR subjects and one RT subject, why might the pooled paired mean differ from the period-adjusted ANOVA contrast?
Does equality of the paired and ANOVA point estimates prove that their p-values are equal? Explain.

Solutions. (1) The estimate is \((0.06+0.10)/2=0.0800\). (2) The TR mean contains the period contrast \(\pi_1-\pi_2\), while the RT mean contains its negative, \(\pi_2-\pi_1\); their sum is zero. (3) The pooled paired mean weights subjects equally and therefore gives more weight to the larger TR sequence. The adjusted contrast gives equal weight to the two sequence means under the additive model. (4) No. The standard errors, degrees of freedom, and p-values can differ because the analyses may model period-related variation differently.