Introduction
Many clinical studies measure the same continuous outcome repeatedly on each participant.
For example, a clinical trial might measure blood pressure at baseline, Week 2, Week 4, Week 8, and Week 12. An oncology study might measure a continuous biomarker at several visits. A rehabilitation study might measure a functional score before treatment and at several follow-up visits.
These observations are not independent because measurements from the same patient tend to be more similar to one another than measurements from different patients.
This creates an important statistical issue. A standard one-way ANOVA assumes independent observations, whereas repeated measurements within the same subject are correlated.
Repeated measures ANOVA provides a classical ANOVA framework that explicitly accounts for this within-subject structure.
What Are Repeated Measures?
A repeated measure occurs when the same experimental unit is observed more than once.
The experimental unit is often a patient, participant, animal, site, or other subject.
For example:
| Patient | Baseline | Week 4 | Week 8 | Week 12 |
|---|---|---|---|---|
| 1 | 82 | 78 | 75 | 73 |
| 2 | 90 | 88 | 84 | 82 |
| 3 | 76 | 74 | 72 | 70 |
| 4 | 95 | 92 | 91 | 89 |
Each patient contributes four observations.
The observations within a row are correlated because they come from the same patient.
The primary question might be: Does the mean outcome change over time?
Why Not Just Use an Ordinary ANOVA?
A conventional one-way ANOVA is based on the assumption that observations are independent.
If all four observations from each patient are treated as though they were independent observations, the analysis ignores an important component of the study design.
For example, suppose 50 patients are measured at four visits.
There are:
observations.
But there are only 50 independent experimental units.
Treating all 200 measurements as independent can lead to incorrect standard errors and hypothesis tests.
The Basic Repeated Measures Question
Suppose each subject is measured at \(k\) time points. Let:
- \(Y_{ij}\) = outcome for subject \(i\) at time \(j\)
- \(i=1,\ldots,n\) = subject index
- \(j=1,\ldots,k\) = repeated-measure index
The primary null hypothesis for a one-factor repeated measures ANOVA is:
where \(\mu_j\) is the population mean at time point \(j\).
The alternative is:
The test therefore asks whether there is evidence that the mean outcome changes across the repeated conditions.
One-Way Repeated Measures ANOVA
The simplest repeated measures ANOVA contains one within-subject factor.
Time is the most common example.
Suppose the factor has four levels:
| Time Level | Visit |
|---|---|
| 1 | Baseline |
| 2 | Week 4 |
| 3 | Week 8 |
| 4 | Week 12 |
Every participant is measured at all four levels in the ideal complete-data case.
The repeated measures factor is therefore within subject.
Between-Subject vs. Within-Subject Factors
This distinction is fundamental.
Within-Subject Factor
A within-subject factor is a factor for which the same subject contributes observations at multiple levels.
Examples include:
- Time
- Dose when each participant receives multiple doses
- Experimental condition
- Measurement method
Between-Subject Factor
A between-subject factor separates subjects into different groups.
Examples include:
- Treatment group
- Sex
- Study site
- Randomization arm
A study can contain both.
Mixed Repeated Measures ANOVA
When a study contains both a between-subject treatment factor and a within-subject time factor, the design is often called a mixed ANOVA, mixed-design ANOVA, or split-plot ANOVA.
For example:
| Patient | Treatment | Baseline | Week 4 | Week 8 |
|---|---|---|---|---|
| 1 | Placebo | 82 | 80 | 79 |
| 2 | Placebo | 88 | 87 | 85 |
| 3 | Drug | 84 | 78 | 73 |
| 4 | Drug | 91 | 84 | 78 |
There are now two factors:
- Treatment: between subjects
- Time: within subjects
This design allows three major questions to be tested.
The Three Major Effects
1. Main Effect of Time
The time effect asks: Do mean outcomes change across visits, averaging across treatment groups?
The null hypothesis is:
2. Main Effect of Treatment
The treatment effect asks: Do the treatment groups have different mean outcomes, averaging across time?
3. Treatment-by-Time Interaction
The interaction asks: Does the pattern of change over time differ between treatment groups?
In many clinical trials, this interaction is particularly important.
Why the Interaction Is Often More Informative Than the Main Effect
Suppose both treatment groups improve over time.
A significant time effect would tell us that outcomes change over time, but it would not tell us whether the treatment caused a different pattern of change.
The interaction addresses the more specific question:
This is often the scientifically important question in longitudinal clinical studies.
A Simple Numerical Example
Suppose the mean outcome is:
| Treatment | Baseline | Week 4 | Week 8 |
|---|---|---|---|
| Placebo | 80 | 79 | 78 |
| Drug | 80 | 72 | 65 |
The placebo change from baseline to Week 8 is:
The drug change is:
The difference in change is:
The treatment-by-time interaction evaluates whether differences of this type are statistically distinguishable from random variation.
Partitioning Variation
ANOVA works by partitioning total variability into meaningful components.
In a one-way repeated measures design, total variation can be separated into variation associated with:
- Differences between subjects
- The repeated-measures factor
- Residual error
Conceptually:
The subject component is important because subjects may have very different baseline levels.
For example, one patient may consistently have blood pressure around 70, while another may consistently have blood pressure around 100.
Repeated measures ANOVA separates these persistent between-subject differences from within-subject changes over time.
Why Repeated Measures Can Be More Efficient
Repeated measurements can provide substantial statistical efficiency because each subject serves as their own reference.
Suppose patient A has a naturally high outcome and patient B has a naturally low outcome.
A simple comparison of their raw measurements could be dominated by this subject-to-subject difference.
Repeated measures analysis instead focuses on how each subject changes relative to their own pattern.
The Statistical Model
A simplified one-way repeated measures ANOVA model can be written as:
where:
- \(\mu\) = overall mean
- \(\alpha_j\) = effect of repeated-measures level \(j\)
- \(s_i\) = subject-specific effect
- \(\epsilon_{ij}\) = residual error
The subject effect represents persistent differences between subjects.
The repeated-measures effect represents systematic differences among the conditions or time points.
Mixed-Design Model
For a treatment-by-time design, a simplified model is:
where:
- \(T_i\) = treatment effect
- \(S_{j(i)}\) = subject nested within treatment
- \(V_k\) = visit effect
- \((TV)_{ik}\) = treatment-by-visit interaction
- \(\epsilon_{ijk}\) = residual component
The exact model representation depends on the experimental design and software implementation, but the conceptual decomposition is the same.
The F Statistic
Repeated measures ANOVA uses F statistics to compare systematic variation with appropriate residual variation.
The general form is:
where \(MS\) denotes a mean square.
A large F statistic indicates that the variation associated with the tested effect is large relative to the relevant error variation.
The corresponding p-value measures the compatibility of the observed result with the null hypothesis.
Degrees of Freedom
For a within-subject factor with \(k\) levels, the uncorrected degrees of freedom for the factor are:
If there are \(n\) subjects, the corresponding error degrees of freedom are:
For example, with 40 subjects measured at four visits:
and:
These degrees of freedom assume the standard uncorrected repeated measures ANOVA formulation.
What Is Sphericity?
One of the most important assumptions in repeated measures ANOVA is sphericity.
Sphericity concerns the variances of the differences between repeated measurements.
For all pairs of repeated conditions, the variance of their pairwise difference should be approximately equal.
For example, with four time points, consider:
and all other pairwise difference variances.
Sphericity requires these variances to be equal.
Why Sphericity Matters
When sphericity is violated, the usual repeated measures F test can become anti-conservative.
In practical terms, the nominal p-value can be too small.
This can increase the risk of falsely declaring a statistically significant time effect.
For this reason, repeated measures ANOVA commonly includes a test or assessment of sphericity.
Mauchly's Test of Sphericity
A commonly used procedure is Mauchly's test of sphericity.
The null hypothesis is that the sphericity assumption holds.
Conceptually:
A small p-value suggests evidence against sphericity.
However, Mauchly's test should not be treated as a perfect diagnostic.
Greenhouse-Geisser Correction
When sphericity is violated, one common approach is the Greenhouse-Geisser correction.
The correction multiplies the relevant degrees of freedom by an estimated epsilon parameter:
and:
The F statistic itself is typically unchanged; the degrees of freedom used to evaluate the F distribution are adjusted.
Because:
the correction generally reduces the effective degrees of freedom.
Huynh-Feldt Correction
Another correction is the Huynh-Feldt correction.
It also adjusts the degrees of freedom using an estimated epsilon.
The Huynh-Feldt correction is often less conservative than Greenhouse-Geisser.
| Approach | Purpose |
|---|---|
| Uncorrected ANOVA | Used when sphericity is appropriate |
| Greenhouse-Geisser | Adjusts degrees of freedom when sphericity is violated |
| Huynh-Feldt | Alternative, generally less conservative correction |
| Lower-bound correction | Uses the most conservative possible epsilon |
A Simple Sphericity Example
Suppose a study has four repeated measurements.
The uncorrected degrees of freedom are:
and:
Suppose the Greenhouse-Geisser estimate is:
The corrected numerator degrees of freedom are:
and the denominator degrees of freedom are:
The F statistic can then be evaluated using these corrected degrees of freedom.
Post-Hoc Comparisons
A significant overall repeated-measures test tells us that not all means are equal.
It does not identify which specific time points differ.
For example, if four visits are compared, a significant omnibus test does not by itself tell us whether the differences are:
- Baseline vs. Week 4
- Baseline vs. Week 8
- Baseline vs. Week 12
- Week 4 vs. Week 8
- Week 4 vs. Week 12
- Week 8 vs. Week 12
Post-hoc or planned pairwise comparisons are therefore needed when the specific differences are scientifically important.
Multiplicity Matters
Multiple pairwise comparisons increase the probability of obtaining at least one false-positive result.
Therefore, multiplicity should be addressed.
Common approaches include:
- Holm adjustment
- Bonferroni adjustment
- Tukey-type procedures where appropriate
- Prespecified contrasts
- Other multiplicity-controlled methods
Planned Contrasts
Sometimes the investigator has a specific hypothesis rather than an interest in every pairwise comparison.
For example, the primary question may be whether Week 12 differs from baseline.
A planned contrast can directly test:
This can be preferable to testing every possible pair.
Repeated Measures ANOVA vs. Paired t Test
A paired t test is appropriate when there are exactly two repeated measurements of interest.
For example:
- Baseline vs. Week 12
- Before treatment vs. after treatment
With more than two repeated conditions, repeated measures ANOVA provides an overall framework.
| Design | Common Method |
|---|---|
| Two measurements on same subjects | Paired t test |
| Three or more repeated measurements | Repeated measures ANOVA |
| Repeated measurements with missing observations | Often mixed-effects model |
| Non-normal or ordinal outcomes | Alternative longitudinal methods may be appropriate |
Repeated Measures ANOVA vs. Independent-Samples ANOVA
The key difference is whether the observations are independent.
| Feature | Independent ANOVA | Repeated Measures ANOVA |
|---|---|---|
| Subjects | Different subjects at each condition | Same subjects repeatedly measured |
| Within-subject correlation | Not applicable | Explicitly accounted for |
| Subject effect | Not modeled in the same way | Separated from within-subject error |
| Typical use | Different groups | Longitudinal or crossover-type measurements |
Complete Worked Example
Consider a hypothetical clinical study evaluating a continuous symptom score over time.
Twenty-four patients are measured at four visits:
- Baseline
- Week 4
- Week 8
- Week 12
Suppose the observed mean scores are:
| Visit | Mean | Standard Deviation |
|---|---|---|
| Baseline | 52.0 | 8.1 |
| Week 4 | 48.5 | 8.0 |
| Week 8 | 44.1 | 7.6 |
| Week 12 | 41.8 | 7.4 |
The descriptive statistics suggest that the outcome decreases over time.
However, the statistical question is whether the observed differences are larger than would reasonably be expected from within-subject variability.
Step 1: State the Hypotheses
The overall time effect is:
versus:
Step 2: Calculate the Mean Changes
Relative to baseline, the observed mean changes are:
The descriptive pattern suggests progressively larger reductions.
However, these mean changes alone do not establish statistical significance.
Step 3: Fit the Repeated Measures Model
In a classical repeated measures ANOVA, visit is specified as the within-subject factor.
The analysis then evaluates whether the visit means differ after accounting for subject-to-subject variability.
Suppose the resulting analysis gives:
| Effect | F | df | p-value |
|---|---|---|---|
| Visit | 18.42 | 3, 69 | <0.001 |
The small p-value provides evidence that the mean symptom score is not constant across the four visits.
Step 4: Examine Sphericity
Suppose Mauchly's test indicates evidence against sphericity.
The uncorrected result should therefore not be interpreted as though the sphericity assumption were unquestionably satisfied.
Suppose the Greenhouse-Geisser epsilon is:
The corrected degrees of freedom become:
and:
The corrected F test might then be reported as:
| Effect | F | Corrected df | p-value |
|---|---|---|---|
| Visit | 18.42 | 2.04, 46.92 | <0.001 |
The conclusion remains that the mean outcome changes significantly over time.
Step 5: Examine Pairwise Differences
Suppose multiplicity-adjusted pairwise comparisons show:
| Comparison | Mean Difference | Adjusted p-value |
|---|---|---|
| Week 4 − Baseline | −3.5 | 0.018 |
| Week 8 − Baseline | −7.9 | <0.001 |
| Week 12 − Baseline | −10.2 | <0.001 |
| Week 8 − Week 4 | −4.4 | 0.012 |
| Week 12 − Week 8 | −2.3 | 0.041 |
These comparisons provide more detailed information about where the overall time effect occurs.
How to Report the Result
A concise report might state:
The exact reporting format should follow the statistical analysis plan, journal requirements, and the software output.
Effect Size
A p-value indicates the strength of evidence against a null hypothesis, but it does not quantify the magnitude of the effect.
Effect sizes can therefore provide additional information.
Common ANOVA effect-size measures include:
- Eta squared, \(\eta^2\)
- Partial eta squared, \(\eta_p^2\)
- Generalized eta squared, \(\eta_G^2\)
Partial Eta Squared
A commonly reported measure is partial eta squared:
It represents the proportion of relevant variance attributable to the effect after accounting for the model's other components.
Assumptions of Repeated Measures ANOVA
Several assumptions should be considered.
1. Continuous Outcome
Classical repeated measures ANOVA is designed primarily for continuous outcomes.
It may not be appropriate for binary, count, or ordinal outcomes without substantial modification.
2. Independence Between Subjects
Measurements from different subjects should be independent, or the dependence must be accounted for in the study design.
3. Approximately Normal Residuals
The residuals should be reasonably compatible with the normality assumptions underlying the F tests.
Repeated measures ANOVA is often reasonably robust to moderate departures from normality, especially in balanced designs with sufficiently large samples.
4. Sphericity
For within-subject factors with more than two levels, the standard uncorrected F test assumes sphericity.
If this assumption is violated, a correction or alternative model should be considered.
What About Equal Variances?
Repeated measures ANOVA involves several different variance concepts.
It is important not to confuse:
- Equality of variances at each time point
- Equality of variances of pairwise differences
- The covariance structure among repeated observations
The classical sphericity assumption concerns the second concept.
Two Time Points Are Special
If there are only two repeated measurements, sphericity is automatically satisfied.
With two time points there is only one unique pairwise difference:
There are no multiple difference variances to compare.
Therefore, a two-time-point repeated measures ANOVA is closely related to a paired t test.
for the corresponding one-degree-of-freedom test.
Repeated Measures ANOVA With a Treatment Group
Now consider a randomized clinical trial with two treatment groups.
| Group | Baseline | Week 4 | Week 8 | Week 12 |
|---|---|---|---|---|
| Placebo | 52.1 | 51.0 | 50.2 | 49.8 |
| Drug | 51.8 | 47.0 | 42.8 | 39.9 |
The mixed ANOVA asks three different questions.
| Effect | Question |
|---|---|
| Treatment | Are the average outcomes different between treatment groups? |
| Time | Do outcomes change over time? |
| Treatment × Time | Do the treatment groups have different trajectories? |
Interpreting a Significant Treatment Effect
A significant treatment main effect means that, averaging over the repeated visits, the groups differ in their mean outcome.
It does not necessarily mean that the treatment groups differ at every visit.
Interpreting a Significant Time Effect
A significant time effect means that, averaging over treatment groups, the mean outcome changes over time.
It does not establish that the treatment caused the change.
Interpreting a Significant Treatment-by-Time Interaction
A significant interaction means that the effect of time depends on treatment group, or equivalently, that the treatment groups have different trajectories.
This is often the most clinically interesting result in a longitudinal randomized trial.
Example of an Interaction
Suppose:
| Group | Baseline | Week 12 | Change |
|---|---|---|---|
| Placebo | 50 | 48 | −2 |
| Drug | 50 | 40 | −10 |
The treatment difference in change is:
The interaction tests whether the observed difference in trajectories is greater than expected from random variation.
Data Format for Repeated Measures Analysis
Repeated-measures data are often stored in either wide or long format.
Wide Format
| Patient | Treatment | Baseline | Week 4 | Week 8 |
|---|---|---|---|---|
| 1 | Drug | 82 | 76 | 71 |
| 2 | Drug | 79 | 74 | 69 |
Each subject occupies one row and each visit occupies a separate column.
Long Format
| Patient | Treatment | Visit | Outcome |
|---|---|---|---|
| 1 | Drug | Baseline | 82 |
| 1 | Drug | Week 4 | 76 |
| 1 | Drug | Week 8 | 71 |
| 2 | Drug | Baseline | 79 |
Long format is generally more flexible for modern statistical modeling and is the preferred structure for many R workflows.
R Implementation: One-Way Repeated Measures ANOVA
Suppose the data are stored in long format with:
id= subject identifiervisit= repeated-measures factorscore= continuous outcome
A classical repeated measures ANOVA can be fit using aov().
fit <- aov( score ~ visit + Error(id/visit), data = dat ) summary(fit)
The expression:
Error(id/visit)
tells R that visit is repeated within subject.
Using the afex Package
The afex package provides a convenient framework for
factorial repeated measures ANOVA.
library(afex) fit <- aov_ez( id = "id", dv = "score", data = dat, within = "visit" ) fit
For a mixed design with treatment as a between-subject factor:
fit <- aov_ez( id = "id", dv = "score", data = dat, within = "visit", between = "treatment" ) fit
This approach can automatically provide standard corrections for violations of sphericity.
Post-Hoc Comparisons in R
The emmeans package is particularly useful for
estimated marginal means and pairwise comparisons.
library(emmeans) emm <- emmeans( fit, ~ visit ) pairs( emm, adjust = "holm" )
For treatment-by-visit comparisons:
emm_tv <- emmeans( fit, ~ treatment * visit ) pairs( emm_tv, adjust = "holm" )
Estimated Marginal Means
Estimated marginal means, sometimes called least-squares means, are model-based means adjusted for the other factors in the model.
They are particularly useful for interpreting factorial repeated measures designs.
For example:
emmeans( fit, ~ treatment | visit )
asks for treatment comparisons separately within each visit.
Conversely:
emmeans( fit, ~ visit | treatment )
asks for visit comparisons within each treatment group.
Checking Residuals
Residual diagnostics should be considered rather than relying solely on a formal normality test.
par(mfrow = c(1, 2)) plot(fit) qqnorm(residuals(fit)) qqline(residuals(fit))
Useful diagnostic questions include:
- Are there extreme outliers?
- Are residuals strongly skewed?
- Does variability change substantially across conditions?
- Are there unusual subjects driving the result?
Why Missing Data Are a Major Limitation
Classical repeated measures ANOVA is most naturally suited to balanced complete-data settings.
Clinical longitudinal studies often have missing observations.
For example:
| Patient | Baseline | Week 4 | Week 8 | Week 12 |
|---|---|---|---|---|
| 1 | 82 | 78 | 75 | 73 |
| 2 | 90 | 87 | — | 82 |
| 3 | 76 | 74 | 72 | — |
If subjects have missing visits, traditional repeated measures ANOVA may require complete cases or special handling.
This can reduce efficiency and potentially introduce bias depending on the missing-data mechanism.
Mixed-Effects Models as an Alternative
Linear mixed-effects models are often preferred for longitudinal clinical trials when repeated observations are incomplete or when the covariance structure requires greater flexibility.
A basic mixed model can be written:
where \(b_i\) is a subject-specific random effect.
In R, a simple model can be fitted using:
library(lme4) fit_lmm <- lmer( score ~ visit + (1 | id), data = dat, REML = TRUE ) summary(fit_lmm)
A random slope can also be considered:
fit_lmm <- lmer( score ~ visit + (visit | id), data = dat, REML = TRUE )
The appropriate random-effects and covariance structure should be based on the study design and scientific assumptions rather than selected mechanically.
Repeated Measures ANOVA Is Not Automatically the Best Longitudinal Model
Repeated measures ANOVA is an important foundational method, but modern clinical trials often use mixed-effects models instead.
This is particularly true when:
- Patients have different numbers of observations.
- Visits are missing.
- Visit timing is irregular.
- Subject-specific slopes are scientifically relevant.
- Different covariance structures are needed.
- Baseline adjustment is required.
Understanding repeated measures ANOVA remains valuable because it provides the conceptual foundation for many longitudinal models.
Baseline Adjustment
In clinical trials, baseline is often treated differently from post-baseline visits.
Simply treating baseline as another repeated time point is not always the most appropriate analysis strategy.
Depending on the estimand and protocol, an analysis may instead use baseline as a covariate and model post-baseline outcomes.
For example, an ANCOVA-style model might include:
For longitudinal analyses, baseline adjustment can be incorporated into a mixed-effects model as well.
Change From Baseline vs. Observed Values
Another common question is whether repeated measures ANOVA should analyze observed values or change from baseline.
Both approaches can be valid in appropriate circumstances, but they answer somewhat different modeling questions.
Change from baseline is:
The choice should be made before analysis based on the scientific estimand and statistical analysis plan.
Common Misinterpretations
- "A significant time effect proves the treatment worked." A time effect can occur because all groups change over time, including placebo.
- "A significant treatment effect proves treatment works at every visit." A main treatment effect averages over the repeated factor.
- "A non-significant interaction proves the treatment trajectories are identical." Failure to reject the interaction null does not prove equality.
- "Mauchly's test is the only assumption check needed." Residual behavior, outliers, design structure, and covariance assumptions also matter.
- "Repeated measures ANOVA handles missing data automatically." Classical repeated measures ANOVA is not as flexible as mixed-effects models for incomplete longitudinal data.
- "More observations means more independent observations." Repeated measurements within a patient remain correlated.
- "If the overall ANOVA is significant, every pairwise comparison is significant." An omnibus test only establishes that at least one mean differs.
- "A p-value describes the size of the treatment effect." Effect sizes and confidence intervals are needed to describe magnitude and precision.
Repeated Measures ANOVA vs. Mixed-Effects Models
| Feature | Repeated Measures ANOVA | Mixed-Effects Model |
|---|---|---|
| Repeated observations | Yes | Yes |
| Balanced data | Best suited | Not required |
| Missing visits | Problematic | More flexible |
| Random subject effects | Implicitly represented | Explicitly modeled |
| Flexible covariance structures | Limited | Yes |
| Random slopes | No | Yes |
| Irregular observation times | Limited | Flexible |
| Classical ANOVA interpretation | Very direct | More model-based |
When Repeated Measures ANOVA Works Well
Classical repeated measures ANOVA can be particularly useful when:
- The outcome is continuous.
- The same subjects are measured at several prespecified conditions.
- The data are reasonably complete.
- The design is balanced.
- The repeated factor has a relatively simple structure.
- The assumptions are reasonably plausible or appropriate corrections are available.
When to Consider a Mixed Model Instead
A mixed-effects model may be preferable when:
- There are missing observations.
- Subjects have different numbers of measurements.
- Follow-up times vary between subjects.
- Subject-specific trajectories are important.
- A particular covariance structure is scientifically justified.
- The study contains multiple levels of clustering.
- Baseline adjustment is required.
A Practical Analysis Workflow
How to Interpret a Mixed ANOVA Table
Suppose a clinical trial produces:
| Effect | F | df | p-value |
|---|---|---|---|
| Treatment | 5.82 | 1, 46 | 0.020 |
| Visit | 21.47 | 2.31, 106.26 | <0.001 |
| Treatment × Visit | 7.16 | 2.31, 106.26 | <0.001 |
The treatment effect suggests that the groups differ in their average outcome across visits.
The visit effect indicates that the outcome changes over time.
Most importantly, the significant treatment-by-visit interaction indicates that the pattern of change differs between treatment groups.
Plotting Longitudinal Means
A plot of group means over time is often one of the most useful ways to understand repeated-measures results.
library(ggplot2)
ggplot(
dat,
aes(
x = visit,
y = score,
group = treatment,
linetype = treatment
)
) +
stat_summary(
fun = mean,
geom = "line"
) +
stat_summary(
fun = mean,
geom = "point"
) +
labs(
x = "Visit",
y = "Mean outcome",
linetype = "Treatment"
)
The plot should ideally show uncertainty as well, such as confidence intervals or standard errors, depending on the intended presentation.
Why Plotting Matters
A statistically significant interaction can be difficult to understand from a table alone.
A trajectory plot can reveal whether:
- Groups start at similar levels.
- One treatment improves more rapidly.
- Groups diverge gradually.
- Groups cross during follow-up.
- Differences occur only at later visits.
The plot should complement, not replace, the formal statistical analysis.
Confidence Intervals Are Important
A p-value tells us about evidence against a null hypothesis.
A confidence interval provides information about the plausible magnitude and precision of an estimated difference.
For a treatment difference at a specific visit:
a confidence interval can describe the uncertainty around \(\widehat{\Delta}\).
Repeated Measures ANOVA and Clinical Trial Estimands
In modern clinical trials, the statistical analysis should be connected to the estimand being targeted.
Questions include:
- What treatment effect is being estimated?
- At what time point?
- Under what intercurrent-event strategy?
- How should treatment discontinuation be handled?
- How should rescue medication be handled?
- How should death or other intercurrent events be handled?
A repeated measures ANOVA is a statistical method; it does not by itself define the clinical estimand.
Handling Treatment Discontinuation
Suppose a patient stops treatment at Week 8 but would otherwise have a Week 12 assessment.
The appropriate treatment of the missing Week 12 value depends on the estimand and the prespecified missing-data strategy.
Possible approaches include:
- Mixed-effects modeling under an assumed missing-at-random framework
- Multiple imputation
- Reference-based imputation
- Pattern-mixture approaches
- Tipping-point analyses
The analysis should not simply discard or replace observations because doing so produces a preferred result.
Common Reporting Structure
A repeated measures analysis section might include:
- Definition of the repeated outcome
- Visit schedule
- Analysis population
- Model specification
- Within-subject factor
- Between-subject factors
- Interaction terms
- Covariates
- Sphericity assessment
- Degrees-of-freedom correction
- Pairwise comparison strategy
- Multiplicity adjustment
- Effect sizes
- Confidence intervals
- Missing-data methodology
Common Mistakes
- Ignoring within-subject correlation. Measurements from the same patient are not independent.
- Using ordinary one-way ANOVA on all repeated observations. This can produce incorrect inference.
- Ignoring sphericity. For within-subject factors with more than two levels, the assumption should be considered.
- Reporting only the uncorrected F test when sphericity is violated. An appropriate correction or alternative modeling strategy should be considered.
- Interpreting a time effect as a treatment effect. Time effects can occur in both treatment groups.
- Ignoring the treatment-by-time interaction. Different treatment trajectories are often the primary scientific question.
- Running every pairwise comparison without multiplicity control. Multiple testing should be addressed.
- Automatically treating baseline as an ordinary repeated visit. Baseline handling should reflect the estimand and analysis plan.
- Ignoring missing data. Longitudinal studies frequently have incomplete follow-up.
- Assuming repeated measures ANOVA is always superior to mixed models. Mixed-effects models often provide greater flexibility for clinical longitudinal data.
Key Formulas
The central repeated measures hypotheses are:
For a within-subject factor with \(k\) levels, the uncorrected numerator degrees of freedom are:
With \(n\) subjects, the corresponding error degrees of freedom are:
The general F statistic is:
Under a Greenhouse-Geisser correction:
Partial eta squared can be expressed as:
Quick Comparison of Common Longitudinal Methods
| Method | Typical Use |
|---|---|
| Paired t test | Two repeated measurements |
| Repeated measures ANOVA | Three or more repeated measurements in a classical balanced design |
| Mixed-effects model | Flexible longitudinal continuous-outcome analysis |
| GEE | Population-averaged longitudinal analysis for correlated outcomes |
| Repeated measures ANCOVA | Repeated outcome with covariate adjustment |
| MANOVA approach | Alternative multivariate treatment of repeated outcomes |
Repeated Measures ANOVA vs. GEE
Repeated measures ANOVA and generalized estimating equations both account for within-subject correlation, but they answer somewhat different modeling questions.
GEE focuses on population-averaged effects and can accommodate several non-normal outcome types.
Repeated measures ANOVA is traditionally associated with continuous outcomes and normally distributed errors.
The appropriate method depends on the outcome, estimand, covariance structure, missingness, and scientific objective.
What Happens If There Are Only Two Groups and Two Visits?
A common clinical trial design has:
- Two treatment groups
- Baseline and one post-baseline measurement
This can often be analyzed more simply using ANCOVA on the post-baseline outcome with baseline as a covariate.
For example:
The choice between change-score analysis, ANCOVA, and other approaches should be prespecified and driven by the estimand and design.
The Most Important Concept
The most important concept in repeated measures ANOVA is simple: the same subject contributes multiple observations, so those observations cannot be treated as independent.
Repeated measures ANOVA separates subject-to-subject variability from within-subject variation and uses the appropriate error structure for testing changes across repeated conditions.
For a simple one-factor design, the primary question is whether the repeated means differ.
For a clinical trial with treatment and time, the key questions become:
- Is there an overall treatment difference?
- Does the outcome change over time?
- Do treatment groups change differently over time?
The third question is represented by the treatment-by-time interaction.
Summary Checklist
| Question | What to Check |
|---|---|
| Are observations repeated? | Identify the subject-level experimental unit. |
| What is the within-subject factor? | Usually visit, time, or condition. |
| Are there between-subject factors? | Examples include treatment group. |
| Is there an interaction? | Evaluate treatment-by-time when scientifically relevant. |
| Is sphericity relevant? | Yes when a within-subject factor has more than two levels. |
| Is sphericity violated? | Consider Greenhouse-Geisser or Huynh-Feldt correction. |
| Are pairwise comparisons needed? | Use prespecified or multiplicity-adjusted comparisons. |
| Are there missing observations? | Consider whether a mixed-effects model is more appropriate. |
| Are effects clinically meaningful? | Report estimates, confidence intervals, and effect sizes. |
References
Box, G.E.P. (1954).
Some theorems on quadratic forms applied in the study of analysis of
variance problems, II. Effects of inequality of variance and of correlation
between errors in the two-way classification.
Annals of Mathematical Statistics, 25, 484–498.
Greenhouse, S.W. & Geisser, S. (1959).
On methods in the analysis of profile data.
Psychometrika, 24, 95–112.
Huynh, H. & Feldt, L.S. (1976).
Estimation of the Box correction for degrees of freedom from sample
data in randomized block and split-plot designs.
Journal of Educational Statistics, 1, 69–82.
Geisser, S. & Greenhouse, S.W. (1958).
An extension of Box's results on the use of the F distribution in
multivariate analysis.
Annals of Mathematical Statistics, 29, 885–891.
Verbeke, G. & Molenberghs, G. (2000).
Linear Mixed Models for Longitudinal Data.
Springer.
Fitzmaurice, G.M., Laird, N.M. & Ware, J.H. (2011).
Applied Longitudinal Analysis, 2nd ed.
Wiley.
Keselman, H.J., Algina, J., Kowalchuk, R.K. & Wolfinger, R.D. (1999).
The analysis of repeated measurements: A quantitative research
synthesis.
British Journal of Mathematical and Statistical Psychology, 52,
87–107.