Why Compare the Two Approaches?
A repeated-measures effect can be tested in more than one way. A corrected univariate test adjusts the degrees of freedom of the usual repeated-measures \(F\) test when sphericity is doubtful. A multivariate test instead analyzes a vector of within-subject contrasts; common summaries are Wilks’ lambda and Pillai’s trace. Both approaches address whether the repeated-condition population means are all equal, but they rely on different assumptions and can lead to different p-values.
The comparison is useful because disagreement is not a signal that one calculation must be wrong. It can reflect covariance patterns, the amount of information available relative to the number of conditions, and the tests’ different sensitivity to departures from their assumptions. The goal is to report and interpret the analyses with those differences in view, not to choose whichever p-value is smaller.
What Each Test Uses
The univariate repeated-measures \(F\) test summarizes variation among condition means relative to within-subject error. Its usual reference distribution requires sphericity: the variances of all pairwise within-subject differences must be equal. When sphericity is questionable, the Greenhouse–Geisser procedure multiplies both numerator and denominator degrees of freedom by an estimated epsilon, \(\hat{\epsilon}_{GG}\), between \(1/(k-1)\) and 1. It does not change the observed \(F\) statistic. The adjusted degrees of freedom generally produce a larger p-value.
The multivariate approach represents each subject’s repeated observations through \(k-1\) linearly independent contrast coordinates, which may be statistically correlated. It tests whether the population mean vector for those differences is zero. Wilks’ lambda compares the determinant of the error variation matrix with the determinant after adding the effect variation; smaller values tend to indicate stronger evidence against \(H_0\). Pillai’s trace summarizes the effect relative to effect-plus-error variation; larger values tend to indicate stronger evidence against \(H_0\). Their p-values are obtained using the appropriate multivariate reference calculation for the design.
These methods are not interchangeable “versions” of one calculation. A univariate correction addresses a particular failure of the univariate reference distribution. A multivariate test avoids the sphericity requirement, but its usefulness can be limited by a small sample or by a covariance matrix that is difficult to estimate. Neither p-value should be interpreted without checking the assumptions relevant to its method.
Worked Example: A Corrected Test and Two Multivariate Summaries
Worked Example: A Corrected Test and Two Multivariate Summaries
Suppose a hypothetical study records a continuous outcome for each of 12 independent participants in three repeated conditions. The repeated-measures analysis gives \(F=4.80\), with uncorrected degrees of freedom \(2\) and \(22\). The estimated Greenhouse–Geisser epsilon is based on a sample covariance summary with \(\operatorname{tr}(S)=4.25\) and \(\operatorname{tr}(S^2)=16.0625\). A multivariate analysis of the same participants reports Wilks’ \(\Lambda=0.49\), \(p=0.028\), and Pillai’s trace \(V=0.51\), \(p=0.028\). Treat the reported multivariate p-values as the output from the appropriate multivariate calculations for this hypothetical design.
Check the univariate correction. With \(k=3\), the Greenhouse–Geisser estimate is
The corrected degrees of freedom are \(0.5623\) times the original values:
The observed \(F=4.80\) remains unchanged. Suppose the corrected univariate output gives \(p\approx0.044\). At \(\alpha=0.05\), this corrected test rejects \(H_0\). In context, the corrected analysis provides evidence at the 5% level that the population mean outcome differs among the three conditions.
Compare the multivariate results. Wilks’ lambda is relatively small, and Pillai’s trace is relatively large in the direction of evidence against equal means. Their reported p-values, 0.028 and 0.028, are both below 0.05, so each multivariate test rejects \(H_0\) at that significance level. Thus both the multivariate tests and the corrected univariate test provide evidence of a condition effect.
The methods reach the same inferential decision here, although their p-values differ. The corrected univariate test reflects its fractional degrees of freedom, while the multivariate tests use the contrast-vector covariance information and do not impose sphericity. The analyst should report the methods, assumptions, and results rather than presenting only the significant conclusion.
Why the Results May Disagree
The univariate test and the multivariate tests weight and summarize the within-subject information differently. The Greenhouse–Geisser adjustment compresses the reference degrees of freedom according to the estimated departure from sphericity. The multivariate tests use the covariance among the contrast coordinates directly. When the covariance structure departs substantially from sphericity, the adjustment can have a noticeable effect, while the multivariate results may retain evidence that is not apparent in the corrected univariate test.
However, “does not require sphericity” does not mean “requires no covariance assumptions.” A multivariate analysis needs enough participants to estimate the covariance structure across the \(k-1\) dimensions. When the sample is small relative to that dimension, estimates can be unstable and the multivariate reference calculations may have limited practical reliability. A multivariate p-value that is smaller than the corrected univariate p-value is not automatically more trustworthy.
Pillai’s trace is often regarded as comparatively robust among the common multivariate criteria, but that does not remove the need to assess the design, sample size, and model assumptions. In general multivariate tests with multiple nonzero roots, Wilks’ lambda and Pillai’s trace can differ because they summarize multivariate evidence differently; for the single repeated-measures factor mean-equality test discussed here, the hypothesis has rank at most 1, so the criteria are complementary transformations of the same sole nonzero root and yield the same p-value. If they yield different decisions in a general multivariate test, the discrepancy is a reason to examine the evidence and assumptions, not a license to select a preferred result.
Worked Example: A Small Sample and Several Conditions
A hypothetical study has \(n=7\) participants measured in \(k=4\) conditions. Assume the participants are independent, but note that there are only 7 subjects for a multivariate analysis involving \(k-1=3\) contrast dimensions. Suppose the uncorrected repeated-measures output is \(F=3.00\) with degrees of freedom \(3\) and \(18\), and \(\hat{\epsilon}_{GG}=0.60\).
Compute the adjusted degrees of freedom.
The test statistic remains \(F=3.00\). Suppose the corrected univariate output reports \(p=0.097\). At \(\alpha=0.05\), this procedure does not reject equal population means. Suppose the same analysis reports Wilks’ \(\Lambda=0.18\), \(p\approx0.057\), and Pillai’s trace \(V=0.82\), \(p\approx0.057\). Neither multivariate test rejects at the 5% level either.
Here the decisions agree, but their agreement does not establish that all assumptions are satisfied. The covariance estimate for the multivariate analysis is based on a small sample in three contrast dimensions, so it warrants caution. The corrected univariate result also depends on the Greenhouse–Geisser adjustment being an appropriate response to the sphericity concern. A responsible interpretation reports the limited sample and avoids treating the matching non-significant results as proof that the condition means are identical.
When Agreement Is Reassuring—and When It Is Not
When the corrected univariate test and both multivariate tests lead to the same decision, the shared conclusion is less dependent on which of these standard approaches is used. This is useful evidence of robustness to the choice of test, but it is not proof that the covariance assumptions hold or that the study has adequate power. In particular, agreement in failing to reject \(H_0\) does not show that the means are equal.
Worked Example: Agreement Across Procedures
In a hypothetical study, 30 independent participants each have an outcome measured in three conditions. Assume the repeated observations are complete and that the analyses are otherwise appropriate. Suppose the usual univariate statistic is \(F=8.10\), and the estimated Greenhouse–Geisser epsilon is \(0.75\). The uncorrected degrees of freedom are \(2\) and \(58\).
The adjusted degrees of freedom are
The \(F\) statistic is still \(8.10\); suppose the corrected univariate output reports \(p=0.001\). The corresponding multivariate output reports Wilks’ \(\Lambda=0.58\), \(p=0.00049\), and Pillai’s trace \(V=0.42\), \(p=0.00049\). All three p-values are below \(\alpha=0.05\), so all three procedures reject \(H_0\). In context, the results provide evidence that the population mean outcome is not the same in all three conditions.
This agreement makes the conclusion less sensitive to choosing between these methods for this analysis. It does not show which particular conditions differ, establish that the effect is clinically important, or prove that the model assumptions are exact. Those are distinct questions that require suitable follow-up analysis and interpretation.
A Practical Comparison
A clear comparison starts with the shared hypothesis and then reports the methods separately. State the uncorrected \(F\), the epsilon estimate and corrected degrees of freedom, and the corrected p-value. Report the multivariate criterion or criteria and their p-values, alongside the sample size and number of repeated conditions. Then describe whether the decisions agree at the stated \(\alpha\), and identify the covariance and sample-size considerations that matter for interpreting that comparison.
Verify that each procedure concerns the same omnibus equality of repeated-condition population means.
Check the estimated epsilon, adjusted degrees of freedom, unchanged \(F\) statistic, and corrected p-value.
Identify whether Wilks’ lambda, Pillai’s trace, or both are reported, and use the p-values from the appropriate multivariate analysis.
Describe agreement or disagreement, then consider sphericity, covariance estimation, and the number of subjects relative to the contrast dimensions.
Common Mistake and Key Takeaway
Common mistake: “The multivariate test is significant and the corrected univariate test is not, so the multivariate method must be the correct one.” This reasoning treats a difference in p-values as a contest and ignores the methods’ distinct assumptions. The correction is to state that both procedures test the same omnibus mean-equality hypothesis but use different covariance and reference-distribution strategies. Assess whether the sample supports the multivariate covariance estimate, whether the univariate correction is suitable, and report the results transparently.
The comparison matters because a single conclusion can depend on how within-subject covariance is handled. Agreement can strengthen confidence that the broad decision is not sensitive to the method, while disagreement calls for careful attention to assumptions and sample size. Neither outcome replaces sound design or makes a p-value a measure of effect size.
Check Your Understanding
Use the ideas in this tutorial to answer the following questions.
- A Greenhouse–Geisser correction changes the degrees of freedom but leaves \(F\) unchanged. Why can the p-value nevertheless change?
- What does it mean if the corrected univariate test has \(p=0.04\) and Pillai’s trace test has \(p=0.08\) at \(\alpha=0.05\)?
- Why can a multivariate result be difficult to rely on when there are few subjects relative to the number of repeated conditions?
- If Wilks’ lambda, Pillai’s trace, and the corrected univariate test all fail to reject \(H_0\), can you conclude that the condition population means are equal? Explain.
- What information should accompany a report that compares corrected univariate and multivariate results?