Why careful reporting matters
Correcting or changing the form of a repeated-measures test changes how its evidence should be evaluated. Several errors can therefore produce a report that looks plausible but gives readers the wrong result: changing the observed F statistic during an epsilon correction, translating Wilks’ lambda into a variance percentage, or reporting a multivariate test that the sample cannot support.
This tutorial focuses on identifying and repairing those errors. The calculations use a standard one-factor repeated-measures setting: \(n\) independent participants each provide an outcome under \(k\) conditions, with complete observations. For the univariate analysis, outcomes are normally distributed within the model and the usual uncorrected F test also requires sphericity. A Greenhouse–Geisser correction adjusts the reference degrees of freedom when sphericity is doubtful; it does not recalculate the observed F statistic. The multivariate approach treats the within-participant response as a vector and requires enough participants to estimate its covariance structure.
Error 1: Changing F during an epsilon correction
For a one-factor repeated-measures analysis with \(k\) conditions and \(n\) complete participants, the uncorrected degrees of freedom are \(k-1\) for the condition effect and \((k-1)(n-1)\) for its error term. If the estimated Greenhouse–Geisser epsilon is \(\hat{\epsilon}_{GG}\), the corrected degrees of freedom are
The F statistic is still the observed ratio of the condition mean square to the error mean square. The correction changes which F distribution is used to calculate its p-value. Because the reference degrees of freedom change, the corrected p-value can differ from the uncorrected p-value, even though the observed F value is identical.
Worked Example: Repairing an F Statistic That Was “Corrected”
In a hypothetical study, \(n=10\) participants are measured under \(k=3\) conditions. The uncorrected analysis gives \(F=5.20\), with degrees of freedom \((2,18)\), and the estimated Greenhouse–Geisser epsilon is \(\hat{\epsilon}_{GG}=0.50\). A draft report says, “The corrected statistic is \(0.50 \times 5.20=2.60\).”
Check the assumptions and setup. Suppose participants are independent, observations are complete, and the outcome model is adequate. The sphericity assumption is not being relied on for the corrected test; the Greenhouse–Geisser adjustment is used to modify the reference degrees of freedom.
Correct the degrees of freedom. The numerator degrees of freedom are \(2\), and the denominator degrees of freedom are \(2(10-1)=18\). Applying the same epsilon to each gives
Keep F unchanged. The corrected test uses \(F=5.20\), not \(2.60\). Its p-value is \(P(F_{1,9}\geq 5.20)\), approximately \(0.0484\). The uncorrected p-value, \(P(F_{2,18}\geq 5.20)\), is approximately \(0.0104\). At \(\alpha=0.05\), both tests are below the significance threshold, although the corrected evidence is less strong.
A suitable report is: “The Greenhouse–Geisser-corrected repeated-measures test was \(F(1,9)=5.20\), \(p=0.0484\).” The draft’s error was multiplying the observed F statistic by epsilon; only the degrees of freedom were adjusted.
A short justification: why the F ratio stays fixed
The observed F ratio is calculated from the data before the degrees-of-freedom correction:
The epsilon correction does not alter either mean square in this ratio. It instead changes the reference distribution used to assess how unusual \(F_{\mathrm{obs}}\) would be under the null hypothesis. In probability notation, the corrected p-value is found from the upper tail of an F distribution with the corrected degrees of freedom:
Thus the value of F remains fixed while its tail probability can change. The distinction is between the observed statistic and the reference distribution used to interpret it. Reporting a corrected p-value alongside an uncorrected pair of degrees of freedom, or reporting corrected degrees of freedom alongside a newly scaled F value, mixes two different pieces of the calculation.
Error 2: Calling Wilks’ lambda an effect size
In a multivariate repeated-measures test, the within-participant response is represented by a vector of dimension \(q=k-1\). Wilks’ lambda compares the error sum-of-squares-and-cross-products matrix \(E\) with the matrix after adding the hypothesis contribution \(H\):
Here, the vertical bars denote determinants. Wilks’ lambda is a test statistic used to assess a multivariate null hypothesis. Smaller values generally indicate that the hypothesis contribution is large relative to the error structure, and so may provide stronger evidence against the null. Its value must be interpreted through the associated test procedure and its p-value.
The common error is to report \(1-\Lambda\) as the proportion of outcome variance explained. The determinant ratio does not generally equal a familiar univariate variance proportion. It depends on the multivariate dimensions and the relationship among the response components. A lambda value is therefore not a stand-alone measure of practical importance.
Worked Example: Interpreting a Wilks’ Lambda Output
Suppose a hypothetical multivariate repeated-measures output reports \(\Lambda=0.72\). A draft summary says, “The conditions explained 28% of the outcome variance because \(1-0.72=0.28\).”
Check what the calculation means. In a simplified two-dimensional illustration, suppose \(E\) has diagonal entries \(4\) and \(9\), while \(E+H\) has diagonal entries \(5\) and \(10\). Then
The arithmetic confirms the reported test statistic. Subtracting from one gives \(1-\Lambda=0.28\), but that is only the complement of this determinant ratio. It does not establish that the conditions explain 28% of outcome variance.
Repair the interpretation. Report \(\Lambda=0.72\) as the multivariate test statistic and give its associated test result, including the transformed test statistic, degrees of freedom, and p-value when available. Interpret the p-value as evidence about the null hypothesis, not as a measure of effect magnitude. To describe the size or practical importance of differences, report appropriate estimates and measures for the study question rather than relabeling Wilks’ lambda.
Error 3: Ignoring the multivariate sample-size constraint
A multivariate test estimates a covariance matrix for the response vector. For \(k\) repeated conditions, the usual multivariate formulation has dimension \(q=k-1\). With \(n\) independent participants, the sample covariance matrix has rank at most \(n-1\). To estimate an invertible \(q\)-dimensional covariance matrix, the sample must have enough independent information: in the standard one-sample formulation, \(n>q\) is required. Having just enough observations to meet this algebraic requirement does not guarantee a stable estimate or a useful test.
A frequent reporting error is to present a multivariate p-value without checking whether the covariance matrix could be estimated. Software warnings, singular matrices, or a test that cannot be computed are not minor formatting details: they may indicate that the sample does not support the analysis. Even when the calculation is technically possible, a small sample relative to the response dimension can make the covariance estimate unstable and the test’s performance fragile.
Worked Example: Checking Whether the Sample Can Support the Test
A hypothetical study measures each participant under \(k=5\) repeated conditions and has \(n=4\) complete participants. The investigator proposes the standard multivariate repeated-measures test.
Find the response dimension. With five conditions, the multivariate response dimension is \(q=k-1=4\). The sample covariance matrix is based on four participant-level response vectors, so its rank is at most \(n-1=3\).
The covariance matrix cannot have full rank in four dimensions. The proposed standard multivariate test is therefore not estimable from these data as specified; a report claiming a valid, ordinary multivariate p-value would need to be checked.
If the study instead had \(n=5\), then \(n>q\) would hold, so full rank could be possible. That boundary case is not a guarantee of a reliable analysis: the covariance estimate would be based on very little information. A defensible report should state the number of participants, the number of repeated conditions, and any limitation due to the sample size, and should not treat a software-generated number as proof that the assumptions and rank requirements were satisfied.
A practical audit for reports
Before interpreting a corrected or multivariate result, check the calculation and the report as separate steps. A well-formatted output can still contain an inappropriate analysis or an incorrect verbal interpretation. Use the following sequence:
Confirm whether the result is a univariate test with an epsilon correction or a multivariate test. Check independence across participants, the completeness and structure of the repeated observations, and the assumptions relevant to the chosen method.
For a corrected univariate test, verify that the same epsilon was applied to both degrees of freedom and that the observed F statistic was not altered. For a multivariate test, check that the sample can support the response dimension.
Use corrected degrees of freedom to interpret F, and use the associated multivariate test and p-value to interpret Wilks’ lambda. Do not present either statistic as a direct percentage of variance explained.
Give the test statistic, its degrees of freedom where applicable, the p-value, the significance level if used, and a conclusion about the condition effect in context. State material limitations, especially a sample-size constraint.
Common mistakes and corrections
| Incorrect reasoning or report | Correction |
|---|---|
| “Epsilon is 0.60, so the observed F should be multiplied by 0.60.” | Keep the observed F unchanged; multiply both univariate degrees of freedom by epsilon. |
| “The corrected F is the same, so the p-value must also be the same.” | The statistic is the same, but the corrected reference degrees of freedom can change its p-value. |
| “Wilks’ lambda is 0.72, so 28% of the variance is explained.” | Lambda is a multivariate test statistic; its complement is not generally a variance-explained measure. |
| “The software produced a multivariate result, so the sample size is adequate.” | Check the response dimension, covariance rank, assumptions, warnings, and the practical stability of the estimate. |
| “A p-value below 0.05 proves the conditions caused the outcome to change.” | A p-value assesses evidence against a statistical null under the model; it does not by itself establish causation. |
These distinctions matter because they affect both numerical conclusions and scientific meaning. A mistaken multiplication of F can change a test result; a mistaken interpretation of lambda can exaggerate what the analysis establishes; and ignoring a sample-size limitation can make an unsupported result appear definitive. Careful reporting keeps the statistic, its reference distribution, and the substantive interpretation separate.
Check Your Understanding
Use the distinctions in this tutorial to answer the following questions.
- A Greenhouse–Geisser epsilon of \(0.70\) is applied to an uncorrected test with \(df=(3,24)\) and \(F=4.10\). What are the corrected degrees of freedom, and should the reported F change?
- Why can the p-value change after an epsilon correction even when the observed F remains fixed?
- A report states that Wilks’ lambda is \(0.65\), so the conditions explained 35% of variance. What is wrong with that interpretation?
- For \(k=6\) repeated conditions, find the multivariate response dimension \(q\). What basic participant-count condition is required in the standard formulation?
- Why is meeting the algebraic sample-size requirement not, by itself, enough to claim that a multivariate test is reliable?