Assumptions and Sample-Size Limits of Multivariate Tests

Why the Assumptions Matter

A multivariate repeated-measures test uses the pattern of outcomes from each participant, not just a separate summary for each condition. This lets the test account for correlations among repeated observations without requiring sphericity. But accounting for those correlations requires estimating a covariance matrix. When the number of within-subject dimensions is large relative to the number of participants, that matrix can be unstable or impossible to invert.

The key checks concern the distribution and independence of participant-level outcome vectors, and whether the data provide enough independent information to estimate the covariance matrix. These are distinct questions: a data set might satisfy independence and look reasonably compatible with a normal model, yet still be too small for the required covariance estimate.

The Multivariate Model and Its Assumptions

Suppose each of \(N\) participants has an outcome measured under \(k\) repeated conditions. Write participant \(i\)'s outcome vector as \(Y_i=(Y_{i1},\ldots,Y_{ik})'\). The repeated-measures effect can be expressed in terms of \(q\) independent within-subject dimensions, often \(q=k-1\) for a test of the overall condition effect. For example, these dimensions can be formed by multiplying \(Y_i\) by a contrast matrix that removes the overall level of the outcomes. The particular choice of independent dimensions does not change the dimension \(q\) of the effect being tested.

For a one-group test, the model concerns whether the mean vector in those \(q\) dimensions is zero. With between-subject groups, it concerns differences among their mean vectors. In the latter case, the usual multivariate analysis assumes the groups have a common covariance matrix in those dimensions. In either setting, the test estimates covariance from variation between participants.

Definition: The standard multivariate repeated-measures test assumes that participant-level vectors are independent across participants; that the relevant vectors (or model residual vectors) follow a multivariate normal distribution within each group; and, when comparing groups, that the groups share a common covariance matrix. The error covariance matrix used by the test must also be full rank so it can be inverted. Correlations among repeated outcomes within a participant are allowed and are represented in that covariance matrix.

These assumptions are about vectors across conditions. Multivariate normality is stronger than saying each condition's measurements are individually normally distributed: it concerns their joint distribution. Conversely, the repeated observations within one participant are generally correlated; independence is required between participants, not between conditions within a participant. Common covariance across groups is relevant when groups are compared, but not a separate requirement for a one-group mean-vector test.

Independence depends primarily on how participants were sampled or assigned and whether observations are clustered. For instance, participants measured as individuals may not be independent if they belong to the same household or clinic and share sources of variation. A larger sample does not correct that design problem. For distributional checks, plots of participant-level vectors or model residuals can reveal unusual observations or patterns, but diagnostics cannot prove multivariate normality. Small samples also make deviations difficult to detect.

Why the Number of Conditions Creates a Sample-Size Limit

Let \(q\) be the number of independent dimensions in the multivariate effect. In the simplest one-group case, estimating their covariance requires a sample covariance matrix based on \(n\) participant vectors. In a model with \(g\) between-subject groups and \(N\) participants, the pooled residual covariance has \(N-g\) residual degrees of freedom. Those residual degrees of freedom, rather than the total number of repeated measurements, determine whether the covariance estimate can have full rank.

\operatorname{rank}(S) \leq \min(q,\nu), \qquad \nu = \begin{cases} n-1, & \text{one group},\\ N-g, & \text{pooled residual covariance across }g\text{ groups}. \end{cases}

Here \(S\) is the estimated covariance matrix in the \(q\) dimensions, and \(\nu\) is its residual degrees of freedom. A necessary condition for \(S\) to be invertible is therefore \(\nu \geq q\). For a one-group test, this becomes \(n \geq q+1\). If the effect uses all \(k-1\) independent dimensions of a \(k\)-condition design, at least \(k\) participants are needed for this mathematical condition in the one-group case.

Center the vectors.
Subtract the estimated mean (or fitted group mean) from each participant's \(q\)-dimensional vector. The centered vectors must sum to zero within a one-group sample, so at most \(n-1\) can be linearly independent.

Form the covariance estimate.
The sample covariance is built from sums of products of the centered vectors. Its rank cannot exceed the number of linearly independent centered vectors, nor the dimension \(q\).

Check the requirement for inversion.
A \(q\)-by-\(q\) covariance matrix needs rank \(q\) to be invertible. Thus, in one group, \(n-1\) must be at least \(q\); with \(g\) groups and pooled residuals, \(N-g\) must be at least \(q\).

This is a necessary estimability condition, not a recommendation for a target sample size. Even when \(\nu=q\), the covariance estimate may be poorly conditioned: a small change in the data can produce a large change in its estimated inverse. A matrix that is technically invertible may still lead to highly unstable test results. The requirement is not that the sample size exceed the number \(q(q+1)/2\) of distinct covariance entries; the rank condition is about the number of independent residual vectors relative to the dimension \(q\).

Worked Examples

Worked Example: Checking the Minimum in a One-Group Study

A hypothetical study measures an outcome under five conditions for each participant and plans a multivariate test of the overall condition effect. There are \(k=5\) conditions, so the full within-subject effect has \(q=k-1=4\) independent dimensions. The study recruits \(n=18\) participants.

For one group, the residual degrees of freedom for the covariance estimate are \(\nu=n-1=18-1=17\). The rank requirement is \(\nu\geq q\), and here \(17\geq4\). Thus, the sample size does not violate the minimum rank condition for estimating a full-rank covariance matrix in four dimensions.

This arithmetic does not establish that the analysis is appropriate. Investigators must still consider whether participants are independent, whether the four-dimensional outcome vectors are reasonably compatible with multivariate normality, and whether unusual observations or near-linear dependence make the covariance estimate unstable. Eighteen participants may be enough for possible invertibility, but the rank calculation alone says nothing about precision.

Worked Example: Calculating a Covariance Estimate From Contrast Data

Consider five hypothetical participants measured in three conditions. Use two independent differences from the first condition to represent the within-subject information. Suppose the resulting participant vectors are \((1,2)\), \((2,1)\), \((3,4)\), \((4,3)\), and \((5,5)\), in outcome units. The dimension is \(q=2\), and the one-group residual degrees of freedom are \(n-1=4\).

The vector of sample means is \((3,3)\). Subtracting it gives centered vectors \((-2,-1)\), \((-1,-2)\), \((0,1)\), \((1,0)\), and \((2,2)\). The sums of squared deviations in each coordinate are both \(10\), and the sum of cross-products is \(2+2+0+0+4=8\). Therefore,

S = \frac{1}{5-1} \begin{pmatrix} 10 & 8\\ 8 & 10 \end{pmatrix} = \begin{pmatrix} 2.5 & 2.0\\ 2.0 & 2.5 \end{pmatrix}.

Its determinant is \(2.5(2.5)-2.0(2.0)=6.25-4.00=2.25\), which is positive, so this particular estimated covariance matrix is invertible. The degrees-of-freedom check also passes: \(4\geq2\). The entries have squared outcome units, since they are variances and covariances of the two differences.

This small example illustrates estimability, not a general endorsement of five participants. With so few observations, the estimate could change substantially if even one participant's vector changed. A positive determinant in a particular sample does not guarantee that the matrix is well estimated or that the normality assumption is plausible.

Worked Example: A Multigroup Design at the Rank Boundary

A hypothetical study compares three treatment groups on ten repeated conditions, with four participants per group. There are \(N=12\) participants and \(g=3\) groups. Testing the full condition effect uses \(q=10-1=9\) dimensions. The pooled residual degrees of freedom are \(\nu=N-g=12-3=9\), exactly equal to \(q\).

The necessary rank condition is met only at its boundary: \(9\geq9\). A full-rank pooled residual covariance is possible, but there is no extra residual degree of freedom beyond the minimum. The estimate may therefore be especially sensitive to the observed data. If the study had only three participants per group, then \(N=9\), \(\nu=9-3=6\), and \(6<9\). In that design the pooled covariance matrix cannot have rank nine, so the standard procedure requiring its inverse cannot be carried out as specified.

The group sizes also matter for assessing the model. With only four participants in each group, it is difficult to evaluate the joint distribution within groups or to identify departures reliably. Satisfying the rank condition does not establish common covariance across groups, and it does not make the small-sample diagnostics conclusive.

What to Assess in Practice

A useful assessment begins with the design and the dimension of the proposed test, before interpreting any multivariate statistic. Count the independent within-subject dimensions, identify the participant-level units that supply independent information, and compute the residual degrees of freedom for the covariance estimate. If that matrix cannot be full rank, the standard test that requires its inverse is not available in that form.

If the rank condition passes, examine whether the assumptions are credible in context. Consider whether participants were sampled independently, whether clustering or repeated recruitment creates dependence, whether outcome vectors or residuals contain influential outliers, and, for group comparisons, whether a common covariance across groups is defensible. Plots and subject-matter knowledge are helpful, but no diagnostic can certify an assumption. In particular, a non-significant normality test in a small sample should not be treated as proof of multivariate normality.

When the dimension is too large relative to the residual degrees of freedom, possible responses include collecting more independent participants or narrowing the planned multivariate effect to a defensible lower-dimensional question. The choice changes what is being tested, so it should be based on the scientific aim rather than made solely to obtain a computable result. Any alternative method has its own assumptions and should not be treated as an automatic repair.

Common Mistake

Wrong reasoning: “There are many repeated observations per participant, so the study has a large sample for estimating the covariance.”
Correction: Repeated observations from the same participant form one correlated vector; they do not supply independent participants. Covariance rank depends on the number of independent participant residual vectors relative to the number \(q\) of dimensions. Count participants and residual degrees of freedom, not the total number of recorded measurements.

Key takeaway: A multivariate repeated-measures test permits within-participant correlations, but it relies on independent participants, an appropriate multivariate model, and an estimable covariance matrix. The condition \(\nu\geq q\) is a minimum rank requirement; it is not evidence that the covariance estimate is stable or that the assumptions are satisfied.

Check Your Understanding

Answer each question using the assumptions and rank arguments above.

A one-group study measures six conditions. How many dimensions are in the full condition effect, and what is the minimum number of participants for the covariance matrix to be potentially full rank?
Why does correlation among conditions within a participant not violate the independence-between-participants assumption?
A study has \(N=20\) participants in four groups and tests an effect with \(q=18\) dimensions. Calculate the pooled residual degrees of freedom and decide whether the rank condition is met.
Why does a full-rank covariance estimate not, by itself, demonstrate that the multivariate test assumptions are satisfied?
Explain why counting every repeated measurement as an independent observation gives a misleading assessment of sample size.