The Friedman Test for Blocked or Repeated Measures

Why Compare Conditions Within Blocks?

Suppose a researcher wants to compare three ways of completing a task, but people differ substantially in their usual speed. Comparing the three groups of times as if they came from different people would ignore a useful feature of the design: each person tried all three methods. The person can serve as a block, so each method is compared with the other methods for the same person.

The Friedman test is a rank-based method for comparing three or more related conditions. It is used with a complete blocked or repeated-measures design: each block has one observation under every condition. Rather than comparing the original measurements directly, it ranks the condition observations separately within each block, then asks whether some conditions tend to receive higher ranks than others.

Definition: In a blocked or repeated-measures design with $b$ blocks and $k$ conditions, each block contributes one observation for every condition. The Friedman test ranks the $k$ observations within each block and tests whether the conditions have systematically different ranks across the $b$ blocks.

A block might be one participant measured under several conditions, one matched set of individuals each assigned to a different condition, or a group of experimental units that are similar in a relevant way. The observations within a block are related; different blocks should be independent of one another. The test can be useful when measurements are at least ordinal but a parametric repeated-measures analysis is not appropriate.

Hypotheses, Ranking, and the Test Statistic

Let $r_{ij}$ denote the rank assigned to condition $j$ within block $i$, where $i=1,\ldots,b$ and $j=1,\ldots,k$. The smallest observation in a block receives rank 1 and the largest receives rank $k$. Tied observations receive their average rank. The rank sum for condition $j$ is $R_j=\sum_{i=1}^{b}r_{ij}$.

The null hypothesis is that the conditions have no systematic effect on the within-block ranks. Under the null, each condition is equally likely to receive any rank within a block, when the condition labels are exchangeable under the design. The alternative is that at least one condition tends to receive different ranks. The test is omnibus: a significant result indicates evidence of a difference somewhere among the conditions, but does not by itself identify which conditions differ.

Friedman statistic: With no ties, calculate the statistic $$ Q=\frac{12}{bk(k+1)}\sum_{j=1}^{k}R_j^2-3b(k+1). $$ For a sufficiently large number of independent blocks, the null distribution of $Q$ is approximated by a chi-square distribution with $df=k-1$. The p-value is the upper-tail probability $P(\chi^2_{k-1}\geq Q)$.

A large $Q$ means the condition rank sums are more separated than would typically be expected if ranks were distributed evenly across conditions. The chi-square approximation becomes more dependable as the number of independent blocks grows. With few blocks, the approximation may be rough; an exact or design-based permutation calculation can be preferable when available.

Why the Formula Measures Rank Differences

Within every block, the ranks are $1,2,\ldots,k$, so their total is $k(k+1)/2$. Across $b$ blocks, the total of all condition rank sums is therefore $bk(k+1)/2$. Under the null, each condition has expected rank $(k+1)/2$ in each block, and thus expected rank sum $b(k+1)/2$.

The Friedman statistic can also be written as a scaled sum of squared deviations of the rank sums from their null expectations:

Q=\frac{12}{bk(k+1)} \sum_{j=1}^{k}\left(R_j-\frac{b(k+1)}{2}\right)^2.

To see that this is the same formula, expand the squared terms. Since $\sum_j R_j=bk(k+1)/2$, the cross-product term is $-b(k+1)\sum_j R_j$, and the constant term is $k b^2(k+1)^2/4$. Multiplying the expanded sum by $12/[bk(k+1)]$ gives

\frac{12}{bk(k+1)}\sum_{j=1}^{k}R_j^2 -6b(k+1)+3b(k+1) = \frac{12}{bk(k+1)}\sum_{j=1}^{k}R_j^2-3b(k+1).

Thus $Q$ is zero when every condition has exactly its expected rank sum, and it grows as the rank sums depart from that balance. Its chi-square reference distribution with $k-1$ degrees of freedom accounts for the fact that the $k$ rank sums have a fixed total, leaving $k-1$ free contrasts.

Worked Example: Comparing Three Conditions Across Participants

In a hypothetical exercise, five participants each rate discomfort on a 0-to-20-point scale after trying three relaxation conditions. Lower ratings indicate less discomfort. The measurements, in points, are shown below. Assume the condition order was randomized for each participant and that the participants are independent of one another.

Participant (block)	Condition A	Condition B	Condition C
1	8	10	12
2	7	9	8
3	11	10	13
4	6	8	7
5	9	12	10

Rank within each participant, with rank 1 assigned to the lowest rating. There are no ties. The ranks for A across participants are $1,1,2,1,1$, giving $R_A=6$. The ranks for B are $2,3,1,3,3$, giving $R_B=12$. The ranks for C are $3,2,3,2,2$, giving $R_C=12$. The rank sums total $6+12+12=30$, which agrees with $b k(k+1)/2=5(3)(4)/2=30$.

Here $b=5$ and $k=3$. Substitute the rank sums into the Friedman formula:

Q=\frac{12}{5(3)(4)}(6^2+12^2+12^2)-3(5)(4) =\frac{12}{60}(324)-60 =64.8-60 =4.8.

The reference distribution has $df=k-1=2$. Its approximate upper-tail p-value is $P(\chi^2_2\geq4.8)\approx0.0907$. At significance level $\alpha=0.05$, this result is not statistically significant. The data do not provide sufficient evidence at that level that the conditions differ systematically in their within-participant ranks. This does not establish that the conditions are equivalent.

Ties and Their Correction

When two or more observations tie within a block, assign each the average of the ranks they would have occupied. For example, if two observations tie for ranks 1 and 2, each receives rank 1.5. Average ranks preserve the total rank in the block, but ties reduce the variability of the ranks. The usual correction adjusts the statistic to account for that reduction.

Tie correction: For block $i$, let each tied group have size $t$. Calculate $$ C=1-\frac{\sum_{i=1}^{b}\sum_{\text{tied groups in block }i}(t^3-t)} {b(k^3-k)}. $$ Use average ranks to calculate the uncorrected $Q$, then calculate $Q_{\text{corrected}}=Q/C$. If there are no ties, $C=1$, so the statistic is unchanged. The corrected statistic is compared with the same approximate chi-square distribution with $k-1$ degrees of freedom.

Worked Example: Applying the Tie Correction

Consider four hypothetical participants who each give a rating under three conditions. The ratings are on the same ordinal scale for every condition. Two participants have a tie within their set of three ratings. Assume the four participants are independent blocks.

Participant (block)	Condition A	Condition B	Condition C
1	2	2	5
2	1	3	3
3	4	2	1
4	5	4	3

The within-block ranks are $(1.5,1.5,3)$, $(1,2.5,2.5)$, $(3,2,1)$, and $(3,2,1)$, in the order A, B, C. Therefore $R_A=8.5$, $R_B=8$, and $R_C=7.5$. The rank sums total 24, as required by $4(3)(4)/2=24$.

The uncorrected statistic is

Q=\frac{12}{4(3)(4)}(8.5^2+8^2+7.5^2)-3(4)(4) =0.25(192.5)-48 =0.125.

There is one tied group of size 2 in block 1 and one of size 2 in block 2. Each contributes $2^3-2=6$, so the total tie contribution is 12. The denominator in the correction is $b(k^3-k)=4(27-3)=96$. Thus

C=1-\frac{12}{96}=0.875,\qquad Q_{\text{corrected}}=\frac{0.125}{0.875}\approx0.143.

With $df=3-1=2$, the approximate p-value is $P(\chi^2_2\geq0.143)\approx0.931$. At $\alpha=0.05$, there is no statistically significant evidence of a systematic difference among the three conditions. The correction changes the statistic to reflect the ties; it does not change the study design or the meaning of the test.

Worked Example: A Significant Friedman Result

In another hypothetical experiment, six independent participants complete a task using each of three protocols. Completion time is measured in seconds, and the order of protocols is randomized for each participant. Every participant's times rank A fastest, B second, and C slowest. For a concrete set of measurements, the rows are $(42,50,57)$, $(38,45,53)$, $(46,51,60)$, $(40,48,55)$, $(35,44,49)$, and $(43,47,58)$, with entries ordered A, B, C. There are no within-participant ties.

Every row therefore has ranks $(1,2,3)$. The rank sums are $R_A=6$, $R_B=12$, and $R_C=18$. With $b=6$ and $k=3$,

Q=\frac{12}{6(3)(4)}(6^2+12^2+18^2)-3(6)(4) =\frac{1}{6}(504)-72 =12.

The chi-square approximation has $df=2$, giving $P(\chi^2_2\geq12)\approx0.00248$. At $\alpha=0.05$, there is strong evidence that the protocols do not all have the same within-participant rank pattern. In this sample, A tends to have the lowest completion times and C the highest. The omnibus test establishes evidence of an overall difference; determining which pairs of protocols differ requires a separate follow-up analysis.

Assumptions and Interpretation

Before using the Friedman test, check that the design and data support its comparison:

Related conditions within blocks. Each block supplies an observation under every condition. The test is not designed for unrelated groups with no matching or repeated measurement.
Independent blocks. Dependence within a participant or matched set is expected, but different blocks should be independent. If blocks are related to one another, the standard reference calculation may not be appropriate.
Comparable measurements. The outcome must be measured on a common scale across conditions and must at least permit meaningful ordering, so within-block ranks make sense.
Exchangeability under the null. Under the null hypothesis, condition labels should be interchangeable within blocks. Randomized condition assignment or order can support this condition; in repeated-measures settings, consider whether time trends or carryover could undermine it.
A suitable approximation. The chi-square p-value is an approximation, and caution is warranted with few blocks, especially when the number of conditions is not small relative to the number of blocks.

A Friedman result concerns systematic differences in ranks across the conditions. It does not automatically test equality of means, and it does not show that one condition causes a change unless the design supports a causal interpretation. Although rank patterns can align with differences in typical values, describing the result simply as a test of medians requires additional assumptions about the distributions. State the outcome, conditions, design, and conclusion in context.

Common Mistakes

Ranking across the whole data set. The Friedman procedure ranks conditions separately within each block, not all observations together. Pooling the values discards the matched structure the test is designed to use.
Counting repeated measurements as independent blocks. Measurements from the same participant belong to one block. Treating each measurement as an independent unit misrepresents the design.
Forgetting the tie correction. When ties occur within blocks, average ranks and the correction factor should be used for the approximate test statistic.
Overstating an omnibus result. A significant result indicates evidence that the conditions do not all have the same rank pattern; it does not reveal which specific conditions differ.
Calling a non-significant result proof of no difference. A large p-value means the data do not provide strong evidence against the null under the test assumptions. It does not prove equality or equivalence.

Key takeaway: The Friedman test compares three or more related conditions by ranking observations within each block, summing ranks by condition, and evaluating how far those sums depart from their null expectations. Check the design, apply the tie correction when needed, and interpret the result as an omnibus comparison of within-block ranks.

Check Your Understanding

Answer these questions using the definitions and procedures in this tutorial.

Why does the Friedman test rank observations separately within each block rather than pooling all observations?
A study has 9 participants, each measured under 4 conditions. What are $b$, $k$, and the chi-square degrees of freedom for the approximate Friedman test?
Within one block, two conditions tie for the lowest measurement. What rank should each receive?
A Friedman test gives a p-value of 0.02 at $\alpha=0.05$. What conclusion is justified, and what does the result not identify?
Name two design or data conditions that should be checked before using the test.