Why Pairwise Agreement Needs a Chance Correction
When a clustering is compared with reference labels, the names assigned to clusters are arbitrary: cluster 1 and cluster 2 could be exchanged without changing the partition. The adjusted Rand index (ARI) avoids dependence on those names by asking whether pairs of cells are grouped together in both partitions. It then subtracts the agreement expected from random labeling with the observed cluster sizes held fixed.
This correction matters because two partitions can share some same-cluster pairs even when their assignments are unrelated. The number of such pairs depends partly on the sizes of the clusters: large clusters contain many pairs. The ARI therefore compares observed shared pairs with a margin-based expectation, rather than treating every raw agreement as evidence of meaningful similarity.
The count \(\binom{k}{2}=k(k-1)/2\) is the number of distinct pairs among \(k\) cells. Thus \(A\) and \(B\) count same-cluster pairs within each partition, while \(S\) counts the intersection of those two sets of pairs. These counts do not require cluster labels to match across the partitions.
Deriving the Fixed-Margin Chance Correction
The chance model used in the ARI holds the row and column totals fixed and randomly reallocates the second partition’s labels among the \(n\) cells. Equivalently, the observed cluster sizes \(b_1,\ldots,b_c\) remain unchanged, but their memberships are uniformly permuted over the cells. This is a specific reference model for pair agreement, not a claim that cells in a biological experiment were generated by random labels.
Consider any pair that is together in the first partition. Under the fixed-margin random-labeling model, the probability that both cells receive the same label in the second partition is the number of pairs together in that partition divided by the number of all pairs, \(B/M\). There are \(A\) pairs together in the first partition, so summing their probabilities gives the expected number \(S\) of pairs together in both:
The same expectation follows by considering a particular cell pair directly. The probability that its two labels are the same is $$ \sum_{j=1}^{c}\frac{b_j}{n}\frac{b_j-1}{n-1} =\frac{\sum_j b_j(b_j-1)}{n(n-1)} =\frac{B}{M}. $$ There are \(A\) pairs together in the first partition, which gives the result above. This derivation requires the fixed-margin random permutation model; changing the null model can change the expected agreement.
Subtracting \(AB/M\) from the observed shared-pair count \(S\) centers the numerator at zero under that model. The denominator is the average of the two same-pair totals, \(A\) and \(B\), after the same expected count is subtracted. This standard normalization sets the score to 1 when the two partitions have identical pair relationships, and to 0 when their shared-pair count equals its fixed-margin expectation.
The expression is symmetric: exchanging the two partitions swaps \(A\) and \(B\) but leaves the ARI unchanged. A value of 1 indicates identical pairwise grouping, even if the cluster names differ. A value near 0 indicates agreement near the fixed-margin expectation; it does not prove that the partitions are unrelated. A negative value means fewer shared same-cluster pairs than expected under this model. The possible negative range depends on the margins, so negative values should not automatically be described as “minus one times as similar.”
Calculating the ARI From a Contingency Table
Worked Example: Fewer Shared Pairs Than the Fixed-Margin Expectation
Suppose six hypothetical cells are assigned by two partitions as follows. The rows represent partition \(U\), and the columns represent partition \(V\).
| \(V_1\) | \(V_2\) | Row total | |
|---|---|---|---|
| \(U_1\) | 2 | 2 | 4 |
| \(U_2\) | 1 | 1 | 2 |
| Column total | 3 | 3 | 6 |
There are \(M=\binom{6}{2}=15\) total pairs. The row totals give \(A=\binom{4}{2}+\binom{2}{2}=6+1=7\), and the column totals give \(B=\binom{3}{2}+\binom{3}{2}=3+3=6\). The shared-pair count must be calculated cell by cell:
The expected shared count is \(AB/M=(7)(6)/15=2.8\), and the denominator is \((A+B)/2-AB/M=6.5-2.8=3.7\). Therefore,
The observed partitions share 2 same-cluster pairs, fewer than the 2.8 expected under fixed-margin random labeling. The negative ARI describes this below-expectation pair agreement; it is not a probability and does not establish that the clustering is biologically wrong.
Worked Example: More Shared Pairs Than Expected
Keep the same six hypothetical cells and the same row and column totals, but consider a different overlap table.
| \(V_1\) | \(V_2\) | Row total | |
|---|---|---|---|
| \(U_1\) | 3 | 1 | 4 |
| \(U_2\) | 0 | 2 | 2 |
| Column total | 3 | 3 | 6 |
The total number of pairs is again \(M=15\), with \(A=7\) and \(B=6\). The shared-pair count is \(S=\binom{3}{2}+\binom{1}{2}+\binom{0}{2}+\binom{2}{2}=3+0+0+1=4\). The fixed-margin expectation remains \(AB/M=42/15=2.8\), and the denominator remains \(6.5-2.8=3.7\). Thus,
This overlap has more shared pairs than expected for the same margins, so its ARI is positive. The margins alone do not determine the index: the arrangement of counts inside the table determines \(S\).
Worked Example: Identical Pair Relationships Despite Different Labels
Suppose eight hypothetical cells have two groups of four under each partition. The overlap table is
| \(V_1\) | \(V_2\) | Row total | |
|---|---|---|---|
| \(U_1\) | 0 | 4 | 4 |
| \(U_2\) | 4 | 0 | 4 |
| Column total | 4 | 4 | 8 |
The labels are reversed, but the partition memberships are identical. Here \(M=\binom{8}{2}=28\), \(A=B=2\binom{4}{2}=12\), and \(S=\binom{4}{2}+\binom{4}{2}=12\). The expected shared count is \(AB/M=144/28=36/7\), and the denominator is \(12-36/7=48/7\). Therefore,
This confirms that the ARI compares partitions, not the literal names attached to clusters. No preliminary relabeling or cluster matching is needed.
What the ARI Does and Does Not Compare
The unadjusted Rand index (RI) is the proportion of all cell pairs on which the partitions agree: a pair agrees if it is together in both or separated in both. The number separated in both is \(M-A-B+S\). Consequently,
The RI is easy to interpret as a fraction of agreeing pairs, but it does not subtract the fixed-margin chance expectation. For the first six-cell example, the number separated in both is \(15-7-6+2=4\), so the RI is \((2+4)/15=0.4\). The ARI instead compares the shared-pair count with its expectation of 2.8 and returns about \(-0.2162\). These are different summaries answering different questions.
Another approach is to match cluster labels and compute a classification accuracy. That requires deciding which cluster corresponds to which reference label, and it can be unsuitable when the partitions have different numbers of clusters. Pair-based ARI avoids that matching step and treats all pairs symmetrically. Neither measure resolves whether the reference labels are complete or biologically appropriate; interpretation still depends on the study’s purpose and the quality of the reference.
The ARI is a descriptive comparison of two partitions on the same cells. Its fixed-margin expected value is a centering device, not by itself a significance test or a p-value. It also does not account for dependence among cells, donor structure, or uncertainty in the reference labels. A high score is evidence of pairwise concordance with those labels, not evidence that the clustering caused a biological outcome or captured every relevant cell type.
Diagnostics, Edge Cases, and Common Mistakes
Before interpreting a computed ARI, check that both partitions cover exactly the same cells, each cell appears once in each partition, and the contingency-table margins sum to \(n\). Then check the pair counts independently: calculate \(A\) and \(B\) from the margins, and calculate \(S\) from the interior cells. A useful consistency check is \(0\leq S\leq\min(A,B)\), since shared pairs cannot exceed the pairs together in either partition.
If the denominator \((A+B)/2-AB/M\) is zero, the formula is undefined. This occurs for degenerate margins, such as both partitions placing every cell in one cluster or both placing every cell in its own cluster. Software may adopt a convention such as returning 1 for these identical degenerate partitions, but that is a convention rather than a value obtained by the displayed ratio. Report the convention used when such a case matters.
- Mistake: counting cells rather than pairs. A cell in a matching table entry is not itself a shared pair. A cell count of 2 contributes \(\binom{2}{2}=1\) pair, while a cell count of 1 contributes none.
- Mistake: using the wrong table for \(S\). Row and column margins determine \(A\), \(B\), and the chance expectation, but not the observed overlap. Compute \(S\) from the actual interior counts.
- Mistake: treating zero as “no similarity.” Zero means the shared-pair count equals its fixed-margin expectation, not that the partitions share no same-cluster pairs.
- Mistake: interpreting the ARI as a probability or a test result. It is a normalized agreement statistic. It is not the probability that a clustering is correct, and it does not provide a p-value.
The introductory material on comparing clusters with reference labels motivates the comparison; the essential refinement here is to make the chance baseline explicit and tied to the observed cluster sizes. In practice, the ARI is useful when cluster labels are arbitrary and cluster counts or sizes differ, but its result should be read alongside the reference-label limitations and other cluster diagnostics.
Confirm both assignments refer to the same \(n\) cells and construct the contingency table.
Calculate \(M\), the margin-based totals \(A\) and \(B\), and the overlap total \(S\).
Find \(AB/M\), subtract it from \(S\), and divide by \((A+B)/2-AB/M\), checking that the denominator is positive.
Describe agreement relative to fixed-margin random labeling, without treating the score as a probability or significance test.
Check Your Understanding
Try each question, then check the solutions below.
- For \(n=5\), one partition has group sizes 3 and 2, and another has group sizes 4 and 1. What are \(M\), \(A\), and \(B\)?
- In the first worked example, why does each overlap cell containing 1 contribute zero to \(S\)?
- For the same margins as the first example, suppose the observed \(S\) equals 2.8. What is the ARI, assuming the denominator is positive?
- What does an ARI of 1 say when the cluster names differ between partitions?
Solutions. (1) \(M=\binom{5}{2}=10\), \(A=\binom{3}{2}+\binom{2}{2}=4\), and \(B=\binom{4}{2}+\binom{1}{2}=6\). (2) A single cell contains no distinct pair, since \(\binom{1}{2}=0\). (3) With \(A=7\), \(B=6\), and \(M=15\), the expectation is \(2.8\) and the denominator is \(3.7\), so \(\mathrm{ARI}=(2.8-2.8)/3.7=0\). (4) The pairwise groupings are identical; the cluster labels may be permuted without changing the partition.