Evaluating Clustering Resolution Across a Parameter Grid

Why Evaluate a Grid of Resolutions?

Graph-clustering methods can return different partitions as their resolution parameter changes. A low resolution may combine cells into a few broad communities; a higher resolution may divide those communities into smaller groups. Neither the largest nor the smallest number of clusters is automatically best. The practical question is whether a range of parameter values gives a coherent description of the graph, and where the structure begins to change sharply.

A grid evaluation is a structured comparison of partitions produced at several specified resolution values. It records at least the number of clusters, the distribution of cluster sizes, and agreement between partitions. These summaries describe how the output changes; they do not by themselves establish that a partition corresponds to biological cell types.

Definition: Let $\gamma_1 < \gamma_2 < \cdots < \gamma_m$ be a prespecified grid of resolution values, and let $P_r$ denote the partition of the same $n$ cells obtained at $\gamma_r$. For each $P_r$, the cluster count is its number of groups and the cluster-size profile is the list of group sizes, usually reported in sorted order. A resolution plateau is a contiguous part of the grid over which these summaries and partition comparisons change little, according to stated criteria. It is a descriptive feature of the evaluated grid, not a guarantee of biological validity.

Interpretation requires a controlled comparison. Keep the cells, features, preprocessing, graph construction, clustering method, and other algorithm settings fixed across the grid. If those change as well as resolution, a difference between partitions cannot be attributed specifically to resolution. Where the algorithm has stochastic elements, use a documented and consistent handling of randomness; a single run at each value describes those runs, not all possible outputs. The earlier material on stochasticity and random seeds is relevant to this design choice.

For a common modularity-based resolution parameter, increasing the parameter tends to favor finer communities. This is a tendency, not a mathematical guarantee that the cluster count must increase at every step. The precise meaning and scale of a resolution parameter are method-dependent, so report the method and the actual grid values rather than assuming that equal increments have equal substantive meaning.

What to Record at Each Resolution

For each grid value, report the cluster count $K_r$ and all cluster sizes, or a compact profile that still reveals small groups. Useful summaries include the smallest cluster size, the number of singleton clusters, and the fraction of cells in clusters below a prespecified size. These complement $K_r$: two partitions can have the same count while differing markedly in whether their sizes are balanced or dominated by tiny groups.

Then compare partitions. The adjusted Rand index (ARI) is a chance-adjusted measure of pairwise agreement between two clusterings of the same cells. It is 1 for identical partitions, is approximately 0 at its chance-adjusted baseline, and can be negative when agreement falls below that baseline. It is symmetric: comparing $P_r$ with $P_s$ gives the same value as comparing $P_s$ with $P_r$. Compute it for neighboring grid values to detect local changes; an all-pairs matrix can additionally show whether a plateau is coherent relative to more distant grid points.

Definition: For two partitions, let $n_{ij}$ be the number of cells in cluster $i$ of the first partition and cluster $j$ of the second. Let $a_i=\sum_j n_{ij}$, $b_j=\sum_i n_{ij}$, and $N={n \choose 2}$. When the denominator below is nonzero, the adjusted Rand index is $$ \operatorname{ARI}= \frac{\displaystyle\sum_{ij}{n_{ij}\choose 2} -\frac{\displaystyle\left(\sum_i{a_i\choose 2}\right)\left(\sum_j{b_j\choose 2}\right)}{N}} {\displaystyle\frac{1}{2}\left(\sum_i{a_i\choose 2}+\sum_j{b_j\choose 2}\right) -\frac{\displaystyle\left(\sum_i{a_i\choose 2}\right)\left(\sum_j{b_j\choose 2}\right)}{N}}. $$ Here ${q\choose 2}=q(q-1)/2$ counts pairs among $q$ cells. The formula is undefined when its denominator is zero; the usual convention assigns ARI $=1$ to identical degenerate partitions, including the partition into one cluster and the partition into all singletons. Check for this case rather than evaluating the displayed fraction as $0/0$.

The numerator subtracts the expected number of pairs placed together in both partitions under the standard fixed-marginal chance model. The denominator is the standard ARI normalization quantity: the midpoint of the two pair totals minus the expected term; it need not be an attainable maximum excess. In particular, $\sum_{ij}{n_{ij}\choose 2}$ counts pairs assigned together in both partitions. The expected term is the product of the two partitions’ together-pair totals divided by $N$. Subtracting that expectation from the midpoint of the two observed together-pair totals gives the denominator in the displayed chance-adjusted comparison.

Worked Example: Reading Cluster Counts and Sizes Across a Grid

Suppose a hypothetical analysis clusters the same 12 cells at six resolution values. Its recorded sizes are as follows.

Resolution	Number of clusters	Cluster sizes, sorted	Singletons
0.20	2	8, 4	0
0.30	2	8, 4	0
0.40	3	6, 4, 2	0
0.50	3	6, 4, 2	0
0.60	5	6, 2, 2, 1, 1	2
0.70	7	6, 1, 1, 1, 1, 1, 1	6

The size entries sum to 12 in every row, as required. For example, at resolution 0.60, $6+2+2+1+1=12$, and $2/12=0.167$ of the cells are singletons. At 0.70, $6+6(1)=12$, so half the cells are singletons. The count and size profile suggest two candidate areas of little change: 0.20–0.30 and 0.40–0.50. The increase from three to five clusters and the appearance of singletons at 0.60, followed by six singletons at 0.70, flag a possible fragmentation region.

These summaries alone do not prove that partitions at 0.20 and 0.30 are identical, or that partitions at 0.40 and 0.50 agree cell by cell. Calculate pairwise ARI before describing either interval as a partition-level plateau. Nor does the table show that the extra small groups at high resolution are errors: that requires substantive interpretation and further evidence.

Using Pairwise Agreement

A contingency table makes the ARI calculation transparent. Its rows are clusters from one partition and its columns are clusters from the other. Large diagonal-like entries indicate that groups mostly correspond, although the clusters need not have matching labels. The following example shows how to calculate agreement without relying on labels such as “cluster 1.”

Worked Example: Calculating Agreement Between Two Partitions

For a hypothetical set of eight cells, partition A has groups $\{1,2,3,4\}$ and $\{5,6,7,8\}$. Partition B has groups $\{1,2,3\}$ and $\{4,5,6,7,8\}$. The contingency table is:

	B: 1, 2, 3	B: 4, 5, 6, 7, 8	Row total
A: 1, 2, 3, 4	3	1	4
A: 5, 6, 7, 8	0	4	4
Column total	3	5	8

The number of cell pairs together in both partitions is $$ \sum_{ij}{n_{ij}\choose 2}={3\choose 2}+{1\choose 2}+{0\choose 2}+{4\choose 2}=3+0+0+6=9. $$ The row pair total is $2{4\choose 2}=12$, and the column pair total is ${3\choose 2}+{5\choose 2}=3+10=13$. There are $N={8\choose 2}=28$ total pairs. Thus $$ \operatorname{ARI}= \frac{9-(12)(13)/28}{(12+13)/2-(12)(13)/28} =\frac{9-5.571}{12.5-5.571} =\frac{3.429}{6.929} \approx 0.495. $$ The partitions have moderate positive agreement above the chance-adjusted baseline, but they are not identical. This value is a comparison of these two partitions; it is not a probability that they are correct.

Locating a Candidate Plateau and a Fragmentation Point

A practical evaluation combines local comparisons with the size summaries. A high adjacent-grid ARI alongside unchanged cluster counts and similar size profiles supports local continuity. A sharp decline in adjacent ARI together with a jump in cluster count, a surge in singletons, or a marked shift in the size distribution provides stronger evidence of a structural transition than any one signal alone. No universal ARI cutoff defines “stable”; state the descriptive criteria used and consider whether conclusions are sensitive to reasonable alternatives.

Worked Example: Interpreting a Split Across Three Resolutions

Consider 12 hypothetical cells. At resolution 0.50, partition A is $\{1,2,3,4\},\{5,6,7,8\},\{9,10,11,12\}$. At 0.60, partition B is $\{1,2\},\{3,4\},\{5,6,7,8\},\{9,10,11,12\}$. At 0.70, partition C is $\{1,2\},\{3,4\},\{5,6\},\{7,8\},\{9,10\},\{11,12\}$. The cluster counts are 3, 4, and 6, respectively.

For A versus B, the nonzero contingency entries are $2,2,4,4$. Therefore the common-pair total is $1+1+6+6=14$; the row and column pair totals are both $3{4\choose2}=18$ and $1+1+6+6=14$, respectively. With $N={12\choose2}=66$, $$ \operatorname{ARI}(A,B)= \frac{14-(18)(14)/66}{(18+14)/2-(18)(14)/66} =\frac{10.182}{12.182}\approx 0.836. $$ For A versus C, six entries equal 2, so the common-pair total is 6. The row pair total is 18, and the column pair total is $6{2\choose2}=6$. Hence $$ \operatorname{ARI}(A,C)= \frac{6-(18)(6)/66}{(18+6)/2-(18)(6)/66} =\frac{4.364}{10.364}\approx 0.421. $$ The first split changes relatively few pair relationships, while by resolution 0.70 each size-4 group has divided into two pairs. The ARI decline and cluster-count increase together indicate growing fragmentation across this grid. A versus B’s high agreement does not mean the fourth cluster is necessarily meaningful; it says that the two partitions retain substantial pairwise structure in common.

A Practical Evaluation and Its Limits

Fix the comparison.
Use the same cells, graph, method, and non-resolution settings throughout. Record how stochasticity is handled and retain the exact grid values.

Summarize each partition.
Report cluster count, sorted sizes, and interpretable small-cluster summaries. Check that sizes sum to the number of analyzed cells.

Compare partitions.
Calculate ARI for adjacent grid values and, when useful, inspect all pairwise comparisons. Handle identical degenerate partitions by the stated convention.

Describe transitions cautiously.
Look for concordant changes in counts, size profiles, and agreement. Report candidate plateaus and abrupt changes as properties of the evaluated grid, not as universal optima.

Diagnostics matter. A sparse grid can miss a narrow transition, while an excessively fine grid can create a false impression of precision in a parameter whose scale is method-specific. Confirm that cell counts and size totals are consistent, investigate unexpected empty or tiny clusters, and verify that the ARI inputs use the same cells in the same comparison. If labels are mismatched or cells are omitted in one partition, the calculation is not a valid same-set comparison.

The main alternatives answer different questions. Cluster validity indices such as silhouette width, Calinski–Harabasz, Davies–Bouldin, and conductance summarize separation or graph structure for a particular partition; they can complement a grid comparison but do not replace tracking how the partition changes. The Gap Statistic compares within-cluster structure with a reference null model to help choose a cluster number; it does not, by itself, describe which cells split as resolution changes. Reference-label comparisons require suitable labels and assess agreement with them, not resolution continuity. Because ARI compares two partitions without reference labels, it can expose changes but cannot decide which resolution is biologically correct.

A stable interval is conditional on the fixed graph, preprocessing, algorithm, and sampled grid. High agreement across nearby resolutions does not establish robustness to resampling, altered features, or preprocessing choices; those are distinct questions. Conversely, abrupt fragmentation is a diagnostic signal, not proof that the finer clusters are spurious. Inspect marker patterns and biological context with appropriate methods, and avoid selecting a resolution solely because it maximizes an agreement score between neighboring grid points.

Common Mistakes

Equating a constant cluster count with an unchanged partition. The same number of clusters can contain different cell memberships. Compare assignments with ARI as well as counts and sizes.
Assuming resolution must increase cluster count monotonically. This is often an expectation for some methods, not a universal guarantee. Record and explain nonmonotone behavior rather than silently rearranging results.
Treating ARI as a probability or a correctness score. It is a chance-adjusted agreement statistic, not the probability that a partition is true and not a measure of biological validity.
Evaluating the ARI formula in a degenerate case. When its denominator is zero, do not report $0/0$; apply the conventional value of 1 for identical degenerate partitions.
Calling an interval robust because neighboring partitions agree. That conclusion concerns the tested resolution grid on the fixed analysis inputs, not changes to data or preprocessing.

Key takeaway: Evaluate resolution as a sequence of partitions: track cluster counts and sizes, compare cell memberships with pairwise ARI, and interpret abrupt changes jointly. A candidate plateau is useful descriptive evidence, not a substitute for biological validation or a guarantee of robustness.

Check Your Understanding

Try each question, then compare your reasoning with the solutions below.

Why should graph construction be held fixed when attributing partition changes to resolution?
Two neighboring resolutions have the same cluster count but different assignments. Which summary can reveal this, and what does it measure?
For two partitions of 10 cells, suppose the common-pair total is 8, the two partition pair totals are 12 and 10, and $N=45$. Calculate the ARI.
What should be done if the displayed ARI denominator is zero for two identical partitions into a single cluster?
A grid shows increasing cluster counts and many singletons at its highest values. What can be concluded, and what cannot be concluded from these summaries alone?

Solutions. (1) Otherwise graph changes are confounded with resolution changes, so the cause of partition differences is unclear. (2) The ARI compares pairwise co-membership in the two partitions, adjusted for its chance baseline; it is not a correctness probability. (3) The expected term is $(12)(10)/45=2.667$, and the denominator is $(12+10)/2-2.667=8.333$. Thus $\operatorname{ARI}=(8-2.667)/8.333\approx0.640$. (4) Do not evaluate the fraction; use the conventional ARI value 1 for identical degenerate partitions. (5) The grid shows increasing fragmentation in the outputs and identifies a region for inspection. It does not prove the new groups are biologically invalid, nor does it establish that a lower resolution is correct.