What Cluster Validity Means in Single-Cell Data

What Does It Mean for a Cluster to Be Valid?

A clustering algorithm always returns groups when asked to do so, but a returned partition is not automatically a meaningful discovery. In single-cell RNA sequencing, cells are grouped using a representation of their measured expression, often after choices about normalization, feature selection, dimensionality reduction, and graph construction. The central question of cluster validity is therefore not simply whether an algorithm produced a result. It is what evidence supports that result, relative to what target, and under what assumptions?

Three forms of evidence are especially useful to distinguish. Internal validity asks whether the clusters fit some structure in the data used to construct them. External agreement asks whether the partition corresponds to labels or measurements not used to create it. Perturbation stability asks whether similar clusters recur after specified changes to the analysis or data. Each addresses a different concern. A compact partition can disagree with known labels; a partition can match labels yet be sensitive to small changes; a stable partition can consistently reflect a technical artifact.

A validity claim is meaningful only after specifying the object being evaluated. The same cells can have different partitions at different resolutions, and the same partition can look different in different representations. A result about Euclidean distances in a selected low-dimensional embedding is not automatically a result about all expression measurements, cell types, or biological mechanisms.

Internal Fit: Structure Relative to the Input

An internal criterion summarizes how well a partition fits a chosen data representation. For example, for points \(y_i\) represented in a Euclidean space, within-cluster sum of squares is

Here \(\bar{y}_k\) is the mean vector in cluster \(k\), and the squared distance measures how far each point is from that cluster's mean. Smaller values indicate more compact clusters in this particular representation. This interpretation assumes that Euclidean distance is meaningful for the chosen features, that their scaling is appropriate, and that a mean is a useful cluster summary. Those assumptions may not hold for every embedding or data type.

An internal criterion is not an independent check on the data used to define it. If a clustering procedure explicitly optimizes compactness, then finding a compact result partly confirms that it optimized its objective. It does not show that the grouping corresponds to cell identity. Nor can raw compactness alone compare partitions with different numbers of clusters fairly: splitting clusters generally reduces within-cluster sum of squares.

Why Splitting Can Improve an Internal Score

The tendency of within-cluster sum of squares to decrease as clusters are split follows from a decomposition of squared deviations. Suppose a cluster \(C\) is split into nonempty subsets \(G_1,\ldots,G_m\), with sizes \(n_g\), means \(\bar y_g\), and overall mean \(\bar y\). For scalar observations, expand each deviation as \(y_i-\bar y=(y_i-\bar y_g)+(\bar y_g-\bar y)\) for \(i\in G_g\). The cross-product sums to zero because \(\sum_{i\in G_g}(y_i-\bar y_g)=0\). Hence

The last term is nonnegative, so the sum of squared deviations after splitting is no greater than before. The same identity holds for vectors with squared Euclidean norms. This justifies why a lower compactness score cannot, by itself, establish that a finer partition is preferable: the score can improve simply because the partition has more groups.

For graph-based clustering, an internal assessment can instead examine the graph structure or the objective used to construct communities. The same caution applies: a partition that scores well on its construction objective fits that objective, but an optimized graph criterion is not an external biological validation. Diagnostics should also ask whether clusters are driven by a few highly connected cells, technical effects, or an uneven-density boundary.

External Agreement: Comparison With a Reference

External agreement compares a clustering with a reference variable that was not used to create it—for example, a set of independently assigned cell annotations or a measured experimental label. The reference must be relevant to the question. It may be incomplete, uncertain, measured at a different level of detail, or reflect a different biological concept from the clustering target.

Cluster identifiers themselves are arbitrary: exchanging the names “cluster 1” and “cluster 2” changes no partition. A label-based comparison must therefore account for how cluster names are matched to reference categories. A simple descriptive approach is to assign each cluster its most frequent reference category and report the fraction of cells whose labels agree with that assignment. This summary is easy to interpret, but it can hide minority populations and depends on the particular reference labels.

Perturbation Stability: Reproducibility Under Defined Changes

Stability concerns whether a clustering feature persists when a specified part of the analysis changes. A perturbation might alter the sampled cells, the selected features, preprocessing choices, or algorithmic randomness. Each perturbation asks a different question. Repeating only a random seed tests sensitivity to that source of randomness; it does not test sensitivity to the data representation or cell sampling.

One simple way to compare runs is to focus on pairs of cells. A pair is co-clustered if its two cells belong to the same cluster. Given a baseline partition and a perturbed run on the same cells, the co-clustering retention fraction is the number of baseline co-clustered pairs that remain co-clustered in the perturbed run divided by the number of co-clustered pairs in the baseline. It ranges from 0 to 1 when the baseline contains at least one co-clustered pair. This one-directional fraction measures retention, not every possible aspect of similarity between partitions.

Interpreting and Comparing the Three Forms of Evidence

The three approaches are complementary, not substitutes. Internal criteria are useful for detecting poor fit to the structure being analyzed, but can favor overly fine partitions. External comparison supplies a distinct reference point, but inherits the limitations of the reference. Stability tests robustness to chosen perturbations, but a result is only as informative as those perturbations: a clustering can be highly stable under repeated runs that all use the same flawed features.

A cautious assessment states the target and assumptions for each check. For internal fit, name the representation and distance or graph structure. For external agreement, describe the source and granularity of the reference. For stability, specify what changed, what was held fixed, and which aspect of the partition was compared. Different answers to these questions are not necessarily contradictory; they may reveal that the partition captures one kind of structure but not another.

Diagnostics, Failure Modes, and Biological Interpretation

Warning signs include a favorable internal score that improves mainly as the number of clusters increases; high reference agreement driven by a dominant category while smaller groups are poorly represented; or apparent stability assessed only by rerunning the same settings. Inspect the cluster sizes, the representation and features that drive separation, and the cells near boundaries. Check whether conclusions change when reasonable analytical choices are varied. A diagnostic flags a potential problem; it does not by itself identify the cause.

A common mistake is to conclude, “The clusters are stable and compact, so they are real cell types.” The reasoning treats two computational properties as proof of biological identity. The correction is to state the narrower evidence: “The partition is compact in this representation and persists under these specified perturbations.” Biological interpretation requires relevant supporting evidence beyond those properties, and cluster labels should not be presented as causal explanations of expression patterns.

No single validity number settles the question. Clustering is a representation-dependent summary of variation among cells; biological categories may be continuous, overlapping, or dependent on context. A partition can still be useful without revealing discrete natural kinds. The appropriate conclusion is proportional to the evidence: describe what was evaluated, what remained consistent, what did not, and which biological claims remain unsupported.