Distance Geometry Can Distort Validity Scores

Why a Validity Score Depends on Geometry

An internal cluster-validity index evaluates a partition using the data representation and a chosen measure of separation. It does not compare clusters with an external truth. Consequently, a high score is evidence that the partition fits a particular geometry—not proof that it captures biologically distinct cell types. In single-cell analyses, the same partition can look compact under one dissimilarity and diffuse under another, especially when the representation is high-dimensional or sparse.

This tutorial focuses on how geometry can affect internal scores, using the silhouette as a concrete example. The central diagnostic is to ask whether the score's conclusion persists under other defensible distance choices. Such comparisons are informative only when each choice has a clear interpretation for the representation being analyzed.

For vectors \(x_i\) and \(x_j\), Euclidean distance measures straight-line separation, while Manhattan distance sums absolute coordinate differences. Cosine dissimilarity measures angular difference and ignores vector magnitude; it is commonly useful as a dissimilarity, though \(1-\)cosine similarity is not generally a metric. Jaccard dissimilarity is often used for binary features, such as presence or absence, and is not a generic choice for continuous expression values.

Here, \(p\) is the number of coordinates in the representation, such as genes or retained components. Cosine dissimilarity is undefined if either vector has zero norm; zero vectors must be handled explicitly rather than assigned an arbitrary cosine value. Euclidean and Manhattan distances are metrics on ordinary real vectors. A metric also depends on the coordinate scale: rescaling a feature changes its contribution to these distances.

Silhouette Scores and Their Assumptions

For a fixed partition into at least two clusters, let \(a_i\) be the mean distance from observation \(i\) to the other observations in its own cluster. For any other cluster, compute the mean distance from \(i\) to the observations in that cluster; let \(b_i\) be the smallest of those means. The silhouette for \(i\) compares its within-cluster distance with its nearest alternative cluster.

The score lies between \(-1\) and \(1\). Values near \(1\) indicate that the observation is, on average, closer to its own cluster than to the nearest alternative; values near \(0\) indicate similar average distances; negative values indicate that another cluster is closer on average. The overall silhouette is commonly the arithmetic mean of the individual \(s_i\). A singleton has no within-cluster comparison, so a common convention assigns it a silhouette of zero; the convention should be stated when singleton clusters occur.

The calculation assumes that the supplied pairwise distances are meaningful for the observations and that the partition is fixed while the score is computed. It does not assume that clusters are spherical or equally sized, but the score still rewards certain distance geometries and can respond to cluster size, density, and shape. It is not a test of a null hypothesis and does not provide a probability that a partition is correct.

Why High Dimensionality Can Flatten Distance Contrast

In high dimensions, pairwise distances can become more alike. A simple derivation shows the mechanism under a specific model. Suppose observations have \(p\) independent coordinates, each distributed as \(N(0,\sigma^2)\). For two independently drawn observations \(X_i\) and \(X_j\), each coordinate difference is distributed as \(N(0,2\sigma^2)\). Therefore, the squared Euclidean distance divided by \(2\sigma^2\) has a chi-squared distribution with \(p\) degrees of freedom.

The coefficient of variation (standard deviation divided by mean) of the squared distance shrinks as \(p\) grows. For the distance itself, a large-\(p\) approximation gives a relative standard deviation of about \(1/\sqrt{2p}\). The derivation assumes independent Gaussian coordinates with equal variance; it is an illustration, not a universal model for expression data. Correlation, unequal feature scales, a low-dimensional signal, or a different distribution can change the behavior. Still, it explains why small relative differences in distances may become less informative as many dimensions contribute noise.

Sparsity, Scaling, and the Choice of Dissimilarity

Single-cell expression data can be sparse: many recorded gene values are zero. In a sparse representation, Euclidean distances may be driven by which coordinates are nonzero, by a few large values, or by the overall scale of a cell's vector. A large shared set of zeros can also make two profiles appear similar under some measures even when their nonzero patterns differ. The exact effect depends on whether the representation contains raw counts, normalized values, transformed values, selected genes, or a lower-dimensional embedding.

No distance is automatically correct. Euclidean distance is interpretable when coordinate magnitudes and scaling are meaningful. Manhattan distance can reduce the influence of a single large coordinate relative to squaring deviations, but it still depends on feature scales. Cosine dissimilarity is appropriate when direction or relative profile is the target and overall magnitude is not; it discards magnitude by construction. Jaccard dissimilarity can compare sets of present features, but binarizing expression discards abundance information and requires a defensible definition of presence.

Changing the distance is not a neutral robustness check if it changes the scientific question. For example, a cosine-based score asks whether profiles point in similar directions; a Euclidean score also reflects their lengths. The metric should be selected based on the representation and target notion of similarity, not because it produces the largest validity score.

A Practical Diagnostic for Metric Dependence

A diagnostic should make the comparison interpretable rather than search across distances until one score looks favorable. For a fixed partition, compute the same index using a small set of scientifically defensible distances on the same stated representation. Record the preprocessing, distance, index, and any special conventions. If the partition's apparent separation changes sharply, report that dependence and investigate which geometric feature drives it.

Useful diagnostics include plotting or summarizing the distribution of pairwise distances, checking how many distances are zero or tied, and examining whether a few features dominate Euclidean or Manhattan distances. In high dimensions, inspect relative contrasts, not only the absolute scale of distances. In sparse data, check whether zeros represent meaningful absence or arise from measurement and preprocessing. Cosine calculations require nonzero vectors, while Jaccard requires an explicit binary definition.

Other internal indices can have different geometric sensitivities. The Davies–Bouldin index compares within-cluster scatter with distances between cluster centers, and the Calinski–Harabasz index is based on between- and within-cluster sums of squares; these are not interchangeable with silhouette scores and commonly rely on Euclidean centroid geometry. Agreement across such indices can be reassuring, but disagreement can reflect different definitions of compactness and separation, not necessarily a computational error. Choosing among them requires understanding their assumptions rather than treating an index as a universal ranking.

Common Pitfalls and Limits

• Calling every dissimilarity a metric. A distance-like formula may fail identity of indiscernibles or the triangle inequality. State whether it is a metric or a general dissimilarity; do not infer metric properties merely from its name.
• Reading a high silhouette as biological truth. The score describes geometric separation for a chosen representation and partition. It does not establish a cell type, causal mechanism, or external validity.
• Changing several choices at once. If representation, scaling, clustering, and distance all change, the source of a score change is unclear. For a distance-dependence check, hold the partition and representation fixed where possible.
• Maximizing the score over many distances. Selecting the most favorable result after trying many choices creates a selectively optimistic summary. Define plausible choices in advance and report the comparison.
• Ignoring degenerate cases. Singleton clusters, zero vectors for cosine calculations, and heavy ties in sparse data require explicit handling. A software default can affect the reported score.

Distance comparisons are diagnostics, not a method for proving which geometry is biologically correct. Internal scores are limited summaries: they compress pairwise structure, can be sensitive to cluster shape and size, and do not by themselves distinguish discrete populations from other patterns. Their strongest use is conditional description alongside a clear account of representation and distance.