Distance Metrics for Cell Neighborhoods

Why the Distance Measure Matters

A cell neighborhood is defined by which other cells are considered close to a given cell. As established in the earlier material on building a k-nearest-neighbor graph, the distance measure helps determine that ordering. Two metrics applied to the same profiles can select different neighbors, because “similar” may mean similar absolute expression, similar relative pattern across genes, or similar pattern after removing a cell-wide baseline.

Here, a cell profile is a vector of values for the same set of features, such as genes, for every cell being compared. The values might be normalized expression measurements or coordinates from a selected feature space. Comparisons are meaningful only when the features are aligned and the values have a defensible, consistent interpretation across cells. These distances do not determine biological similarity by themselves: they encode particular geometric choices about the data.

Definition: For two profiles \(x=(x_1,\ldots,x_p)\) and \(y=(y_1,\ldots,y_p)\), Euclidean distance is \(d_E(x,y)=\sqrt{\sum_{j=1}^p(x_j-y_j)^2}\). If both vectors are nonzero, cosine distance is \(d_C(x,y)=1-\frac{x^\top y}{\lVert x\rVert\lVert y\rVert}\). If both profiles are nonconstant, correlation distance is \(d_R(x,y)=1-r_{xy}\), where \(r_{xy}\) is their Pearson correlation across the \(p\) features.

Euclidean distance measures absolute coordinate differences, so both the size and pattern of expression differences matter. Cosine distance compares directions from the origin: multiplying one nonzero profile by a positive constant leaves its cosine distance to any other profile unchanged. Correlation distance compares centered profiles, removing each profile’s mean across features before measuring their linear association. Thus, positive rescaling and adding a constant to an entire profile do not change its Pearson correlation with another nonconstant profile.

What the Formulas Imply

For Pearson correlation, let \(\bar{x}\) and \(\bar{y}\) be the means of the profile coordinates. The centered vectors are \(x-\bar{x}\mathbf{1}\) and \(y-\bar{y}\mathbf{1}\), where \(\mathbf{1}\) is a vector of ones. Correlation is their normalized dot product:

r_{xy} = \frac{\sum_{j=1}^p (x_j-\bar{x})(y_j-\bar{y})} {\sqrt{\sum_{j=1}^p (x_j-\bar{x})^2} \sqrt{\sum_{j=1}^p (y_j-\bar{y})^2}}

This gives a direct justification for the distinction between cosine and correlation distance. Cosine uses the uncentered vectors \(x\) and \(y\); correlation uses their centered versions. A shared pattern of variation can therefore produce high correlation even when the profiles have different overall baselines. Conversely, two profiles with similar directions from the origin need not have similar centered patterns.

Euclidean distance is a metric: it is nonnegative, symmetric, zero only for identical vectors, and satisfies the triangle inequality. Cosine distance \(1-\cos(x,y)\) and correlation distance \(1-r_{xy}\) are commonly used dissimilarities, but they do not in general satisfy the triangle inequality; either can be zero for distinct profiles. For example, positive multiples have cosine distance zero, and positively related profiles that differ only by an additive constant have correlation distance zero. They can still be used to rank neighbors, but should not be treated as ordinary Euclidean geometry.

Worked Example: Different Metrics, Different Nearest Cell

Suppose a target cell has profile \(x=(1,2,0)\). Two candidate cells have profiles \(A=(2,4,0)\) and \(B=(1,0,0)\). All profiles use the same three features. Calculate each distance and compare the ranking.

Euclidean distance. The coordinate differences from \(x\) to \(A\) are \((-1,-2,0)\), so \(d_E(x,A)=\sqrt{1+4+0}=\sqrt{5}\approx2.236\). The differences from \(x\) to \(B\) are \((0,2,0)\), so \(d_E(x,B)=\sqrt{4}=2\). Euclidean distance ranks \(B\) closer.

Cosine distance. For \(A\), \(x^\top A=1(2)+2(4)+0(0)=10\), while \(\lVert x\rVert=\sqrt{5}\) and \(\lVert A\rVert=\sqrt{20}\). Thus, \(\cos(x,A)=10/(\sqrt{5}\sqrt{20})=1\), and \(d_C(x,A)=0\). For \(B\), \(x^\top B=1\), \(\lVert B\rVert=1\), and \(\cos(x,B)=1/\sqrt{5}\approx0.4472\). Hence \(d_C(x,B)\approx0.5528\). Cosine distance ranks \(A\) closer because \(A\) is a positive multiple of \(x\).

Correlation distance. The profile means are \(\bar{x}=1\), \(\bar{A}=2\), and \(\bar{B}=1/3\). The centered vectors are \(x-\bar{x}\mathbf{1}=(0,1,-1)\), \(A-\bar{A}\mathbf{1}=(0,2,-2)\), and \(B-\bar{B}\mathbf{1}=(2/3,-1/3,-1/3)\). The correlation between \(x\) and \(A\) is 1 because the centered vectors are positive multiples. For \(B\), their centered dot product is \(0(2/3)+1(-1/3)+(-1)(-1/3)=0\), so \(r_{xB}=0\). Therefore \(d_R(x,A)=0\) and \(d_R(x,B)=1\). Correlation, like cosine, ranks \(A\) closer, while Euclidean distance ranks \(B\) closer.

There is no arithmetic error or contradiction: each measure answers a different question. \(A\) has the same relative profile pattern as \(x\), but its coordinate values are farther away in absolute terms than those of \(B\).

Scaling Changes the Geometry

Euclidean distance gives more influence to features with larger numerical differences. If one gene’s values vary over a much wider range than another’s, that gene can dominate the distance even if the wider range reflects measurement scale rather than the biological distinction of interest. Rescaling features changes Euclidean distances and can change the neighbor ranking. Standardizing features is one possible response, but it is not automatic: it changes the scientific meaning of “close” and can amplify noisy or rarely varying genes.

Worked Example: Feature Rescaling Reverses an Euclidean Ranking

Consider a target \(x=(0,0)\) and candidates \(A=(3,0)\) and \(B=(0,2)\), where the coordinates represent two features in their original units. Their Euclidean distances are \(d_E(x,A)=\sqrt{3^2+0^2}=3\) and \(d_E(x,B)=\sqrt{0^2+2^2}=2\). Thus \(B\) is closer in the original coordinate system.

Now rescale the first feature by \(0.5\) and the second by \(2\), applying the same transformation to every cell. The transformed profiles are \(x'=(0,0)\), \(A'=(1.5,0)\), and \(B'=(0,4)\). The transformed distances are \(d_E(x',A')=1.5\) and \(d_E(x',B')=4\), so \(A\) is now closer. The reversal follows directly from the changed feature weights, not from a change in the cells.

A useful diagnostic is to calculate or inspect the contribution of each feature to squared Euclidean distance, \((x_j-y_j)^2\), before taking the square root. If a small number of high-range features dominate, ask whether those features should dominate the neighborhood definition. Any centering, scaling, or variance-based feature selection should be chosen deliberately and applied consistently.

Sparse Profiles and Centering

Single-cell expression profiles often contain many zeros. A zero can reflect no detected expression, low abundance, or measurement processes; it is not automatically evidence that two cells are biologically alike. Euclidean distance counts coordinate differences, cosine distance focuses on the direction of the nonzero vectors, and correlation distance centers each profile across the selected features. With many zeros, that centering step can substantially affect the relationship being measured.

Sparsity also creates edge cases. Cosine distance is undefined if either profile has zero norm. Correlation distance is undefined if either profile is constant across the selected features, including an all-zero profile. Software may omit such cells, return a missing value, or apply a convention; those choices should be checked rather than assumed. A distance calculation cannot recover information absent from an all-zero profile.

Worked Example: A Sparse Profile and Baseline Sensitivity

Let the target profile be the sparse vector \(x=(0,0,0,1)\). Compare \(A=(10,10,10,11)\), which shares a one-unit increase in the fourth feature over its baseline, with \(B=(0,0,1,1)\). The example illustrates that a high correlation can coexist with a relatively low cosine similarity when profiles have different baselines.

Cosine distance. For \(A\), the dot product is \(x^\top A=11\), and \(\lVert A\rVert=\sqrt{10^2+10^2+10^2+11^2}=\sqrt{421}\). Since \(\lVert x\rVert=1\), \(\cos(x,A)=11/\sqrt{421}\approx0.5361\), and \(d_C(x,A)\approx0.4639\). For \(B\), \(x^\top B=1\) and \(\lVert B\rVert=\sqrt{2}\), so \(\cos(x,B)=1/\sqrt{2}\approx0.7071\) and \(d_C(x,B)\approx0.2929\). Cosine distance ranks \(B\) closer.

Correlation distance. The centered target is \((-0.25,-0.25,-0.25,0.75)\). The centered \(A\) is \((-0.25,-0.25,-0.25,0.75)\), so \(r_{xA}=1\) and \(d_R(x,A)=0\). The mean of \(B\) is \(0.5\), giving centered \(B=(-0.5,-0.5,0.5,0.5)\). The centered dot product is \(0.125+0.125-0.125+0.375=0.5\); the centered norms are \(\sqrt{0.75}\) for \(x\) and \(1\) for \(B\). Therefore \(r_{xB}=0.5/\sqrt{0.75}\approx0.5774\), so \(d_R(x,B)\approx0.4226\). Correlation distance ranks \(A\) closer.

The rankings reverse because correlation ignores the shared baseline offset between \(x\) and \(A\), while cosine compares their directions from the origin. In real expression data, such a contrast should prompt a check of the preprocessing and feature set: the result depends on whether overall baseline differences are intended to matter.

Choosing and Checking a Distance

No one distance is universally best for cell neighborhoods. The decision should follow the intended notion of similarity, the representation being compared, and the transformations already applied to the data. Earlier material on choosing a distance metric for cell embeddings considered geometric choices in an embedding; here the same general principle applies to cell profiles, but profile scaling and sparsity deserve particular attention.

Distance	What it emphasizes	Important checks
Euclidean	Absolute differences in feature values	Check feature scales and dominant squared-difference contributions.
Cosine	Direction from the origin; insensitive to positive profile magnitude	Check for zero vectors and whether magnitude should be ignored.
Correlation	Centered linear pattern across features	Check for constant profiles and whether baseline differences should be removed.

A practical comparison is to compute neighbor rankings under plausible alternatives and identify cells whose neighbors change substantially. This is a sensitivity analysis, not a contest in which one ranking must be declared correct. Investigate whether changes are concentrated among low-count or unusually scaled profiles, and whether the selected features or transformations explain the differences. Also inspect the actual profiles of cells that appear unexpectedly close.

The three distances have limits. Euclidean distance can be dominated by scale and outliers. Cosine distance discards positive magnitude information and is not defined for zero profiles. Correlation distance discards cell-wide offsets and can become unstable when profiles have little variation across the selected features; it is undefined for constant profiles. All are dependent on the representation and feature set, so a neighborhood is a property of the chosen analysis as well as of the cells.

Common Pitfalls

“Cosine and correlation are interchangeable.” They are not: cosine uses uncentered profiles, while correlation centers each profile first.
“Standardizing features is always an improvement.” Scaling changes the geometry and can give noisy features greater influence. Justify it for the intended comparison.
“A zero distance means identical profiles.” Not for cosine or correlation distance: distinct positive multiples can have zero cosine distance, and positively affine-related profiles can have zero correlation distance.
“Shared zeros prove similarity.” Many zeros can shape a distance calculation without establishing shared biological activity. Consider the measurement process and the metric’s treatment of zeros.
“The software returned a number, so the distance is valid.” Check how zero-norm and constant profiles are handled, and confirm the input features and transformations are consistent.

Key takeaway: Euclidean distance compares absolute profile values, cosine distance compares uncentered directions, and correlation distance compares centered patterns. Scaling, baselines, and sparsity can alter their rankings, so the chosen distance should match the scientific meaning of a neighborhood and be checked for failure cases.

Check Your Understanding

Answer each question, then compare your reasoning with the solutions.

For \(x=(1,2)\) and \(y=(2,4)\), which of the three distances is zero, and why?
Why can multiplying one feature by a constant reverse an Euclidean neighbor ranking even when the same transformation is applied to all cells?
What makes cosine distance undefined for a profile, and what makes correlation distance undefined?
A zero-heavy target has a high correlation distance similarity to a profile with a much higher baseline. What feature of correlation explains this, and what should be checked before interpreting the result?

Solutions. (1) Cosine distance and correlation distance are zero: \(y\) is a positive multiple of \(x\), and their centered versions are also positive multiples. Euclidean distance is \(\sqrt{(1-2)^2+(2-4)^2}=\sqrt{5}\), not zero. (2) Rescaling changes that feature’s contribution from \((x_j-y_j)^2\) to the square of the rescaled difference, changing the relative influence of coordinates and potentially the ordering. (3) Cosine is undefined when either vector has zero norm; correlation is undefined when either profile has zero variation across features. (4) Correlation centers each profile, so a constant baseline shift does not affect it. Check whether baseline differences should be ignored, whether the profiles are sufficiently variable, and whether zeros and feature selection make the pattern meaningful.