Why Compare Embeddings on the Same Cells?
A two-dimensional cell embedding is a representation of a high-dimensional expression dataset, not a direct picture of the underlying biology. PCA, t-SNE, and UMAP can produce different layouts from the same cells because they optimize different aspects of the data. A useful comparison therefore asks a specific question: which properties of the original representation does each layout preserve, and which claims are justified by that preservation?
The comparison is informative only when the methods start from a common basis. Hold the cells, genes or input features, normalization and scaling choices, distance metric, and any intermediate representation fixed. Then compare the resulting embeddings with reference to that shared input. If preprocessing or input dimensions change between methods, a difference in the plots cannot be attributed to the embedding method alone.
PCA centers the input and finds orthogonal directions that successively maximize variance. Its coordinates on those directions are principal-component scores, and the associated loadings describe how input features contribute to each direction. t-SNE constructs a low-dimensional layout to represent local affinities, using a KL-divergence objective. UMAP builds a weighted nearest-neighbor representation and optimizes a low-dimensional layout using a cross-entropy objective. These objectives are not interchangeable, so a single numerical score cannot establish that one method is best for every purpose.
For a fair, interpretable comparison, distinguish three questions. First, do nearby cells in the input remain nearby in the embedding? Second, does the embedding preserve meaningful broader geometry, such as the ordering of selected pairwise distances? Third, can a plot be interpreted in relation to the data and its annotations without assigning unsupported meaning to its axes, gaps, or apparent clusters?
What the Methods Can and Cannot Summarize
| Method | What its layout is designed to represent | What is especially interpretable | Important caution |
|---|---|---|---|
| PCA | Linear directions of high variance in the centered input | Scores, component loadings, and variance explained | A two-component view can omit substantial variation and need not preserve nearest neighbors. |
| t-SNE | Local affinities between cells | Whether some input-space neighbors appear near one another | Distances between separated groups and apparent group sizes are not reliable measures of input-space geometry or density. |
| UMAP | Weighted local relationships in a nearest-neighbor representation, with a low-dimensional layout optimized to match them | Local relationships represented in the input graph | Global distances and group separations are not guaranteed to reproduce input-space distances. |
A PCA plot has a special advantage for interpretation: its axes have defined relationships to the input features, and the fraction of input variance explained by each component can be reported. The nonlinear axes of t-SNE and UMAP do not have this variance-explained interpretation. Their orientation and coordinates are layout coordinates, not direct gene-expression measurements. Nor does a visually tidy plot, by itself, demonstrate that the corresponding method preserves more useful information.
Worked Example: Keeping the Input Fixed
Suppose a hypothetical analysis represents 1,200 cells using the same 30 centered input features for all three methods. PCA, t-SNE, and UMAP are each used to produce two-dimensional layouts. The first two PCA components explain 31% and 18% of the total variance, respectively.
For the PCA plot, the two-dimensional variance fraction is \(31\%+18\%=49\%\). Thus, the plot displays directions accounting for 49% of the variance in this specified input representation. This does not mean that it preserves 49% of cell neighborhoods, nor does it mean that the other methods preserve 51%.
For t-SNE and UMAP, a two-dimensional variance-explained number is not supplied by their layout objectives. Comparing their plots using the PCA variance fraction would therefore be invalid. Instead, assess each two-dimensional layout against the same 30-dimensional input distances or neighborhoods. This comparison isolates a meaningful question: how well does each layout preserve selected properties of the common reference?
Measuring Neighborhood Preservation
A direct measure is the fraction of each cell’s input-space neighbors that also appear among its embedding-space neighbors. Let \(N_X^{(k)}(i)\) be the \(k\) nearest cells to cell \(i\) in the input representation, and let \(N_Z^{(k)}(i)\) be its \(k\) nearest cells in an embedding. The mean neighborhood overlap is
The value lies between 0 and 1, with larger values indicating more shared neighbors at the selected \(k\). Its meaning depends on the reference metric, the chosen \(k\), and how ties are handled. It is a simple measure of shared membership, but it does not distinguish a near miss from a severe displacement among neighbors.
Trustworthiness addresses one particular failure: an embedding may place cells together that are not close in the input. For cell \(i\), let \(U_i^{(k)}\) be the cells that are in its embedding \(k\)-nearest-neighbor set but not in its input \(k\)-nearest-neighbor set. Let \(r(i,j)\) be the rank of cell \(j\) among the other cells ordered by input-space distance from \(i\), with rank 1 closest. For \(k<2n/3\), trustworthiness is
An embedding neighbor that is just outside the input \(k\)-nearest-neighbor set incurs a small penalty; one that is far down the input ranking incurs a larger penalty. The normalization follows from the largest possible penalty: for each cell, the \(k\) intruding neighbors could occupy input ranks \(n-k,\ldots,n-1\), whose total excess over \(k\) is \(k(2n-3k-1)/2\). Across \(n\) cells this gives the normalizing denominator. Trustworthiness therefore decreases as the embedding introduces more or more distant false neighbors. It does not, by itself, measure every way an input neighbor can be lost.
Worked Example: Calculating Trustworthiness
Consider five hypothetical cells and set \(k=1\). In the input representation, the nearest neighbor of A is B, of B is A, of C is D, of D is C, and of E is D. Suppose the nearest neighbors in one hypothetical embedding are B, C, A, B, and D for A, B, C, D, and E, respectively. Their input-space ranks are 1, 2, 3, 4, and 1.
A and E have embedding neighbors that are also their input nearest neighbors, so they contribute no penalty. B’s embedding neighbor C has input rank 2, contributing \(2-1=1\). C’s neighbor A has input rank 3, contributing \(3-1=2\). D’s neighbor B has input rank 4, contributing \(4-1=3\). The total penalty is \(1+2+3=6\).
Here \(n=5\), \(k=1\), and \(2n-3k-1=10-3-1=6\). Therefore
This hypothetical embedding has trustworthiness 0.600 at \(k=1\), reflecting several embedding neighbors that rank poorly in the input. It is not a probability that a neighbor is correct, and it does not summarize behavior at other neighborhood sizes.
Assessing Global Geometry and Interpretation
A global-geometry check should be stated narrowly. For example, select a set of cell pairs before inspecting the layouts, compute their distances in the fixed input representation and in each embedding, and compare the distance rankings. Spearman’s rank correlation summarizes the association between two rankings. A high value indicates similar ordering for those selected pairs; it does not imply that the numerical distances are equal, nor that all pairwise relationships are preserved.
Pairwise distances are dependent because many pairs share cells. A rank correlation across pairs is therefore a descriptive comparison, not automatically a valid hypothesis test with independent observations. The selection of pairs also matters: pairs concentrated within one region answer a different question from pairs spanning the full dataset. Report the reference representation, distance metric, pair selection, and whether ties affect the ranking.
Worked Example: Comparing Pairwise Distance Ranks
Suppose four prespecified cell pairs have input-distance ranks 1, 2, 3, and 4. In a hypothetical embedding, their distance ranks are 1, 3, 2, and 4. There are no ties. The rank differences \(d\) are 0, \(-1\), 1, and 0, so \(\sum d^2=0+1+1+0=2\). Spearman’s rank correlation is
The ranking of these four pair distances is fairly similar in this example, but the result concerns only the selected pairs. It does not prove that the embedding preserves global geometry throughout the dataset. In particular, a strong local-neighborhood score and this rank correlation answer different questions and should be reported separately.
Interpretability also depends on what the plot is used to support. PCA loadings can help identify features associated with a principal direction, but a component is not automatically a biological mechanism. In t-SNE and UMAP, colors for known annotations can help reveal whether a layout separates or mixes labeled cells. Such visual patterns describe the embedding and the annotations; they do not establish that the embedding causes a biological distinction, that every visible island is a discrete cell type, or that the apparent distance between islands has a direct biological meaning.
A Practical Comparison and Its Failure Modes
Use the same cells, input features, preprocessing, distance metric, and input dimensions. Record these choices so the comparison has a defined target.
Compare layouts in the same output dimension and calculate neighborhood measures at prespecified values of \(k\). Do not compare a two-dimensional layout to an unstated reference.
Report neighbor overlap or trustworthiness for local structure, and use an explicitly defined distance-ranking check for any selected global question.
Use the plot and known annotations as descriptive evidence. Treat PCA’s variance summary as specific to PCA; do not transfer it to t-SNE or UMAP.
Several warning signs make a comparison unreliable. If the methods received different input representations, their scores do not isolate method differences. If a neighborhood score is reported without \(k\), a metric, or a reference space, its meaning is incomplete. If a conclusion about broad separation rests only on a local-structure objective, it exceeds what that objective guarantees. If a method is declared best because its plot looks clearest, the judgment is subjective unless the scientific purpose and evaluation criteria were specified in advance.
Alternative approaches answer different questions rather than providing a single hierarchy. PCA is often preferable when a linear variance summary, reproducible component axes, or feature loadings are central. t-SNE or UMAP may be useful when a low-dimensional view of local relationships is the priority. Quantitative neighborhood diagnostics can supplement all three, while global rank comparisons can check a limited aspect of distance ordering. None of these checks alone establishes that an embedding is biologically correct.
The measures also have limitations. Neighborhood scores depend on \(k\) and the input metric, and a favorable value at one scale need not persist at another. Trustworthiness focuses on false neighbors; a complementary question is whether true input neighbors were omitted. Global distance ranks depend on the selected pairs and can hide differences in absolute distance. Finally, a low-dimensional layout necessarily compresses information. The right conclusion is therefore conditional: a method performed better for a stated criterion, at a stated scale, relative to a stated input representation.
Common Mistake
Check Your Understanding
Answer each question using the definitions and comparison principles above.
- Why is it important to hold the input representation fixed when comparing PCA, t-SNE, and UMAP?
- In a hypothetical analysis, PCA’s first two components explain 54% of the input variance. Can this number be used to say that PCA preserves more information than a t-SNE embedding? Explain.
- For \(n=10\) cells and \(k=1\), an embedding has one false neighbor whose input rank is 3, and all other cells have no trustworthiness penalties. Calculate \(T(1)\).
- A high Spearman correlation between selected input and embedding distance ranks is observed. Give one conclusion it supports and one conclusion it does not support.
Solutions. 1. If inputs differ, a change in results may be due to preprocessing, features, or metric rather than the embedding method; a common input makes the comparison conditional on one reference. 2. No. The 54% is PCA’s variance fraction for its two components, not a common information-preservation scale for t-SNE, whose objective is based on local affinities. 3. The penalty is \(3-1=2\). The normalizing denominator is \(10(1)(20-3-1)=160\), so \(T(1)=1-2(2)/160=0.975\). 4. It supports similar ordering of distances for the selected pairs. It does not establish equal distances, preservation for all pairs, or biological validity of the layout.