A Diagnostic Workflow for Single-Cell Embeddings

A Workflow for Diagnosing Embeddings

A cell embedding is the result of a sequence of choices, not a direct picture of cell identity. The expression representation, distance metric, dimension-reduction method, and tuning parameters all affect what appears in two dimensions. A useful diagnostic workflow therefore asks, in order: what information enters the method, what notion of similarity does it use, and how much does the displayed neighborhood structure persist under reasonable alternatives?

The earlier material on distance metrics, embedding geometry, parameter effects, and run stability supplies the underlying concepts. Here the emphasis is on using those concepts together to decide what an embedding can support. The workflow is descriptive: it can assess whether a visual pattern is robust to analysis choices, but it does not by itself establish a cell type, a biological mechanism, or a causal effect.

Definition: Let \(x_i\) denote the selected, processed expression-feature vector for cell \(i\), and let \(z_i\) denote its coordinates in an embedding. An input-space neighbor of cell \(i\) is another cell among the closest cells to \(x_i\) under a specified input distance. An embedding-space neighbor is among the closest cells to \(z_i\) under a specified distance in the displayed coordinates. These are distinct relationships; agreement between them is something to check, not assume.

Step 1: Specify the Representation and Distance

Before fitting an embedding, state what each cell vector contains and how it was prepared. For example, the features might be selected genes with normalized and transformed expression values. Filtering, normalization, transformation, and feature selection determine which differences are available to the embedding. A method cannot recover a distinction that the input representation has removed, and a few high-variance or poorly scaled features can dominate a distance.

Choose a distance to match the intended comparison. Euclidean distance responds to both direction and magnitude; cosine distance, commonly \(1-\)cosine similarity, emphasizes the angle between vectors and is insensitive to positive rescaling of a vector. Neither is universally preferable. A distance appropriate for normalized expression profiles may not answer a question about total expression magnitude. State the metric, the representation, and the scientific comparison it is intended to capture.

Worked Example: Magnitude Versus Profile Similarity

Suppose three hypothetical cells have two-feature vectors \(A=(8,2)\), \(B=(4,1)\), and \(C=(2,8)\). Compare Euclidean distance and cosine distance from \(A\).

For \(A\) and \(B\), the Euclidean distance is \(d_E(A,B)=\sqrt{(8-4)^2+(2-1)^2}=\sqrt{17}=4.123\). Their cosine similarity is \((8\cdot4+2\cdot1)/(\sqrt{68}\sqrt{17})=34/34=1\), so their cosine distance is \(0\). They have the same profile direction but different magnitude.

For \(A\) and \(C\), the Euclidean distance is \(d_E(A,C)=\sqrt{(8-2)^2+(2-8)^2}=\sqrt{72}=8.485\). Their cosine similarity is \((8\cdot2+2\cdot8)/(\sqrt{68}\sqrt{68})=32/68=0.4706\), so their cosine distance is \(1-0.4706=0.5294\).

Euclidean distance ranks \(B\) closer to \(A\) than \(C\), and cosine distance does too here, but for different reasons: \(B\) is a rescaled version of \(A\), while the cosine metric ignores that scale difference. If magnitude is biologically important, treating \(A\) and \(B\) as identical under cosine distance may be undesirable. This calculation does not choose a metric for all data; it makes explicit which distinction each metric retains.

Step 2: Fit a Linear Reference and Inspect It

PCA is a useful reference because its coordinates are linear combinations of the input features and its variance summary is explicit. For a centered feature matrix, PCA finds orthogonal directions of successively maximal sample variance. The proportion of total variance explained by component \(j\) is its eigenvalue divided by the sum of all eigenvalues. PCA does not require normally distributed expression values for this descriptive decomposition. Its interpretation does, however, depend on choices such as feature scaling and centering; Euclidean geometry of the chosen, scaled input is the geometry being summarized.

A PCA plot is not a neutral baseline if the inputs were scaled in a way that changes the relative contribution of genes. Inspect whether a small number of features dominate the leading components, whether the leading coordinates align with known technical variables, and whether the apparent pattern persists across a sensible range of components. A high variance proportion is not evidence that the component captures biology, and a low proportion does not imply that the remaining structure is meaningless.

Derivation: If the centered data have covariance matrix \(S\) with eigenvalues \(\lambda_1,\ldots,\lambda_p\), then total feature variance is \(\mathrm{tr}(S)\). Because the trace equals the sum of eigenvalues, \(\mathrm{tr}(S)=\sum_{j=1}^{p}\lambda_j\). The fraction explained by component \(j\) is therefore \(\lambda_j/\sum_{\ell=1}^{p}\lambda_\ell\); the first \(r\) components explain \(\sum_{j=1}^{r}\lambda_j/\sum_{\ell=1}^{p}\lambda_\ell\). This is a variance accounting identity, not a measure of biological importance.

Worked Example: Interpreting PCA Variance

Consider a hypothetical centered dataset whose covariance eigenvalues for the first three components are \(4, 1,\) and \(1\), with no other components. The total variance is \(4+1+1=6\) in the input’s squared units.

The first component explains \(4/6=0.6667\), or \(66.67\%\), of total variance. The second explains \(1/6=0.1667\), or \(16.67\%\). Together, the first two explain \((4+1)/6=0.8333\), or \(83.33\%\). The third explains the remaining \(16.67\%\).

These values describe variance in the chosen features and preprocessing scale. They do not show that PC1 is a cell-state axis, nor do they imply that a two-dimensional nonlinear embedding should preserve \(83.33\%\) of variance. Those methods have different objectives.

Step 3: Fit Nonlinear Embeddings as Alternatives

UMAP and t-SNE can represent local relationships that a low-dimensional linear projection does not display well, but their coordinates are optimized for objectives different from PCA’s variance decomposition. Their input metric and neighborhood-related parameters matter. UMAP’s neighborhood-size and minimum-distance settings, for example, affect the scale of structure emphasized and the compactness of points in the display. In t-SNE, perplexity affects the neighborhood scale used in the affinity construction. These parameters should be selected as analysis choices to examine, not as controls that reveal a uniquely correct map.

A practical comparison starts with a primary setting justified by the number and scale of cells, then repeats the fit over a modest set of plausible settings. Keep the input representation and distance fixed while varying embedding parameters if the purpose is to understand parameter sensitivity. If both the metric and parameters change at once, a changed plot cannot be attributed to either choice. Conversely, deliberately vary one choice at a time when diagnosing the sensitivity to that choice.

Worked Example: Comparing Plausible UMAP Settings

Suppose a hypothetical analysis uses the same normalized feature matrix and cosine distance for two UMAP fits. The first uses neighborhood size 15 and minimum distance 0.1; the second uses neighborhood size 40 and minimum distance 0.3. The first plot shows several compact groups, while the second shows a more continuous arrangement.

Because the input and metric were held fixed, the contrast is evidence that the display is sensitive to these parameter settings. It is not evidence that one setting is the biological truth. The analyst next checks whether particular local relationships of interest recur in both fits and in the input-space neighborhoods. If only the visual separation of groups changes, the defensible conclusion is that the apparent separation is parameter-sensitive.

A second run with the same settings but a different random seed addresses a different question: stochastic run variability. It should not be substituted for a parameter comparison, since it does not assess sensitivity to the chosen neighborhood scale or compactness setting.

Step 4: Check Neighborhood Agreement and Run Stability

Visual coordinate comparison is often misleading because an embedding may rotate, reflect, translate, or rescale without changing its neighborhood relations. A more direct diagnostic is to compare neighbor sets. For a chosen number \(k\), let \(N_k^{(a)}(i)\) and \(N_k^{(b)}(i)\) be the \(k\) nearest other cells to cell \(i\) in two representations or runs. Their Jaccard similarity is the size of the intersection divided by the size of the union. It is 1 for identical sets and 0 for disjoint sets. Compare input-space neighborhoods with embedding-space neighborhoods, and compare neighborhoods across repeated fits or plausible settings.

The value of \(k\), the distance used in each space, and the cells being summarized must be reported. A single average can conceal unstable regions, so inspect the distribution of similarities and identify whether low agreement is concentrated among particular cells or regions. There is no universal Jaccard cutoff that certifies a good embedding; the diagnostic is comparative and tied to the analysis purpose.

Worked Example: Quantifying Stability for One Cell

For a hypothetical cell \(i\), suppose the four nearest neighbors in one run are \(N_4^{(a)}(i)=\{B,C,D,E\}\), and in a second run they are \(N_4^{(b)}(i)=\{B,C,D,F\}\). The intersection contains \(B,C,D\), so its size is 3. The union contains \(B,C,D,E,F\), so its size is 5.

Thus the Jaccard similarity is \(3/5=0.6000\). Three of the four nearest neighbors are shared, but the overlap is not complete. This is a per-cell diagnostic, not a significance test and not a probability that the run is correct. To summarize an analysis, calculate this quantity for many cells and report an appropriate distributional summary alongside the value of \(k\).

A defensible neighborhood statement is narrow and tied to these checks: for example, “these cells repeatedly appear near one another under the stated metric and across the tested settings.” If a neighborhood is present only in one display, report it as a feature of that fit rather than a robust property of the input data. Do not infer quantitative distances between separated islands from their displayed spacing.

Putting the Workflow Into Practice

Document the input.
State the cell features, preprocessing, feature selection, and any scaling. Explain which differences are retained or deemphasized.

Choose and justify a distance.
Connect the metric to the intended notion of cell similarity. Avoid changing the metric and embedding parameters simultaneously when interpreting a comparison.

Establish a PCA reference.
Inspect leading components, variance explained, and potential technical alignment. Treat the result as a linear summary of the selected input, not a biological validation.

Compare nonlinear fits.
Fit a justified primary setting, then vary parameters and random seeds in planned comparisons. Keep track of which choice changes in each comparison.

Report stability and scope.
Compare neighborhood sets across input space, settings, and runs. State which local claims persist, which are sensitive, and what the embedding cannot establish.

Diagnostics, Alternatives, and Limits

Warning signs include major changes in apparent neighborhoods across modest parameter changes, a pattern aligned strongly with a technical variable, or low neighborhood overlap across repeated runs. These findings do not automatically invalidate an embedding. They indicate that the affected feature needs cautious interpretation and that the source of variation should be investigated. Also check whether conclusions depend on a small subset of cells or on the chosen value of \(k\).

PCA is the main linear reference: its axes and variance accounting are interpretable, but a small number of components may fail to display nonlinear local structure. UMAP and t-SNE emphasize neighborhood relationships in different ways and can be useful visual alternatives, but neither provides a general preservation guarantee for every pairwise distance. The appropriate choice depends on whether the goal is a compact linear summary, a local exploratory map, or a stable claim about particular cell neighborhoods.

An embedding is a low-dimensional representation, not an independent validation dataset. Its visible clusters do not by themselves establish discrete populations; apparent density and distances between separated regions can be distorted; and robustness to seeds or settings does not rule out confounding, poor input choices, or batch effects. Use embedding diagnostics to describe the reliability and limits of the visualization. Further analysis is needed to substantiate biological labels or other substantive conclusions.

Common pitfall: “The clusters are clearly separated in the UMAP, so the cells form distinct biological groups.” This reasoning treats display separation as proof of discrete biology. Instead, check whether the relevant local neighborhoods persist across reasonable settings and runs, compare them with the chosen input-space relationships, inspect technical covariates, and describe only the structure supported by those checks.

Check Your Understanding

Answer each question, then compare with the brief solutions.

Two cells have proportional feature vectors. Which distance, Euclidean or cosine, treats their profile directions as identical, and what information does that choice discard?
A PCA analysis has eigenvalues 3, 2, and 1. What proportion of variance is explained by the first two components?
In two runs, a cell has neighbor sets \(\{A,B,C\}\) and \(\{A,B,D\}\). Calculate their Jaccard similarity.
A group appears as a separated island under one parameter setting but is connected to other cells under another. What conclusion is justified, and what conclusion is not?
Why is visual coordinate alignment alone a poor measure of whether two embedding runs are stable?

Solutions. 1. Cosine distance is zero for vectors in the same positive direction; it discards their magnitude difference. 2. The total variance is \(3+2+1=6\), so the first two explain \((3+2)/6=5/6=0.8333\), or \(83.33\%\). 3. The intersection has size 2 and the union has size 4, giving \(2/4=0.5000\). 4. The apparent separation is parameter-sensitive; do not claim that the island is a distinct biological population based on the map alone. 5. Rotation, reflection, translation, or rescaling can change coordinates without changing neighbors, so compare neighborhood relations rather than raw coordinate positions.