Random Seeds and Embedding Variability

Why Repeat an Embedding?

A t-SNE or UMAP plot is the result of an optimization, not a unique coordinate map guaranteed by the data. Different random seeds can change the starting coordinates, the order of stochastic updates, or other randomized steps. Approximate neighbor-search procedures may introduce further variation. As a result, two runs on the same cells can look different even when the input data and nominal parameters are unchanged.

Some visible differences are only changes of orientation: one plot may be rotated, reflected, or shifted relative to the other. Other differences are more substantive: a cell may acquire different nearby cells, or groups of cells may be arranged or separated differently. A careful comparison distinguishes these cases rather than judging stability by whether two plots look alike on screen.

To attribute differences to random initialization or optimization, hold the cells, input representation, preprocessing, distance metric, parameters, software implementation, and other settings fixed. If any of these change, a difference between plots cannot be attributed to the seed alone. Record the seed and settings for each run. Exact reproducibility may also depend on software versions, hardware, parallel computation, and deterministic implementation details.

Coordinate Symmetries and Meaningful Change

Both t-SNE and UMAP optimize objectives based on relationships among points in the low-dimensional embedding. For their usual Euclidean layouts, translating every point by the same vector, rotating all points, or reflecting them preserves all pairwise distances. A plot can therefore have different coordinates while representing exactly the same geometry. In two dimensions, a rotation or reflection is represented by an orthogonal matrix \(Q\), satisfying \(Q^\mathsf{T}Q=I\). Translation adds a vector \(b\).

Because \(\lVert y_i'-y_j'\rVert=\lVert Q(y_i-y_j)\rVert=\lVert y_i-y_j\rVert\), every pairwise distance is unchanged. The objectives used by t-SNE and UMAP depend on the embedded pairwise relationships, so these rigid transformations do not change the objective value. This explains why orientation and position on the page are not substantive evidence of a changed embedding.

Scaling is different: multiplying all coordinates by a constant generally changes pairwise distances and is not one of these symmetries. Nor does the invariance result imply that two optimization runs will reach the same solution up to a rigid transformation. The objectives can have multiple configurations with different neighborhoods or layouts. Thus, after accounting for translation, rotation, and reflection, remaining differences still need to be assessed rather than automatically dismissed.

Comparing Runs Fairly

A useful comparison has two distinct parts. First, determine whether plots differ mainly by a rigid coordinate transformation. Second, compare structural summaries that do not depend on orientation, especially which cells are neighbors. Cell identifiers must match across runs; comparing points by row position is unsafe if cell ordering changed.

One way to assess orientation is to align one set of coordinates to the other using a rigid Procrustes alignment: choose a translation and an orthogonal transformation that minimize the summed squared coordinate differences for corresponding cells. Allowing reflections is appropriate when the purpose is to remove orientation ambiguity. Do not allow arbitrary scaling if the question is whether runs differ only by a distance-preserving symmetry. Small residual differences after alignment are not, by themselves, a complete measure of biological or structural agreement; the scale and geometry of the embedding also affect coordinate residuals.

For a direct local-structure comparison, define the \(k\)-nearest-neighbor set \(N_i^{(r)}(k)\) for cell \(i\) in run \(r\): the \(k\) other cells closest to \(i\) in that run’s embedding. A standard comparison for that cell is the Jaccard similarity, the size of the intersection divided by the size of the union of the two neighbor sets. Averaging this value across cells summarizes neighborhood retention. This is a descriptive comparison for a specified \(k\), not an inferential test or a universal threshold for acceptable stability.

What to Compare Beyond Coordinates

Neighborhood overlap is often more informative than visual similarity because it is unchanged by translation, rotation, or reflection. Still, one summary can conceal local differences: a high average may coexist with a small set of cells whose neighbors change substantially. Inspect the distribution of \(J_i(k)\), including low values, and consider whether conclusions change over a small, prespecified range of \(k\). The purpose is to check whether the local pattern of interest persists, not to search for a favorable value.

Island layout also requires care. Compare whether apparent groups contain similar cells and whether separations or mergers recur across runs, but do not treat a change in the page position of an island as a change in the data. Distances between embedding islands are unreliable measures of between-group similarity, so their relative spacing or orientation should not be used as a stability criterion on its own. A split or merge accompanied by changed local neighborhoods is more substantive than a rigid movement of an otherwise unchanged island.

Repeated runs of t-SNE should be compared with other t-SNE runs using the same settings; repeated UMAP runs should likewise be compared within UMAP. Differences between algorithms are not seed variability. The algorithms optimize different objectives, so a cross-method comparison answers a different question and should not be substituted for a controlled seed comparison.

Diagnostics, Pitfalls, and Limits

A run comparison is most useful when it can distinguish random variation from a change in the analysis itself. Check that the same cells and cell identifiers were used, the same features and preprocessing were applied, and all algorithm settings besides the intended seed were held fixed. Then inspect both the aligned plots and neighborhood summaries. If neighbor overlap is poor, determine whether the differences are widespread or concentrated in particular regions or cells; do not explain them away merely because the overall plot appears familiar.

• Confounded settings: changing perplexity, neighborhood size, metric, input representation, or other parameters alongside the seed prevents attributing the difference to randomness alone.
• Misleading overlays: plotting raw coordinates without alignment can make a simple rotation or shift look like a major change. Conversely, alignment can make plots look deceptively similar if local neighborhoods have changed.
• Cell-order errors: matching cells by row number rather than a stable identifier can produce meaningless coordinate residuals and neighbor comparisons.
• Overinterpreting a single summary: a mean overlap conceals cell-level variability and depends on \(k\); report what was compared and inspect the distribution.
• Assuming a seed guarantees exact replication: software, hardware, parallel execution, and approximate computations can matter. Record these details when reproducibility is important.

Run-to-run comparisons are diagnostic, not a formal test of a biological hypothesis. A small number of seeds does not define a sampling distribution for the embedding, and a mean overlap does not provide a confidence interval for a population parameter. These checks also cannot establish that a stable pattern is biologically meaningful: they assess reproducibility under chosen computational settings, not scientific validity.