Why Branches Pose a Different Geometric Problem
A principal curve is a flexible way to summarize a cloud of cells by one smooth, nonbranching path. This can be useful when cell states vary continuously along one progression. But a smooth curve is still one path: at a regular point it has two local directions, one on either side. A biological state that leads into three distinct directions has a different local structure. A single curve may pass through the common state and follow two directions, but it cannot also represent the third as a separate arm without folding, doubling back, or leaving some cells poorly represented.
The distinction matters because a curve is often used not only as a visual summary, but also to project cells onto a path and assign them a scalar pseudotime. If two branches are forced onto one path, cells from different paths can receive the same or misleading pseudotimes. The resulting ordering may hide, distort, or falsely sequence branch-specific cell states.
These definitions describe the geometry in the reduced space where the curve is fitted. They do not assert that the reduced coordinates capture all biological variation, or that a geometric junction proves a biological fate decision. For the examples below, assume the synthetic coordinates are measured on a common scale, Euclidean distance is an appropriate local measure, and the apparent arms are supported by cells rather than isolated outliers.
Why One Curve Cannot Represent a Three-Arm Junction
There is a simple topological reason for the limitation. Remove an interior point from a regular, locally non-self-intersecting curve. The remaining nearby part has two connected pieces: the part on one side of the point and the part on the other. Remove the junction from a three-arm structure, and three nearby pieces remain. Because these local structures have different numbers of connected pieces, they cannot be the same structure under a continuous, one-to-one reparameterization.
This argument concerns the local shape of the representation, not the biological meaning of the junction. It also relies on the curve being locally non-self-intersecting. A curve can be made to revisit a location or fold back on itself, but that does not create a clean branch: it introduces overlapping curve segments and potentially ambiguous projections instead.
Principal-curve fitting seeks a smooth path that summarizes the data. When the data occupy several diverging arms, a fitted path must trade off where it goes. Depending on initialization, smoothing, density, and the geometry of the arms, it may follow one arm, pass through the densest region, or bend between arms. None of these outcomes supplies a genuine third outgoing direction. The fitting algorithm may converge successfully while the chosen representation remains unsuitable.
Worked Example: A Y-Shaped Synthetic Dataset
Consider hypothetical cells in a two-dimensional embedding. The shared state is near \((0,0)\). Cells continue along an incoming arm at \((-1,0)\) and \((-2,0)\), and diverge along two outgoing arms at \((1,1)\), \((2,2)\), \((1,-1)\), and \((2,-2)\). For clarity, take these coordinates as exact and consider the candidate curve \(\gamma(t)=(t,0)\), the horizontal axis.
The projection of a point \((x,y)\) onto this curve is \((x,0)\), because that point has the same horizontal coordinate and minimizes squared Euclidean distance. Its curve parameter is \(t=x\), and its residual is \(|y|\). Thus, \((1,1)\) and \((1,-1)\) both project to \(t=1\), each with residual \(1\). Likewise, \((2,2)\) and \((2,-2)\) both project to \(t=2\), each with residual \(2\). The two cells at each horizontal coordinate are on different outgoing arms, yet receive identical curve positions.
For all six cells, the squared residuals are \(0,0,1,4,1,4\). The sum of squared residuals is \(10\), and the root mean squared residual is \(\sqrt{10/6}=1.291\) coordinate units. A small value is not guaranteed merely because a curve was fitted; here the residuals on the outgoing arms increase with distance from the junction. A curve that bends toward one arm might reduce that arm’s residuals, but could increase the other arm’s residuals or misrepresent the incoming path.
How Branches Distort Projection and Pseudotime
Projection converts a cell’s position in the embedding into its nearest curve location. If the curve passes between diverging arms, cells from separate paths can map to the same parameter value. Even if their projected values differ, the curve’s scalar ordering cannot record which outgoing direction a cell follows. This is a structural limitation of reducing a branching arrangement to one coordinate, not a failure of arithmetic in the projection step.
Self-consistency provides another way to see why fitting can be misleading. In the standard principal-curve idea, a curve is self-consistent when, at a given curve position, the conditional mean of the data projected there lies on the curve. If cells from two symmetric arms are assigned to the same position, their transverse deviations can cancel in that conditional mean. The resulting average can lie between the arms, even if few or no cells occupy that intermediate location.
This average is the horizontal-axis point at \(t=1\). It summarizes the two symmetric observations but does not identify either outgoing path. More generally, branch-specific signals may average to an unrepresentative intermediate profile when cells from distinct arms are combined at the same pseudotime.
Worked Example: Branch-Specific Values Hidden at One Pseudotime
Use the two hypothetical cells \((1,1)\) and \((1,-1)\) from the previous dataset. Suppose their vertical coordinate is also a simple synthetic marker measurement, in arbitrary measurement units: the upper-arm cell has value \(1\), and the lower-arm cell has value \(-1\). On the horizontal candidate curve, both cells project to \(t=1\). The mean marker value among cells at that projected position is
The mean is \(0\) units, although neither cell has that value. It conceals the contrast between the two arms. The same averaging problem can arise when many cells are grouped by a common pseudotime: a smooth average trend may appear even when branch-specific trends differ. The remedy is not to interpret the pooled mean as a profile shared by both paths; first check whether the cells at that pseudotime belong to distinct geometric or biological directions.
Diagnostics for a Misleading Single-Curve Fit
A low overall fitting error alone does not establish that one curve is appropriate. Diagnostics should ask whether the residuals, projections, and local geometry are consistent with a single path. In a reduced embedding, inspect the fitted curve together with the cells, and examine residuals in relation to projected position rather than relying only on an overall summary.
- Look for repeated arms around a junction. A dense region that opens into three well-supported directions is incompatible with a regular single-path representation at that location.
- Plot residuals against projected position. A sharp increase, systematic pattern, or change in residual direction near a suspected junction can reveal cells that the curve does not represent well. Residuals need not be random in a formal test; the point is to detect structured lack of fit.
- Check whether distinct groups collapse onto the same curve segment. Cells with the same projected position but different local directions, labels, or marker patterns may have been combined by the projection.
- Assess sensitivity to initialization and smoothing. If plausible fitting choices produce curves that follow different arms or give substantially different projections, the single-curve summary is not robust to those choices.
- Check data support and scale. An apparent extra arm may be caused by a few outliers or an unevenly scaled embedding. Confirm that it contains a coherent set of cells and that the chosen distance measure is meaningful.
These are diagnostic clues, not automatic branch tests. An apparent fork can reflect noise, a batch effect, a poorly chosen embedding, or an unmodeled covariate rather than a true branching cell-state structure. Conversely, limited sampling may make a real branch difficult to see. Interpret geometry alongside cell annotations and other relevant evidence, without treating a curve fit as proof of lineage or causation.
Worked Example: Quantifying Projection Collapse and Residual Growth
Return to the four synthetic outgoing-arm cells \((1,1)\), \((1,-1)\), \((2,2)\), and \((2,-2)\), projected onto \(\gamma(t)=(t,0)\). Their projected parameters are \(1,1,2,2\), respectively. Their residuals are \(1,1,2,2\) coordinate units. The root mean squared residual across these four cells is
At \(t=1\), the two cells are separated by \(2\) coordinate units in the vertical direction; at \(t=2\), they are separated by \(4\) coordinate units. Yet each pair has one shared pseudotime. Both the increasing residuals and the growing separation between same-pseudotime cells flag a poor single-curve representation. These calculations diagnose this candidate curve on this synthetic dataset; they do not prove that every possible fitted curve has the same numerical residuals.
Choosing a Representation and Recognizing Its Limits
When the evidence supports one nonbranching progression, a principal curve remains a useful compact summary. It provides a smooth path and a one-dimensional coordinate, subject to the curve’s fit and projection assumptions. When the data support multiple diverging paths, the representation should be able to preserve the distinct directions rather than force them into one scalar ordering.
| Approach | What it can represent | Main caution |
|---|---|---|
| One principal curve | A smooth, nonbranching progression | Can collapse distinct arms or leave structured residuals |
| Separate curves for specified lineages | Different paths after a chosen division | Requires a defensible way to identify the lineages and their shared region |
| A branch-capable graph or trajectory representation | A shared region with multiple outgoing paths | Can add unsupported branches or depend on choices about connectivity and pruning |
| Separate state comparisons without a continuous path | Differences among groups of cells without asserting an ordering | Does not provide a continuous pseudotime along a path |
A branch-capable representation is not automatically correct: it can overfit noise, reflect uneven sampling, or impose a junction where none exists. Separate curves also need careful interpretation near their shared region, and state comparisons avoid a path claim but answer a different question. The choice should follow the supported geometry and scientific aim, not a preference for a particular output format.
Common Mistake
Check Your Understanding
Answer each question before reading the brief solutions.
- Why can a regular, locally non-self-intersecting curve not represent all three arms of a genuine junction?
- For the curve \(\gamma(t)=(t,0)\), what are the projected parameter and residual for a cell at \((3,-2)\)? Include units in words if the coordinates have units.
- Two cells at \((2,2)\) and \((2,-2)\) are projected onto \(\gamma(t)=(t,0)\). What is their shared projected parameter, and what geometric information does that parameter omit?
- Why is a converged principal-curve fit not sufficient evidence that the data follow a nonbranching path?
- Name two diagnostics that could flag a misleading single-curve fit, and state one alternative representation or analysis.
Solutions. (1) Removing an interior point from a regular curve leaves two local pieces, while removing a three-arm junction leaves three; the local structures differ. (2) The nearest point is \((3,0)\), so the projected parameter is \(t=3\) and the residual is \(\sqrt{(3-3)^2+(-2-0)^2}=2\) coordinate units. (3) Both have \(t=2\); the scalar projection omits whether the cell lies on the upper or lower arm, and the cells are separated by \(4\) coordinate units. (4) Convergence only indicates that the algorithm met its stopping rule; it does not show that a single nonbranching curve matches the data’s topology. (5) Possible diagnostics include residuals against projected position, same-pseudotime cells on distinct directions, or sensitivity to initialization. Alternatives include a branch-capable graph representation, separately specified lineage curves, or state comparisons that do not assert a continuous path.