Identifiability in Cell-Type Deconvolution

Why Identifiability Matters

A deconvolution algorithm can return a precise-looking vector of cell-type proportions even when the observed spot does not contain enough information to distinguish those cell types. The central question is therefore not only how to estimate proportions, but whether the data and reference profiles determine them uniquely. This is a question of identifiability, separate from the numerical method used to fit the model.

Use the linear mixture model from the earlier material on spot deconvolution. Let \(y\) be the vector of expression measurements for the selected genes in one spot, and let \(S\) be the reference matrix whose \(k\)th column gives the profile for cell type \(k\). The unknown mixture weights are \(p=(p_1,\ldots,p_K)^T\). On the scale where the reference profiles and spot measurements are comparable, the noiseless model is \(y=Sp\). The weights are proportions: \(p_k\geq 0\) and \(\sum_{k=1}^{K}p_k=1\). In observed data, measurement error or model mismatch may be represented by \(y=Sp+\varepsilon\).

Identifiability concerns the mapping from proportions to expected measurements, not whether the measurements are noisy or whether an optimizer converges. It also depends on what the weights represent. If reference profiles encode RNA contribution per cell rather than calibrated expression per cell, fitted weights need not equal cell-number proportions. The interpretation must match the scale and construction of the reference profiles.

The Uniqueness Condition

Two mixtures \(p\) and \(q\) produce the same noiseless observation exactly when \(Sp=Sq\), or equivalently when \(S(p-q)=0\). At the same time, both vectors sum to one, so their difference satisfies \(\sum_k(p_k-q_k)=0\). Thus an ambiguity is a nonzero vector in the null space of \(S\) whose entries sum to zero, provided the corresponding alternative mixtures are feasible.

This gives an exact global criterion: the only vector \(h\) satisfying \(Sh=0\) and \(\sum_k h_k=0\) must be \(h=0\). If a nonzero such \(h\) exists, choose an interior mixture \(p\), with all entries positive. For a sufficiently small nonzero \(t\), both \(p\) and \(p+th\) remain nonnegative and sum to one. Since \(S(p+th)=Sp+tSh=Sp\), they yield the same expected observation, contradicting uniqueness.

Affine independence means that no column can be represented as an affine combination of the others, where the coefficients sum to one. Geometrically, the possible noiseless observations form the convex hull of the reference profiles: every mixture is a convex combination of the columns. Global identifiability means each point in that convex hull has only one representation as a convex combination. If every column of \(S\) has been normalized to the same total, then any \(h\) with \(Sh=0\) also has entries summing to zero. In that case, full column rank is necessary as well as sufficient for global uniqueness.

The criterion concerns exact uniqueness. Near dependence is different: it may leave the solution technically unique while making small changes in measurements produce large changes in the estimated proportions. That is a stability problem, and it matters in practice just as much.

Collinear Signatures and Exact Ambiguity

If two cell types have identical profiles on the selected genes, the data can reveal their combined contribution but cannot separate their proportions. More generally, any affine dependence among the columns creates at least one mixture with multiple feasible representations. A rank check alone needs care: the sum-to-one constraint can make a set of columns affinely independent even when it is linearly dependent. The relevant test for proportions is the null-space condition above, not simply the number of genes compared with the number of cell types.

Why Near-Collinearity Makes Estimates Unstable

When signatures are almost, but not exactly, redundant, a unique answer may exist but be sensitive to noise. For two cell types with profiles \(s_1\) and \(s_2\), write \(d=s_1-s_2\). Since \(p_2=1-p_1\), the model is \(y=s_2+p_1d+\varepsilon\). If least squares is used without an active boundary constraint, minimizing the squared residual gives the following estimate.

The first expression follows by differentiating \(\|y-s_2-p_1d\|^2\) with respect to \(p_1\), setting the derivative to zero, and solving for \(p_1\). Substituting \(y=s_2+p_1d+\varepsilon\) gives the second expression. When \(d\) is small, the denominator \(d^Td\) is small, so noise aligned with the difference between the profiles can produce a large change in the estimated proportion. Constraining the estimate to \(0\leq \widehat p_1\leq 1\) may clip an unstable result to a boundary; it does not recover information missing from the data.

Missing Cell Types Can Mimic a Good Fit

The identifiability condition assumes that the reference matrix includes the cell types that contribute to the spot and that the reference profiles represent them appropriately. If a type is omitted, fitting the reduced model may attribute its expression to included types. A small residual does not necessarily expose the omission: if the missing profile lies within the convex hull of the included profiles, its expression can be represented exactly as a mixture of them.

Diagnostics, Alternatives, and Limitations

Identifiability should be checked before interpreting individual fitted proportions. In practice, profiles are estimated from reference data and observations are noisy, so a mathematical rank test is only one part of the assessment. Useful checks include the following:

• Inspect signature similarity. Look for duplicate or highly correlated profiles on the selected genes. Correlation is a warning rather than a complete identifiability test because the proportion constraint also matters.
• Assess singular values or affine rank. Small singular values signal directions in which distinct mixtures have very similar predicted measurements. Check the affine condition for the simplex, not only full rank, especially when there are more cell types than genes.
• Examine sensitivity. Refit after modest changes to the reference profiles, selected markers, or observed measurements. Large proportion changes with little change in fitted expression indicate weak identification.
• Review residuals and reference coverage. Systematic residual patterns can indicate a missing type or mismatched reference, but a good residual fit cannot exclude a missing profile inside the reference convex hull.

When signatures are too similar, one alternative is to collect or select additional genes that distinguish the relevant types. A better-matched reference panel can address missing or poorly estimated profiles. Regularization or prior information can also select a stable solution from a broad set of plausible mixtures, but the resulting uniqueness comes partly from those added assumptions, not from the expression data alone. Spatial constraints may contribute information across spots, but they do not repair an unsuitable reference matrix automatically and should be justified separately.

These approaches have limits. Adding genes helps only if their profiles provide new, reliable contrasts; redundant or noisy markers may add little. Regularization can reduce variability while introducing bias and can hide ambiguity if its assumptions are not reported. Better numerical conditioning cannot correct a systematic mismatch between reference and tissue. Finally, identifiability is specific to the chosen genes, cell types, reference scale, and model: a result for one setup does not guarantee uniqueness under another.