Diagnosing Kinetic Model Misfit

Why Diagnose Kinetic Model Misfit?

An RNA velocity estimate is conditional on a kinetic model: it describes how unspliced and spliced RNA are assumed to change over time. A fitted curve may look plausible overall while systematically missing a subset of cells, a phase of the trajectory, or a change in transcriptional behavior. When that happens, the inferred rates and velocity directions can be misleading even if an optimizer has returned precise numerical estimates.

This tutorial focuses on diagnosing that mismatch. The central idea is to compare the model’s implications with the observed phase portrait, the residuals left by the fit, and the fitted parameter values. These are complementary checks: a pattern in one may be difficult to see in another. A diagnostic is evidence to investigate, not automatic proof of a particular biological mechanism.

Model Assumptions and Diagnostic Residuals

For a gene, let \(u(t)\) and \(s(t)\) denote unspliced and spliced RNA abundance at time \(t\), respectively. A common kinetic model is

Here \(a(t)\) is the transcription rate, \(\beta\) is the splicing rate, and \(\gamma\) is the spliced-RNA degradation rate. The model assumes nonnegative rates, a specified form for how transcription changes over time, and constant splicing and degradation rates over the fitted trajectory. A steady-state phase portrait additionally assumes that the relevant observations are adequately described by equilibrium under a constant transcription rate. A dynamical fit instead allows the abundance to change over time according to the specified transcriptional history, often with a switch between transcriptional regimes.

These assumptions concern the mean behavior represented by the model. Observed counts or normalized abundances vary around that behavior. Accordingly, a residual is a discrepancy between an observation and the model’s prediction; it can reflect model failure, observation variability, or both. For a fitted phase-portrait curve \(s=f(u;\theta)\), where \(\theta\) denotes its parameters, a useful vertical residual is

A positive residual means the observed spliced abundance is above the fitted curve at that unspliced abundance; a negative residual means it is below. For a time-indexed dynamical fit, residuals can instead be defined separately for unspliced and spliced abundance by subtracting the fitted trajectory at the assigned time. These definitions are not interchangeable: vertical residuals in a phase portrait describe distance in the \(s\)-direction, while trajectory residuals compare both coordinates with predictions at a specified time.

What a Steady-State Phase Portrait Implies

At equilibrium with constant transcription, both derivatives in the kinetic equations equal zero. The first equation gives \(a=\beta u^*\), and the second gives \(\beta u^*=\gamma s^*\), where \(u^*\) and \(s^*\) are equilibrium abundances. Thus, when the rates are positive, the steady-state phase portrait has a linear relationship through the origin:

The slope is a ratio of rates, not a direct measure of either rate alone. This relationship is a useful reference, but points departing from it do not by themselves establish that the steady-state model is wrong: cells changing expression over time are not necessarily at equilibrium. A departure may instead reflect transient dynamics, a different regime, an influential observation, or variability in the measurements.

A short derivation also clarifies why residual structure matters. Suppose a model correctly describes the conditional mean of the observed spliced abundance at each \(u_i\), so that \(s_i=f(u_i;\theta)+\epsilon_i\), with \(E(\epsilon_i\mid u_i)=0\). If the parameters were known, then

Consequently, a persistent sequence of positive residuals at high abundance, for example, is not what this mean model predicts. It suggests the curve may be missing structure. In practice, parameters are estimated from the same observations, so residuals are not independent and the argument is a diagnostic principle rather than a formal test. Correlated observations and uneven variability also make simple residual thresholds unreliable.

Reading Residuals From a Dynamical Fit

For a dynamical model, inspect discrepancies in both abundance coordinates and in the order of the fitted trajectory. If the observations are assigned times \(t_i\), define \(r_{u,i}=u_i-\hat{u}(t_i)\) and \(r_{s,i}=s_i-\hat{s}(t_i)\). Plot these residuals against fitted time, fitted abundance, and any relevant cell grouping. A run of residuals with the same sign over part of the trajectory may indicate a missed switch, a poorly specified transcriptional history, or a rate that is not adequately represented as constant.

The differential equations also provide an instantaneous check. At a given state and transcription rate, the model predicts the velocity components \(a-\beta u\) and \(\beta u-\gamma s\). If locally estimated changes repeatedly disagree with those predictions, the assumed dynamics deserve scrutiny. Such local estimates can be noisy, particularly when cells are not observed repeatedly over time; a discrepancy in one location should therefore be checked against the full trajectory and the uncertainty of the fit.

Parameter Estimates and Multiple Regimes

Parameter estimates are another diagnostic, not merely the final output of a fit. Estimates should be interpreted in the units used for time and abundance. Rates \(\beta\) and \(\gamma\) should be nonnegative and have units of inverse time; a negative estimated rate contradicts the model’s biological rate interpretation. Extremely large or small rates, a switch time outside the observed time range, or estimates that change markedly when a small set of cells is removed can signal weak information in the data or a poor fit. These signs do not alone establish which parameter or assumption is responsible.

A single regime can also conceal several systematic patterns. For example, cells from two groups may follow different phase-portrait relationships, or one part of the trajectory may follow a transient path that a steady-state line cannot represent. Compare residuals across groups and trajectory regions. If a pattern is concentrated in one group, investigate whether the grouping is scientifically meaningful and whether a shared-rate assumption is tenable. Do not infer separate biological mechanisms just because a more flexible fit can reduce residuals: added flexibility can fit random variation as well as real structure.

A Practical Diagnostic Sequence

Diagnostics, Alternatives, and Limits

A phase-portrait check is quick and communicates geometry; residual plots make systematic deviations easier to locate; parameter and sensitivity checks reveal instability that may be hard to see visually. None replaces the others. A formal goodness-of-fit procedure, when its assumptions are justified, can summarize discrepancy relative to a specified variability model. But a single test statistic does not show where the model fails, and calibration can be unreliable if the observations are dependent or the variability model is wrong. A more flexible model may capture multiple regimes, but it also adds assumptions and parameters. Comparing alternatives should therefore include the structure of residuals and parameter stability, not only whether one fit looks better.

A particularly common mistake is to treat every point away from a steady-state line as a bad observation. That reasoning assumes all cells are at equilibrium, even though a dynamic trajectory can depart from equilibrium while following the kinetic equations. The correction is to ask whether the deviation is consistent with the model’s predicted dynamics and whether it is isolated or systematic. Likewise, a large residual is not proof that a cell should be discarded. It may be a measurement issue, a genuine distinct regime, or evidence that the fitted model is incomplete.

These diagnostics have limits. Residual patterns can be hard to distinguish from variability, fitted parameters may be weakly determined, and conclusions depend on how cells are assigned to a trajectory or regime. A good fit does not prove that the biological assumptions are true; it shows only that the model has not displayed an obvious mismatch under the checks used. When diagnostics disagree, report the uncertainty and avoid presenting a single fitted velocity as an unquestionable description of the dynamics.