Pearson Residuals as Variance-Stabilized Features

Why Standardize Counts Before Ranking Genes?

A raw count difference does not have the same meaning at every expression level. A difference of three molecules can be substantial for a gene whose expected count is near one, but modest for a gene whose expected count is near one hundred. A gene-ranking method should therefore account for both the fitted mean and the expected variability at that mean.

Pearson residuals do this by dividing each observed-minus-fitted count difference by its model-based standard deviation. Across cells, unusually variable residuals can identify genes whose observed variation exceeds what the fitted count model predicts. This is a way to construct features on a common scale; it does not make counts normally distributed or prove that a high-ranked gene is biologically important.

Definition and Model

Let \(Y_{ig}\) be the observed count for gene \(g\) in cell \(i\), and let \(\hat{\mu}_{ig}\) be its fitted mean under a count model. For a negative-binomial model, use the dispersion parameterization \(\operatorname{Var}(Y_{ig}\mid\text{model})=\mu_{ig}+\phi_g\mu_{ig}^2\), where \(\phi_g\geq 0\). A fitted mean may account for cell-specific library size through an offset and may include other modeled covariates. The Pearson residual uses the fitted mean and variance.

The model is conditional on whatever offsets and covariates were used to fit it. Interpreting residuals as standardized deviations assumes that the mean structure is adequate, the dispersion captures the relevant conditional variability, and the observations are sufficiently independent for the intended comparison. In single-cell data, these are working assumptions, not guarantees. Estimating parameters from the same data also means residuals need not have exactly zero mean or unit variance.

Why the Standardization Works

The basic justification follows from the first two conditional moments. If the model parameters were known and the model were correct, then \(E(Y_{ig}\mid\text{model})=\mu_{ig}\) and \(\operatorname{Var}(Y_{ig}\mid\text{model})=V_{ig}\), where \(V_{ig}=\mu_{ig}+\phi_g\mu_{ig}^2\). Thus:

This is the sense in which Pearson residuals are variance-standardized. It is a statement about their first two moments under a correct model, not a claim that each residual is normally distributed. Counts can be discrete and strongly skewed, especially at low means. Fitting parameters, filtering cells, or misspecifying the mean or variance can also change the residual distribution.

For a gene, a simple variability score is the sample variance of its residuals across \(n\) cells: \(s^2_{r,g}=\frac{1}{n-1}\sum_{i=1}^{n}(r_{ig}-\bar r_g)^2\), where \(\bar r_g=\frac{1}{n}\sum_i r_{ig}\). Larger values mean greater residual spread relative to the fitted model. Another possible summary is the mean squared residual, \(Q_g=\frac{1}{n}\sum_i r_{ig}^2\). Unlike the centered variance, \(Q_g\) also reflects a nonzero average residual, so a ranking based on it can mix systematic mean misfit with cell-to-cell variability. Inspect the residual mean as well as the spread.

Using Residual Variability to Rank Genes

To compare genes, first calculate each cell–gene residual using that gene's fitted mean and dispersion. Then summarize how much the residuals vary across cells. A high residual-variance score indicates that the gene varies more across the observed cells than its fitted mean–variance relationship predicts. If the score is low, the observed differences are broadly compatible with that relationship.

The interpretation depends on the model. For example, if a biological covariate was included in the mean, residual variability describes variation remaining after accounting for that covariate; it does not summarize the total biological variation. Conversely, omitting a relevant cell group or technical effect can inflate residual spread. A ranking is therefore a model-dependent ordering, not a model-free property of a gene.

Extreme Residuals and Model Diagnostics

An extreme residual can arise because a cell genuinely expresses a gene unusually strongly or weakly. It can also arise because the fitted mean is too small, the dispersion is underestimated, a relevant covariate is missing, or the observation reflects a technical problem. The residual alone cannot distinguish these explanations.

Check residuals in context rather than treating a large absolute value as an automatic exclusion rule. Useful checks include:

• Plotting residuals against fitted means or library size to look for systematic trends.
• Checking whether residuals are consistently shifted above or below zero for a gene or group of cells.
• Examining whether a few extreme values dominate a gene's variability score.
• Comparing residual patterns across known batches or biological groups that were not included in the fitted mean.
• Checking whether the dispersion estimates and mean–variance trend are plausible for the range of expression being analyzed.

For example, residual spread that increases systematically with fitted mean can indicate that the variance model does not adequately capture the mean–variance relationship. A cluster of positive residuals in one batch may point to an omitted batch effect, but it could also reflect a true difference between the cells in that batch. The data collection and study design are needed to evaluate the alternatives.

At low fitted means, the discreteness and asymmetry of counts are particularly important: a residual of a given magnitude does not necessarily have the same tail probability across genes or cells. Pearson residuals are not automatically standard normal, so normal-theory cutoffs or p-values should not be assigned to them without an appropriate distributional argument. Extreme values can be flagged for review, but their interpretation requires diagnostics and biological context.

How Pearson Residuals Differ From Other Rankings

Negative-binomial deviance, the subject of the earlier material on negative-binomial deviance for feature ranking, also compares observed counts with fitted means. A deviance residual is based on the likelihood loss relative to a saturated model; a Pearson residual is based directly on the signed difference divided by the fitted standard deviation. Both depend on the count model, but they weight discrepancies differently, so their gene rankings need not agree.

Dispersion-based rankings instead summarize excess variability relative to a mean–variance trend. Pearson-residual rankings express deviations on the scale of fitted standard deviations and can retain cell-level information before a gene summary is formed. Neither approach removes model dependence. A high score under either method can reflect omitted structure as well as the variation of interest.

Common Pitfalls and Limitations

• Assuming variance stabilization means normality. Dividing by the fitted standard deviation standardizes the model-based variance; it does not make discrete counts normally distributed.
• Treating every extreme residual as an error. A large residual can reflect a real signal, a technical issue, or a misspecified mean or variance. Investigate rather than automatically discard it.
• Interpreting residual variance as biological variability alone. The score includes any unmodeled technical or biological structure, and its meaning depends on which covariates were included.
• Ignoring a nonzero residual mean. A score based on squared residuals can be high because of a systematic shift, not only cell-to-cell spread. Inspect both the mean and centered variance.
• Reading a ranking as a significance test. Residual scores order genes under a fitted model; they do not supply a p-value or a probability that a gene is important.

Pearson residuals are useful when the goal is to compare count deviations on a scale that accounts for expected mean-dependent variance. Their strengths—simple standardization and a direct connection to the fitted variance—come with a central limitation: a misspecified model can create misleading residuals and rankings. Model checks are part of interpreting the score, not an optional step after it.