Choosing a Feature Count and Selection Threshold

Why the Selection Rule Matters

After filtering low-expression genes and choosing an expression scale, a feature-selection procedure still needs a rule for deciding which genes to retain. A rule that selects the highest-ranked genes always returns a prespecified number, whereas a score cutoff returns however many genes meet the threshold. These choices affect the size and composition of the selected set, so the rule should be described alongside the resulting gene list.

This tutorial compares three choices: selecting a fixed number of genes by rank, selecting genes above a dispersion cutoff, and selecting genes above a residual-variability cutoff. The scores must be computed on the chosen expression scale and with a specified scoring procedure. The introductory material on expression scales for variability modeling motivates that choice; here the focus is how the selection rule uses the resulting scores.

The score is not itself the selection rule. For example, regularized dispersion scores and residual-based scores can both rank genes, but a value of 1.5 on one score scale does not generally mean the same thing as 1.5 on another. A cutoff is interpretable only with its scoring method and scale specified. Neither a high score nor selection by a cutoff is, by itself, a hypothesis test or a claim that a gene is biologically important.

Fixed Counts and Score Cutoffs

Write the scores for the \(G\) eligible genes in descending order as \(S_{(1)} \geq S_{(2)} \geq \cdots \geq S_{(G)}\). A fixed-count rule selects the genes occupying the first \(K\) ranks. If scores tie at the boundary, a tie-breaking convention is needed to return exactly \(K\) genes. A deterministic convention, such as ordering tied genes by a prespecified gene identifier, makes the count reproducible but does not make the tied genes biologically distinguishable.

A cutoff rule instead selects scores satisfying \(S_g \geq c\). Its selected count is determined by the observed score distribution. If the boundary is tied, every gene meeting the cutoff is selected, so the result may contain more genes than a target count. Changing the inequality to \(S_g>c\) changes what happens to genes exactly at the threshold; state which convention is used.

The result follows directly from the ordering. For \(1 \leq K < G\), the condition \(c \leq S_{(K)}\) includes the first \(K\) ranked genes, while \(S_{(K+1)} < c\) excludes every gene ranked below them. Together they select exactly the first \(K\). For \(K=G\), there is no \((G+1)\)st score: all genes are selected precisely when the cutoff is no greater than the smallest eligible score, \(S_{(G)}\). These conditions concern counts, not the substantive adequacy of the selected genes.

Dispersion and Residual Cutoffs in Practice

A dispersion-cutoff rule selects genes whose chosen dispersion measure reaches a stated threshold. Depending on the scoring procedure, the measure may be adjusted for the mean-variance relationship rather than being raw variance. The selection rule does not determine how dispersion was estimated; that definition and any trend adjustment must be reported separately. This distinction matters because a cutoff on unadjusted variance can favor genes with larger means, even when the scientific aim is to identify variability beyond that associated with the mean.

A residual-cutoff rule acts on a summary of residual behavior after the specified model has been fitted. One transparent example is the sample variance of a gene’s Pearson residuals across \(n\) cells. If \(r_{gc}\) is the residual for gene \(g\) in cell \(c\), define \(Q_g = \frac{1}{n-1}\sum_{c=1}^{n}(r_{gc}-\bar{r}_g)^2\), where \(\bar{r}_g\) is that gene’s mean residual across cells. A rule using \(Q_g \geq c\) selects genes with sufficiently large residual variance under this particular definition. Other procedures may summarize residuals differently, so the formula must accompany the threshold.

A residual score is not automatically a p-value, and a cutoff such as \(Q_g \geq 1\) does not establish statistical significance. Residuals depend on the fitted model, its assumptions, the expression-scale and normalization choices, and any residual-processing convention. A residual cutoff describes a feature-selection decision on the resulting scores; it does not certify that all model assumptions hold.

Dataset Size Changes What a Rule Means

The number of eligible genes \(G\) can vary because datasets differ in measured genes or because filtering retains different numbers. A fixed \(K\) then represents different fractions of the eligible set. Conversely, a score cutoff can select different numbers of genes in different datasets because their score distributions differ. Neither rule guarantees the same fraction or the same biological composition across datasets.

The number of cells also matters. With fewer cells, gene-level variability scores may be estimated less precisely; with more cells, those scores may be more stable, though this depends on the sampling and model. Increasing cell count does not, by itself, imply that a particular gene should pass a fixed numerical cutoff. The cutoff must remain tied to its score definition and analysis purpose.

Diagnostics, Comparisons, and Limitations

Before using a selection rule, verify that the eligible-gene set, score definition, direction of ranking, and cutoff inequality are explicit. Then examine the score distribution near the selection boundary. A large gap between \(S_{(K)}\) and \(S_{(K+1)}\) means a range of thresholds yields the same fixed-count set; a tie at the boundary means a score cutoff cannot reproduce exactly \(K\) genes under the \(S_g \geq c\) convention. For a cutoff rule, report both \(c\) and the number of genes selected, since the threshold alone does not communicate set size.

For residual scores, also inspect whether the fitted model and residual calculations are behaving as intended. Systematic residual patterns or remaining library-size effects can undermine the interpretation of a residual-variability score. A numerical cutoff cannot correct model misspecification or substitute for residual diagnostics. Because no single cutoff is universally calibrated across score definitions and datasets, any comparison should state whether the same scale, scoring method, filtering rule, and threshold were used.

These rules answer different operational questions. A fixed count is useful when an analysis requires a specified number of features, but it does not ensure that the last selected gene is meaningfully separated from the next one. A dispersion cutoff makes the minimum score explicit, but selected-set size can vary. A residual cutoff likewise states a minimum residual score, but its meaning is conditional on how residuals were obtained and summarized. In every case, the threshold and count are choices for feature selection, not evidence of causation, biological importance, or inferential significance.

Common Mistake

A common mistake is to say, “I selected the top 200 genes, so the cutoff is the score of the 200th gene,” and then treat that as an exact score threshold without checking ties. If the 200th and 201st genes have the same score, a rule \(S_g \geq S_{(200)}\) selects at least 201 genes. The correction is to distinguish the rank-based rule, which may use a stated tie-break to return exactly 200, from the score-cutoff rule, which includes every gene meeting the threshold. Also check the endpoint: the condition involving \(S_{(K+1)}\) applies only when \(1 \leq K < G\); selecting all \(G\) genes by cutoff requires \(c \leq S_{(G)}\).