Why Filter Before Variability Ranking?
A gene recorded as zero in nearly every cell provides little stable information for a variability ranking. It may be unexpressed in the sampled population, or its expression may be too sparse to distinguish from sampling and measurement limitations. Either way, including thousands of such genes can make the later feature-selection step less useful and more computationally demanding.
Low-expression filtering is a preliminary screen, not a method for deciding which genes are biologically variable. It asks whether the observed counts provide enough support for a gene to be considered at all. A variability ranking asks a different question: among genes with adequate support, which show informative differences across cells? Filtering too aggressively can remove a gene that is consistently detected in a small but meaningful cell group. The goal is therefore not to make the retained genes common in every cell; it is to remove genes with inadequate evidence while preserving plausible subset-specific signal.
The threshold \(c=1\) counts every nonzero observation as detected. A larger \(c\) requires more observed counts in a cell before that cell contributes to the detection count. A rule such as “retain genes detected in at least 10 cells” is a minimum-cell rule; a rule such as “retain genes with at least 20 total counts” is a total-count rule. These rules are not interchangeable. Total counts can be concentrated in one or two cells, while a detection count records how broadly the gene is observed.
Specify the Filter and Its Target
There is no universal cutoff appropriate for every dataset. A useful filter makes its unit and target explicit: which count matrix is used, what counts as detected, how many cells must meet the threshold, and whether the criterion applies across all cells or within relevant groups. For example, one might retain a gene if it has at least one count in 10 or more cells. This is transparent, but the same threshold has different implications in a dataset of 100 cells and one of 100,000 cells.
Observed counts also depend on sequencing depth. A cell with a larger library may be more likely to show a nonzero count for a weakly expressed gene than a shallowly sequenced cell. Thus a raw-count detection rate is an operational summary of observed data, not a direct measurement of the fraction of cells with biological expression. If detection is defined using a higher count threshold, the dependence on depth can be stronger. Apply the same rule consistently and examine whether filtering is disproportionately removing genes from shallow libraries or particular samples.
A practical screen often combines criteria. For example, retain a gene if it is detected at least once in \(k\) cells and has a total count of at least \(t\). The first condition prevents a few high counts from carrying the entire decision; the second excludes genes with many isolated single counts but little total support. The values of \(k\) and \(t\) should reflect dataset size, assay sensitivity, and the intended analysis, rather than being treated as universal constants.
Why a Global Detection Rate Can Miss Subset-Specific Genes
Suppose a gene is detected in a fraction \(q\) of cells from a small cell subset and is not detected elsewhere. Even if it is reliably detected within that subset, its pooled detection rate is diluted by the subset’s size. If the subset contains fraction \(r\) of the cells and the probability of observing a count meeting the chosen threshold in an expressing subset cell is \(s\), then the approximate pooled detection probability is \(rs\), assuming the other cells do not meet the threshold.
This simple model explains why a global minimum-cell rule can be unsuitable for rare populations. Under independent, equal-probability detection across sampled cells, the probability of observing no detections is \((1-rs)^n\). The formula is only an approximation when cells have different depths, cell types are sampled unevenly, or observations are dependent. It nevertheless makes the central point: a low pooled rate does not by itself show that a gene is uninformative.
A safeguard is to assess support within known or plausible cell groups as well as across the full dataset. Define \(N_{gh}(c)\) as the number of cells in group \(h\) whose count for gene \(g\) is at least \(c\). A group-aware rule can retain a gene when it meets the global criterion or a suitably chosen within-group criterion. This does not mean accepting any gene observed in one cell: group-specific support should still require enough observations to be credible for the group’s size and assay sensitivity.
Worked Example: A Gene Concentrated in a Small Subset
Consider 1,000 hypothetical cells, of which 50 belong to a cell group of interest. A gene has at least one count in 30 of those 50 cells and in none of the other 950 cells. Suppose the proposed global rule retains genes detected in at least 40 cells.
The pooled detection count is \(N_g(1)=30\), so the gene fails the global rule. Its pooled detection rate is \(30/1000=0.03\), or 3%. Within the group of interest, its detection rate is \(30/50=0.60\), or 60%.
The global rule would discard the gene despite detection in a majority of the relevant group. A group-aware screen could instead require, for example, detection in at least 20 cells within that group. The gene passes that criterion because \(30\geq 20\). The calculation does not prove that the gene is a useful feature; it shows why pooled detection alone is insufficient for the filtering decision.
Worked Filtering Decisions
Worked Example: Combining Detection and Total-Count Thresholds
In a hypothetical dataset of 120 cells, use the following illustrative rule: retain a gene if it has at least one count in 8 or more cells and a total count of at least 20. Three genes have these observed patterns:
- Gene A has counts of 1 in 10 cells and 0 in the other 110.
- Gene B has a count of 12 in one cell, a count of 1 in 3 other cells, and 0 in the remaining 116.
- Gene C has a count of 2 in 8 cells and 0 in the other 112.
For Gene A, \(N_A(1)=10\), \(D_A(1)=10/120=0.0833\), and \(T_A=10\). It meets the minimum-cell condition but fails the total-count condition because \(10<20\). It is removed.
For Gene B, \(N_B(1)=4\), \(D_B(1)=4/120=0.0333\), and \(T_B=12+3=15\). It fails both criteria: \(4<8\) detected cells and \(15<20\) total counts. It is removed. This illustrates why a few large counts do not necessarily indicate broad support.
For Gene C, \(N_C(1)=8\), \(D_C(1)=8/120=0.0667\), and \(T_C=8\times2=16\). It meets the minimum-cell condition but fails the total-count condition, so it is removed under this rule. That decision may be too strict if the eight cells form a biologically important small group; the rule should be checked against cell-group context before being applied.
Worked Example: Applying a Group-Aware Criterion
A hypothetical dataset contains 300 cells: 270 from a common group and 30 from a smaller group. A gene is detected with at least one count in 2 cells from the common group and 18 cells from the smaller group. Consider a global criterion of at least 20 detected cells, alongside a smaller-group criterion of at least 15 detected cells.
Across all cells, the gene has \(N_g(1)=2+18=20\), so it just meets the global criterion. Within the smaller group, \(N_{g,\mathrm{small}}(1)=18\), which also meets its group-specific criterion. Its pooled detection rate is \(20/300=0.0667\), while its rate in the smaller group is \(18/30=0.60\).
The group-specific result makes clear that most observed support comes from the smaller group. If the global threshold had instead been 25 cells, the gene would fail globally but still meet the stated group criterion. This is a reason to preserve it for later assessment, not a reason to claim that its expression is specific to that group or that it ranks highly in variability.
Justifying the Detection-Rate Caution
A short sampling calculation formalizes the risk of using a pooled cutoff without considering cell composition. Suppose each sampled cell has probability \(\pi\) of yielding a count at least \(c\), and detections are independent. Then each indicator \(I(Y_{ig}\geq c)\) is Bernoulli with probability \(\pi\), so \(N_g(c)\) has a binomial distribution with mean \(n\pi\) and variance \(n\pi(1-\pi)\). The detection-rate estimate has mean \(\pi\) and variance \(\pi(1-\pi)/n\).
If expression is restricted to a group occupying fraction \(r\) of the sample and the probability of meeting the count threshold in that group is \(s\), then \(\pi=rs\) under the simplifying assumptions above. The expected number detected is \(nrs\), which can be small even when \(s\) is high. For example, a group that makes up 2% of 500 cells contains about 10 cells; even perfect detection in that group cannot yield more than about 10 pooled detections. A global threshold requiring 20 detections would necessarily remove the gene.
The independence and equal-probability assumptions are approximations, not guarantees. Unequal library sizes, batch structure, related cells, and uncertainty in group membership can all affect the observed count. The derivation justifies inspecting subgroup support; it does not supply a universal threshold or turn filtering into a formal hypothesis test.
Diagnostics, Alternatives, and Limitations
Before finalizing a filter, inspect how many genes are removed under the proposed rule and whether removal is concentrated in particular samples, batches, or cell groups. Compare the number of detected cells with total counts: genes with high totals but very few detections may be driven by a small number of cells, while genes with modest totals and broad detection may carry more distributed support. Examine a few genes near the cutoff and determine whether the result changes sharply under nearby thresholds.
Filtering choices have different trade-offs:
| Approach | What it captures | Main caution |
|---|---|---|
| Minimum detected-cell count | How many cells meet a specified count threshold | A pooled cutoff can remove genes limited to a small group. |
| Minimum total count | Overall observed count support | A few cells with large counts can satisfy it. |
| Combined global and group-aware rules | Broad support or adequate support in a relevant group | Groups and thresholds must be chosen thoughtfully; tiny groups give unstable evidence. |
| No preliminary filter | Avoids removal by a chosen cutoff | Leaves many nearly unsupported genes for downstream ranking and computation. |
A filter based on normalized expression or depth-aware detection may reduce some dependence on library size, but it changes the meaning of “detected” and still requires explicit criteria. A raw-count threshold is easy to reproduce and interpret, but can favor deeper cells. Neither approach establishes biological absence. Filtering can also affect downstream conclusions if it is performed separately by condition in a way that removes different genes from different groups; the retained feature universe should be chosen with the intended comparison in mind.
Most importantly, do not use variability as the criterion for this preliminary screen. Ranking genes by a variability measure before excluding genes with negligible support is a different operation, and the later ranking methods address how to compare variation among genes with adequate expression. Conversely, an aggressive expression threshold should not be used as a substitute for variability ranking: a gene can be expressed broadly and nearly constantly, or be expressed in a subset and highly informative.
A Practical Filtering Sequence
State whether a cell is counted as detected at \(Y_{ig}\geq1\) or at a higher threshold, and specify the count matrix being screened.
Choose a minimum detected-cell count, total count, or combination appropriate to the dataset size and assay; record the values used.
For known or plausible small groups, inspect detection counts within each group and decide whether a group-aware retention condition is warranted.
Check whether conclusions depend on a small change in cutoff or on one deep library, sample, or group.
Treat retention as eligibility for later assessment, not as evidence that a gene is variable or biologically important.
Check Your Understanding
Answer the questions, then compare your reasoning with the solutions.
- In 400 cells, a gene has at least one count in 12 cells. Calculate its detection rate. What additional information would you want before deciding whether to remove it?
- Explain why a minimum total-count rule can retain a gene that has very limited support across cells.
- A cell group makes up 5% of 600 cells. A gene is detected in 24 cells, all in that group. Calculate its pooled and within-group detection rates and explain why a global cutoff of 30 detected cells would be risky.
- Under the independent-detection model, what is the expected number of detections among 200 cells if the probability of detection per cell is 0.04?
- State one difference between low-expression filtering and variability ranking.
Solutions. (1) The rate is \(12/400=0.03\), or 3%. Check whether the detections are concentrated in a relevant cell group, whether the cells have comparable sequencing depth, and what total counts the gene has. (2) A gene could have a large count in one or a few cells, producing a high total while remaining undetected in most cells. (3) The group contains \(0.05\times600=30\) cells. The pooled rate is \(24/600=0.04\), or 4%; the within-group rate is \(24/30=0.80\), or 80%. A cutoff of 30 pooled detections removes it despite strong observed support in the small group. (4) The expected count is \(n\pi=200\times0.04=8\) detections. (5) Filtering screens for adequate observed support; variability ranking compares eligible genes by how their expression differs across cells.