Why Filter Genes Before Testing?
RNA-seq differential-expression analysis can involve thousands of genes, but many have such low expression that their counts provide little information about a condition difference. Testing every gene includes these low-information hypotheses in the multiple-testing procedure. A filter based on expression can remove some of them before testing, leaving a smaller set of hypotheses and potentially increasing the power to detect changes among the genes that remain.
The key constraint is that the filter must not select genes because their observed test results look favorable. A useful filter is instead based on a statistic such as mean normalized count, chosen because it is related to the precision of a test. For valid inference, its relationship to the test statistic under the null hypothesis matters: under the null, the filter statistic should be independent of the test result, or satisfy a suitable conditional validity property.
For a retained null gene, the p-value should remain valid conditional on passing the filter: for every \(t\) between 0 and 1, \(P(p_i \leq t \mid F_i \geq c, H_{0i}) \leq t\). Exact tests often give equality; calibrated approximate tests may give the conservative inequality. This condition is what lets the multiple-testing procedure continue to treat retained null p-values as valid.
How Filtering Can Increase Detection Power
A low-abundance gene usually has noisier count measurements relative to its mean. In a count model, that often means a larger standard error for its estimated condition coefficient. For a fixed true effect, a larger standard error makes the test statistic less likely to exceed its rejection threshold. Filtering can therefore remove tests that are unlikely to yield discoveries, while reducing the number of hypotheses included in a multiple-testing adjustment.
For example, the Benjamini–Hochberg (BH) procedure for \(m\) p-values orders them as \(p_{(1)} \leq \cdots \leq p_{(m)}\), finds the largest \(k\) for which \(p_{(k)} \leq \alpha k/m\), and rejects the hypotheses corresponding to ranks 1 through \(k\). Under independence, or suitable positive dependence, this procedure controls the false discovery rate at the specified level when the p-values are valid. Filtering reduces \(m\) to the number retained, but it does not guarantee more rejections in every data set: the retained p-values and their ranks also matter.
Worked Example: Comparing Power at Two Expression Levels
Suppose a hypothetical gene has a true condition coefficient \(\beta = 0.585\) on the natural-log scale. Consider a two-sided Wald test at \(\alpha = 0.05\), with critical value approximately 1.96. Assume the estimated coefficient is approximately normal with mean \(\beta\) and standard error \(SE\). This normal approximation is the working model for this calculation; in an actual analysis, count-model fit and the adequacy of its standard error also need checking.
Under this model, the test rejects when the estimate divided by its standard error is greater than 1.96 or less than \(-1.96\). Equivalently, if the estimate has distribution \(N(\beta, SE^2)\), its power is:
For a lower-expression gene with \(SE=0.50\), the noncentrality ratio is \(0.585/0.50=1.17\). Substitution gives:
The power is about 0.216, or 21.6%. For a higher-expression gene with \(SE=0.20\), the ratio is \(0.585/0.20=2.925\), so:
The power is about 83.3% in the second case. The true effect is the same in both calculations; the difference comes from precision. This illustrates why an expression filter may remove tests with very low power. It does not establish that every higher-expression gene has a larger biological effect.
Choosing and Applying an Expression Filter
A common filter statistic is the mean normalized count across samples. Normalized counts account for differences in library size or other estimated sample scaling factors, so genes can be compared on a common scale. The filter should be defined independently of the observed condition effect—for example, by calculating the mean across all samples rather than selecting genes whose treatment counts exceed their control counts. Its threshold may be set in advance or chosen by a procedure with appropriate validity safeguards.
Worked Example: Applying a Mean-Count Filter
Suppose a hypothetical experiment has four samples with equal size factors, so the counts below are already on the same normalized scale. A prespecified rule retains a gene if its mean normalized count is at least 10.
| Gene | Counts across four samples | Mean normalized count | Decision |
|---|---|---|---|
| G1 | 1, 2, 3, 2 | \((1+2+3+2)/4=2\) | Filter out |
| G2 | 8, 12, 9, 11 | \((8+12+9+11)/4=10\) | Retain |
| G3 | 20, 17, 19, 24 | \((20+17+19+24)/4=20\) | Retain |
G1 falls below the threshold, while G2 and G3 meet or exceed it. This calculation uses the overall expression level, not the direction or size of a condition contrast. For this filter to support valid testing, its null behavior must also be appropriate for the test being used; a threshold and a calculation alone do not establish independence.
Here is the conditional argument underlying the method. Let \(S_i\) indicate that gene \(i\) passes a fixed filter, and suppose a null gene has a valid p-value independent of its filter statistic. Then:
Thus, retaining the gene has not made its null p-value anti-conservative. If filtering reduces the number of hypotheses from \(m\) to \(m_F\), where \(m_F<m\), the BH boundary at rank \(k\) changes from \(\alpha k/m\) to \(\alpha k/m_F\), which is larger. This creates the possibility of more rejections at a given rank. It does not prove that the final number of rejections must increase: that depends on the actual ordered p-values as well.
Worked Example: How the Testing Burden Changes
Consider eight hypothetical p-values. A filter based on expression, applied without looking at the p-values, retains four genes with p-values \(0.001, 0.020, 0.030,\) and \(0.150\). The four genes removed by the filter have p-values \(0.40, 0.50, 0.70,\) and \(0.90\). Use BH at \(\alpha=0.05\), and assume the procedure's p-value and dependence conditions are satisfied.
Without filtering, \(m=8\). The first two ordered p-values are 0.001 and 0.020. Their BH boundaries are \(0.05(1)/8=0.00625\) and \(0.05(2)/8=0.0125\). The first p-value passes its boundary, but the second does not. The later p-values also fail their corresponding boundaries, so the largest passing rank is \(k=1\); BH rejects one hypothesis.
After filtering, \(m_F=4\). The ordered retained p-values have boundaries \(0.0125, 0.0250, 0.0375,\) and \(0.0500\). The first three comparisons are \(0.001 \leq 0.0125\), \(0.020 \leq 0.0250\), and \(0.030 \leq 0.0375\); the fourth fails because \(0.150>0.0500\). The largest passing rank is \(k=3\), so BH rejects three hypotheses. The filter has increased the number of discoveries in this example, not because it selected small p-values, but because it removed four tests and the retained p-values satisfy the conditions for the procedure.
Assumptions, Diagnostics, and Failure Modes
The argument depends on more than having a plausible expression statistic. The test p-values must be calibrated under their null hypotheses, the filter must preserve that calibration among retained null genes, and the multiple-testing procedure must be used under conditions that support its error-rate guarantee. For BH, the standard conditions include independent p-values or a suitable positive dependence structure. In RNA-seq applications, a mean-count filter is commonly used because expression level is related to test precision and can be approximately independent of the test statistic under the null. That property should not be assumed for every design, test, or normalization scheme.
- Check that the filter does not encode the observed effect. A rule such as “retain genes with larger treatment than control counts” selects on the direction of the contrast and is not an independent expression filter.
- Check null calibration after filtering. In null simulations or appropriate diagnostic checks, retained null p-values should not accumulate excessively near zero. A filter that systematically leaves unusually small null p-values can invalidate the intended error control.
- Check the filter distribution and retained set. A threshold that removes almost no genes cannot materially reduce the testing burden; one that removes nearly all genes may discard genes that could have been detected. Report the rule and the number retained.
- Check model diagnostics separately. Filtering does not repair poor dispersion estimates, extreme observations, or a misspecified count model. Those problems can make the p-values invalid whether or not filtering is used.
A further subtlety arises when the threshold is chosen from the data. A fixed filter can be independent of each null p-value, but selecting a threshold by trying many possibilities and keeping the one that produces the most discoveries can couple the selection rule to the p-values. Independence of a filter statistic under the null does not automatically validate every data-driven threshold search. Use a threshold-selection procedure with appropriate theoretical or empirical safeguards; do not treat an unrestricted search for the most favorable result as harmless.
Comparison and Limitations
| Approach | What determines which genes are tested? | Main implication |
|---|---|---|
| No filtering | All genes with analyzable data | Simple and avoids filter selection, but low-information tests remain in the multiplicity burden. |
| Independent filtering | An expression or information statistic with suitable null independence | Can reduce the testing burden without selecting null genes for small p-values. |
| Effect- or p-value-based screening | Observed fold change, test statistic, or p-value | Uses evidence from the test itself to select hypotheses and can invalidate ordinary downstream error control. |
Independent filtering is not a substitute for testing, and it does not create information that the experiment did not collect. It may fail to help if nearly all genes are sufficiently expressed, if the expression statistic is weakly related to precision, or if the filter removes genuinely detectable genes. It can also be inappropriate when the filter is strongly associated with the null test statistic in the particular design. Always distinguish the filter rule from any later biological interpretation of estimated effects.
Common Mistakes
- “The filter is independent, so any threshold is safe.” A data-chosen threshold can depend on test results even when the underlying filter statistic is null-independent. Use a justified threshold-selection procedure.
- “Filtering guarantees more discoveries.” It can increase the BH boundary by reducing the number tested, but the retained p-values determine whether additional hypotheses pass.
- “Higher expression means a larger effect.” Higher expression often improves precision and power; it does not imply a larger biological condition effect.
- “A filter based on observed fold change is equivalent.” Selecting on the observed contrast uses the same data that drive the test and can distort inference. An expression filter should not be a disguised effect screen.
Check Your Understanding
Answer each question, then compare your reasoning with the solution immediately below it.
- A fixed filter retains genes with mean normalized count at least 8. For a null gene, the p-value is uniform and independent of the filter statistic. What is \(P(p_i\leq 0.05 \mid \text{gene retained}, H_{0i})\)?
Solution: Independence gives \(P(p_i\leq 0.05 \mid \text{retained}, H_{0i})=P(p_i\leq 0.05\mid H_{0i})=0.05\). Filtering has not increased the probability of a small null p-value. - At \(\alpha=0.05\) and rank \(k=2\), compare the BH boundary for \(m=1000\) tests with the boundary after filtering to \(m_F=500\).
Solution: Before filtering, the boundary is \(0.05(2)/1000=0.0001\). After filtering it is \(0.05(2)/500=0.0002\), twice as large. This change permits, but does not guarantee, a rejection at rank 2. - Why is retaining genes only when their observed treatment count exceeds their control count not the same as independent filtering?
Solution: That rule uses the direction of the observed condition contrast, which is part of the test evidence. It can select genes based on their observed effects and therefore need not preserve the null p-value distribution conditional on retention. - A researcher tries 20 expression thresholds and reports the one yielding the most discoveries, without a validity argument for this search. Is null independence of the expression statistic alone enough to justify the result?
Solution: No. Choosing among thresholds using the observed discoveries can couple the selected threshold to the p-values. Null independence for a fixed rule does not by itself justify unrestricted data-driven threshold selection.