The Benjamini-Hochberg Step-Up Procedure in Genomic Data

Why Genomic Analyses Need a Step-Up Rule

A genomic analysis may test thousands of hypotheses, such as whether each gene is associated with a measured phenotype. Even when every individual test is conducted correctly, examining many p-values creates a multiple-testing problem. The Benjamini-Hochberg (BH) procedure is a rule for selecting discoveries while targeting a specified false discovery rate, rather than controlling the probability of making any false rejection.

The false discovery proportion is the number of false rejections divided by the total number of rejections, with the proportion defined as zero when there are no rejections. The false discovery rate is the expected value of that proportion. The BH algorithm takes a family of p-values and a target level \(q\), then determines which hypotheses to reject. It is important to distinguish this algorithmic decision rule from its guarantee: the usual BH FDR guarantee holds when the null p-values are valid and the tests are independent, and it also holds under certain positive dependence conditions. The procedure can be computed without independence, but its standard guarantee does not automatically extend to arbitrary dependence.

The family must be specified before interpreting the results. Here \(m\) is the number of hypotheses in that family, and all \(m\) p-values—not only the small ones or the subset that looks interesting—are used to set the BH thresholds. A target \(q\) is chosen, with \(0<q<1\). It is the nominal FDR level for the procedure, not a claim that each selected result has probability \(q\) of being false.

The BH Step-Up Algorithm

The thresholds increase with rank: \(t_1=q/m\), \(t_2=2q/m\), and \(t_m=q\). The step-up feature is the crucial part of the rule. After identifying the largest passing rank, the procedure rejects every hypothesis ranked up to that point. It does not require every one of those p-values to pass its own rank-specific comparison.

When p-values are sorted for calculation, their original hypothesis labels should be retained. The ordered list identifies the cutoff, but a final scientific report needs to name the genes, variants, or other hypotheses associated with the rejected p-values. Sorting does not change which p-value belongs to which hypothesis.

Why the Largest Passing Rank Determines the Set

The rank-specific comparisons are used to locate the cutoff. Once \(k\) is found, the rejection set is defined by rank, not by repeating a separate acceptance rule for every smaller rank. This distinction matters because \(t_i\) increases with \(i\): a p-value can be too large for its own early threshold but still be included among the first \(k\) when a later rank passes.

The following short justification makes the selection logic explicit. Since the p-values are ordered, \(p_{(i)}\le p_{(k)}\) for every \(i\le k\). Since \(k\) passes, \(p_{(k)}\le kq/m\). It follows that every p-value among the first \(k\) is at most \(kq/m\). This does not show that \(p_{(i)}\le iq/m\) for every \(i\le k\), because \(iq/m\) can be smaller than \(kq/m\). The procedure's definition is to reject the entire prefix of ranks up to the largest passing rank.

Mapping the Decisions Back to Genomic Results

In a genomic application, the family might consist of one test per gene for a specified phenotype. For the algorithm to apply as stated, each test must produce a p-value for its corresponding null hypothesis, and the same family size \(m\) must be used throughout the threshold calculation. The example below illustrates how ordering and the final rejection set work when the input is recorded by gene rather than by rank.

Implementation Checks, Alternatives, and Limits

A reliable implementation keeps the hypothesis family, p-values, ranks, thresholds, and labels aligned. Useful checks include confirming that the ordered p-values are nondecreasing, that the thresholds increase from \(q/m\) to \(q\), and that the reported rejection set contains exactly the first \(k\) ordered hypotheses. If no comparison passes, the result is an empty rejection set; do not force a cutoff by choosing the smallest p-value.

A frequent mistake is to compare every p-value only with its own threshold and reject exactly those that pass. That is the wrong reasoning because BH is a step-up rule: first find the largest passing rank, then reject the entire prefix through that rank. The second worked example shows why these answers can differ.

BH is not interchangeable with familywise error procedures such as Bonferroni or Holm. Those procedures target familywise error—the probability of at least one false rejection—whereas BH targets the expected false discovery proportion. A familywise procedure can therefore be more conservative when many discoveries are sought. Conversely, applying BH does not provide the same protection against any false rejection that familywise control provides.

The standard BH FDR guarantee requires appropriate conditions on the p-values, including independence for the basic guarantee. Genomic tests can be dependent because genes may be co-regulated, share biological pathways, or be affected by linkage. The step-up calculations remain well-defined in that setting, but the nominal target alone does not establish that the FDR is controlled under every dependence structure. The procedure also depends on defining the family appropriately: omitting tests after seeing their p-values changes the analysis and can undermine its intended error-rate interpretation.