Tied And Discrete P-Values In BH

Why Ties and Discreteness Matter

Genomic analyses often produce p-values with repeated values. Some ties arise because different tests happen to give the same result; others are common when a test has a discrete outcome, such as a small-sample count test or a permutation test with a limited number of possible outcomes. The Benjamini–Hochberg (BH) procedure still uses the same rank-based rule in either case. But ties raise a practical question about ranks, while discreteness can leave gaps between attainable p-values and the thresholds BH uses.

The distinction matters: ties do not require a special version of BH, and a valid discrete p-value remains valid. However, a sparse set of attainable p-values can make it harder for observed values to fall below BH thresholds. In that sense, the procedure can be more conservative than its nominal target.

BH With Ordered and Tied P-Values

Suppose a family contains \(m\) hypotheses, with p-values \(p_1,\ldots,p_m\). For a target false discovery rate level \(q\), where \(0<q<1\), arrange the p-values in nondecreasing order: \(p_{(1)}\le \cdots \le p_{(m)}\). The BH threshold at rank \(i\) is \(qi/m\). The procedure finds the largest rank \(k\) for which \(p_{(k)}\le qk/m\), then rejects the hypotheses corresponding to the first \(k\) ordered p-values. If no rank meets its threshold, it makes no rejections.

For independent, valid p-values, BH controls the false discovery rate at no more than \((m_0/m)q\), where \(m_0\) is the number of true null hypotheses. In particular, this bound is at most \(q\). This is the relevant guarantee when interpreting the calculations below. The step-up rule itself can be applied to an ordered list whether or not its p-values are tied or discrete.

A tied value occupies consecutive ranks. The rank labels within that block can be assigned in any order because the tied hypotheses have identical p-values. More importantly, the BH rule cannot end by rejecting only some members of a tied block.

Discrete P-Values and Conservatism

For a continuous test statistic under a true null, a suitably calibrated p-value is often uniformly distributed on the interval from 0 to 1. A valid discrete p-value need not be uniform. Its attainable values may skip intervals, and its cumulative probabilities can be strictly below the super-uniform bound \(P(P_i\le t)\le t\). That inequality protects validity, but it can also mean fewer small p-values than a continuous uniform calibration would produce.

BH compares observed values with thresholds on the grid \(q/m,2q/m,\ldots,q\). If the attainable p-values are sparse, a threshold may fall below the smallest attainable value, or between two attainable values with the next one above the threshold. In that case, no observed p-value at that rank can pass. Across repeated testing, such gaps can make the achieved FDR lower than the bound \(q\); the same feature can also reduce the chance of making discoveries. This is a possible effect of discreteness, not a guarantee that every discrete-data analysis will have fewer rejections than every continuous-data analysis.

The discrete setting does not change the BH threshold formula. Nor does it justify treating an attainable p-value as though it were smaller than its actual value. The key check is validity under the null: exact discrete p-values are commonly super-uniform, whereas a modified value that is not super-uniform may not retain the usual guarantee.

Adjusted P-Values When Values Are Tied

The BH-adjusted p-value for the hypothesis at ordered rank \(i\) is the smallest target level at which that hypothesis would be rejected by BH, accounting for the step-up rule. It can be calculated by taking the minimum of \(mp_{(j)}/j\) over ranks \(j\ge i\), and capping the result at 1. The adjusted values are then nondecreasing in rank. When a p-value is tied, every hypothesis in its tied block gets the same adjusted p-value.

Diagnostics, Alternatives, and Common Pitfalls

Before interpreting BH results from a discrete testing procedure, check what p-values the test can actually produce. A short list of attainable values, or a histogram showing spikes and gaps, can make the discreteness visible. Confirm that the reported values are valid p-values under the null and retain enough numerical precision to order them correctly. If values are rounded for display, use the unrounded values for ranking and threshold comparisons whenever they are available. Rounding can create artificial ties or change which side of a threshold a value falls on.

A frequent mistake is to break a tie arbitrarily and then report that only some hypotheses with the same p-value were rejected. The correction is to apply the step-up rule to the full ordered list: if the largest qualifying rank falls within a tied block, the later ranks in that block qualify too, so the whole block is included. Another mistake is to conclude that a discrete p-value is invalid merely because it is not uniformly distributed. Validity requires super-uniformity, not an exactly uniform distribution.

Mid-p values, which reduce an exact tail probability by assigning part of the probability at the observed outcome, can be less conservative than ordinary exact p-values. But a mid-p value is not guaranteed to satisfy \(P(P_i\le t)\le t\) for every \(t\). Replacing exact p-values with mid-p values may therefore remove the validity condition behind the usual BH guarantee. Do not assume that a less conservative-looking value has the same error-control properties.

Holm and Bonferroni procedures are alternatives when the goal is familywise error rate control rather than false discovery rate control. They use different thresholds and answer a different error-rate question; they do not make a discrete p-value continuous or eliminate the need to check its validity. For BH, the practical sequence is to verify p-value validity, sort the unrounded values, compare each rank with its own threshold, and report the full rejected prefix, including every tied value it contains.