Negative Dependence And Limits Of BH Guarantees

Why Dependence Can Change an FDR Guarantee

In genomic analyses, many tests may be based on overlapping samples, correlated genes, or shared biological pathways. The resulting p-values can therefore be dependent. Dependence alone does not mean the Benjamini–Hochberg (BH) procedure must fail: BH has a standard false discovery rate guarantee under independence and under a specified positive dependence condition. But marginally valid p-values are not enough to extend that guarantee to every possible dependence structure. A dependence pattern can coordinate small p-values in a way that increases the chance of making discoveries when all tested null hypotheses are true.

The distinction is important. A p-value can be correctly calibrated for its own null hypothesis while its relationship with the other p-values still affects a multiple-testing procedure. We will use a two-test construction to calculate this effect directly, then compare it with the independent case.

For BH at target level \(\alpha\), order the p-values as \(P_{(1)}\le\cdots\le P_{(m)}\). Find the largest \(k\) for which \(P_{(k)}\le k\alpha/m\), and reject the hypotheses corresponding to \(P_{(1)},\ldots,P_{(k)}\). If no such \(k\) exists, make no rejections. This rule uses all the p-values, so their joint behavior can matter even when each marginal p-value is valid.

Which Dependence Conditions Support BH?

Under mutual independence of all the p-values, if every true-null p-value is marginally valid, BH controls the FDR at no more than \((m_0/m)\alpha\). The condition concerns the joint collection of p-values: independence among true-null p-values alone is not a sufficient general statement. In particular, a true-null p-value may be dependent on nonnull p-values in a way that affects the number of rejections.

A second standard guarantee applies under positive regression dependence on a subset (PRDS): the p-values corresponding to true null hypotheses must satisfy the required PRDS condition, and the true-null p-values must be valid. Informally, PRDS rules out certain forms of dependence in which making one true-null p-value larger makes an increasing event in the full p-value vector less likely. More precisely, for each true-null index \(i\) and each increasing set \(D\) of p-value vectors, \(P(\mathbf P\in D\mid P_i\le t)\) is nondecreasing in \(t\), wherever the conditional probability is defined. “Increasing” means that if a vector belongs to \(D\), then any vector with every coordinate at least as large also belongs to \(D\). Under the appropriate PRDS condition, BH has the same \((m_0/m)\alpha\) upper bound.

Neither guarantee says that any dependence is harmless. Arbitrary dependence does have a standard alternative: the Benjamini–Yekutieli (BY) procedure uses a more stringent threshold, with \(\alpha\) divided by \(H_m=\sum_{j=1}^m 1/j\), and controls FDR under arbitrary dependence when the true-null p-values are valid. This can cost power. The important point here is that the ordinary BH threshold is not automatically entitled to the independence or PRDS guarantee when neither condition is established.

A Two-Test Construction Where BH Exceeds Its Target

The previous example is not a counterexample to BH control. The next construction is. It keeps both p-values marginally uniform, but pairs their values so that BH’s rejection regions occur more often than under independence. All hypotheses are true, so the FDR is exactly the probability of at least one rejection.

Comparison With the Independent Case

The same two-test setting provides a useful contrast. Suppose \(P_1\) and \(P_2\) are independent uniform p-values, both null hypotheses are true, and BH is again used at \(\alpha=0.10\). BH rejects if the minimum p-value is at most \(0.05\), or, if that condition fails, if both p-values are at most \(0.10\). These disjoint events have probabilities \(1-(0.95)^2=0.0975\) and \((0.05)^2=0.0025\), respectively. Therefore:

The independent and dependent examples have the same number of tests, the same marginal distributions, and the same BH thresholds. Their FDRs differ because the joint distribution differs. Under independence, the calculation meets the nominal target; in the constructed dependent case, it exceeds it. Dependence does not imply that BH will always exceed its target, as the perfectly opposed example also demonstrated. Rather, without the needed condition, the standard upper bound is not guaranteed.

Why the Independence Bound Holds

A short justification makes clear where independence enters. Write \(I_i\) for the indicator that true null \(i\) is rejected. Then the false discovery proportion is \(\sum_{i\in I_0} I_i/\max(R,1)\), where \(I_0\) is the set of true-null indices. For a given \(i\), replace \(P_i\) by zero and let \(R_i\) be the number of BH rejections in that modified vector. The step-up rule implies that if \(i\) is rejected in the original vector, then the rejection count is \(R_i\) and \(P_i\le \alpha R_i/m\). Conversely, if \(P_i\le \alpha R_i/m\), the step-up rule rejects \(i\), with rejection count \(R_i\). Thus the contribution from null \(i\) can be written using the event \(P_i\le \alpha R_i/m\).

Under mutual independence, \(P_i\) is independent of the other p-values, and hence of \(R_i\). Conditioning on those other p-values fixes \(R_i\). Marginal validity then bounds the conditional chance that \(P_i\le\alpha R_i/m\) by \(\alpha R_i/m\). Dividing by \(R_i\) bounds the expected contribution of this null by \(\alpha/m\):

when \(R_i>0\); when \(R_i=0\), the contribution is zero. Summing over the \(m_0\) true nulls gives \(FDR\le(m_0/m)\alpha\). This argument relies on independence of each true-null p-value from the other p-values, not merely independence among the true-null p-values themselves. The PRDS guarantee is a separate result under its stated positive-dependence condition; it does not follow from this independence argument.

Practical Interpretation and Common Pitfalls

In a genomic analysis, it is often possible to describe plausible sources of dependence, such as linkage disequilibrium or shared measurements, without being able to establish a formal condition like PRDS. A sample correlation matrix or a visual pattern of p-values does not by itself prove that PRDS holds. Nor does failure to observe a particular pattern prove that it does not hold. The design, test construction, and relevant dependence assumptions need to be considered together.

If arbitrary dependence is a serious concern, BY provides a formal FDR guarantee under arbitrary dependence for valid true-null p-values, at the cost of more stringent thresholds. Other approaches may be appropriate when their assumptions are justified, but they do not turn marginal validity alone into an ordinary BH guarantee.

• Mistake: “Every null p-value is uniform, so BH controls FDR at \(\alpha\).” Correction: Marginal validity is necessary for the stated guarantees, but BH’s standard bound also requires an appropriate joint-dependence condition.
• Mistake: “The true-null p-values are independent of one another, so the independence result applies.” Correction: That statement does not ensure independence from nonnull p-values. A sufficient condition is mutual independence of all p-values with valid true-null p-values, or independence of each true-null p-value from the vector of all the others.
• Mistake: “Negative dependence always makes BH exceed its target.” Correction: It need not; the perfectly opposed two-p-value example had FDR equal to its target. The point is that arbitrary dependence does not guarantee the bound.