Local FDR From P-Values

Why a P-Value Has a Posterior Null Probability

A p-value is evidence against a particular null hypothesis, but it is not itself the probability that the null hypothesis is true. In a large genomic analysis, a useful question is instead: among tests with a p-value near \(p\), what proportion are null? The two-group local false discovery rate answers this question by combining a model for the fraction of null tests with the p-value densities expected under the null and the alternative.

For an exact continuous test under a correctly specified null, the p-value is uniformly distributed on \([0,1]\). Small p-values are therefore uncommon among null tests. If alternative tests tend to produce small p-values, their density is concentrated near zero, and an observed small p-value can have a low posterior probability of coming from the null. The calculation is a Bayes rule calculation; its interpretation depends on the mixture model being appropriate.

For exact continuous null p-values, \(f_0(p)=1\) for \(0\leq p\leq1\), so the local FDR becomes

Here \(f_1(p)\) is a density, not the probability that an alternative test has a p-value exactly equal to \(p\). For a continuous variable, that probability is zero; the density describes relative concentration near \(p\). A density value can exceed 1 without being a probability, provided the density is nonnegative and integrates to 1.

Deriving the Posterior Formula

The formula follows by comparing the joint density of a null test and its p-value with the marginal p-value density. For a small interval from \(p\) to \(p+\Delta\), the probability of a null test whose p-value falls in that interval is approximately \(\pi_0 f_0(p)\Delta\). The probability of any test in the same interval is approximately \(f(p)\Delta\). Their ratio tends, as the interval narrows, to the conditional probability of a null test at p-value \(p\).

The cancellation of \(\Delta\) gives the density version of Bayes’ rule. This derivation requires a valid two-group model with \(0\leq\pi_0\leq1\), nonnegative densities integrating to 1, and a positive marginal density at the p-value being considered. The uniform-null simplification additionally requires exact continuous null p-values from a correctly specified null distribution. If null p-values are conservative, discrete, or affected by a misspecified null, their density need not be 1 and that simplification is not justified.

This is a posterior probability conditional on the observed p-value and the stated model. It is not a frequentist probability that a fixed null hypothesis is true, and it does not by itself establish that a biological effect is absent. The posterior interpretation also depends on treating the null proportion and densities as known; when they are estimated from many tests, the result is an estimated local FDR.

Worked Calculations Across the P-Value Range

Local FDR and Tail-Area Error Rates

The local FDR is a posterior probability at a particular p-value. A tail-area false discovery rate concerns a collection of tests selected by a rule such as \(P\leq t\). Under the same continuous two-group model with a uniform null, the expected fraction of null tests in that tail, expressed as a population conditional probability, is

The numerator is the probability of a null test with p-value at most \(t\), since the null cumulative probability is \(t\). The denominator is the marginal probability of being in that tail. In contrast, \(\operatorname{lfdr}(p)\) compares densities at a particular value \(p\). A tail probability averages evidence over an interval; it is not obtained simply by substituting \(t\) into a local FDR formula. The two summaries answer different questions and can differ substantially.

A useful way to read the local formula is through posterior odds. For a uniform null and positive \(f_1(p)\), the posterior odds of null to alternative are \(\operatorname{lfdr}(p)/[1-\operatorname{lfdr}(p)]=\pi_0/[(1-\pi_0)f_1(p)]\). Thus, at a point where the alternative density is high relative to the null density, the posterior null probability falls. When the alternative density is low, it rises. A nonmonotone alternative density can produce a nonmonotone local FDR; no general monotonicity claim follows from the two-group formula alone.

Diagnostics, Assumptions, and Limitations

Before interpreting local FDR values, check the ingredients of the calculation. The null p-value distribution should be uniform when the continuous null is exact; systematic departures can indicate calibration problems or a model that does not represent the null tests. The estimated alternative density should be nonnegative, integrate to 1, and be estimated stably over the p-value range where local FDRs are reported. A density estimate that is negative, poorly normalized, or erratic at the boundaries cannot be inserted uncritically into Bayes’ formula.

The mixture interpretation also requires that the tests being pooled can reasonably be described by a common null proportion and component densities. Tests with very different calibration or alternative behavior may need a more appropriate stratified or otherwise structured model. Dependence among tests does not, by itself, change the one-test Bayes identity when the marginal mixture model is correct. However, dependence can matter for estimating the mixture from a collection of tests and for assessing how stable the fitted quantities are.

Boundary behavior deserves attention. Continuous p-values equal exactly to 0 with probability zero under the model, so an observed zero often reflects rounding or finite numerical precision. If an alternative density becomes unbounded as \(p\) approaches zero, the local FDR may have a well-defined limit there even though the density at zero is not finite. Do not treat a rounded zero as an ordinary interior p-value without considering how it was produced. Conversely, when the alternative density is very small, the local FDR can be close to 1; that is not a calculation failure if the fitted model supports it.

A common mistake is to say, “The p-value is 0.01, so there is a 1% probability the null is true,” or to equate a small p-value directly with a small local FDR. A p-value is a tail probability calculated under a null model, whereas the local FDR is a posterior probability that also depends on \(\pi_0\) and the alternative density. Another mistake is to use a tail-area error rate as though it were the local FDR at its threshold. The correction is to state which conditional probability is being calculated, use the matching density or cumulative probability, and report the model assumptions that support its interpretation.