Two Different Questions About False Discoveries
A genomic analysis may report a test statistic and a list of tests that pass a threshold. Those lead to related but distinct questions. For one test with statistic \(z\), what is the probability that its null hypothesis is true, given this value? For all tests beyond a threshold, what fraction of that selected tail is expected to come from null hypotheses? The first is a pointwise, posterior question; the second averages over a region of the statistic’s distribution.
The distinction matters because a value just beyond a threshold and a much more extreme value need not have the same posterior null probability. A tail-area quantity averages over both, as well as over every other value in the selected region. It does not, in general, give the posterior null probability for each individual result or the posterior expected null fraction in a particular observed list.
Here \(P_0\) denotes probability under the null distribution \(f_0\), while \(P\) denotes probability under the marginal mixture. A common two-sided selection region is \(A_t=\{z:|z|\ge t\}\); a one-sided region might be \(A_t=\{z:z\ge t\}\). The region must be stated because the tail-area quantity changes with the threshold and with the direction of selection.
Why the Tail Quantity Is an Average
The link between the two quantities follows directly from Bayes’ rule and integration. At each statistic value, local FDR is the posterior null probability. Averaging it over the marginal distribution restricted to the selection region gives the posterior null probability conditional on selection:
Thus tail-area FDR is a density-weighted average of local FDR over the selected region, not the local FDR at the boundary and not usually the local FDR at any one observed value. The weighting is by the marginal density \(f(z)\): regions where statistics occur more often contribute more to the average.
This identity is a marginal, single-test result and does not require independence between tests. It describes the probability that a test drawn from the mixture is null conditional on its statistic falling in \(A\). For a collection of tests, interpreting a tail quantity as the posterior expected null fraction given the full observed vector requires additional care. Dependence can make the other tests’ statistics informative about a test’s null status.
Worked Example: Local FDR at a Particular Statistic
Suppose, hypothetically, that \(\pi_0=0.90\), and under the alternative the statistic has the symmetric mixture \(\frac12N(2,1)+\frac12N(-2,1)\). Under the null, \(Z\sim N(0,1)\). Calculate the local FDR at \(z=2\). The model assumes these are the correct null and alternative densities and prior null proportion; the calculation treats them as known.
Write \(\phi(x)\) for the standard normal density. At \(z=2\), \(f_0(2)=\phi(2)=0.05399\). The alternative density is \(f_1(2)=\frac12\phi(0)+\frac12\phi(4)=\frac12(0.39894)+\frac12(0.0001338)=0.19954\).
The null contribution to the mixture density is \(\pi_0f_0(2)=0.90(0.05399)=0.04859\), and the alternative contribution is \(\pi_1f_1(2)=0.10(0.19954)=0.01995\). Therefore,
Under this model, a test whose statistic is \(2\) has posterior null probability about \(70.9\%\). This is a pointwise statement at \(z=2\), not a statement about all tests with \(|Z|\ge2\).
Calculating a Tail-Area FDR
For a fixed two-sided threshold \(t\), the tail-area calculation uses probabilities of entering the entire region \(|Z|\ge t\), rather than densities at one point. Under the hypothetical mixture above, the null tail probability at \(t=2\) is \(P_0(|Z|\ge2)=2\Phi(-2)\), approximately \(0.04550\). Under \(N(2,1)\), the probability of \(|Z|\ge2\) is \(0.5+\Phi(-4)\), approximately \(0.50003\). The symmetric alternative mixture has the same probability.
Worked Example: Tail-Area FDR Beyond an Absolute Threshold
Using the same hypothetical model, find the tail-area FDR for the selection region \(|Z|\ge2\). The assumed densities are normalized normal densities, the mixture weights sum to one, and the threshold is fixed before considering the statistic.
The marginal probability of selection is the weighted sum of the null and alternative tail probabilities: \(P(|Z|\ge2)=0.90(0.04550)+0.10(0.50003)=0.04095+0.05000=0.09095\). The joint probability of being null and selected is \(0.90(0.04550)=0.04095\). Hence,
In this model, among tests whose statistics fall in the two-sided tail \(|Z|\ge2\), the null fraction is about \(45.0\%\) in the mixture-probability sense. It is not the posterior null probability for a test known to have \(Z=2\), which was about \(70.9\%\). The tail average includes more extreme values as well as values near the cutoff.
Worked Example: A More Stringent Tail and a More Extreme Value
Keep the same model and raise the threshold to \(t=3\). The null probability is \(P_0(|Z|\ge3)=2\Phi(-3)\approx0.002700\). For a component \(N(2,1)\), the tail probability is \(P(Z\ge3)+P(Z\le-3)=\Phi(-1)+\Phi(-5)\approx0.15866\); the reflected component gives the same value.
Thus the null contribution is \(0.90(0.002700)=0.002430\), and the alternative contribution is \(0.10(0.15866)=0.01587\). The tail-area FDR is
For comparison, at the particular value \(z=3\), \(f_0(3)=\phi(3)=0.004432\) and \(f_1(3)=\frac12\phi(1)+\frac12\phi(5)\approx0.12099\). Therefore, \(\operatorname{lfdr}(3)=0.90(0.004432)/[0.90(0.004432)+0.10(0.12099)]\approx0.2479\). The local value at the boundary is about \(24.8\%\), whereas the average over the full tail is about \(13.3\%\). In this model, more extreme values have lower local null probability and pull down the tail average.
From a Model Tail to an Observed List
A tail-area probability is not automatically the posterior expected false fraction in a particular list after every statistic has been observed. This distinction is especially important in genomic settings, where tests can be dependent. A single-test mixture specifies the marginal distribution of each test’s statistic and status; by itself, it does not specify how statuses and statistics across tests are jointly distributed.
Under an explicit independence assumption, a list-specific posterior calculation is possible. Suppose the pairs \((H_{0i},Z_i)\) are independent across tests and each pair follows the stated two-group model. Then conditioning on the full statistic vector leaves each test’s posterior null probability equal to its local FDR at its own statistic. If a fixed threshold selects \(R>0\) tests with observed values \(z_1,\ldots,z_R\), linearity of conditional expectation gives
Here \(V\) is the number of null tests among the selected tests. The independence assumption matters: without it, the relevant quantities are \(P(H_{0i}\mid Z_1,\ldots,Z_m)\), which need not equal \(\operatorname{lfdr}(z_i)\). The average of marginal local FDR values is then not justified by linearity alone.
Worked Example: Posterior Risk for a Small Observed List
Suppose three independent tests follow the hypothetical model used above, and the observed statistics are \(1,2,3\). Use the fixed selection rule \(|Z|\ge1\), so all three are selected. The independent-pairs assumption ensures that conditioning on all three statistics does not change each test’s posterior null probability beyond its own local FDR.
At \(z=1\), \(f_0(1)=0.24197\) and \(f_1(1)=\frac12\phi(1)+\frac12\phi(3)=0.12320\), giving \(\operatorname{lfdr}(1)=0.90(0.24197)/[0.90(0.24197)+0.10(0.12320)]\approx0.9465\). The preceding calculations give \(\operatorname{lfdr}(2)\approx0.7089\) and \(\operatorname{lfdr}(3)\approx0.2479\). Therefore the posterior expected fraction of null tests in this observed three-test list is
This \(63.4\%\) is conditional on these particular observed values under the stated independent model. It differs from the tail-area FDR for \(|Z|\ge1\), which averages over all possible values in that tail. If tests were dependent, this list-specific calculation would require the joint posterior probabilities given the full vector; the marginal local FDRs alone would not establish it.
How the Quantities Differ in Practice
| Quantity | Conditioning information | Interpretation |
|---|---|---|
| Local FDR | A particular statistic value \(Z=z\) | Posterior null probability at that value under the mixture model |
| Tail-area FDR | Membership in a fixed region such as \(|Z|\ge t\) | Null probability averaged over the selected tail |
| List-specific posterior null fraction | The full observed statistic vector and selection rule | Expected null fraction in this realized list, requiring the appropriate joint model |
| Procedure-level FDR | The repeated-sampling behavior of a testing procedure | \(E[V/(R\vee1)]\), an expectation over repeated data sets |
The last row uses the conventional procedure-level false discovery rate, with \(R\) the number of rejections and \(R\vee1=\max(R,1)\). A tail-area posterior probability and this procedure-level expectation are not definitions of the same quantity. For example, with independent, identically distributed tests and a fixed threshold, each selected test has null probability \(q=P(H_0\mid Z\in A)\). Conditional on a nonempty set of selection indicators, the expected null fraction is \(q\); with FDP defined as zero when no tests are selected, the procedure-level FDR is \(qP(R>0)\). Different procedures, dependence structures, or heterogeneous test distributions can further separate these summaries.
Assumptions, Diagnostics, and Common Errors
Both local and tail-area calculations rely on a suitable two-group model: the null and alternative densities must describe the relevant test statistics, the mixture weight must represent the null proportion for the population being analyzed, and the selection region must match the actual rule. If the model is misspecified, the Bayes calculations can be arithmetically correct while their interpretation is misleading. For example, a null distribution with the wrong center or scale changes both the pointwise density ratio and the null tail probability.
Useful checks include comparing the modeled and observed distribution of test statistics, examining whether conclusions shift sharply under modest threshold changes, and checking whether a reported tail region matches the actual one- or two-sided selection rule. In dependent genomic data, linkage or other structure can affect joint selection and list-specific posterior calculations. Dependence does not invalidate the marginal identity \(P(H_0\mid Z\in A)=\pi_0P_0(A)/P(A)\) for a correctly specified marginal mixture, but it does prevent treating that identity alone as a joint posterior for an observed discovery list.
- Confusing point and tail conditioning. The local FDR at the cutoff is not generally the tail-area FDR. The latter integrates local FDR over the entire selected region.
- Calling a tail average the risk for every discovery. A selected test’s statistic may be much more extreme than the threshold. Its pointwise posterior risk can differ from the average across the tail.
- Using marginal probabilities to claim a list-specific posterior. Linearity of expectation applies to posterior probabilities conditional on the information at hand. If the full statistic vector is conditioned on, use \(P(H_{0i}\mid Z_1,\ldots,Z_m)\), unless a joint-model assumption justifies replacing it with \(\operatorname{lfdr}(z_i)\).
- Equating tail-area FDR with procedure-level FDR. One conditions on a test entering a region; the other averages a false discovery proportion over repeated data sets. State which quantity is being reported.
The practical choice follows the question. Use local FDR to describe the posterior null risk attached to a particular statistic under a specified mixture. Use tail-area FDR to summarize the null fraction among statistics in a defined tail under that mixture. Use a procedure-level error rate when the question concerns the repeated-sampling performance of a multiple-testing rule. These summaries can complement one another, but none should be substituted for another without stating the conditioning and assumptions.
Check Your Understanding
Answer each question, then compare with the solutions below.
- In one sentence, distinguish local FDR at \(z\) from tail-area FDR for a region \(A\).
- For a mixture with \(\pi_0=0.8\), \(P_0(Z\in A)=0.05\), and \(P_1(Z\in A)=0.40\), calculate \(P(H_0\mid Z\in A)\).
- Why does the marginal two-group model alone not guarantee that the average local FDR in an observed list is its posterior expected null fraction given the full statistic vector?
- Is the local FDR at the threshold necessarily equal to the tail-area FDR? Explain.
- Under independent, identically distributed tests and a fixed selection threshold, let \(q=P(H_0\mid Z\in A)\). If the procedure-level FDP is set to zero when no tests are selected, express the procedure-level FDR in terms of \(q\) and \(R\).
Solutions. (1) Local FDR conditions on one statistic value, whereas tail-area FDR conditions on membership in a region and averages over values in that region. (2) The null-selected probability is \(0.8(0.05)=0.04\); the total selection probability is \(0.8(0.05)+0.2(0.40)=0.12\), so the result is \(0.04/0.12=0.3333\). (3) Under dependence, other statistics can change the posterior null probability for a given test; marginal densities do not specify that joint posterior. (4) No. Tail-area FDR is a density-weighted average throughout the selected region, while local FDR is evaluated at one point. (5) The FDR is \(qP(R>0)\), because conditional on a nonempty set of selected tests the expected null fraction is \(q\), and the FDP is zero when \(R=0\).