Why Rank by Posterior Null Risk?
A genomic analysis may produce thousands of test statistics, while a follow-up experiment can investigate only a fixed number of candidate genes. The practical question is then not merely which tests look most unusual, but which set of a specified size has the smallest expected number of false discoveries. Local false discovery rates provide a direct answer: rank tests from smallest to largest posterior probability of being null, then select the first \(k\).
The reason is a decision-theoretic one. Each selected test contributes its posterior null probability to the expected false-selection count. Consequently, for a fixed selection size, replacing a selected test with a less risky unselected one cannot increase that expectation. The result is conditional on the observed data and the posterior model; it does not require the null indicators to be independent.
Posterior Risk and the Fixed-Size Objective
Here, \(\ell_i\) is a probability between 0 and 1, conditional on the observed data and the model used to obtain the posterior. In an empirical Bayes analysis, it may be calculated using estimated mixture parameters. The ranking result below is exact when the \(\ell_i\) are the model's posterior probabilities; using fitted estimates gives the corresponding result for the fitted model, without automatically accounting for uncertainty in the fitted parameters.
Suppose exactly \(k\) hypotheses must be selected, where \(k\) is fixed after observing \(D\). The target is to minimize the posterior expected number of false selections, \(E\{V(S)\mid D\}\), among all sets \(S\) containing \(k\) tests. If \(k>0\), the posterior expected false discovery proportion for that set is \(E\{V(S)/k\mid D\}\). Since \(k\) is constant, minimizing either quantity gives the same ranking.
Why the Smallest Local FDRs Are Optimal
Let \(Z_i=\mathbf{1}\{H_i\text{ is null}\}\), so \(V(S)=\sum_{i\in S}Z_i\). Conditional expectation is linear, whether or not the \(Z_i\) are dependent:
Thus the fixed-size problem is to choose \(k\) numbers from the list of local false discovery rates so that their sum is as small as possible. The next argument proves that the \(k\) smallest do this.
Write the values in nondecreasing order, \(\ell_{(1)}\leq\ell_{(2)}\leq\cdots\leq\ell_{(m)}\). The proposed set contains tests with risks \(\ell_{(1)},\ldots,\ell_{(k)}\).
Any other set of \(k\) tests must omit at least one of those first \(k\) positions and include the same number of tests from positions \(k+1,\ldots,m\). Each included risk from beyond position \(k\) is at least as large as each omitted risk among the first \(k\).
Pair each omitted risk among the first \(k\) with one added risk beyond position \(k\). For every pair, the change in the sum is \(\ell_{\text{added}}-\ell_{\text{omitted}}\geq0\). Adding these inequalities shows that the competing set's total risk is no smaller.
This exchange argument also covers ties: when an added and an omitted risk are equal, that replacement changes the expected count by zero. The theorem concerns the sum of marginal posterior risks. It neither assumes that false discoveries occur independently nor guarantees that one chosen set has the fewest false discoveries for every possible realization of the unknown null indicators.
Worked Example: Selecting Three Genomic Tests
Suppose a hypothetical analysis has fitted a two-group model and, conditional on its observed data, assigns six genes the following local false discovery rates. Assume these values are the posterior null probabilities used for this decision.
| Gene | Local false discovery rate |
|---|---|
| A | 0.04 |
| B | 0.11 |
| C | 0.18 |
| D | 0.31 |
| E | 0.52 |
| F | 0.74 |
The follow-up budget permits exactly three genes, so select A, B, and C. The posterior expected number of false selections is
For this set, whose size is three, the posterior expected false discovery proportion is \(0.33/3=0.11\). This is an expectation under the fitted posterior model; it is not a claim that exactly 0.33 genes will be false in the realized set.
For comparison, selecting A, D, and E instead gives \(0.04+0.31+0.52=0.87\) expected false selections. Both sets contain three genes, but the second set has a higher posterior expected false count. The ranking theorem guarantees that no three-gene set has a smaller sum than A, B, and C.
Worked Exchanges and Ties
Worked Example: Checking a Proposed Replacement
In a separate hypothetical analysis, suppose the local false discovery rates for five tests, in increasing order, are \(0.08,\ 0.12,\ 0.19,\ 0.25,\ 0.41\). Exactly three tests must be selected.
The smallest-risk set has expected false count
Consider replacing the test with risk \(0.19\) by the test with risk \(0.25\). The new sum is \(0.08+0.12+0.25=0.45\), an increase of \(0.25-0.19=0.06\). Replacing both \(0.12\) and \(0.19\) with \(0.25\) and \(0.41\) increases the sum by \((0.25-0.12)+(0.41-0.19)=0.35\), giving \(0.74\). Each replacement of a selected risk by a larger unselected risk raises or leaves unchanged the expected false count.
The arithmetic illustrates the exchange proof rather than relying on a particular shape for the data distribution. The posterior risks themselves are the inputs to the fixed-size decision.
Worked Example: Tied Risks and Dependence
Suppose three hypothetical tests have local false discovery rates \(0.06,\ 0.10,\ 0.10\), and exactly two must be selected. Selecting the first test and either one of the tied tests gives
The two available tied choices have identical expected false counts. No ranking based only on these posterior risks can prefer one over the other.
Now consider two selected tests, each with posterior null probability \(0.20\). Their expected number of false selections is \(0.20+0.20=0.40\), regardless of their dependence. For example, if their null indicators are perfectly correlated, the number of false selections is 0 with probability 0.80 and 2 with probability 0.20, giving mean \(0(0.80)+2(0.20)=0.40\). If instead at most one is null, with probability 0.20 for each test to be the sole null and probability 0.60 that neither is null, the mean is \(1(0.20)+1(0.20)+0(0.60)=0.40\). Dependence changes the distribution of the false count, but not its expectation when the marginal posterior risks are held fixed.
What the Ranking Does—and Does Not—Optimize
The theorem answers a specific decision question: among sets of the same fixed size, which has the smallest posterior expected number of false selections? It does not establish that the chosen set minimizes the probability of at least one false selection, the variance of the false count, or the chance of exceeding a particular false-count limit. Those criteria depend on more than the sum of marginal risks and can be affected by dependence.
It also differs from choosing a local false discovery rate threshold. A threshold rule includes every test whose local risk is below a chosen cutoff, so the number selected can vary with the data. A fixed-size rule instead requires exactly \(k\) selections and uses rank order; its \(k\)-th risk may be high or low. Earlier material on local FDR versus tail-area FDR distinguishes an individual posterior null probability from an average over a selected group. For the fixed-size result here, the sum of the individual posterior risks is the relevant quantity.
A ranking by raw test statistics, p-values, or estimated effect sizes optimizes this same objective only if that ranking agrees with the ordering of the local false discovery rates. That agreement is not automatic: posterior null risk depends on the null and alternative distributions and their mixture proportions, not solely on how extreme a statistic appears under a null reference distribution.
Assumptions, Diagnostics, and Limitations
The mathematical result requires a well-defined set of hypotheses, posterior null probabilities for the tests, and a fixed number \(k\) to select. Its statistical interpretation additionally depends on the model used to calculate those probabilities being appropriate for the data and the null hypotheses. Independence among tests is not required for the expected-count proof.
In practice, local false discovery rates are often estimated rather than known. Check that the computed values are finite and lie between 0 and 1, that the ranking corresponds to the intended posterior null risks, and that the selected set really contains the specified number \(k\). If the estimates are close or tied near the selection boundary, small changes in model fitting may change which individual tests are selected even when the expected false-count sums are similar. A clean ranking is not evidence by itself that the mixture model is well specified.
The guarantee is only as reliable as the posterior risks supplied to it. If a fitted model understates null probabilities for some tests, the ranking can be optimal for that fitted model but not for the true data-generating process. Parameter-estimation uncertainty can also make reported point estimates of \(\ell_i\) look more certain than they are. The theorem does not remove either limitation; it identifies the optimal decision conditional on the risk values being used.
Common Mistake
A related overstatement is to say that the selected set will contain fewer false discoveries in every realization. The theorem concerns the conditional expectation, not a guarantee about the unknown truth in one realized data set. It also does not say that a posterior expected count such as 0.33 is a possible realized count; the actual count is an integer, while its expectation need not be.
Check Your Understanding
Answer each question using the fixed-size posterior-risk criterion. Solutions follow the questions.
- Four tests have local false discovery rates \(0.03,\ 0.14,\ 0.27,\ 0.60\). If exactly two are selected, which set minimizes expected false selections, and what is that expected count?
- For a selected set of size four, the local false discovery rates sum to \(0.52\). What is its posterior expected false discovery proportion?
- Why does the fixed-size ranking proof not require independent null indicators?
- Two tests have equal local false discovery rates at the selection boundary. What can be concluded about choosing either one?
- Does selecting the smallest local false discovery rates necessarily minimize the probability of at least one false selection? Explain.
Solutions. (1) Select the tests with risks 0.03 and 0.14; the expected count is \(0.03+0.14=0.17\). (2) The expected proportion is \(0.52/4=0.13\). (3) Linearity of conditional expectation gives \(E\{\sum_{i\in S}Z_i\mid D\}=\sum_{i\in S}E(Z_i\mid D)\) without an independence assumption. (4) Either tied test gives the same expected count, so the posterior-risk criterion alone does not distinguish them. (5) No. The theorem minimizes the expected number of false selections at a fixed size; the probability of one or more false selections can depend on the joint distribution of the null indicators.