Local FDR Thresholds and Decision Loss

From Posterior Probabilities to Decisions

A local false discovery rate expresses the posterior probability that a tested feature is null, given its observed evidence and the model used to analyze it. That probability describes uncertainty; it does not, by itself, say whether a feature should be declared a discovery. A decision also requires specifying the consequences of the two possible errors: declaring a null feature non-null, or failing to discover a non-null feature.

In genomic analysis, those consequences can differ. A false positive might lead to costly follow-up experiments, while a false negative might cause a potentially useful signal to be overlooked. A rule that makes the two errors equally costly can therefore be inappropriate when their practical consequences differ. Decision theory makes the trade-off explicit by assigning costs to errors and choosing the action with the lower posterior expected cost.

In applications, the available data may be the feature’s test statistic under a two-group model, so that \(\ell_i\) is computed from that statistic and the model’s null proportion and densities. More generally, the decision rule below uses the posterior null probability conditional on whatever information is being used for the decision. The costs must be expressed on a common scale: multiplying both by the same positive constant leaves the decision unchanged.

Deriving the Local-FDR Threshold

For a given feature, a false positive occurs if it is declared a discovery while \(H_i=0\), an event with posterior probability \(\ell_i\). A false negative occurs if it is not declared a discovery while \(H_i=1\), an event with posterior probability \(1-\ell_i\). Thus the two possible actions have the following conditional expected losses.

Choose the action with smaller expected loss. A discovery is preferred when its loss is no greater than the alternative:

The rule is a direct consequence of minimizing conditional expected loss, not an extra assumption about the distribution of the test statistics. For equal costs, \(\tau=1/2\): discover when the posterior probability of being non-null exceeds one half. If false positives are more costly, the denominator is unchanged but the threshold falls as \(C_{\mathrm{FP}}\) grows relative to \(C_{\mathrm{FN}}\). If false negatives are more costly, the threshold rises. These comparisons hold when the posterior probabilities are appropriate for the information used and the stated costs correctly represent the decision problem.

Worked Decisions With Specified Costs

Many Features, Assumptions, and Alternatives

If the total loss is the sum of feature-level losses, the posterior expected total loss is also the sum of the feature-level posterior expected losses. This follows from linearity of conditional expectation; independence among features is not required for this addition. Therefore, minimizing each feature’s conditional expected loss minimizes the sum. In particular, dependence does not change this calculation when the posterior null probabilities used for each decision are valid conditional probabilities for the information being used.

The assumptions that matter are chiefly about the decision setup. There are two actions per feature; costs are specified and comparable; incorrect discoveries and missed non-null features incur the stated costs; and the posterior probabilities are suitable for the decisions. If a false positive’s cost differs by feature, use its feature-specific cost in the comparison. If the cost of a missed signal also varies, use that feature-specific cost too. The same derivation then gives a separate threshold for each feature.

This loss-minimizing rule answers a different question from a procedure designed to control an error rate, such as a false discovery rate target. Error-rate procedures seek a specified property of errors across a collection of tests; the rule derived here seeks the lowest expected cost under the stated cost model and posterior probabilities. A conventional \(p\)-value cutoff is also not automatically a local-FDR decision threshold: it summarizes evidence under a null model, while the rule compares posterior probabilities and costs. One approach is not universally preferable; the appropriate choice depends on the objective.

A practical diagnostic is to recompute the threshold and the action-specific expected losses for borderline features. If a reported decision conflicts with those calculations, check whether the costs were reversed, whether the local FDR is a posterior null probability rather than a tail-area error rate, and whether the inequality has been applied in the correct direction. Also assess whether the fitted null and alternative model plausibly represents the data: a numerically precise threshold cannot correct a poorly calibrated posterior probability. Estimated posterior probabilities and chosen costs may themselves be uncertain, so decisions close to the threshold deserve particular caution.

The two-action rule has limitations. It does not account for costs that depend on how many features are selected, such as a penalty for exceeding a follow-up budget, unless those effects are added to the decision problem. Nor does it directly represent graded actions, such as prioritizing features for different levels of validation. In such settings, the possible actions and their costs need to be specified more fully; the simple threshold applies to the binary, additive-loss setup described here.