Common Errors in Local FDR Analysis

Why Local FDR Errors Matter

Local false discovery rate (local FDR) is useful because it assigns a posterior null probability to a test result, rather than only labeling results significant or not significant. But that interpretation depends on the model and on calculating the right quantity on the right scale. Confusing local FDR with a tail-area error rate, using an unsupported estimate of the null proportion, or interpreting a null probability as an effect-size probability can change the scientific conclusion.

The empirical Bayes setup used here has a two-group model. A randomly selected test is null with probability $\pi_0$, and non-null with probability $1-\pi_0$. Conditional on being null, its statistic $X$ has density $f_0$; conditional on being non-null, it has density $f_1$. The marginal density of $X$ is the mixture $f$. For genomic tests, $X$ might be a z-score or a p-value. The model treats the null label as a latent indicator; it does not make a non-null result automatically biologically important.

Definition: Under the two-group mixture model, the local FDR at an observed value $x$ is the posterior probability that the test is null given that value: $$ \operatorname{lfdr}(x)=P(\text{null}\mid X=x)=\frac{\pi_0 f_0(x)}{\pi_0 f_0(x)+(1-\pi_0)f_1(x)}=\frac{\pi_0 f_0(x)}{f(x)}. $$ The densities in this expression must be defined on the same scale and with respect to the same variable.

The formula is Bayes’ rule expressed using densities: the numerator is the joint density contribution from null tests, and the denominator is the density contribution from all tests. As with any continuous observation, $P(X=x)=0$; the conditional probability is defined through the density model, not by treating an exact point as an event of positive probability.

Local FDR Is Not Tail-Area FDR

A common error is to calculate the posterior null probability at one observed statistic and describe it as the false discovery rate among all tests beyond a threshold. These quantities condition on different events. Local FDR conditions on the test statistic being at a particular value. A tail-area quantity conditions on the statistic falling in a chosen rejection region.

Definition: For a rejection region $A$ in the statistic’s sample space, the mixture-model tail-area null fraction is $$ \operatorname{Fdr}(A)=P(\text{null}\mid X\in A) =\frac{\pi_0 P_0(X\in A)}{\pi_0P_0(X\in A)+(1-\pi_0)P_1(X\in A)}, $$ where $P_0$ and $P_1$ are probabilities under $f_0$ and $f_1$. For a threshold $t$, $A$ might be $\{x:|x|\geq t\}$.

For a fixed rejection region and independent tests from this mixture, the expression is the expected fraction of null tests among tests in that region when the number of tests is large. It is not, in general, identical in finite samples to the frequentist false discovery rate $E[V/(R\vee1)]$, where $V$ is the number of rejected nulls and $R$ is the number of rejections. Nor is it the local FDR at a boundary point. The distinction is not merely terminology: the tail-area calculation aggregates probability over the entire rejection region.

Worked Example: Same Mixture, Different Error Rates

Suppose hypothetical z-scores follow a mixture with $\pi_0=0.80$, null density $f_0$ equal to the standard normal density, and alternative density $f_1$ equal to the density of $N(0,4)$. Compare local FDR at $z=2$ with the tail-area null fraction for $|Z|\geq2$. The example assumes the stated mixture is correctly specified and uses known parameters.

At $z=2$, the null density is $\phi(2)=0.0540$, and the alternative density is $\frac{1}{2}\phi(1)=0.1210$. Thus the mixture density is $f(2)=0.80(0.0540)+0.20(0.1210)=0.0674$, giving

\operatorname{lfdr}(2)=\frac{0.80(0.0540)}{0.0674} =\frac{0.0432}{0.0674}\approx 0.641.

For the tail region, the standard normal probability of $|Z|\geq2$ is approximately $0.0455$. Under $N(0,4)$, the standard deviation is 2, so this probability is $P(|N(0,1)|\geq1)\approx0.3173$. The null contribution to the mixture tail is $0.80(0.0455)=0.0364$; the alternative contribution is $0.20(0.3173)=0.0635$. Therefore,

\operatorname{Fdr}(\{|Z|\geq2\}) =\frac{0.0364}{0.0364+0.0635}\approx0.364.

The local FDR at exactly $z=2$ is about 0.641, while the null fraction in the full two-sided tail is about 0.364. Both calculations use the same mixture, but answer different questions. Neither number is the probability that a biological effect is absent or present in a broader scientific sense than the model’s null label.

Estimate the Null Proportion With Its Assumptions Visible

The null proportion $\pi_0$ is a mixture weight, not the observed fraction of tests that fail to reach a significance threshold. A p-value threshold does not reveal the null labels: some null tests can have small p-values, and some non-null tests can have large p-values. Estimating $\pi_0$ therefore requires a model or an estimator with assumptions.

One illustrative estimator uses the upper tail of p-values. Under a continuous, correctly specified null, p-values are uniform on $[0,1]$. At a chosen $\lambda\in[0,1)$, the population tail probability is $P(P>\lambda)=\pi_0(1-\lambda)+(1-\pi_0)P_1(P>\lambda)$. Dividing by $1-\lambda$ gives the population analogue of the estimator $\hat{\pi}_0(\lambda)=\#\{p_i>\lambda\}/[m(1-\lambda)]$, where $m$ is the number of tests. The calculation estimates $\pi_0$ only when alternative p-values contribute little to the selected upper tail, or when their contribution is otherwise accounted for. If alternatives have appreciable upper-tail mass, this simple estimator tends to be too large. A low estimate is not automatically trustworthy either: finite-sample variation or an invalid null model can distort it.

Worked Example: An Upper-Tail Estimate Is Not Automatically the Truth

Consider a hypothetical set of $m=10{,}000$ p-values. At $\lambda=0.50$, suppose 3,400 p-values exceed 0.50. The plug-in estimate is

\hat{\pi}_0(0.50)=\frac{3400}{10{,}000(1-0.50)} =\frac{3400}{5000}=0.680.

This is an estimate, not a direct count of true nulls. To see why its assumptions matter, suppose for illustration that the actual mixture has $\pi_0=0.60$, null p-values are uniform, and the alternative distribution has $P_1(P>0.50)=0.10$. The expected upper-tail proportion is $0.60(0.50)+0.40(0.10)=0.34$; hence the expected count above 0.50 is 3,400. The estimator’s population value is then $0.34/0.50=0.68$, above the true 0.60, because alternative tests contribute to the upper tail. The observed calculation is arithmetically correct, but interpreting it as an assumption-free estimate of the true null fraction is not.

This reasoning assumes valid null p-values and a meaningful two-group representation. If the null distribution is miscalibrated, its p-values need not be uniform, so even the null contribution to the upper tail can be wrong. Inspecting the p-value distribution and considering the estimation method’s assumptions are part of the analysis, not optional decorations.

Keep Density Scales Consistent

A density is measured relative to its variable. The density of a z-score and the density of the corresponding p-value are not interchangeable numbers. If a statistic is transformed, its density changes by a Jacobian factor. In a local FDR ratio, omitting that factor can create a false result because the numerator and denominator no longer refer to densities on the same scale.

For a two-sided p-value $P=2\{1-\Phi(|Z|)\}$, let $U=|Z|$. For $0<p<1$, the absolute z-score corresponding to $p$ is $u=\Phi^{-1}(1-p/2)$, and $|dp/du|=2\phi(u)$. If $g_0$ and $g_1$ are the densities of $U$ under the null and alternative, respectively, then the p-value densities are $f_{0,p}(p)=g_0(u)/(2\phi(u))$ and $f_{1,p}(p)=g_1(u)/(2\phi(u))$. The same transformation factor applies to both groups, so it cancels in their local FDR ratio. It does not mean that a density value on the z scale can be compared directly to a p-value density.

Worked Example: Transforming a Two-Sided z-Score to a p-Value Density

Use the normal mixture from the first example at $z=2$. Here $u=2$, and the corresponding two-sided p-value is approximately $p=0.0455$. Under the null, the density of $U=|Z|$ at 2 is $g_0(2)=2\phi(2)=0.1080$. Under the $N(0,4)$ alternative, it is $g_1(2)=2\{(1/2)\phi(1)\}=0.2420$. The Jacobian denominator is $2\phi(2)=0.1080$, so the densities with respect to p are approximately

f_{0,p}(0.0455)=\frac{0.1080}{0.1080}=1.000,\qquad f_{1,p}(0.0455)=\frac{0.2420}{0.1080}\approx2.241.

The local FDR on the p-value scale is therefore $\{0.80(1.000)\}/\{0.80(1.000)+0.20(2.241)\}\approx0.641$, matching the z-scale result. The agreement follows because both densities were transformed consistently. A frequent mistake is to insert a z-scale density for one component and a p-scale density for another; the resulting ratio has no valid posterior interpretation.

A Null Probability Is Not an Effect-Size Probability

Under the specified mixture, $\operatorname{lfdr}(x)$ is the posterior probability that the test belongs to the model’s null component, conditional on its observed statistic. Thus $1-\operatorname{lfdr}(x)$ is the posterior probability of belonging to the model’s alternative component. Neither quantity gives the probability that the effect exceeds a biological threshold, has a particular direction, or is large enough to matter clinically.

That distinction depends on how the model defines the null. If the null is a point mass at zero and every nonzero effect belongs to the alternative, then “non-null” does not mean “large.” The alternative may include effects arbitrarily close to zero. If a scientific question concerns effects larger than a meaningful threshold, it requires a parameter model and a posterior probability for that event; local FDR alone does not supply it. Likewise, an estimated non-null probability is not evidence that the treatment caused the effect.

Worked Example: A Small Local FDR Does Not Establish a Large Effect

Suppose a hypothetical test has estimated $\operatorname{lfdr}(x)=0.08$. Under the fitted two-group model, the posterior probability of the null component at this result is 0.08, and the posterior probability of the alternative component is $1-0.08=0.92$. That is the full direct interpretation.

It would be incorrect to conclude that there is a 92% probability the effect is biologically important. For example, the alternative distribution could put most of its probability on effects close to zero, while still assigning the observed statistic relatively little density under the null. To calculate the posterior probability that an effect exceeds a threshold, one would need a model for the effect parameter and would compute a probability such as $P(|\theta|>\delta\mid\text{data})$, where $\delta$ is a scientifically chosen threshold. The local FDR value alone does not determine that probability.

Diagnostics, Comparisons, and Practical Corrections

When an analysis looks surprising, first identify the conditioning event and the scale. A local FDR value should be based on densities at the observed value. A tail-area value should use probabilities integrated over the specified rejection region. The two can differ substantially, as the first example shows. State which quantity is being reported and do not substitute one for the other.

Next check whether the null proportion and null model are credible. Under a valid null p-value model, the null component is uniform; systematic departures in regions believed to be null can indicate calibration problems. A p-value histogram can reveal broad patterns, but it cannot by itself certify the null proportion or the mixture model. For z-scores, compare the observed distribution with the specified null model and examine whether a fitted alternative and null together provide a plausible mixture. Sensitivity to modeling choices is a warning that conclusions rely on those choices.

The main alternatives answer different questions. A thresholded tail-area FDR summarizes a rejection set; local FDR ranks or describes individual results by posterior null risk under the model. A p-value threshold tests a null hypothesis using a tail probability, not a posterior probability. None is a substitute for the others without an explicit decision rule. The earlier material on local FDR thresholds and decision loss connects posterior risks to decisions; the cost assigned to errors matters when converting risks into actions.

Local FDR also has limitations: it depends on the null and alternative densities and on $\pi_0$, and its numerical precision cannot be read as certainty about those fitted quantities. Dependence among tests, null misspecification, weak mixture identifiability, and boundary estimates can all affect empirical Bayes calibration. Treating an estimated value as exact or as a universal property of a gene ignores those modeling limitations.

Common Mistakes to Avoid

Wrong: “The local FDR at $z=2$ is the false discovery rate among every test with $|z|\geq2$.” Correction: The first conditions on the observed value; the latter aggregates over the tail region.
Wrong: “The fraction of p-values above 0.5 equals the null proportion.” Correction: Convert that fraction using the estimator and account for alternative tail mass and null calibration.
Wrong: “The density of z is the density of p at the corresponding value.” Correction: Transform densities using the Jacobian and use one scale consistently.
Wrong: “A local FDR of 0.05 means a 95% chance of a large, important effect.” Correction: It means a 5% posterior probability of the model’s null component, conditional on the statistic; effect magnitude needs a separate model and event.
Wrong: “A smaller local FDR proves a causal genomic effect.” Correction: It is a model-based posterior classification probability, not a causal conclusion.

Key takeaway: Before reporting a local FDR, specify the mixture, the null proportion, the statistic scale, and the conditioning event. Interpret the result only as a posterior null probability under that model.

Check Your Understanding

Answer each question, then compare with the solutions below.

In one sentence, distinguish local FDR at $x$ from tail-area FDR for $X\in A$.
At $\lambda=0.80$, 1,600 of 10,000 hypothetical p-values exceed $\lambda$. What is the upper-tail estimate of $\pi_0$, and what assumption could make it too large?
Why must a z-score density and a p-value density not be inserted together into the same local FDR ratio?
A fitted model gives local FDR $0.12$. What posterior probability of the alternative component follows, and what does it not establish?

Solutions. (1) Local FDR conditions on the observed statistic value, whereas tail-area FDR conditions on membership in a rejection region and integrates probabilities over that region. (2) The estimate is $1600/[10{,}000(1-0.80)]=1600/2000=0.800$. Alternative p-values in the upper tail can raise the estimate above the true null proportion. (3) The densities are with respect to different variables; the change-of-variable Jacobian is required to compare them consistently. (4) The posterior probability of the alternative component is $1-0.12=0.88$. This does not establish a large or biologically meaningful effect, nor does it establish causation.