Interpreting Local FDR Calibration Plots

What a Local FDR Calibration Plot Can—and Cannot—Show

A local false discovery rate (local FDR) is a posterior probability: under a two-group model, it is the probability that a test is null given its observed statistic. A calibration plot can help assess whether the fitted model describes the distribution of the test statistics and whether the resulting posterior probabilities are plausible. These are related checks, but they are not the same check.

The first comparison is between the observed distribution of the test statistics and the density implied by the fitted mixture. The second summarizes fitted local FDR values in bins and, when null labels are known, compares predicted null fractions with the actual null fractions. In ordinary genomic data, the true null status of each test is hidden. Therefore, a plot of fitted local FDR values alone cannot directly verify that those probabilities are calibrated against truth.

Definition: In a two-group model, a test is null with prior probability $\pi_0$, and its statistic $Z$, conditional on being null, has density $f_0$. Conditional on being nonnull, it has density $f_1$. The marginal density and local FDR are $$ f(z)=\pi_0 f_0(z)+(1-\pi_0)f_1(z), \qquad \operatorname{lfdr}(z)=P(\text{null}\mid Z=z)=\frac{\pi_0 f_0(z)}{f(z)}. $$ The model assumes these densities and the null proportion adequately describe the marginal distribution of the tests being assessed. A fitted calibration plot assesses their empirical adequacy; it does not establish that the model is correct.

The mixture-density comparison is usually made by drawing a histogram or smoothed estimate of the observed statistics and overlaying the fitted density, or by comparing observed and fitted counts in bins. It asks whether the model reproduces the overall shape: center, spread, shoulders, and tails. A good overall fit is necessary for credible local FDR estimates, but it does not guarantee that the null component and alternative component have been separated correctly.

Comparing Observed Counts With the Fitted Mixture

For a bin $B_j$, the fitted model gives probability $q_j=\int_{B_j}f(z)\,dz$. If there are $m$ test statistics, the model's expected count in that bin is $m q_j$. The observed count can then be compared with this expectation. Bins must form a genuine partition: each observed value belongs to exactly one bin. For example, the intervals $(-\infty,-2)$, $[-2,-1)$, $[-1,1)$, $[1,2)$, and $[2,\infty)$ are disjoint and cover the real line, including their endpoints.

Worked Example: Reading a Mixture-Density Overlay

Suppose 1,000 hypothetical test statistics are modeled using $f_0=N(0,1)$, $f_1=\tfrac12N(-2,1)+\tfrac12N(2,1)$, and $\pi_0=0.8$. Numerical integration of the fitted mixture over the five disjoint bins $(-\infty,-2)$, $[-2,-1)$, $[-1,1)$, $[1,2)$, and $[2,\infty)$ gives probabilities approximately $0.068$, $0.143$, $0.578$, $0.143$, and $0.068$, respectively. These rounded probabilities sum to 1.000.

Multiplying each probability by $m=1{,}000$, the fitted expected counts are $68$, $143$, $578$, $143$, and $68$. Suppose the hypothetical observed counts are $74$, $132$, $586$, $139$, and $69$. They sum to $1{,}000$, as they should. The observed-minus-expected differences are $+6$, $-11$, $+8$, $-4$, and $+1$.

These differences are modest relative to the counts, so this binned comparison does not reveal a gross mismatch. It is not proof of a correct model: the differences could still reflect systematic shape problems, and a formal goodness-of-fit test would need to account for estimated parameters and any dependence among tests. The histogram overlay and the bin comparison are two views of the same marginal fit, not independent confirmation.

Inspect more than the center of the plot. A mixture can match the central peak while missing the shoulders or tail counts, and a density estimate can hide local discrepancies through smoothing. Conversely, a few uneven bins need not imply failure: sampling variation is expected. The sample size, bin widths, and the fact that model parameters were estimated all affect how much variation is unsurprising.

Binning Posterior Null Probabilities

Once a model is fitted, each observed statistic $z_i$ receives a local FDR value $r_i=\operatorname{lfdr}(z_i)$. For a bin containing $n_B$ tests, the mean local FDR is $\bar r_B=n_B^{-1}\sum_{i: z_i\in B}r_i$. Under the model, this is the predicted null fraction among tests whose statistics fall in that bin. The corresponding expected number of null tests is $\sum_{i:z_i\in B}r_i=n_B\bar r_B$.

Justification: Let $H_i$ indicate that test $i$ is null, with $H_i=1$ for a null test. Since $E(H_i\mid Z_i=z_i)=\operatorname{lfdr}(z_i)$, conditioning on membership in a bin $B$ gives $$ E(H_i\mid Z_i\in B)=E\{\operatorname{lfdr}(Z_i)\mid Z_i\in B\}. $$ Thus the average fitted local FDR in the bin is the model-based expected null fraction there. This identity follows from the fitted posterior probabilities; by itself, it is not evidence that the model's probabilities match the unknown null labels.

Worked Example: Calculating and Binning Local FDR Values

Use the hypothetical model $f_0=N(0,1)$, $f_1=N(0,4)$, and $\pi_0=0.9$. The alternative has standard deviation 2. At $z=2$, the null density is $\phi(2)=0.05399$, and the alternative density is $\phi(2;0,4)=0.12099$. Therefore, $$ \operatorname{lfdr}(2)= \frac{0.9(0.05399)} {0.9(0.05399)+0.1(0.12099)} =\frac{0.04859}{0.06069} \approx 0.8007. $$

For context, the same calculation gives $\operatorname{lfdr}(0)\approx0.9473$, $\operatorname{lfdr}(1)\approx0.9252$, and $\operatorname{lfdr}(3)\approx0.3812$. For example, at $z=3$, the weighted null density is approximately $0.0039887$, while the weighted alternative density is approximately $0.0064759$. Their sum is approximately $0.0104645$, so the null share is $0.0039887/0.0104645\approx0.3812$.

Suppose six hypothetical tests have statistics $0,0,1,1,2,2$, and use the disjoint bins $[-0.5,0.5)$, $[0.5,1.5)$, and $[1.5,2.5)$. Each bin has two tests. Their mean local FDR values are approximately $0.9473$, $0.9252$, and $0.8007$, so the model-based expected null counts are $2(0.9473)=1.895$, $2(0.9252)=1.850$, and $2(0.8007)=1.601$. Across all six tests, the expected null count is approximately $5.346$, not a claim that exactly 5.346 tests are null.

The estimates decline as the statistic moves away from zero in this model, but that pattern is a property of the fitted densities, not a universal calibration requirement. Bins chosen by local FDR value rather than by $Z$ can also be useful for summarizing risk, but they do not reveal true null proportions when null labels are unobserved.

Direct Calibration When Null Status Is Known

A direct calibration comparison is possible in a simulation or other setting where the true null status is known. Group tests into bins of predicted local FDR, calculate the mean prediction in each bin, and compare it with the observed fraction of null tests. Agreement is expected on average under a correctly specified model, but the fractions in a finite sample fluctuate.

Worked Example: Interpreting a Simulation Calibration Check

In a hypothetical simulation, 30 tests with known null labels are divided into three bins of predicted local FDR. Each bin contains 10 tests. The mean predicted null probabilities are $0.90$, $0.60$, and $0.20$, giving expected null counts $10(0.90)=9$, $10(0.60)=6$, and $10(0.20)=2$. Suppose the known null counts are 9, 5, and 3. The observed null fractions are therefore $0.90$, $0.50$, and $0.30$.

The first bin agrees exactly. The second has one fewer null than expected, and the third has one more. The aggregate observed count is $9+5+3=17$, equal to the aggregate expected count $9+6+2=17$, but the bin-level deviations show why an overall total can conceal imperfect calibration. With only 10 tests per bin, these differences are not compelling evidence of systematic miscalibration; repeated simulation or larger bins would give a more stable assessment.

In real genomic analyses, the true null labels are generally not observed. A plot of mean local FDR against the fraction of tests assigned to a bin is not a direct calibration plot unless that fraction comes from known labels or a defensible external validation source. The fitted model can report an expected null count, but that is a model-based expectation, not an empirically counted number of null tests.

Diagnostics, Alternatives, and Limitations

Read the plots together. A mixture-density overlay that misses the central peak, shoulders, or tails warns that the fitted marginal distribution may be inadequate. A binned local FDR summary can show whether posterior risks change in an interpretable way across regions of the statistic, but it cannot repair a poor density fit. Nor can a visually smooth decline in local FDR establish that the prior null proportion or null density is correct.

Look for structured residuals. Repeated excesses on one side, or systematic underprediction in the tails, are more concerning than isolated bin differences.
Check bin definitions and counts. Use bins that are disjoint and cover the intended range. Sparse bins produce noisy observed counts and unstable summaries.
Separate fit from truth. Matching the observed marginal density supports the overall model fit, but different combinations of null and alternative components can sometimes produce similar marginal shapes.
Account for estimation. The same data are often used to estimate mixture parameters and assess the fitted density. The overlay is therefore a diagnostic, not an independent validation sample.
Remember the unit of interpretation. A local FDR is a posterior probability for a test with its observed statistic under the model; it is not a frequentist error rate guaranteed for every selected set.

The density overlay is often the most direct visual check of the marginal model. Binned residual or count comparisons make discrepancies easier to quantify, while known-label simulation checks whether posterior probabilities recover null frequencies in settings where truth is available. These approaches answer different questions. A tail-area FDR summarizes risk over a selected region rather than the posterior null probability at a particular statistic, so it is not a substitute for checking local FDR calibration.

A common mistake is to say, “The average local FDR in this bin is 0.20, so exactly 20% of the tests in the real data are null.” The correction is that 0.20 is the fitted model's expected null fraction for that bin. The actual fraction is unknown unless null status is known, and finite-sample variation remains even when the model is correct.

Key takeaway: A local FDR calibration assessment has two distinct targets: whether the fitted mixture reproduces the observed statistic distribution, and whether predicted posterior null probabilities agree with known null frequencies when those are available. Neither a good-looking overlay nor a smooth binned curve alone proves calibration.

Check Your Understanding

Use the definitions and distinctions above to answer each question. Brief solutions follow.

A fitted mixture assigns probability $0.25$ to a bin containing 400 tests. What is the expected count in that bin, and what does it not establish?
Why are $[-1,1]$ and $[1,2)$ not disjoint bins? Give a corrected partition around those endpoints.
A bin contains five tests with local FDR values $0.1, 0.2, 0.2, 0.5,$ and $0.5$. Find the mean local FDR and expected null count.
In a simulation bin, the mean predicted local FDR is $0.30$ and 20 tests have known null labels, of which 8 are null. Compare the prediction and observation.
What conclusion is justified if the observed histogram closely follows the fitted mixture density?

Solutions. (1) The expected count is $400(0.25)=100$. It does not establish that exactly 100 tests are null or that the component model is correct. (2) Both intervals contain 1. One corrected pair is $[-1,1)$ and $[1,2)$; the shared endpoint 1 belongs only to the second interval. (3) The mean is $(0.1+0.2+0.2+0.5+0.5)/5=1.5/5=0.30$, so the expected null count is $5(0.30)=1.5$. (4) The model predicts $20(0.30)=6$ nulls, while 8 are observed, a difference of 2; this one bin alone does not establish systematic miscalibration. (5) The overlay supports the fitted model's marginal description of the observed statistics, but does not prove that the null and alternative components, or the local FDR probabilities, are correctly specified.