Estimating the Null Proportion

Why Estimate the Null Proportion?

In a large genomic analysis, each tested feature has a p-value, but the analyst usually does not know which hypotheses are truly null. The null proportion summarizes the fraction of null hypotheses in the family being analyzed. It is useful in empirical Bayes models and can affect estimates of false discovery rates. Estimating it is difficult because a p-value near one may come from a null test, but it may also come from a nonnull test with little power.

The question here is how to estimate that fraction from a collection of p-values, and what assumptions make an estimator informative. In particular, the upper tail is attractive because valid null p-values are uniform, so the expected fraction of null p-values exceeding a threshold \(\lambda\) is \(1-\lambda\). This gives a simple estimator. Its limitation is equally important: alternative p-values can also exceed \(\lambda\), making the estimate too large.

Two-Group Model and Threshold Estimator

For a p-value \(P\), write \(\overline{G}(\lambda)=P(P>\lambda\mid\text{alternative})=1-G(\lambda)\). The p-value density, when it exists, is \(f(p)=\pi_0+(1-\pi_0)g(p)\), where \(g\) is the alternative density. The density expression is not required for the threshold estimator, but makes the two components explicit.

For a chosen threshold \(0\leq\lambda<1\), count how many observed p-values exceed it. If those p-values were all null, their expected fraction would be \(1-\lambda\). Dividing the observed fraction by that null-tail probability gives the raw threshold estimate:

Because a proportion cannot exceed one, a commonly reported version is \(\min\{1,\widehat{\pi}_0(\lambda)\}\). This truncation is a practical constraint, not a change to the raw estimator’s expectation formula. In particular, expectations derived for the raw estimator do not automatically describe the truncated version.

The simple calculation relies on marginally uniform null p-values. For the variance formula below, assume in addition that the \(m\) p-values are independent draws from the same mixture. Independence is not needed for the expectation calculation if each test has the stated marginal mixture, but dependence changes the variance. In applications, p-values that are merely super-uniform under the null can make the upper-tail null probability larger than \(1-\lambda\); the exact-uniform identity used by this estimator then need not hold.

Bias and Sampling Variability

Let \(I_i\) equal one if \(p_i>\lambda\), and zero otherwise. Under the mixture model, its success probability is \(q(\lambda)=\pi_0(1-\lambda)+(1-\pi_0)\overline{G}(\lambda)\). The first term is the null contribution to the upper tail, and the second is the alternative contribution. Since the raw estimator is \(\sum_{i=1}^m I_i/[m(1-\lambda)]\), its expectation follows directly.

Thus the raw estimator’s bias is \((1-\pi_0)\overline{G}(\lambda)/(1-\lambda)\). It is upward biased when \((1-\pi_0)\overline{G}(\lambda)>0\), and unbiased when that product is zero. The latter includes the boundary case \(\pi_0=1\), even if the alternative tail probability is positive. If almost all alternative p-values fall below \(\lambda\), then \(\overline{G}(\lambda)\) is small and the contamination bias is small. The model alone does not guarantee this.

Under the additional independent, identically distributed mixture assumption, the indicators \(I_i\) are independent Bernoulli variables with success probability \(q(\lambda)\). Their sum has variance \(m q(\lambda)\{1-q(\lambda)\}\), giving:

This formula shows the threshold trade-off. Increasing \(\lambda\) can reduce alternative contamination, but it also reduces the null-tail probability in the denominator and leaves fewer p-values in the count. Consequently, the estimator can become more variable near one. The two effects must be considered together.

Choosing a Threshold in Sparse and Dense Settings

A sparse-signal setting has a large \(\pi_0\); a dense-signal setting has a smaller \(\pi_0\). Sparsity by itself does not guarantee an accurate threshold estimate. The amount of alternative probability above the chosen threshold matters as well. When alternatives tend to produce small p-values, a high threshold may have little alternative contamination. When alternative p-values often remain in the upper tail, the estimate can be inflated even if the threshold seems high.

A common practical approach computes estimates over several candidate thresholds and looks for a region where they are reasonably stable. A flat region can be reassuring, but it does not prove that the estimates are unbiased: alternative p-values may contribute similarly across that region. Conversely, a sharply changing sequence signals sensitivity to the threshold and should prompt caution. Thresholds chosen after inspecting the same data also make simple uncertainty calculations harder to interpret.

Another upper-tail approach estimates the p-value density near one, where the null density is one. For a narrow interval \((1-h,1]\), the observed fraction in that interval divided by \(h\) estimates the mixture density averaged over that interval. If the alternative density is negligible there, this quantity can approximate \(\pi_0\). If alternatives contribute in that interval, the density estimate is inflated. Narrow intervals have fewer observations and greater sampling variability; wider intervals are more stable but can include more alternative mass. This is related to the threshold idea but uses a local interval rather than the full tail above \(\lambda\).

Comparison, Diagnostics, and Limitations

The choice \(\widehat{\pi}_0=1\) avoids estimating the null proportion and is often used as a conservative input to procedures that permit it. It is not an estimate tailored to the observed p-value distribution. Threshold and upper-tail density methods use the data, but rely on alternatives contributing little to the selected upper-tail region if they are to estimate \(\pi_0\) accurately. More elaborate mixture fitting is a different strategy; it requires additional distributional assumptions and is not a substitute for checking whether the upper-tail premise is plausible.

Several checks help reveal trouble. Plot or tabulate estimates over a prespecified range of thresholds; large movement indicates threshold sensitivity. Inspect the upper-tail p-values and the number of observations supporting each estimate; a very small tail count is inherently unstable. Check that the null p-values are calibrated for the analysis, and consider the dependence structure: correlated tests may make realized tail counts more variable than the independent formula suggests. Neither a stable estimate nor a large number of tests removes systematic inflation from alternative p-values in the upper tail.

A frequent mistake is to argue, “Only null p-values exceed a high threshold, so the fraction above it estimates \(\pi_0(1-\lambda)\).” The conclusion assumes what needs to be checked. Under the mixture model, the tail probability is \(\pi_0(1-\lambda)+(1-\pi_0)\overline{G}(\lambda)\); the second term vanishes only if the alternative tail probability is zero or there are no alternatives. A high threshold may make that term small, but does not make it exactly zero by definition.