Choosing Lambda For Storey’s Pi0 Estimate

Why Lambda Matters

Storey’s estimate of the proportion of true null hypotheses uses the fraction of p-values in the upper tail. The threshold \(\lambda\) determines which p-values count as upper-tail observations. A low threshold includes more alternatives, which can make the estimate too large. A high threshold reduces that contamination in some settings, but leaves fewer p-values to estimate the null proportion and can make the estimate noisy. Choosing \(\lambda\) is therefore a bias-variance decision, not a cosmetic setting.

This tutorial focuses on that decision. The estimate is used in Storey’s q-value approach, but the question here is how to choose its threshold, not how to calculate q-values from an estimate. In the formulas below, \(m\) is the number of tested hypotheses, \(p_i\) is the p-value for test \(i\), and \(\pi_0\) is the proportion of hypotheses that are truly null.

The Upper-Tail Estimator and Its Assumptions

The basic mixture model supposes that a fraction \(\pi_0\) of the hypotheses are null, with null p-values uniformly distributed on \([0,1]\), and that the remaining fraction have alternative p-values with cumulative distribution function \(F_1\). This exact-uniform null assumption is appropriate for continuous tests calibrated under the null. It is stronger than merely requiring valid, super-uniform p-values: conservative or discrete null p-values need not have upper-tail probability \(1-\lambda\), so the displayed estimator may require adjustment in those settings.

For the expectation calculation, the model can be read as sampling each p-value from the same mixture: a null p-value with probability \(\pi_0\), or an alternative p-value with probability \(1-\pi_0\). Independence is not needed for this expectation. The simple variance formula used below additionally assumes that the \(m\) p-values are independent and identically distributed under this mixture. Those assumptions are useful for understanding the tradeoff, but genomic tests may violate independence.

How Lambda Controls Bias and Variance

Write \(S_1(\lambda)=P(p_i>\lambda\mid\text{alternative})=1-F_1(\lambda)\), the alternative probability of falling above the threshold. Under the mixture model, the probability that a p-value exceeds \(\lambda\) is

Dividing the expected upper-tail fraction by \(1-\lambda\) gives the estimator’s expectation. The difference between that expectation and \(\pi_0\) is its bias under this model.

The bias is nonnegative because alternative p-values that remain above \(\lambda\) are counted alongside null p-values. If most alternatives produce small p-values, \(S_1(\lambda)\) can be small, and this source of upward bias is reduced. If alternatives have substantial upper-tail probability, the estimate can remain appreciably above \(\pi_0\), even at a seemingly high threshold.

Under the independent, identically distributed mixture model, the number of p-values above \(\lambda\) has a binomial distribution with size \(m\) and probability \(q(\lambda)\). Therefore,

The denominator shrinks as \(\lambda\) approaches 1. There are fewer upper-tail observations, so the estimate generally becomes more variable. The formula makes the tradeoff explicit: raising \(\lambda\) can reduce the alternative contribution to bias, but can increase variance. It does not guarantee that variance rises at every possible step for every p-value distribution; it shows why an extreme threshold is often unstable.

A Short Derivation of the Tradeoff

Let \(I_i\) equal 1 when \(p_i>\lambda\), and 0 otherwise. Then \(\hat{\pi}_0(\lambda)=\sum_{i=1}^m I_i/[m(1-\lambda)]\). Under the mixture assumptions, \(E[I_i]=q(\lambda)\), and hence

Substituting the mixture tail probability gives

This derivation separates the null contribution, which is calibrated to \(\pi_0\), from the excess contributed by alternatives in the tail. For the variance, independence gives \(\operatorname{Var}(\sum I_i)=m q(\lambda)\{1-q(\lambda)\}\); dividing by the square of \(m(1-\lambda)\) yields the variance formula above. If p-values are dependent, covariance terms enter \(\operatorname{Var}(\sum I_i)\), so that independent-binomial formula no longer applies.

Choosing a Threshold in Practice

There is no universally optimal \(\lambda\). A defensible choice considers the anticipated alternative tail and the number of tests. The aim is to place the threshold high enough that relatively few alternatives remain above it, while retaining enough observations to estimate the null tail with useful precision. A fixed value such as 0.5 is sometimes used as a practical convention, not as a theorem that it is optimal for every data set.

If there is independent scientific or pilot information about the alternative p-value distribution, it can inform the tradeoff. The following hypothetical calculation shows how a threshold can be chosen by comparing approximate mean squared error, the sum of variance and squared bias.

Diagnostics, Alternatives, and Limitations

Before relying on an estimate, check whether its assumptions are plausible and whether the upper tail contains enough information. A p-value histogram can reveal gross departures from a uniform-looking null background or a shortage of upper-tail values, but it cannot by itself verify that null p-values are exactly uniform or identify the true labels of null tests. A very small count above \(\lambda\) is a warning that the estimate may be unstable. Estimates above 1, before truncation, can also arise from sampling variation or model mismatch; truncating them does not repair the underlying cause.

A fixed threshold is transparent and reproducible, but may leave considerable bias if alternatives retain substantial upper-tail mass. Raising the threshold can reduce that contamination, but may make the estimate sensitive to a handful of p-values. Model-based choices can use a scientifically justified alternative distribution, as in the preceding example, but are only as reliable as that model. More adaptive methods are possible, yet they do not eliminate the need to state assumptions and assess uncertainty. A systematic examination of how estimates change across thresholds is a separate diagnostic task.

Dependence is an important limitation in genomic applications. The expectation formula follows from marginal mixture probabilities, but the independent variance formula does not account for correlated tests. Likewise, if null p-values are discrete or conservative, replacing their actual tail probability with \(1-\lambda\) can distort the calibration. The estimator is not a direct count of true null hypotheses: it estimates a proportion under a model, using upper-tail p-values as indirect evidence.

Common Mistake

Another poor practice is to select \(\lambda\) because it produces a preferred number of discoveries. That makes the estimate outcome-driven and can understate uncertainty. A threshold should be justified by the estimator’s bias-variance behavior and the analysis context, rather than by the desired conclusion.