Null Proportion and Alternative Density

Why the Mixture Components Matter

A collection of genomic test statistics contains results from both null and non-null hypotheses. The overall distribution is therefore a mixture: its shape depends on how many tests are null and on the distribution of statistics among non-null tests. A statistic value may be common under the null but also common under the alternative, or it may be much more plausible under one component than the other. Local false discovery rate (local FDR) expresses this comparison as a posterior null probability at the observed value.

The null proportion and alternative density have distinct roles. The null proportion sets the prior weight assigned to null tests, while the alternative density describes where non-null statistics tend to fall. Neither component alone determines local FDR: the relevant comparison is between their weighted densities at the observed statistic.

Definition: Let \(Z\) denote the unobserved status of a randomly selected test, with \(Z=0\) for a null test and \(Z=1\) for a non-null test. Write \(\pi_0=P(Z=0)\) for the null proportion and \(\pi_1=1-\pi_0\) for the alternative proportion. Conditional on \(Z=0\), let the test statistic \(X\) have density \(f_0(x)\); conditional on \(Z=1\), let it have density \(f_1(x)\). The marginal density of \(X\) is \(f(x)=\pi_0 f_0(x)+(1-\pi_0)f_1(x)\). At values where \(f(x)>0\), the local FDR is \(\operatorname{lfdr}(x)=P(Z=0\mid X=x)=\pi_0 f_0(x)/f(x)\).

Each density is nonnegative and integrates to one over the statistic's support. The weights satisfy \(0\leq \pi_0\leq1\). These are model assumptions, not conclusions drawn from a particular observed statistic. The mixture formula describes the marginal distribution across tests; it does not require the tests to be independent. Dependence can affect uncertainty and estimation from a collection of tests, but it does not alter this marginal identity when the stated two-group model applies.

For a continuous statistic, \(P(X=x)=0\) at any single value. The conditional probability in the definition is therefore understood through the density form of Bayes' rule: the null contribution to the density at \(x\), divided by the total density there. The densities have the same units, so their ratio and the local FDR are dimensionless.

How the Components Determine Local FDR

The mixture density is a weighted average of the two component densities at every value \(x\). A high \(f_1(x)\) relative to \(f_0(x)\) gives more of the marginal density at \(x\) to the alternative component. Conversely, a high null proportion or a high null density relative to the alternative tends to increase the posterior null probability. The odds form makes the balance explicit:

\frac{\operatorname{lfdr}(x)}{1-\operatorname{lfdr}(x)} = \frac{\pi_0}{1-\pi_0}\frac{f_0(x)}{f_1(x)}.

The posterior null odds equal the prior null odds multiplied by the null-to-alternative density ratio. This is a pointwise comparison: a change in the alternative density matters according to whether it raises or lowers \(f_1(x)\) at the particular value being interpreted.

Justification: By the mixture formula, \(f(x)=\pi_0 f_0(x)+(1-\pi_0)f_1(x)\). Bayes' rule gives \(\operatorname{lfdr}(x)=\pi_0 f_0(x)/f(x)\), while \(1-\operatorname{lfdr}(x)=(1-\pi_0)f_1(x)/f(x)\). Dividing these expressions yields the odds identity. If both densities are positive and the densities are fixed, differentiating the local FDR with respect to \(\pi_0\) gives \(f_0(x)f_1(x)/f(x)^2\), which is positive. Thus increasing \(\pi_0\) increases local FDR at that value. Differentiating with respect to \(f_1(x)\), holding \(\pi_0\) and \(f_0(x)\) fixed, gives \(-\pi_0 f_0(x)(1-\pi_0)/f(x)^2\), which is negative. These derivatives describe changes at a fixed \(x\); a valid alternative density must still integrate to one.

That final qualification matters. Raising an alternative density in one region generally requires lowering it elsewhere or changing its overall shape while preserving its integral. A change in \(f_1(x)\) at one value is not a free change to the whole density. Therefore, the local effect can vary across the statistic's range even when the null proportion remains unchanged.

Worked Calculations

Worked Example: A Null Proportion and a Broad Alternative

Suppose, for illustration, the null statistic has the standard normal density and the alternative statistic has a normal density with mean 0 and variance 4. Let \(\pi_0=0.90\), and consider an observed statistic \(x=2\). These are hypothetical model values, not results from a study.

At \(x=2\), the null density is \(f_0(2)=\frac{1}{\sqrt{2\pi}}\exp(-2^2/2)=0.05399\). The alternative density is \(f_1(2)=\frac{1}{2\sqrt{2\pi}}\exp[-(2-0)^2/(2\cdot4)]\). Substituting the values gives \(f_1(2)=0.19947\exp(-0.5)=0.19947(0.60653)=0.12099\).

The marginal density at 2 is therefore \(f(2)=0.90(0.05399)+0.10(0.12099)=0.04859+0.01210=0.06069\). The local FDR is \(\operatorname{lfdr}(2)=0.90(0.05399)/0.06069=0.04859/0.06069=0.8007\). Under this specified mixture model, a test with statistic 2 has a posterior null probability of about 0.801. This is a model-based conditional probability, not a probability that a fixed null hypothesis is true.

Worked Example: Changing the Null Proportion

Keep both densities and the observed value \(x=2\) from the previous calculation, but change the null proportion from 0.90 to 0.70. The component densities remain \(f_0(2)=0.05399\) and \(f_1(2)=0.12099\).

The new marginal density is \(f(2)=0.70(0.05399)+0.30(0.12099)=0.03779+0.03630=0.07409\). The new local FDR is \(\operatorname{lfdr}(2)=0.70(0.05399)/0.07409=0.03779/0.07409=0.5101\).

With the null proportion reduced, the posterior null probability at 2 falls from about 0.801 to about 0.510. The marginal density at 2 also changes, because its two component weights changed. The result illustrates why a statistic value cannot be interpreted from the density shapes alone: the prior mixture weights matter as well.

Worked Example: Changing the Alternative Density

Now keep \(\pi_0=0.90\) and \(f_0(2)=0.05399\), but replace the alternative density with a normal density having mean 2 and variance 1. At \(x=2\), this alternative density is at its mean: \(f_1(2)=\frac{1}{\sqrt{2\pi}}\exp(0)=0.39894\).

The marginal density becomes \(f(2)=0.90(0.05399)+0.10(0.39894)=0.04859+0.03989=0.08849\). Thus \(\operatorname{lfdr}(2)=0.04859/0.08849=0.5491\).

The local FDR is lower than the value in the first example, even though the null proportion is unchanged. The alternative density puts more mass near 2, increasing the alternative contribution to the marginal density there. In contrast, at values far from 2 this alternative density may be relatively small; local FDR can then rise. Changing the alternative distribution reshapes local evidence across the statistic range rather than lowering local FDR uniformly.

Assumptions, Diagnostics, and Limits

The two-group interpretation is useful only when the components are meaningful for the statistics being modeled. The null density must describe the distribution of statistics for null tests, and the alternative density must describe the distribution for non-null tests, on the same statistic scale. The mixture weights must represent the relevant collection of tests. If tests from distinct populations or analysis pipelines are pooled despite having different null distributions, a single pair of components may fail to describe the data.

Several checks help identify problems in a proposed mixture description:

Check the component densities and weights. Each density must be nonnegative and integrate to one; the weights must lie between zero and one and sum to one.
Check the implied marginal shape. The weighted combination should plausibly describe the observed distribution across the full range, not just at a selected statistic value.
Check the local calculation. Wherever \(f(x)>0\), the numerator \(\pi_0f_0(x)\) cannot exceed the denominator \(f(x)\), so the local FDR must lie in \([0,1]\).
Watch for weak component separation. If \(f_0(x)\) and \(f_1(x)\) are similar over much of the support, local FDR may be driven substantially by \(\pi_0\), and the statistic value provides little discrimination between the two groups.

A marginal distribution alone does not always identify its components uniquely. Different choices of \(\pi_0\), \(f_0\), and \(f_1\) can produce similar or even identical marginal shapes while assigning different posterior null probabilities. Consequently, local FDR values depend on the component model, not solely on the observed histogram. In applications, component estimation and its uncertainty need careful attention; the calculations here treat the components as specified.

Local FDR is also distinct from tail-area FDR. Local FDR evaluates the posterior null probability at a particular statistic value. A tail-area quantity averages null probabilities over a selected region of statistics. The two can differ because the null-to-alternative density ratio need not be constant across that region. Neither quantity should be read as a causal effect or as a guarantee about a particular finding.

Common Mistakes

Using only the null proportion. A large \(\pi_0\) does not by itself determine the local FDR. The density ratio \(f_0(x)/f_1(x)\) also matters at the observed value.
Assuming any alternative makes local FDR smaller everywhere. An alternative density helps explain a statistic only where it has appreciable density. A changed alternative can lower local FDR in one region and raise it in another.
Interpreting the marginal density as a component density. The observed distribution is the weighted sum of both components, not a direct display of the null or alternative density separately.
Calling local FDR the probability that the null is true in a fixed frequentist test. It is a posterior probability under the specified two-group model and its assumptions.
Confusing local FDR with a tail-area error rate. A pointwise posterior and an average over a rejection region answer different questions.

Key takeaway: At a statistic value \(x\), local FDR is determined by the weighted null density relative to the total mixture density. The null proportion sets the prior weight; the alternative density determines how strongly that value is represented among non-null tests.

Check Your Understanding

Use the two-group mixture formulas to answer these questions. Solutions follow the questions.

At a fixed statistic value, suppose \(\pi_0=0.80\), \(f_0(x)=0.04\), and \(f_1(x)=0.16\). Find the marginal density and local FDR.
Suppose \(f_0(x)=0.05\), \(f_1(x)=0.10\), and \(\pi_0=0.60\). What is the local FDR at \(x\)?
At a fixed value \(x\), what happens to local FDR if \(f_1(x)\) increases while \(f_0(x)\) and \(\pi_0\) remain fixed?
Why can two mixture models with similar marginal distributions yield different local FDR values?

Solutions. (1) The marginal density is \(0.80(0.04)+0.20(0.16)=0.032+0.032=0.064\). The local FDR is \(0.032/0.064=0.500\). (2) The null contribution is \(0.60(0.05)=0.030\), and the alternative contribution is \(0.40(0.10)=0.040\), so the marginal density is \(0.070\) and the local FDR is \(0.030/0.070=0.4286\). (3) It decreases, because a larger alternative contribution increases the denominator while the null contribution stays fixed. (4) Local FDR depends on the component weights and the component densities at the particular value, and different component choices can divide a similar marginal shape differently between null and alternative contributions.