Sensitivity to Null-Model Misspecification

Why Null-Model Sensitivity Matters

A local false discovery rate (local FDR) is a model-based estimate of the probability that a tested feature is null, conditional on its observed statistic. In genomic applications, a large collection of statistics may have a central distribution that differs from the theoretical null because of unmodeled variation or other features of the analysis. The empirical-null material and Q-Q diagnostics for empirical null models address how such departures can be detected and estimated. Here the focus is what happens to local FDR when the null model used in the calculation is wrong.

The practical issue is that local FDR depends directly on the null density at the observed statistic. A small change to the center, scale, or tail behavior can therefore matter substantially for statistics near a decision threshold. The numerical comparisons below isolate each change: they hold the null proportion and alternative density fixed while changing only the candidate null density. This makes the direct effect interpretable, though it is not a substitute for refitting a complete model to data.

Model and Definition

This is a posterior probability under the specified mixture model. It has that interpretation when the model and its parameters adequately describe the distribution of statistics and null status. A local FDR calculated from a misspecified null density is still a number between zero and one, but it need not be a calibrated posterior probability under the actual data-generating process.

For controlled comparisons, write \(\pi_1=1-\pi_0\) and hold \(\pi_0\) and \(f_1\) fixed. If the assumed null density is \(g_0\), the resulting calculation is

The comparison is pointwise: it asks how the local FDR at the same observed statistic changes when the null density changes. It does not assert that the marginal density remains the same after changing a component of the mixture. In an actual analysis, fitting the null, alternative, and null proportion together can change all three estimates.

A Direct Sensitivity Identity

Let \(L(z)\) be the local FDR calculated under a reference null density \(f_0\), and suppose a candidate null density is \(g_0(z)=r(z)f_0(z)\), with \(r(z)>0\). Holding \(\pi_0\) and \(f_1\) fixed, the null-to-alternative posterior odds under the reference model are \(L/(1-L)\). Under the candidate null, the null contribution is multiplied by \(r\), while the alternative contribution is unchanged. Thus the odds become \(rL/(1-L)\), which gives the following identity:

Equivalently, the log posterior odds change by \(\log r(z)\). This shows why the density ratio, not merely a parameter difference such as a change of 0.1 in the null mean, determines the local effect at a particular statistic. For a small change in the null density,

To justify this differential expression, take the derivative of \(L=A/(A+B)\), where \(A=\pi_0 f_0(z)\) and \(B=\pi_1 f_1(z)\) are fixed except for \(A\). Then \(dL=B\,dA/(A+B)^2\). Since \(dA=A\,d\log f_0(z)\), and \(AB/(A+B)^2=L(1-L)\), the expression follows. The factor \(L(1-L)\) is largest at \(L=0.5\), so small relative changes in the null density can have their largest absolute effect on local FDR around intermediate posterior risks.

Worked Comparisons

Interpreting and Diagnosing Sensitivity

For a normal null \(N(\mu,\sigma^2)\), the log density at a fixed \(z\), apart from a constant, is \(-\log\sigma-(z-\mu)^2/(2\sigma^2)\). Its parameter sensitivities are

Combined with \(dL=L(1-L)d\log f_0(z)\), these expressions show that small center changes matter most directionally according to how far and on which side \(z\) lies from the null center. A scale increase lowers the density near the center but can raise it far into the tails. These are local statements about density at a fixed statistic, not claims that every statistic's local FDR moves in the same direction.

A useful sensitivity analysis evaluates a grid of plausible null centers, scales, or tail forms and reports the range of local FDR estimates for statistics relevant to the decision. It should state what was held fixed and what changed. A broad range signals that conclusions depend on null specification. A narrow range is reassuring about this particular source of uncertainty, but does not establish that the model is otherwise correct.

Diagnostics include checking whether the candidate nulls plausibly represent the central statistics, examining whether sensitivity is concentrated in extreme tails or near a decision cutoff, and comparing estimates across a deliberately stated range of null parameters. Q-Q diagnostics for empirical null models can help reveal distributional departures, but visual agreement in the center does not guarantee correct tail behavior. In particular, tails may be poorly informed when few observations fall there.

A common mistake is to say, “The null standard deviation fell from 1 to 0.8, so its density at \(z=2\) must be about 20% lower,” and then use that rough change in a local FDR calculation. The density includes both the rescaling factor \(1/\sigma\) and the changed standardized distance \((z-\mu)/\sigma\). Here the correct value is \(\phi(2.5)/0.8\approx0.02191\), not \(0.0439\). Always calculate the full density before forming the mixture ratio.

Other approaches answer related but different questions. Refitting an empirical null and alternative uses the data to estimate model components rather than holding them fixed; that can reveal joint changes but introduces estimation variability and possible instability. A sensitivity grid is transparent about assumed alternatives for the null, but it does not select a uniquely correct model. A single theoretical null is simpler, but can give misleading posterior risks if its location, scale, or tails are inadequate. No pointwise comparison alone resolves model uncertainty.