Central Matching for Empirical Null Estimation

Why Match the Center?

In a genomic analysis, many test statistics may be close to zero even when a smaller group show strong departures from the null. The central part of their distribution can therefore contain useful information about the null distribution. Central matching estimates an empirical null by fitting a scaled null density to the observed test-statistic density near its center, where the null component is assumed to dominate.

This is not a claim that every statistic near zero is a true null, or that every alternative lies in the tails. It is a modeling strategy whose accuracy depends on whether alternative density is small enough in the chosen central region. The introductory material on empirical nulls distinguishes the theoretical null from a null distribution estimated from the observed statistics; here the focus is the central density fit used to estimate that empirical null.

Definition: Let $Z$ denote a test statistic with marginal density $f(z)=\pi_0 f_0(z)+(1-\pi_0)f_1(z)$, where $\pi_0$ is the null proportion, $f_0$ is the null density, and $f_1$ is the alternative density. In central matching, one assumes that $f_1(z)$ is negligible over a selected central region $C$, so that $f(z)\approx\pi_0 f_0(z)$ there. For a normal empirical null, $f_0(z)=\frac{1}{\sigma_0}\phi((z-\mu_0)/\sigma_0)$, where $\mu_0$ and $\sigma_0>0$ are the empirical-null center and scale, and $\phi$ is the standard normal density.

Fitting a Scaled Normal Density

Suppose the central region is adequately described by a scaled normal density: $f(z)\approx \pi_0 f_0(z)$. The scale factor $\pi_0$ is needed because the null component accounts for only a fraction of all tests. Taking logarithms gives a quadratic function of $z$:

\log f(z)\approx \log\pi_0-\log(\sigma_0\sqrt{2\pi})-\frac{(z-\mu_0)^2}{2\sigma_0^2} =a+bz+cz^2.

Expanding the square identifies the coefficients: $c=-1/(2\sigma_0^2)$, $b=\mu_0/\sigma_0^2$, and $a=\log\pi_0-\log(\sigma_0\sqrt{2\pi})-\mu_0^2/(2\sigma_0^2)$. Thus a fitted quadratic can be translated back into estimates of all three quantities. In practice, the log density must first be estimated from the observed statistics, for example by smoothing or by using a histogram with bins fine enough to represent the central shape.

Central-matching estimates: If the fitted central log-density curve is $a+bz+cz^2$, with $c<0$, then $$ \hat{\sigma}_0=\sqrt{-\frac{1}{2c}},\qquad \hat{\mu}_0=b\hat{\sigma}_0^2,\qquad \hat{\pi}_0=\exp\left(a+\log(\hat{\sigma}_0\sqrt{2\pi})+ \frac{\hat{\mu}_0^2}{2\hat{\sigma}_0^2}\right). $$ These estimates are meaningful only if the central fit is plausible and the resulting $\hat{\pi}_0$ lies in the allowable range from 0 to 1.

The coefficient formulas follow directly from equating the constant, linear, and quadratic terms in the expanded expression. In particular, a downward-opening quadratic is required because a normal log density curves downward. If the fitted quadratic does not curve downward, it cannot represent a normal density with positive scale. Even when it does, a quadratic fit to a badly chosen region can produce implausible parameter estimates.

Worked Example: Recovering the Empirical Null From a Central Fit

Suppose a hypothetical smoothed density of 1,500 test statistics has a central log-density fit $q(z)=-1.1254+0.2469z-0.6173z^2$. Interpret its coefficients using central matching.

The quadratic coefficient gives $\hat{\sigma}_0=\sqrt{-1/(2(-0.6173))}=\sqrt{0.8100}=0.9000$. The linear coefficient then gives $\hat{\mu}_0=0.2469(0.9000)^2=0.19999$, or approximately $0.2000$.

For the null proportion, $\hat{\pi}_0=\exp[-1.1254+\log(0.9000\sqrt{2\pi})+(0.2000)^2/(2(0.9000)^2)]$. Here $0.9000\sqrt{2\pi}\approx2.2560$, its log is approximately $0.8136$, and the final squared-center term is $0.02469$. Therefore $\hat{\pi}_0\approx\exp(-1.1254+0.8136+0.02469)=\exp(-0.2871)\approx0.7504$. The fitted empirical null is centered at about $0.2000$, has scale $0.9000$, and represents about 75.0% of the test-statistic distribution under the central-dominance model.

This numerical fit alone does not verify central dominance. It only shows how fitted coefficients translate into estimates; the choice of central region and the density shape there still need scrutiny.

Estimating the Null Proportion From the Center

If $\mu_0$ and $\sigma_0$ are specified or estimated separately, the density at the null center provides a simpler matching calculation. Since $f_0(\mu_0)=1/(\sigma_0\sqrt{2\pi})$, central dominance implies $f(\mu_0)\approx\pi_0/(\sigma_0\sqrt{2\pi})$. Rearranging yields $\pi_0\approx f(\mu_0)\sigma_0\sqrt{2\pi}$. The density estimate at the center must use the same scale and units as the test statistic; it is a density, not a probability.

With observed statistics $z_1,\ldots,z_m$, one crude local density estimate at a point $z$ uses a narrow interval of width $h$ centered at $z$: if $N_h(z)$ observations fall in that interval, $\hat f(z)=N_h(z)/(mh)$. A smoother density estimate can reduce sensitivity to bin boundaries, but does not remove the central-dominance assumption. If the result exceeds 1, it is not a valid estimate of a proportion; that outcome is a warning that the density estimate, null parameters, or model assumption needs attention, not a reason to report a proportion greater than 1.

Worked Example: Estimating the Null Proportion From a Central Bin

Consider 2,000 hypothetical test statistics. The empirical null is estimated as $N(0.10,1.10^2)$, and 116 statistics fall in the narrow interval $[0,0.20]$, centered at the estimated null center. Use the midpoint density approximation to estimate $\pi_0$.

The observed density estimate at the center is $\hat f(0.10)=116/(2{,}000\times0.20)=116/400=0.2900$ per test-statistic unit. The matching estimate is $\hat{\pi}_0=0.2900(1.10)\sqrt{2\pi}\approx0.2900(2.7573)=0.7996$. Thus central matching estimates that about 80.0% of the statistics belong to the null component, conditional on the assumed null and on negligible alternative density in this interval.

The interval width is small relative to the null scale, which makes treating its average density as approximately the density at its midpoint more defensible. It is still an approximation: a wider interval or one not centered at $\mu_0$ would generally need an explicit integrated-density calculation rather than this midpoint calculation.

Why Central Contamination Biases the Fit

The key approximation $f(z)\approx\pi_0 f_0(z)$ fails when alternatives contribute appreciable density in the fitting region. The exact density is $f(z)=\pi_0f_0(z)+(1-\pi_0)f_1(z)$, so at the center the density-based estimate satisfies $\hat{\pi}_0\approx \pi_0+(1-\pi_0)f_1(\mu_0)/f_0(\mu_0)$, assuming the null parameters are correct and the density is estimated accurately. If $f_1(\mu_0)>0$ and $\pi_0<1$, this expression exceeds $\pi_0$. The alternative component is mistakenly counted as null mass, biasing the null-proportion estimate upward.

Worked Example: A Concentrated Alternative Inflates the Center Estimate

Suppose a hypothetical mixture has $\pi_0=0.80$, a null density $N(0,1)$, and an alternative density $N(0,0.5^2)$. Both components are centered at zero, but the alternative is more concentrated. Calculate the central-matching estimate that would result from the true density at zero.

The null density at zero is $\phi(0)=1/\sqrt{2\pi}\approx0.3989$. The alternative density at zero is $1/(0.5\sqrt{2\pi})\approx0.7979$. Hence the mixture density at zero is $f(0)=0.80(0.3989)+0.20(0.7979)=0.3191+0.1596=0.4787$. Matching this to a scaled standard normal gives $\hat{\pi}_0=f(0)\sqrt{2\pi}=0.4787(2.5066)\approx1.200$.

The estimate is impossible as a proportion. The calculation correctly exposes a model failure: the central alternative density is not negligible, so the mixture has more density at zero than any scaled version of the assumed null with scale factor at most 1. Truncating the answer to 1 would conceal, rather than fix, the problem.

Assumptions, Diagnostics, and Alternatives

Central matching relies on several linked assumptions. The null component must be adequately represented by the chosen density family, such as a normal distribution with center $\mu_0$ and scale $\sigma_0$. The statistics used for the fit must be on a common scale and sufficiently comparable for one marginal mixture density to be meaningful. Most importantly, the selected central region must be dominated by the null component. A broader central interval supplies more data for density estimation but is more likely to include alternative mass; an extremely narrow interval reduces contamination risk in some settings but makes the density estimate noisy.

Diagnostics should target the fit, not just the reported $\hat{\pi}_0$. Plot or inspect the estimated density and fitted scaled-null curve over the whole central interval. Look for systematic curvature or asymmetry that the fitted normal cannot reproduce, unstable estimates when the interval or smoothing bandwidth changes modestly, a non-downward quadratic, or $\hat{\pi}_0$ outside $[0,1]$. Also examine whether a central peak or shoulder could reflect alternatives concentrated near the null center. These signals do not identify the unique cause of failure, but they undermine a literal central-match interpretation.

Central matching differs from estimating $\pi_0$ by counting large p-values: it uses the shape and height of the test-statistic density near its center, and it can accommodate an empirical null whose center or scale differs from the theoretical null. A theoretical-null analysis fixes the reference distribution rather than estimating its central location and spread from the observed statistics. A full two-group mixture analysis instead models the null and alternative components together; it may use more of the distribution, but depends on assumptions about both components and is not equivalent to assuming alternatives vanish centrally. No approach removes the need to check whether its model is suitable.

Dependence among tests also matters for uncertainty and stability. Central matching describes a marginal density; it does not require that test statistics be independent to define that density. But strong dependence can make the observed central histogram or smoother less stable than an independent-sample calculation would suggest. Consequently, a good-looking fit to one large set of correlated statistics should not be treated as proof that its estimated parameters are precise.

Choose a candidate central region.
Specify which range of test statistics will be used and why it is plausibly null-dominated.

Estimate the observed central density.
Use a stated histogram or smoothing procedure, recognizing its bin-width or bandwidth sensitivity.

Fit and translate the null curve.
Fit the scaled normal shape when appropriate, then calculate $\hat{\mu}_0$, $\hat{\sigma}_0$, and $\hat{\pi}_0$.

Challenge the assumptions.
Compare nearby central regions and density choices; investigate poor fit, implausible estimates, and possible alternative mass at the center.

Key limitation: Central matching cannot distinguish null observations from alternatives that have similar test-statistic values in the center using central density alone. Its estimates are therefore conditional on a substantive assumption about the alternative density, not merely a consequence of having many tests.

Common Mistake

A common mistake is to say, “The center is where null statistics should occur, so every observation near zero is null.” That reasoning ignores the mixture: alternatives can also lie near zero, especially when effects are weak or the alternative distribution is concentrated centrally. The correction is to state central dominance as an assumption, assess how plausible it is, and report the resulting estimates as model-dependent. Another error is treating the fitted density height as a probability; density has units of inverse test-statistic units, and it must be multiplied by $\sigma_0\sqrt{2\pi}$ to obtain the central-matching proportion.

Check Your Understanding

Answer each question, then compare with the brief solutions below.

In central matching, what condition on $f_1(z)$ in the fitting region allows $f(z)$ to be approximated by $\pi_0 f_0(z)$?
A central fit is $q(z)=a+bz+cz^2$ with $c=-0.5$ and $b=0.20$. Find the implied $\hat{\sigma}_0$ and $\hat{\mu}_0$.
Why can central alternative mass inflate a density-at-the-center estimate of $\pi_0$?
A central-matching calculation gives $\hat{\pi}_0=1.08$. Should it be reported as 1.00? Explain.

Solutions. (1) The alternative density must be negligible relative to the null contribution in that region, so $(1-\pi_0)f_1(z)$ is small compared with $\pi_0f_0(z)$. (2) $\hat{\sigma}_0=\sqrt{-1/(2(-0.5))}=1$, and $\hat{\mu}_0=b\hat{\sigma}_0^2=0.20$. (3) The observed center density includes both null and alternative contributions; matching all of it to the null peak attributes some alternative density to the null component. (4) No. A proportion cannot exceed 1, and the result flags a problem with the density estimate, fitted null, or central-dominance assumption; simply capping it hides that evidence.