Empirical Null Calibration for Association Statistics

Why Estimate an Empirical Null?

Association tests are often interpreted by comparing their statistics with a theoretical null distribution: the distribution expected if the tested variant has no association with the outcome. In a well-calibrated analysis, the bulk of statistics from null variants should be compatible with that reference. But a systematic shift or excess spread in the observed central statistics can make nominal p-values misleading.

An empirical null estimates the center and scale of the null distribution from the central part of the observed association statistics. It can therefore allow for a location shift as well as a change in spread. That flexibility is useful, but it is not automatic protection against confounding or model misspecification: the central statistics must be a reasonable source of information about the null.

Definition: For signed association statistics \(Z_1,\ldots,Z_m\), an empirical null model represents the null statistics approximately as \(Z_j \sim N(\mu_0,\sigma_0^2)\). The parameter \(\mu_0\) is the null center and \(\sigma_0\) is the null scale. Under the ideal standard normal null, \(\mu_0=0\) and \(\sigma_0=1\). The empirical-null-adjusted statistic is \(Z_j^*=(Z_j-\hat{\mu}_0)/\hat{\sigma}_0\).

Here, \(Z_j\) is a signed statistic, such as a z statistic whose sign indicates the direction of the estimated association. If results are reported only as one-degree-of-freedom chi-square statistics, the sign has been discarded. A signed z statistic can be recovered only if the direction of effect is also available: it is the square root of the chi-square statistic with that direction's sign. Estimating a nonzero center requires signed statistics; unsigned chi-square values alone do not identify it.

The empirical null concerns calibration of individual association statistics. It does not replace the choice of a testing family or a multiple-testing procedure. After obtaining calibrated p-values, investigators still need to use an appropriate family-wise error or false discovery rate method for their analysis.

Estimating the Center and Scale From Central Statistics

A practical approach is to estimate the center and spread using central quantiles of the signed statistics. The motivation is that genuinely associated variants often contribute more strongly to the tails than to the center. If that pattern holds, central quantiles can be less affected by those signals than the mean and standard deviation of all observed statistics.

For a normal distribution, the population median is its center, and the interquartile range is a known multiple of its standard deviation. Thus, central quantiles give direct estimates of \(\mu_0\) and \(\sigma_0\). The empirical estimates are not guaranteed to equal the true null parameters: they are useful only to the extent that the selected central part represents the null.

\hat{\mu}_0 = \widehat{Q}_{0.50}, \qquad \hat{\sigma}_0 = \frac{\widehat{Q}_{0.75}-\widehat{Q}_{0.25}} {\Phi^{-1}(0.75)-\Phi^{-1}(0.25)} \approx \frac{\widehat{Q}_{0.75}-\widehat{Q}_{0.25}}{1.349}.

Here, \(\widehat{Q}_p\) is the estimated \(p\)-th quantile of the central statistics, and \(\Phi\) is the cumulative distribution function of a standard normal random variable. This quantile-matching method uses the median and interquartile range rather than the full-sample mean and standard deviation. Other methods fit a normal density to a central histogram region or maximize a likelihood for a truncated normal distribution.

Important condition: If observations are selected because they fall inside a fixed window, such as \(-c \leq Z_j \leq c\), their distribution is truncated. A normal likelihood for those selected values must account for that selection. For a proposed center \(\mu\) and scale \(\sigma\), the density within the window is the normal density divided by the probability of falling in the window:

f(z \mid -c \leq Z \leq c;\mu,\sigma) = \frac{\phi((z-\mu)/\sigma)} {\sigma\left[\Phi((c-\mu)/\sigma)-\Phi((-c-\mu)/\sigma)\right]}, \qquad -c \leq z \leq c,

where \(\phi\) is the standard normal density. The denominator adjusts for the probability of being selected. Treating the selected values as though they were an unselected normal sample can distort estimates, particularly when the window is narrow. Quantile matching and truncated-likelihood fitting are alternative estimation strategies; the method and the central region should be reported clearly.

Worked Example: Estimating an Empirical Null From Central Quantiles

Suppose a hypothetical association scan has a large collection of signed z statistics. After examining its central region, the estimated central quantiles are \(\widehat{Q}_{0.25}=-0.40\), \(\widehat{Q}_{0.50}=0.175\), and \(\widehat{Q}_{0.75}=0.75\). Estimate the empirical-null center and scale by quantile matching.

The estimated center is the median:

\hat{\mu}_0 = 0.175.

The estimated interquartile range is \(0.75-(-0.40)=1.15\). Therefore:

\hat{\sigma}_0 = \frac{1.15}{1.349} \approx 0.8525.

The fitted central null is approximately \(N(0.175,0.8525^2)\). Its estimated center is above zero, and its estimated scale is below one. These are features of the fitted central distribution, not proof of a particular cause. For example, the estimates could reflect calibration problems, features of the analysis, or the way the central region was selected.

Adjusting a Statistic and Its P-Value

For a two-sided association test, standardize the observed signed statistic using the estimated empirical-null center and scale. The adjusted statistic is then compared with a standard normal reference distribution. The resulting p-value measures how extreme the adjusted statistic is under that reference model.

Z_j^* = \frac{Z_j-\hat{\mu}_0}{\hat{\sigma}_0}, \qquad p_j^* = 2\Phi\left(-|Z_j^*|\right).

This adjustment accounts for both estimated location and scale. A positive estimated center shifts statistics downward before rescaling; a negative center shifts them upward. A scale estimate above one reduces their absolute standardized magnitudes, while a scale estimate below one increases them. Those descriptions apply to the calculation, not to the cause of the fitted parameters.

Worked Example: Calculating an Empirical-Null-Adjusted P-Value

Use the hypothetical estimates \(\hat{\mu}_0=0.175\) and \(\hat{\sigma}_0=0.8525\) from the preceding example. Suppose a variant has an observed signed statistic \(Z=3.20\). Calculate its empirical-null-adjusted two-sided p-value.

First, standardize relative to the fitted center and scale:

Z^* = \frac{3.20-0.175}{0.8525} = \frac{3.025}{0.8525} \approx 3.548.

The two-sided p-value is \(2\Phi(-|3.548|)\), which is approximately \(0.00039\). Under the fitted empirical-null model, the observed statistic is therefore about 3.55 standard deviations above the estimated null center. This is a calibrated p-value conditional on the adequacy of the fitted model; it is not the probability that the null hypothesis is true.

If the fitted empirical null is used to adjust many variants, the same estimates \(\hat{\mu}_0\) and \(\hat{\sigma}_0\) are generally applied to all of them. The adjusted p-values can then be passed to a multiple-testing procedure. Their interpretation depends on the empirical-null approximation, and using estimated parameters introduces uncertainty that the simple formula does not include.

How Empirical-Null Adjustment Differs From Genomic Control

Genomic control estimates a scale adjustment from the median of one-degree-of-freedom association chi-square statistics. Under the central chi-square null, the median is approximately \(0.4549\). The genomic inflation factor is the observed median divided by that reference median. A chi-square statistic is then divided by this factor before calculating its p-value.

\hat{\lambda}_{GC} = \frac{\operatorname{median}(\chi_j^2)}{0.4549}, \qquad \chi_{j,\mathrm{adj}}^2 = \frac{\chi_j^2}{\hat{\lambda}_{GC}}.

This adjustment is based on the median chi-square and rescales the chi-square statistics. In contrast, an empirical null fitted to signed z statistics can estimate both a center and a scale. If the center is not zero, adjusting only the chi-square scale does not remove that signed location shift. The two procedures also rely on different working assumptions and estimates; neither adjustment by itself establishes why the observed statistics differ from their theoretical reference.

Worked Example: Applying Genomic Control

In a separate hypothetical scan, suppose the median of the one-degree-of-freedom chi-square statistics is \(0.500\). For a variant with \(\chi^2=10.24\), calculate the genomic control factor and adjusted statistic.

The estimated factor is:

\hat{\lambda}_{GC} = \frac{0.500}{0.4549} \approx 1.099.

Divide the variant's chi-square statistic by this factor:

\chi_{\mathrm{adj}}^2 = \frac{10.24}{1.099} \approx 9.318.

For one degree of freedom, this corresponds to an absolute z statistic of \(\sqrt{9.318}\approx 3.053\), and a two-sided p-value of approximately \(0.0023\). This is a scale adjustment based on the scan's median chi-square. It does not estimate a signed center, because the chi-square statistics do not retain the direction of association.

A Location Shift Can Matter

The ability to estimate a center is one reason to distinguish empirical-null calibration from a scale-only correction. A center shift can change the calibrated statistic differently on opposite sides of zero. The next example isolates that arithmetic and shows why retaining the sign matters.

Worked Example: Adjusting a Negative Signed Statistic

Suppose a hypothetical central fit gives \(\hat{\mu}_0=0.20\) and \(\hat{\sigma}_0=1.10\). A variant has signed statistic \(Z=-2.40\). Calculate its adjusted statistic and approximate two-sided p-value.

The adjusted statistic is:

Z^* = \frac{-2.40-0.20}{1.10} = \frac{-2.60}{1.10} \approx -2.364.

Thus, \(p^*=2\Phi(-2.364)\approx 0.0181\). The fitted positive center makes this negative statistic more extreme relative to the estimated null center. This conclusion follows from the specified model and estimates; it does not show that a positive center arose from any one biological or technical cause.

Why the Central Fit Needs Care

The empirical null is most persuasive when the central statistics are dominated by null variants and their distribution is reasonably approximated by a normal location-scale family. It can be misleading when those conditions fail. A large number of small genuine effects may influence the center or spread, especially if their statistics overlap the central region. In that case, fitting an empirical null can absorb some real polygenic signal and make associations appear less significant.

Conversely, a central fit cannot guarantee that all confounding has been removed. Different sources of distortion can affect the tails and center differently, and a single normal distribution may not describe them. A fitted center or scale is a calibration summary, not a diagnosis. Interpret it alongside the study design, test construction, and the pattern of the full association results.

Uncertainty also matters. The estimated center and scale are calculated from observed statistics, so they are not known constants. The adjusted p-value formula treats them as fixed and does not account for their estimation uncertainty. This approximation is often used in practice, but it should not be confused with a procedure that fully propagates uncertainty in the calibration parameters.

Key distinction: Empirical-null calibration estimates a reference distribution for association statistics; it does not identify the cause of miscalibration or replace multiple-testing control. Report the signed statistic used, the central region and estimation method, the fitted center and scale, and the subsequent p-value and error-control procedure.

Common Mistake

A common mistake is to treat an empirical-null adjustment as proof that the remaining p-values are correctly calibrated. For example, one might reason, “The central statistics now match a standard normal distribution, so every adjusted p-value is valid.” That conclusion does not follow. The fit was estimated from the data, the central region may contain genuine effects, and the tails may not follow the same model. The correction is to state the assumptions, assess whether the central statistics plausibly represent null behavior, and interpret adjusted p-values as conditional on that model. Calibration also does not control a family-wise error rate or false discovery rate on its own.

Check Your Understanding

Use these questions to check your understanding of empirical-null calibration.

What do \(\mu_0\) and \(\sigma_0\) represent, and what are their ideal standard-normal null values?
Why can the median and interquartile range be useful for estimating the empirical-null center and scale?
Why must a likelihood fit to statistics selected within a fixed central window account for truncation?
In what way can an empirical-null adjustment differ from genomic control?
Why does empirical-null adjustment not replace a multiple-testing procedure?