Genome-Wide Significance Versus Strength of Evidence

Significance Is a Decision Threshold, Not an Effect Size

A genome-wide association study may test hundreds of thousands or millions of genetic variants. A p-value that would look small in a single planned test can therefore be unsurprising somewhere in a genome-wide scan. A genome-wide significance threshold addresses this multiplicity: it sets a rule for deciding which results are sufficiently unusual under the testing procedure to be flagged. It does not measure how large an association is, how useful it is, or the probability that it is true.

Those questions require reading the association result as a whole. The p-value describes compatibility with a specified null model; the estimated effect describes the association's magnitude and direction on a stated scale; and the standard error and confidence interval describe its precision. Evidence from an independent replication can add support, but its interpretation depends on the replication design and its prespecified criteria.

What the P-Value Says

For a particular variant, suppose the association model estimates an effect \(\hat{\beta}\), with standard error \(SE\), and tests \(H_0:\beta=0\). In a large sample, a Wald test uses the statistic \(Z=\hat{\beta}/SE\), which is approximately standard normal under the null and the model's assumptions. A two-sided p-value is the probability, assuming that null model, of obtaining a test statistic at least as far from zero as the observed statistic:

where \(\Phi\) is the standard normal cumulative distribution function. The calculation assumes the chosen association model is appropriate, the standard error is valid, and the large-sample approximation is adequate. In a GWAS, sample structure, genotype quality, phenotype definition, and other design features matter because they affect whether the model and its uncertainty estimates are credible.

The p-value is not the probability that \(H_0\) is true, nor the probability that chance alone produced the data. It is calculated conditional on \(H_0\) and the statistical model. A smaller p-value indicates that the result is less compatible with that null model, but it does not by itself establish a causal effect or quantify the effect's importance.

From a Testing Family to a Threshold

The testing family must be defined before choosing a genome-wide cutoff: it specifies which hypotheses are being treated as one family for error control. For example, the family might include all tested variants for one phenotype under a stated analysis, or it might include a broader set of phenotypes and analyses. Bonferroni control provides a simple reference: for \(m\) tests and family-wise error target \(\alpha\), test each hypothesis at \(\alpha/m\). By the union bound, the probability of at least one false rejection is no greater than the sum of the individual probabilities, and hence no greater than \(\alpha\), when each null test is valid.

This bound does not require independent tests. In genetic data, linkage disequilibrium makes tests correlated, so Bonferroni can be conservative; nevertheless, its threshold gives a transparent benchmark. The commonly used \(5\times10^{-8}\) is a convention motivated by genome-wide testing, not an exact consequence of a fixed universal number of independent tests. A threshold is meaningful only in relation to the family and procedure it was chosen for.

Prespecification matters. If analysts select a threshold after seeing results, or silently expand the family to include additional phenotypes or analyses, the nominal error guarantee may no longer describe the full search. Conversely, crossing a stringent threshold says that the result passed a decision rule designed for a broad search; it does not imply a large effect.

Worked Examples: Threshold, Effect, and Precision

Effect Estimates and Confidence Intervals

An estimated effect needs a scale and a direction. For a quantitative trait, \(\hat{\beta}\) might be the mean trait difference per additional allele, measured in trait units or standard deviations. For another outcome, the association model may use a different scale. A p-value alone does not supply that information. Reporting the estimate with its units, standard error, and interval makes the result interpretable beyond the threshold decision.

For a large-sample Wald test of \(H_0:\beta=0\), the usual two-sided 95% Wald interval is \(\hat{\beta}\pm1.96SE\). Under the same approximation, zero is outside this interval exactly when \(|\hat{\beta}/SE|>1.96\), corresponding to a two-sided p-value below approximately 0.05. This relationship is a feature of that test and interval construction; it does not mean that an interval excluding zero is genome-wide significant. A genome-wide cutoff is much more stringent than 0.05.

An interval is also more informative than a binary threshold decision when the estimate is imprecise. A narrow interval can rule out many effect sizes, while a wide one may include both negligible and potentially meaningful effects. Neither interval width nor p-value settles whether the effect matters clinically or biologically; that judgment requires a relevant scale and substantive context.

Worked Example: Reading a Discovery Result Alongside Replication

Why the Distinction Matters

A significance threshold is useful because it makes a decision rule explicit in a large search. But reducing a result to “significant” or “not significant” discards information. A result just below a cutoff is not categorically different in evidential content from one just above it; the cutoff is a chosen boundary for a decision, not a natural dividing line between true and false associations. Likewise, failure to cross a stringent threshold does not prove no association exists. The estimate and interval show what effect sizes remain compatible with the data.

Replication is valuable because evidence in a separate cohort can test whether a result is seen again, rather than relying only on the discovery scan. It should be interpreted on a comparable allele and phenotype scale and with attention to its own uncertainty. A similar direction and plausible effect estimate are informative, but differences in precision mean that p-values can differ even when estimates are similar. A replication p-value should be judged against the replication plan, not automatically against the discovery threshold.

Common Mistake

Wrong reasoning: “Variant A has the smaller p-value, so it must have the larger and more important genetic effect.” This confuses evidence against a null with effect magnitude. A p-value depends on both the estimate and its standard error, as \(Z=\hat{\beta}/SE\) shows. A precise small estimate can have a smaller p-value than a larger, uncertain estimate. The correction is to compare effect estimates on the same scale, inspect their intervals, and use the p-value to describe evidence against the specified null—not as a ranking of biological importance.