P-Value Thresholds and Score Sparsity

Why a P-Value Threshold Changes Score Sparsity

A clumping-and-thresholding score combines association results from a GWAS with genotype data for the people whose scores will be calculated. Clumping limits redundancy among correlated variants; thresholding then determines which of the remaining variants are included. A stringent p-value cutoff produces a shorter, sparser score. A more permissive cutoff generally includes more variants, including variants whose estimated associations are weak or compatible with chance.

The central question is not simply how many variants a cutoff admits. A score with more terms is not necessarily more accurate, and a score with fewer terms is not necessarily better. Threshold choice interacts with the genetic architecture: how many variants affect the trait and how large their effects are. It also depends on the GWAS information available to distinguish signals from noise. This tutorial focuses on that interaction, taking the clumping procedure and score definition as given.

Definition: Let $C$ be a fixed set of $m$ variants retained after clumping, and let $p_j$ be the GWAS p-value for variant $j$ in that set. At threshold $t$, where $0\leq t\leq 1$, variant $j$ is included if $p_j\leq t$. The included-variant count is $$ K(t)=\sum_{j=1}^{m} I(p_j\leq t), $$ where $I(\cdot)$ equals 1 when its condition is true and 0 otherwise. The score is formed from the included variants and their GWAS weights.

The definition is conditional on a specified clumped set. Comparing thresholds while holding that set and its clumping settings fixed makes the effect of thresholding clear: if $t_1\leq t_2$, then every variant included at $t_1$ is also included at $t_2$. Therefore $K(t_1)\leq K(t_2)$. Some workflows apply clumping separately after changing the threshold; the selected lead variants may then change as well. In that case, the sets need not be nested, and a count difference reflects both thresholding and a changed clumping result.

Expected Counts Under Null and Alternative Associations

For each variant, the probability of inclusion is the probability that its p-value falls at or below $t$. If $F_j(t)=P(p_j\leq t)$, then the expected count in a repeated-GWAS view of the procedure is

E[K(t)]=\sum_{j=1}^{m}P(p_j\leq t)=\sum_{j=1}^{m}F_j(t).

This result follows from adding the expectations of the indicator variables. It does not require the p-values to be independent. That matters in genetic data, where association statistics can be dependent because of linkage disequilibrium, even among variants that remain after clumping.

Under a null association, a p-value is often described as valid when it is super-uniform: for every $u$ between 0 and 1, $P(p_j\leq u)\leq u$. Exact equality $P(p_j\leq u)=u$ holds for an exactly uniform null p-value, as with a continuous exact test. Validity alone does not imply exact uniformity; p-values may be discrete or conservative. Thus the expected-count equality under the global null requires the exact-uniform assumption. With only super-uniform null p-values, the corresponding bound is an inequality.

\begin{aligned} \text{Exact-uniform global null:}\quad &E[K(t)]=mt,\\ \text{Super-uniform global null:}\quad &E[K(t)]\leq mt. \end{aligned}

These are expectations, not guarantees for a particular GWAS. Even under exact uniformity, the realized count can differ from $mt$. Further, the expectation formula does not require independence, but calculating the variance of $K(t)$ generally does depend on the joint behavior of the p-values.

Worked Example: Expected Count Under a Global Null

Suppose a hypothetical analysis has $m=100{,}000$ clumped variants, all null, and uses $t=0.0005$. First assume every null p-value is exactly uniform on $[0,1]$. The expected included count is

$$ E[K(t)]=mt=100{,}000(0.0005)=50. $$

So 50 is the expected number passing the threshold over repeated analyses under these assumptions; it is not a prediction that exactly 50 will pass in this particular analysis. Independence is not needed for this expectation.

If instead the null p-values are only known to be super-uniform, the justified calculation is the bound

E[K(t)]\leq mt=50.

It would be incorrect to claim that the expectation must equal 50 in this second case. For example, a p-value that is always 1 is super-uniform but has zero probability of being at most 0.0005.

When some variants are associated with the trait, their p-values need not follow the null distribution. For an alternative variant $j$, $F_j(t)$ depends on the association strength and the precision of the GWAS, among other factors. The same count identity still applies, but the global-null formula $mt$ does not describe the expected total count. In general, it is useful to separate the expected contributions from null and non-null variants.

E[K(t)] = \sum_{j\in\mathcal{N}}F_j(t) + \sum_{j\in\mathcal{A}}F_j(t),

where $\mathcal{N}$ is the set of null variants and $\mathcal{A}$ is the set of associated variants. This decomposition describes inclusion, not prediction quality: it does not say whether the retained weights estimate effects accurately or whether the resulting score predicts well in another sample.

Worked Example: Counting Variants at Several Thresholds

Consider a hypothetical fixed clumped set of 12 variants with the following GWAS p-values, already ordered from smallest to largest: 0.00002, 0.0004, 0.0018, 0.006, 0.009, 0.012, 0.021, 0.047, 0.08, 0.20, 0.63, and 0.91. Use the inclusion rule $p_j\leq t$.

At $t=0.001$, the first two p-values pass, so $K(0.001)=2$. At $t=0.01$, the first five pass, including 0.009 but not 0.012, so $K(0.01)=5$. At $t=0.05$, the first eight pass, including 0.047 but not 0.08, so $K(0.05)=8$.

The score at any of these thresholds uses the genotype and GWAS weight only for included variants. For example, at $t=0.01$, its form is

S_i(0.01)=\sum_{j:p_j\leq 0.01}G_{ij}\hat{\beta}_j,

where $G_{ij}$ is the appropriately coded genotype for individual $i$ at variant $j$, and $\hat{\beta}_j$ is its GWAS effect estimate. The example establishes how many terms are retained; it does not establish which threshold gives the most useful score.

Thresholds and Polygenic Architecture

A threshold acts on association evidence, not directly on the true effect size. A small p-value can result from a sufficiently large estimated effect, a sufficiently precise estimate, or both. Precision depends on features such as GWAS sample size and allele frequency. A p-value is not the probability that a variant is causal or that the null hypothesis is true. Consequently, a stringent threshold may exclude genuinely associated variants when their effects are small or their estimates imprecise.

The implications depend partly on architecture. In a relatively sparse architecture, a small number of variants may account for much of the genetic signal. A stringent cutoff may retain a useful proportion of the strongest signals while excluding many null variants. In a more diffuse architecture, many variants may contribute small effects. A stringent cutoff can then discard many contributing variants, while a permissive cutoff can bring more of them into the score. However, a permissive threshold also admits more null or poorly estimated variants. More included terms can therefore add useful information, estimation noise, or both.

The following calculation illustrates expected counts under two hypothetical architectures. It uses a fixed set of 1,000 clumped variants. For the null variants, assume exact-uniform p-values. For associated variants, suppose the p-value probabilities at the listed thresholds are as stated. These probabilities are illustrative assumptions, not universal values.

Worked Example: Comparing Sparse and Diffuse Architectures

In the sparse scenario, assume 20 associated variants and 980 null variants. Suppose an associated variant has probability 0.15, 0.55, or 0.95 of passing thresholds 0.001, 0.05, or 0.5, respectively. In the diffuse scenario, assume 200 associated variants and 800 null variants, with corresponding probabilities 0.005, 0.08, and 0.35. For each threshold, the expected null contribution is the number of null variants multiplied by $t$, and the expected associated contribution is the number of associated variants multiplied by the assumed alternative probability.

For the sparse scenario at $t=0.001$, the null contribution is $980(0.001)=0.98$, and the associated contribution is $20(0.15)=3$. Thus $E[K(0.001)]=0.98+3=3.98$. At $t=0.05$, the contributions are $980(0.05)=49$ and $20(0.55)=11$, giving $E[K(0.05)]=60$. At $t=0.5$, they are $980(0.5)=490$ and $20(0.95)=19$, giving $E[K(0.5)]=509$.

For the diffuse scenario at $t=0.001$, the contributions are $800(0.001)=0.8$ and $200(0.005)=1$, so $E[K(0.001)]=1.8$. At $t=0.05$, they are $800(0.05)=40$ and $200(0.08)=16$, so $E[K(0.05)]=56$. At $t=0.5$, they are $800(0.5)=400$ and $200(0.35)=70$, so $E[K(0.5)]=470$.

The counts show that a cutoff changes the balance between associated and null variants, and that the expected count depends on the assumed p-value distributions as well as on how many variants are associated. They do not show which scenario yields higher predictive accuracy. That depends on effect estimates, genotype correlations, and other features of the score and target population, not on the count alone.

Interpreting Sparsity and Comparing Thresholds

The threshold determines a discrete inclusion rule. With a fixed candidate set, raising $t$ cannot remove an included variant, so the score becomes less sparse in the sense that it has more nonzero-weight terms, or at least more terms selected for inclusion. It does not follow that the numerical score for every individual increases: genotype coding and effect-weight signs can make added terms positive or negative.

Thresholds are often considered as a set of candidate values rather than selected by an abstract rule that works for every trait. A threshold should be understood alongside the GWAS, the clumping choices, and the genetic architecture that is plausible for the trait. The same numerical cutoff can produce very different counts when the number of tested or clumped variants differs, or when GWAS sample size and association strength differ. In a fixed analysis, report the threshold and the number of retained variants so the score's sparsity is transparent.

A useful comparison keeps the clumping set and other construction choices fixed while varying $t$. If clumping is rerun for each threshold, report that fact, because the set being thresholded may also change. Likewise, p-values rounded to zero can make a strict cutoff comparison misleading: use unrounded p-values when available, since a displayed zero may represent a small positive value rather than exactly zero.

Specify the candidate set.
Record the variants retained after clumping and the p-value used for each variant.

Apply the cutoff consistently.
Include variant $j$ exactly when $p_j\leq t$, and count the included variants.

Interpret the count conditionally.
The count describes score sparsity for these GWAS results and clumping settings; it is not a direct measure of prediction accuracy.

Common Mistake and Key Takeaway

Common mistake: “The cutoff is more permissive, so the score must be better because it contains more genetic information.” This reasoning treats every additional term as useful information. In fact, some added variants may be null, and estimated weights may be noisy. The correction is to distinguish inclusion from usefulness: a larger $K(t)$ means a less sparse score, not necessarily a more accurate one. Conversely, a stringent threshold may omit many small but real effects under a diffuse architecture.

A related mistake is to assert that $mt$ is the expected number of null variants passing a threshold merely because their p-values are valid. That equality requires exact uniformity of each null p-value. Under super-uniform validity, the expected number is at most $mt$, and dependence among p-values does not change this expectation bound.

Key takeaway: Conditional on a fixed clumped set, threshold $t$ includes variants with $p_j\leq t$, and the expected count is the sum of their inclusion probabilities. Thresholds control score sparsity, while polygenic architecture and GWAS precision shape the trade-off between retaining associated variants and adding noise. Variant count alone does not identify the best threshold.

Check Your Understanding

Answer each question using the inclusion rule and assumptions stated in the tutorial.

A fixed set contains 4,000 null variants with exactly uniform p-values. What is the expected number passing $t=0.002$? Which assumption makes this an equality?
If the null p-values are only super-uniform, what can be concluded about the expected count in the same example?
Why does the expected-count formula not require independent p-values, and why might variance calculations require more information?
Explain why raising the threshold increases the included-variant count when the clumped set is fixed, but does not guarantee that every individual's score increases.
Why might a stringent threshold be problematic for a highly diffuse architecture? Why might a permissive threshold also be problematic?