Why a Frequency Cutoff Changes a Burden Test
A burden test combines information across variants in a defined set. A minor allele frequency (MAF) cutoff is one way to define that set: include variants whose MAF is at or below a chosen value. Changing the cutoff can add or remove variants, so it changes the burden score and its null variance. The resulting test may become more significant, less significant, or remain similar.
This change is not simply a matter of gaining or losing sample size. Each included variant contributes a score in a direction determined by its observed association with the phenotype. If an added variant's score points against the scores already in the burden, the aggregate can shrink. If its score points in the same direction, the aggregate can grow. The interpretation also depends on the weights and covariance used in the test.
MAF is the frequency of the less common allele in the population or sample being analyzed. For a biallelic variant with allele frequency \(f_j\), its MAF is \(\min(f_j,1-f_j)\). In practice, the frequency estimate, ancestry group, quality-control rules, and treatment of variants with missing or uncertain genotypes should be specified. The boundary convention here is \(p_j\leq c\): a variant exactly at the cutoff is included.
The cutoff defines the tested set; it does not establish that every included variant is functional or causal. Nor does it make the included variants' effects identical. The burden test's directional-effect assumption and its potential vulnerability to opposing effects were covered in the earlier material on directional effects and power loss. Here the focus is how changing the MAF rule changes the test.
Statistic and Assumptions for Comparing Cutoffs
Let \(U_j\) be the score contribution for variant \(j\) under the null model, and let \(U_S\) be the vector of contributions for the selected variants. Let \(w_S\) be the vector of their prespecified burden weights and \(V_S\) the null covariance matrix of \(U_S\). A burden score statistic is
Under the null hypothesis of no association, the score vector is approximately multivariate normal with mean zero and covariance \(V_S\). If the model and covariance are correctly specified and the score approximation is adequate, the standardized aggregate in the numerator is approximately standard normal, so \(Q(c)\) is approximately distributed as chi-square with one degree of freedom. This is the usual large-sample calibration for a one-dimensional burden score.
For the examples below, each variant has equal weight, each score \(U_j\) has null variance 1, and distinct score contributions have covariance 0. These simplifying assumptions give \(Q(c)=(\sum_{j\in S(c)}U_j)^2/|S(c)|\). Real variants can be correlated through linkage disequilibrium, so their covariance must generally be included; setting it to zero when it is not zero can miscalibrate the statistic. The examples use hypothetical score summaries to isolate the effect of set membership.
Worked Comparisons
Worked Example: Adding an Opposing Variant Weakens the Result
Suppose three hypothetical variants have MAFs \(0.004\), \(0.009\), and \(0.012\), with score contributions \(2.2\), \(1.8\), and \(-2.0\), respectively. Use the equal-weight, independent-score assumptions above. Compare cutoffs \(c=0.005\), \(0.010\), and \(0.015\).
At \(c=0.005\), only the first variant is selected. Thus \(Q=(2.2)^2/1=4.84\), giving a chi-square-one p-value of approximately \(0.0278\).
At \(c=0.010\), the first two variants are selected. Their scores add to \(2.2+1.8=4.0\), so \(Q=(4.0)^2/2=8.00\). The chi-square-one p-value is approximately \(0.00468\).
At \(c=0.015\), all three variants are selected. The aggregate score is \(2.2+1.8-2.0=2.0\), and \(Q=(2.0)^2/3=1.333\), with p-value approximately \(0.248\). The third variant's negative score offsets part of the previous aggregate, while the null variance also increases. In this hypothetical example the largest cutoff produces the weakest evidence against the null.
These p-values are calibrated separately under the stated assumptions. They do not show that one cutoff is inherently preferable; the cutoffs would need scientific justification and, ideally, prespecification.
Worked Example: Adding an Aligned Variant Strengthens the Result
Now suppose three hypothetical variants have MAFs \(0.003\), \(0.008\), and \(0.0104\), with score contributions \(1.5\), \(1.4\), and \(1.3\). Keep the same equal-weight, independent-score assumptions. Compare \(c=0.010\) with \(c=0.011\).
At \(c=0.010\), the first two variants are included, and the third is excluded because \(0.0104>0.010\). The aggregate is \(1.5+1.4=2.9\), so \(Q=(2.9)^2/2=4.205\), with p-value approximately \(0.0403\).
At \(c=0.011\), all three variants are included. The aggregate is \(1.5+1.4+1.3=4.2\), and \(Q=(4.2)^2/3=5.88\), with p-value approximately \(0.0153\). Here the newly included score is aligned with the existing aggregate; despite the larger null variance, the test statistic increases.
The comparison illustrates why the direction and size of a variant's score matter, not just its frequency. It does not justify choosing \(0.011\) because it gives a smaller p-value. If the cutoff was selected after examining results, the reported p-value does not account for that selection.
Worked Example: A Variant Close to the Cutoff
In a hypothetical sample of 500 diploid individuals, suppose a variant is observed on 10 of the 1,000 chromosome copies. Its estimated MAF is \(10/1000=0.010\). Under the stated \(p_j\leq c\) convention, it is included at \(c=0.010\). If instead 11 copies carry the minor allele, the estimate is \(11/1000=0.011\), and it is excluded at \(c=0.010\) but included at \(c=0.011\).
To see the statistical consequence, suppose three hypothetical variants have minor-allele counts 8, 10, and 11, with score contributions \(1.6\), \(1.7\), and \(-1.2\). The first two frequencies are \(0.008\) and \(0.010\); the third is \(0.011\). At \(c=0.010\), the aggregate is \(1.6+1.7=3.3\), giving \(Q=(3.3)^2/2=5.445\) and a p-value of approximately \(0.0196\). At \(c=0.011\), the third variant enters, the aggregate becomes \(3.3-1.2=2.1\), and \(Q=(2.1)^2/3=1.47\), with p-value approximately \(0.225\).
The arithmetic shows the impact of the stated cutoff rule, not the uncertainty in the frequency estimate. With finite samples, an estimated MAF near the boundary may be sensitive to a small number of allele counts, genotype errors, or ancestry composition. A variant's inclusion can therefore be unstable even when the rule itself is unambiguous.
Why the Null Calibration Has This Form
The one-degree-of-freedom calibration follows directly from the null distribution of the aggregate score. For a fixed selected set and weights, define \(A=w_S^{\mathsf T}U_S\). Under the score approximation, \(U_S\) has mean zero and covariance \(V_S\), so
Consequently, \(A/\sqrt{w_S^{\mathsf T}V_Sw_S}\) is approximately standard normal under the null. Squaring it gives \(Q(c)\), approximately chi-square with one degree of freedom. This justification requires a nonzero aggregate variance and a sufficiently accurate score approximation. A poorly estimated covariance matrix, extreme sparsity, or a small effective number of carriers can make the approximation unreliable; the simple examples do not address such complications.
Diagnostics, Alternatives, and Limitations
When results change across fixed cutoffs, first verify the variant lists rather than interpreting p-values alone. Check the MAF source and ancestry definition, the exact boundary rule, genotype quality, missingness, and whether all tests used the same participants and phenotype model. Confirm that the covariance accounts for dependence among variants and that weights were handled consistently. A discrepancy caused by different samples or weights is not solely a cutoff effect.
The most direct comparison is between a small number of scientifically justified, prespecified cutoffs, reporting the selected variants, aggregate score, variance, statistic, and p-value for each. A more permissive cutoff includes more variants but may add variants with opposing effects or poorly supported frequency estimates. A stricter cutoff excludes those variants, but can also discard genuinely associated variants and reduce the amount of information in the set. Neither direction guarantees greater power.
A fixed-cutoff burden test is also different from testing variants one at a time: the burden test targets a combined, directional signal for the selected set, whereas individual tests ask about separate variants and may be noisy for rare alleles. Defining separate sets by a prespecified biological or functional criterion is another option, but it changes the scientific target and can create additional testing. Methods that search across cutoffs are a distinct approach and require calibration for that search; they are not equivalent to reporting the ordinary p-value at whichever cutoff looks most favorable.
A cutoff does not resolve heterogeneity in effect direction, uncertainty about which variants are functional, or differences in allele frequency across ancestry groups. A single pooled frequency can yield a different inclusion set from ancestry-specific frequencies. The selected set and its interpretation should therefore be tied to the population and frequency definition used. Association of a burden score is not, by itself, evidence that every included variant affects the phenotype or that any association is causal.
Common Mistake
Wrong reasoning: “The higher cutoff gave a smaller p-value, so it is the correct cutoff and proves the added variants are associated.” This confuses a result with a selection rule. The statistic changed because the set and its aggregate changed; choosing the cutoff after inspecting the results introduces a search that the ordinary p-value does not reflect.
Correction: Set the cutoff using a scientific rationale before examining the association results, or account for the search over cutoffs with an appropriate procedure. Report how the set, weights, and test statistic were defined. A smaller p-value is stronger evidence against the null only for the test as specified; it does not validate the cutoff or identify causal variants.
Check Your Understanding
Answer each question using the fixed-cutoff rule \(p_j\leq c\). Solutions are given below.
- A variant has MAF \(0.006\). Is it included at cutoffs \(0.005\), \(0.006\), and \(0.010\)?
- Under the equal-weight, independent-score setup, two included variants have scores \(1.0\) and \(-1.0\). Calculate \(Q\). What does the cancellation illustrate?
- A variant's estimated MAF is \(0.0102\). Explain why it is excluded at \(c=0.010\), and name one reason its inclusion could be sensitive in a real analysis.
- Why is it not valid to choose the cutoff that gives the smallest ordinary p-value and report that p-value as if the cutoff had been fixed in advance?
Solutions. (1) It is excluded at \(0.005\), included at \(0.006\) because equality is included, and included at \(0.010\). (2) \(Q=(1.0-1.0)^2/2=0\). The opposing scores cancel in the aggregate, even though each score is nonzero. (3) Since \(0.0102>0.010\), the variant is excluded. Its estimated frequency could be affected by a small number of allele counts, genotype error, or the ancestry composition used to estimate frequency. (4) Searching over cutoffs and selecting the smallest p-value creates selection beyond the single fixed-cutoff test; the ordinary p-value does not account for that search.