What Does a Null Result Tell Us?
A burden test and SKAT can both return nonsignificant results for a gene, or disagree about whether the evidence is significant. Neither outcome can be interpreted by looking at the p-value alone. The tests summarize variant-level information in different ways, and a null result may reflect weak effects, limited information, an effect pattern poorly matched to the test, or the definition of the tested variant set.
A nonsignificant result means that the test did not reject its specified null hypothesis at the chosen significance threshold. It does not show that the null hypothesis is true, establish that every variant has no effect, or rule out an effect large enough to matter scientifically. To make a claim against effects of practical importance requires more: a clearly defined effect of interest, adequate precision, and an analysis designed to assess whether effects of that size are incompatible with the data.
For a set of variants, the global association null is typically that all modeled variant effects are zero. A burden test focuses on a weighted aggregate of variant effects. SKAT evaluates a variance-component alternative that can detect a range of effect patterns, including effects that differ in direction. Their p-values are calibrated for their respective test statistics under the null; the tests need not have equal power under any particular alternative.
The interpretations also depend on the analysis being valid. In particular, the phenotype model, covariate adjustment, genotype and variant-set definitions, and null calibration must be appropriate. Population structure, relatedness, or other features not adequately handled by the model can distort test calibration. A small p-value from a miscalibrated test is not reliable evidence, and a nonsignificant p-value from a low-power analysis is weak evidence against an effect.
Why the Tests Can Disagree
A simplified score model makes the difference in sensitivity clear. Suppose the set contains \(m\) variants, and the standardized variant score vector \(U=(U_1,\ldots,U_m)\) has approximately independent coordinates. Under the global null, take \(U_j\sim N(0,1)\). For this illustration, use equal weights and define a burden statistic by summing the scores, while the SKAT statistic sums their squares:
Under these simplifying assumptions, \(Z_B\sim N(0,1)\) under the null, and \(Q\) has a chi-square distribution with \(m\) degrees of freedom. In actual analyses, weights and correlations among variant scores affect the statistics and their null distributions; a calibrated burden or SKAT procedure must account for those features. The simplified model is useful for understanding why the tests can respond differently, not for substituting for the analysis-specific calibration.
Suppose the mean score under an alternative is \(\delta_j\), so \(U_j\sim N(\delta_j,1)\). Then the expected burden score is \(\sum_j\delta_j/\sqrt{m}\), while
The burden’s mean depends on the signed sum of effects: positive and negative contributions can cancel. The expected SKAT statistic depends on squared effects, so opposite signs do not cancel in this simplified expression. This does not make SKAT uniformly more powerful; the tests’ power depends on the effect pattern, weights, sample size, allele frequencies, and the full score covariance structure.
Worked Example: Opposing Scores and a Burden–SKAT Disagreement
Consider two equally weighted variants with observed standardized scores \(U_1=3.0\) and \(U_2=-3.0\). Use the simplified independent-score model above, with \(m=2\).
The burden statistic is
Its two-sided standard-normal p-value is \(1.000\). The SKAT statistic is
Under the null, \(Q\sim\chi^2_2\), whose upper-tail probability at 18.0 is \(e^{-18/2}=e^{-9}=0.0001234\). Thus, in this hypothetical illustration, SKAT is significant at a 0.05 threshold while the burden test is not. The opposing score directions cancel in the burden sum, whereas both contribute positively to \(Q\). This pattern can be consistent with heterogeneous or bidirectional effects, but the test results alone do not establish that the two variants have causal effects.
A burden result can also be significant while SKAT is not. A burden test can be well matched to a pattern in which many effects point in a common direction and contribute to a coherent aggregate. SKAT’s statistic instead asks whether the weighted squared score is unusually large relative to its null distribution. Because that reference distribution depends on the number and dependence of the scores, a moderate, coordinated shift can be strong evidence for a burden alternative without producing an extreme SKAT statistic.
Worked Example: A Coherent Burden Signal Without a Large SKAT Statistic
Suppose a hypothetical set has \(m=25\) equally weighted, independent standardized scores, each observed as \(U_j=0.50\). Then
The two-sided normal p-value is approximately \(0.0124\). For SKAT,
Under this simplified null, \(Q\sim\chi^2_{25}\). Since 6.25 is far below the null mean of 25, its upper-tail p-value is greater than \(0.99\), not small. The two tests therefore give very different results at the 0.05 level. This example illustrates the distinct test statistics under stated assumptions; it is not a claim that the same pattern or p-values would occur with dependent scores or unequal weights.
Reading Two Nonsignificant Results
If both tests are nonsignificant, the immediate conclusion is limited: neither test detected association at its chosen threshold for this set, phenotype, model, and sample. The result does not identify which explanation is correct. For example, the effects may be absent, too small for the study to detect, concentrated in a few variants that receive little effective weight, or arranged in a pattern that neither test captures well.
Power is specific to a test and an alternative. A power calculation is the probability of rejecting the null under a specified effect pattern and analysis, at a specified significance threshold. A study can have reasonable power for effects aligned in one direction but limited power for effects with mixed directions, or the reverse. Saying “the study had high power” without saying for what effect pattern and magnitude is therefore incomplete. In rare-variant analyses, the number of carriers and the distribution of their genotypes can also limit information, even when the overall sample size is large.
Worked Example: Two Nonsignificant Tests Are Not Proof of No Effect
Suppose a hypothetical analysis of a prespecified variant set gives a burden p-value of \(0.16\) and a calibrated SKAT p-value of \(0.35\). The analysis uses a significance threshold of \(0.05\). Both tests fail to reject their null hypotheses because \(0.16>0.05\) and \(0.35>0.05\).
The defensible conclusion is that neither test detected association at that threshold. It is not correct to conclude that the gene has no effect. To judge what the result says about effects of practical importance, one would also need information such as the study’s power for relevant effect patterns and the uncertainty around effect estimates. If the study had poor power for plausible patterns, these p-values provide little basis for excluding those patterns. If the study had strong power for a prespecified, scientifically important range, a null result could be more informative about that range—but a nonsignificant p-value alone still does not perform that evaluation.
Variant-Set Boundaries and Interpretation
The set definition is part of the question being tested. Changing which variants are included can change the burden aggregate, the SKAT statistic, and the null distribution used to calibrate each test. Adding variants with opposing effects can reduce a burden sum. Adding variants can also change SKAT’s statistic and its reference distribution; a larger raw \(Q\) does not necessarily mean stronger evidence because its calibrated tail probability depends on the set and score covariance. A null result for one boundary therefore does not automatically generalize to a broader or narrower set.
Worked Example: A Boundary Change Alters Both Tests
Consider a hypothetical set of independent standardized scores with equal weights. For the first two variants, let \(U=(2.0,0.2)\). Under the simplified model, the burden statistic and two-sided p-value are
The SKAT statistic and its null p-value are
Now suppose a prespecified expansion of the set adds a third variant with score \(-1.8\). The burden calculation becomes
For SKAT, \(Q=4.04+(-1.8)^2=7.28\). Using the \(\chi^2_3\) null distribution, the upper-tail p-value is approximately \(0.0634\). Neither test reaches 0.05 for either set in this illustration, but the statistics and p-values change when the boundary changes. These comparisons assume independent standardized scores and the corresponding chi-square null distributions. With real data, one must recalculate the appropriate null distribution for each set rather than carry over the calibration from the smaller set.
This example illustrates why a null result is conditional on the variant-set definition. Before interpreting it, check what biological and statistical criteria defined inclusion, whether those criteria were applied consistently, and whether the analysis’s calibration corresponds to the actual tested set. A boundary chosen after examining results can also make a nominal p-value harder to interpret as confirmatory evidence.
Diagnostics, Comparisons, and Common Mistakes
When burden and SKAT disagree, treat the difference as a prompt to inspect the analysis, not as a reason to select whichever result is more attractive. Useful checks include confirming that both tests used the intended variant set and comparable covariates; reviewing null calibration; checking whether the variant scores show opposing directions or are driven by a small number of variants; and verifying that the weights and score dependence used in calibration match the analysis. These checks help distinguish a result that reflects the tests’ different alternatives from one caused by a data or modeling problem.
The earlier material on variant-level score contributions and influence diagnostics motivates examining which variants contribute to a gene-level statistic. Such examination can clarify a burden–SKAT discrepancy, but searching among variants or alternative sets after seeing the results changes the inferential problem. Any selected follow-up analysis should be interpreted with that selection in mind.
A common mistake is to say, “The burden test is nonsignificant, so the gene has no association,” or “SKAT is significant, so the gene has a real effect.” The first statement overinterprets failure to reject; the second treats a test result as proof and ignores model assumptions, calibration, and the analysis plan. The correction is to name the test, set, phenotype, threshold, and result, then describe the conclusion as evidence for or against the test’s specified null—not as proof about every possible genetic effect.
The main contrast is not that one test is generally superior. Burden tests are most responsive to alternatives with a coherent weighted aggregate, while SKAT can be more responsive to heterogeneous effects that do not align in direction. Neither necessarily has high power for every plausible architecture. When both are null, report that neither detected association and qualify the conclusion using power and set definition. When one is significant and the other is not, explain how their different summaries could produce that pattern, while checking that both tests were validly calibrated.
Check Your Understanding
Answer each question, then compare your reasoning with the solutions below.
- A burden test is nonsignificant and SKAT is significant. Give one reason this pattern can occur and one conclusion that is not justified.
- Why can effects in opposite directions weaken a burden signal while still contributing to SKAT?
- Both tests are nonsignificant. What can be concluded, and what additional information is needed to assess evidence against effects of practical importance?
- Why must SKAT be recalibrated if a variant is added to the tested set?
Solutions. (1) Opposing variant effects can cancel in the burden aggregate while their squared score contributions remain positive in SKAT. The result does not prove that the gene has an effect or that any variant is causal. (2) A burden sum preserves signs, so positive and negative contributions may offset; a sum of squared scores does not cancel by sign. (3) Neither test detected association at its chosen threshold for the tested set and model. Assessing effects of practical importance also requires information about power for relevant effect patterns and uncertainty in estimates. (4) Adding a variant changes the statistic and generally changes the score covariance and null distribution, so the original calibration may no longer apply.