Why the Effect Pattern Matters
A gene-level test combines evidence across variants, but different tests combine that evidence in different ways. If the included variants are expected to affect a trait in the same direction, a burden test can add their contributions together. If effects may point in opposing directions, those contributions can cancel; SKAT instead accumulates squared variant-level scores. When the direction pattern is not known confidently, SKAT-O provides a calibrated way to combine burden-like and SKAT-like evidence.
The choice is about which alternative patterns a test is likely to detect efficiently, not about which test is universally best. It should be informed by the scientific context and made without treating a test’s eventual p-value as proof that its assumptions about effects were correct.
Set-Up and Test Definitions
Consider \(m\) variants in a gene and \(n\) individuals. Let \(G_{ij}\) be the allele count for individual \(i\) at variant \(j\), usually coded as 0, 1, or 2 copies of the effect allele. Let \(\beta_j\) denote the variant’s effect on the trait under the chosen model. The trait may be quantitative or binary; the outcome model and its covariates must be appropriate for that trait. Under the null hypothesis of no association between the variants in the tested set and the trait, all \(\beta_j=0\).
After fitting the null model with appropriate covariates, let \(U_j\) be the score for association at variant \(j\). A score is a data-based quantity that tends to have a consistent sign with the fitted variant effect, but it is noisy: its sign alone does not establish the true direction of an effect. Let \(v_j\geq 0\) be a prespecified weight for variant \(j\). For clarity, the definitions below use \(v_j\) as the multiplier of each variant score.
These statistics are evaluated under the null using their appropriate null distributions. In particular, the variant scores are generally correlated, for example because variants are in linkage disequilibrium. A raw statistic from one test cannot be compared directly with a raw statistic from another to decide which gives stronger evidence: each needs its own null calibration, and SKAT-O’s calibration must also account for searching across \(\rho\).
The standard power motivations for the tests can be summarized as follows. A burden test is especially well matched when effects are expected to act in a common direction after allele coding and weighting. SKAT is designed to be sensitive to a range of effect patterns, including mixtures of positive and negative effects, and to situations where only some variants have effects. SKAT-O is useful when there is uncertainty about which of these patterns is more plausible. These are statements about power under alternatives, not additional assumptions needed for a correctly calibrated null test.
| Plausible effect pattern | Why a test may suit it | Main caution |
|---|---|---|
| Effects tend to share a direction | A weighted sum can accumulate their score contributions. | Opposing or poorly matched contributions can reduce the sum. |
| Effects may have different directions or only some variants may matter | Squaring variant-level contributions avoids cancellation between signs. | Do not assume every variant has an effect just because the test includes it. |
| The direction and configuration are uncertain | SKAT-O combines burden-like and SKAT-like evidence with calibrated search. | Its p-value must reflect the search; it is not the unadjusted minimum p-value. |
Worked Examples: How the Scores Combine
Worked Example: Effects Expected to Act in One Direction
Suppose a hypothetical gene contains three variants with equal weights \(v_1=v_2=v_3=1\). The observed score vector is \(U=(2,2,2)\). This pattern is compatible with concordant effects, although the score signs by themselves do not prove that the true effects share a direction.
The burden score before squaring is \(2+2+2=6\), so \(Q_B=(2+2+2)^2=6^2=36\). For SKAT, \(Q_S=2^2+2^2+2^2=4+4+4=12\).
The larger numerical value of \(Q_B\) does not, by itself, show that the burden test has stronger evidence. The two statistics have different null distributions. The scenario explains why a burden test may be efficient when effects are concordant; the calibrated p-values are needed to assess evidence in data.
Worked Example: Opposing Scores and Cancellation
Now suppose the hypothetical score vector for three equally weighted variants is \(U=(3,-3,0)\). Opposing effects are plausible in this scenario. The burden score is \(3+(-3)+0=0\), giving \(Q_B=(3-3+0)^2=0^2=0\).
SKAT instead gives \(Q_S=3^2+(-3)^2+0^2=9+9+0=18\). The two nonzero scores cancel in the burden sum but contribute positively after squaring. This shows the mechanism by which SKAT can retain evidence when directions differ. It does not mean that this particular observed score vector establishes an association; the statistic must still be compared with its null distribution.
Worked Example: Uncertain Pattern and SKAT-O
Suppose a third hypothetical analysis has equal weights and score vector \(U=(2,-1,1)\). Neither a fully concordant pattern nor a simple all-effects-in-one-direction assumption is persuasive in advance. The burden statistic is \(Q_B=(2-1+1)^2=2^2=4\), while the SKAT statistic is \(Q_S=2^2+(-1)^2+1^2=4+1+1=6\).
At a specified illustrative value \(\rho=0.5\), the mixed statistic is \(Q_{0.5}=(1-0.5)(6)+0.5(4)=3+2=5\). That arithmetic gives the statistic at this one value of \(\rho\); it does not give the SKAT-O p-value. In an actual SKAT-O analysis, the candidate \(\rho\) values and the search procedure are calibrated under the null. For illustration, suppose a valid analysis reports calibrated p-values of 0.31 for burden, 0.24 for SKAT, and 0.28 for SKAT-O. At a prespecified significance level of 0.05, none is statistically significant. The p-values, not the numerical ordering of 4, 6, and 5, are the relevant summaries of evidence against the null.
Why Opposing Effects Change the Burden Test
The contrast between the tests follows directly from their formulas. Define \(z_j=v_jU_j\), the weighted score contribution from variant \(j\). Then \(Q_B=(\sum_j z_j)^2\), whereas \(Q_S=\sum_j z_j^2\). Expanding the burden statistic gives
Each cross-product \(z_jz_k\) is positive when two contributions have the same sign and negative when they have opposite signs. Thus, cross-products can reinforce a burden statistic when weighted scores are aligned and reduce it when scores oppose one another. SKAT contains the squared contributions but not these cross-products, so a contribution cannot subtract from another merely because their signs differ. This algebra explains the different sensitivity; it does not establish which pattern is present in a real gene.
A Practical Selection Strategy
Selection should begin with a scientific description of plausible effects, not a search through results for whichever p-value happens to be smallest. A disciplined choice is particularly important because the variants, weights, covariates, and analysis plan can all affect the evidence. The following sequence makes the reasoning explicit.
Define which variants are in the tested set and how \(v_j\) is chosen. The weights should reflect the intended analysis, rather than being changed after seeing which choice gives the smallest p-value.
Ask whether external biological knowledge supports effects acting in a shared direction, whether different directions or sparse effects are plausible, or whether there is substantial uncertainty. This is a scientific judgment, not something established by the signs of noisy scores alone.
Use burden when a common direction is a defensible expectation; use SKAT when mixed directions or a range of variant effects is plausible; use SKAT-O when uncertainty between burden-like and SKAT-like alternatives motivates a hybrid test.
Fit an appropriate null model, adjust for relevant covariates, account for the correlation among variant scores, and use a valid p-value procedure for the chosen test. For SKAT-O, the procedure must include the calibration for the \(\rho\) search.
Explain why the test was used, give its calibrated result, and describe the tested variant set and weights. Avoid interpreting a significant gene-level result as identifying a particular causal variant or proving a particular direction pattern.
The validity of a test result also depends on the analysis conditions, not just the choice among these three statistics. The null model must represent the trait and account for relevant covariates; dependence among variant scores must be represented in calibration. In a binary-trait analysis, for example, the null model must be suitable for the case-control outcome. If these conditions or the method’s small-sample calibration are inadequate, choosing the test that seems best matched to the effects does not repair the problem.
Common Mistakes and Interpretation
- Choosing by observed score signs alone. A score is noisy, especially for rare variants. The correction is to motivate the test from prior scientific knowledge and the intended alternative, not to treat observed signs as known true effects.
- Assuming that a burden test always has more power when many variants are included. A large set can contain effects in opposing directions. The correction is to consider whether weighted contributions are plausibly aligned; if they are not, their cross-products can reduce the burden statistic.
- Assuming SKAT proves that effects differ in direction. SKAT can be sensitive to mixed directions, but a significant SKAT result alone does not establish that the true effects have both signs. The correction is to report the association evidence separately from claims about the underlying configuration.
- Comparing raw statistics across tests. The burden, SKAT, and SKAT-O statistics do not share a common reference distribution. The correction is to compare calibrated p-values or other appropriate inferential summaries, not their raw numerical sizes.
- Taking the smallest p-value from a SKAT-O search without adjustment. Searching across \(\rho\) affects the null behavior of the selected result. The correction is to use SKAT-O’s calibrated p-value, not the unadjusted minimum among candidate tests.
- Reading a gene-level result as a variant-level conclusion. A test evaluates the combined signal for the chosen set; it does not by itself identify which variant drives that signal. The correction is to keep the conclusion at the level supported by the test.
Check Your Understanding
For each question, explain the reasoning rather than naming a test alone.
- A set of variants is expected, from prior biological evidence, to affect a quantitative trait in the same direction after allele coding. Which test is naturally motivated, and what could make its aggregate score smaller than expected?
- For equal weights and scores \(U=(2,-2)\), calculate \(Q_B\) and \(Q_S\). What feature of the score pattern explains the difference?
- Why is a larger raw SKAT statistic than burden statistic not enough to conclude that SKAT gives stronger evidence against the null?
- When is SKAT-O a useful choice, and why is the minimum unadjusted p-value across its candidate \(\rho\) values not its calibrated p-value?
- A significant SKAT result is reported for a gene. What can be concluded about the tested gene-level set, and what cannot be concluded about the directions or identities of the causal variants?