Why the Permutation Scheme Matters
A pathway test compares an observed pathway statistic with what would be expected under a specified null hypothesis. A permutation test constructs that reference distribution by rearranging some part of the data while keeping other parts fixed. The rearrangement is not a neutral computational choice: it defines which features of the data are treated as irrelevant under the null.
In pathway analysis, this matters because gene statistics may depend on shared phenotypes, correlated genotypes, linkage disequilibrium (LD), gene size, and annotation coverage. A permutation that breaks a dependence present in the data can produce an inappropriate reference distribution. Conversely, a permutation that preserves too much may fail to represent the null being tested. The scheme should follow the estimand and the data-generating assumptions, not just the convenience of the available table.
Specify What Is Being Permuted
Let \(T\) be the pathway statistic, such as a set-level sum or a rank-based statistic, and let \(T_{\mathrm{obs}}\) be its value in the observed data. A permutation scheme must specify which object is rearranged, what remains fixed, and which arrangements are allowed. Three common choices are phenotype permutation, genotype permutation, and gene-label permutation.
| Scheme | What is rearranged | What it typically preserves | Central assumption |
|---|---|---|---|
| Phenotype permutation | Phenotypes across individuals | Genotypes and their LD structure; fixed gene and pathway definitions | Phenotypes are exchangeable across the permuted individuals under the null |
| Genotype permutation | Whole genotype profiles across individuals | Phenotypes; within-person relationships among the jointly permuted variants | Genotype profiles are exchangeable across the permuted individuals under the null |
| Gene-label permutation | Gene scores or gene labels relative to pathway membership | The observed score distribution and usually pathway size | Genes are exchangeable under the null, possibly only within defined strata |
For an upper-tail test, an exact permutation p-value is the fraction of allowed arrangements with a statistic at least as large as the observed statistic. If \(K\) arrangements are allowed, including the observed arrangement, then
When only \(B\) random permutations are sampled, a common Monte Carlo estimate is
The added one accounts for the observed arrangement and prevents a reported Monte Carlo p-value of zero. For a lower-tail or two-sided test, the comparison must instead match the chosen alternative and the definition of extremeness. Ties are counted as at least as extreme, making the usual test conservative when the permutation distribution is discrete.
Phenotype Permutations
A phenotype permutation keeps each person's genotype data fixed and reallocates phenotype values among individuals. In a pathway analysis, a replicate should generally rerun the association analysis and gene-scoring steps affected by the phenotype, then recalculate the pathway statistic. Permuting only a final list of significant genes does not reproduce the variation in the statistics that generated that list.
Phenotype permutation can retain the observed LD structure because the genotype data are not changed. It also keeps fixed the gene definitions and pathway membership. Its basic exchangeability assumption is that, under the null being tested, the phenotype values could have been assigned to the permuted individuals in any allowed order with the same probability. This may hold in a simple, unstratified setting, but often fails when phenotype distributions differ by covariates, ancestry, family, site, or other design features.
Worked Example: An Exact Phenotype Permutation
Hypothetical data. Four unrelated individuals have genotypes \(G=(0,0,1,1)\) at a variant and phenotypes \(Y=(1,2,3,4)\). For illustration, use the difference in mean phenotype between genotype groups as the association statistic, and use its absolute value for a two-sided test. The observed difference, mean for genotype 1 minus mean for genotype 0, is
Under the simple null that the four phenotypes are exchangeable across individuals, hold \(G\) fixed and assign every pair of phenotype values in turn to the two individuals with genotype 1. There are \(\binom{4}{2}=6\) assignments. The absolute differences for assignments \(\{1,2\}\), \(\{1,3\}\), \(\{1,4\}\), \(\{2,3\}\), \(\{2,4\}\), and \(\{3,4\}\) are, respectively, \(2,1,0,0,1,\) and \(2\). Thus,
The observed assignment is one of the two equally extreme arrangements. This p-value is not evidence that the null hypothesis has a 33.33% probability of being true; it is the null probability of an equally or more extreme statistic under the stated permutation scheme. In a real pathway test, the statistic would be recalculated for each permuted phenotype assignment across the relevant genes and pathway.
Genotype Permutations
A genotype permutation keeps phenotypes fixed and reallocates genotype profiles among individuals. To retain LD among variants, permute each individual's whole genotype row jointly, rather than independently shuffling each variant column. Joint row permutation preserves the observed combinations of genotypes across the permuted variants. Independently permuting columns can create genotype combinations that were not present and destroy correlations that affect gene and pathway statistics.
This scheme requires genotype profiles to be exchangeable over the individuals being permuted under the null. Population structure can invalidate unrestricted permutation because genotype distributions vary by ancestry, while phenotype distributions may also vary by ancestry. Relatedness, family-based sampling, and covariate adjustment can create additional constraints. Permuting within appropriate strata may help only when exchangeability within those strata is defensible; it does not automatically correct a misspecified null.
Worked Example: Permuting Whole Genotype Profiles
Hypothetical data. Suppose two variants in a gene have genotype rows \((0,0),(1,1),(1,2),(2,2)\) for four individuals, and the fixed phenotype values are \(Y=(0,1,2,3)\). For a simple illustration, define the gene statistic as the sum of the absolute centered cross-products across the two variants:
Here \(\bar{Y}=1.5\). The first variant has mean \(1\), so its centered cross-product is \(({-1.5})({-1})+({-0.5})(0)+(0.5)(0)+(1.5)(1)=3\). The second variant has mean \(1.25\), giving \(({-1.5})({-1.25})+({-0.5})({-0.25})+(0.5)(0.75)+(1.5)(0.75)=3.5\). Therefore, \(T_{\mathrm{obs}}=|3|+|3.5|=6.5\).
Now permute the genotype rows in the order \((2,1,4,3)\), giving rows \((1,1),(0,0),(2,2),(1,2)\), while leaving \(Y\) fixed. The first variant is now \((1,0,2,1)\), with mean \(1\); its centered cross-product is \(0+0.5+0.5+0=1\). The second is \((1,0,2,2)\), with mean \(1.25\); its centered cross-product is \(0.375+0.625+0.375+1.125=2.5\). Thus this permutation gives \(T=|1|+|2.5|=3.5\).
The same four two-variant genotype rows remain in the permuted data, so their within-row pattern is retained. This one rearrangement illustrates a replicate, not a p-value: inference would require comparing \(T_{\mathrm{obs}}\) with the statistics from all allowed permutations or a suitable random sample of them. A pathway statistic may require recalculating gene scores for many genes in each replicate.
Gene-Label Permutations
A gene-label permutation leaves the gene scores fixed but changes which scores are assigned to pathway membership, or equivalently reassigns pathway labels relative to the scores. For a pathway containing \(m\) genes, an unrestricted version samples \(m\) genes from the eligible gene universe and compares their statistic with the observed pathway statistic. This preserves the score distribution and set size, but it does not, by itself, preserve relationships between a gene's score and its properties.
Unrestricted gene exchangeability is a strong assumption. Genes can differ in length, number of mapped variants, annotation coverage, and LD-related structure. Those features can affect the distribution and dependence of gene scores. If the probability of a high score differs across such features, treating every gene label as equally exchangeable can produce a poorly calibrated reference distribution. One response is to permute within prespecified strata or match genes on relevant features, but the resulting null is conditional on that matching scheme and is valid only if exchangeability within the strata is reasonable.
Worked Example: Exact Gene-Label Permutation
Hypothetical data. Five eligible genes have fixed scores \(3.2, 2.4, 1.1, 0.7,\) and \(0.2\). A pathway contains two genes, and its statistic is the sum of their scores. Suppose the observed pathway contains the genes scored \(3.2\) and \(2.4\), so
Under unrestricted gene-label exchangeability, there are \(\binom{5}{2}=10\) equally likely two-gene subsets. Their sums are \(5.6, 4.3, 3.9, 3.4, 3.5, 3.1, 2.6, 1.8, 1.3,\) and \(0.9\). Each is obtained by adding one pair: for example, \(2.4+1.1=3.5\) and \(2.4+0.7=3.1\). Only one sum is at least as large as \(5.6\), so the exact upper-tail p-value is
This calculation is exact only for the stated null in which every two-gene subset is equally likely. If genes with different annotation coverage or variant counts have systematically different score distributions, that assumption may fail; a matched or stratified permutation would answer a different, conditional question.
Choosing and Checking a Null
The three schemes answer related but distinct questions. Phenotype permutation asks what pathway statistic would arise if phenotypes could be reassigned among individuals while genotypes remained fixed. Genotype permutation asks what would arise if genotype profiles could be reassigned while phenotypes remained fixed. Gene-label permutation asks what would arise if pathway membership were unrelated to the observed score pattern among exchangeable genes. These are not interchangeable ways to obtain a generic “random” pathway result.
The key justification for a permutation test can be stated directly. Under the null, suppose every allowed rearrangement of the observed data is equally likely. Then the rank of the observed statistic among the \(K\) rearranged statistics is uniform when there are no ties. Counting the observed arrangement and all arrangements at least as extreme as it gives a p-value whose null probability of being no greater than any threshold \(\alpha\) is at most \(\alpha\). With ties counted as extreme, the test is generally conservative. This argument fails if the null does not make the allowed arrangements exchangeable.
Before running a permutation test, make the following decisions explicit:
Specify the pathway estimand, the direction of the test, and exactly how \(T\) is calculated.
Decide whether the null concerns phenotype assignment, genotype assignment, or gene membership relative to scores.
For example, joint genotype-row permutation preserves the observed variant combinations within a row, whereas independent variant shuffles do not.
Consider covariates, ancestry, relatedness, study design, gene properties, and any constraints that make some rearrangements implausible under the null.
For each allowed replicate, repeat the association and scoring steps that depend on the permuted data before calculating the pathway statistic.
Covariate-adjusted analyses need particular care. Raw phenotype values are generally not exchangeable across individuals when their distributions differ by covariates. Permuting residuals from a fitted model is sometimes used, but is not automatically an exact solution: its validity depends on the model, error structure, and permutation procedure. Likewise, permuting genes within broad bins does not ensure exchangeability if important score-generating differences remain within those bins. State the assumptions and interpret the resulting p-value relative to that null.
Common Pitfall and Practical Interpretation
Wrong reasoning: “All three approaches randomize the data, so they should give equivalent pathway p-values.” Correction: each approach holds different information fixed and therefore represents a different null. Phenotype permutation retains the observed genotype structure; whole-profile genotype permutation retains joint genotype patterns; gene-label permutation retains the score distribution while changing its relationship to pathway membership. Their p-values need not agree, and agreement is not required.
A permutation p-value is calibrated only to the extent that the specified rearrangements are valid under the null. In particular, breaking LD or ignoring annotation-related differences may make a null distribution too narrow or otherwise misshapen. The aim is not to preserve every feature of the data, but to preserve the dependencies that remain relevant under the particular null being tested. Report the permuted object, the constraints on rearrangements, the number of permutations, and the statistic used.
Check Your Understanding
For each question, identify the relevant null assumption or data feature.
- In a phenotype permutation, why should association and gene-scoring steps affected by the phenotype generally be rerun for every replicate?
- What relationship among variants is preserved by permuting whole genotype rows, and what can be lost by shuffling each variant independently?
- For gene-label permutation, name two gene properties that can make unrestricted exchangeability questionable.
- In the four-person example, how many phenotype assignments had an absolute mean difference at least as large as the observed value, and what was the exact p-value?
- Why does the claim “a permutation p-value is valid because labels were randomized” omit an essential assumption?