Why the Permutation Unit Matters
A permutation test compares an observed statistic with statistics computed after rearranging data under a null hypothesis. The rearrangement is not merely a computational convenience: it specifies which observations could plausibly have been reassigned under that null. In genomic studies, a row in the data table is not automatically a valid permutation unit. Several rows may come from related people, one person measured repeatedly, or individuals belonging to ancestry groups with systematically different outcome distributions.
If a shuffling scheme breaks relationships that should remain under the null, its reference distribution describes the wrong experiment or probability model. The resulting p-value can be too small or too large. The key question is therefore not “What can software shuffle?” but “Under the null, which transformations leave the joint data distribution unchanged?”
Two Routes to a Valid Permutation
The earlier material on exchangeability and permutation tests distinguishes randomization-based and model-based justifications. In a randomized study, the assignment mechanism supplies the allowable rearrangements: if treatment was assigned to clinics, permute clinic assignments, not patient labels within clinics. In an observational analysis, the assignment was not randomized. A permutation test then needs a model-based claim that the relevant observations or errors are exchangeable under the null, possibly conditional on design information. Unrestricted shuffling does not create that claim.
For a statistic \(T\), an exact permutation p-value is a tail probability under the null distribution induced by the allowed transformations. For an upper-tail test with finite transformation set \(\mathcal{G}\), it is
where \(D\) denotes the observed data and \(g\cdot D\) the data after transformation \(g\). This calculation is exact only when the chosen transformations are valid under the null. A large number of permutations cannot repair an invalid choice of \(\mathcal{G}\).
Related Individuals and Ancestry Structure
Relatives share genetic background and often share environmental influences. Consequently, their outcomes or test statistics may be correlated. If family members are treated as independent exchangeable rows, a shuffle can separate each person from the family structure that generated the data. In a study where families, rather than individuals, were assigned to conditions, the assignment unit is the family. In an observational genetic analysis, family membership instead signals dependence that must be represented in the null model or preserved by a defensible permutation scheme.
Ancestry can cause a different problem. Allele frequencies can vary across ancestry groups, while a phenotype can also differ between those groups for environmental, social, or other reasons. A pooled genotype-label shuffle can then compare unlike mixtures of groups. The apparent association may reflect group composition rather than a within-group genotype relationship. Simply permuting within broad ancestry categories is not automatically enough: the categories must capture the relevant structure, and observations within them must still satisfy the exchangeability assumptions.
Worked Example: An Apparent Genotype Difference From Group Composition
Consider a hypothetical observational data set with two ancestry groups, A and B, and a quantitative phenotype measured in arbitrary units. In group A, the eight phenotypes are 0, 2, 0, 2, 0, 2, 0, and 2. In group B, the eight phenotypes are 9, 11, 9, 11, 9, 11, 9, and 11. Within each group, genotype 1 and genotype 0 have the same phenotype mean, but the genotype counts differ by group.
| Group | Genotype 1 phenotypes | Genotype 0 phenotypes |
|---|---|---|
| A | 0, 2 | 0, 2, 0, 2, 0, 2 |
| B | 9, 11, 9, 11, 9, 11 | 9, 11 |
For genotype 1, the total is \(0+2+9+11+9+11+9+11=62\), so its mean is \(62/8=7.75\) units. For genotype 0, the total is six group-A values summing to 6 plus two group-B values summing to 20, or \(6+20=26\); its mean is \(26/8=3.25\) units. The pooled difference is therefore \(7.75-3.25=4.50\) units.
Yet the group-specific means are identical by genotype: in group A, both means are \((0+2)/2=1\) unit; in group B, both are \((9+11)/2=10\) units. Genotype 1 includes six people from B and two from A, whereas genotype 0 includes two from B and six from A. The pooled difference is entirely due to the different group mixtures in this constructed data. Unrestrictedly shuffling genotype labels across all 16 people would ignore that structure and use a null reference distribution that treats the labels as freely exchangeable across ancestry groups. The arithmetic does not itself establish a valid alternative permutation: that requires a justified conditional null model and adequate handling of family dependence if relatives are present.
Repeated Measurements and the Subject as a Unit
Repeated measurements create multiple rows per person, but those rows share subject-specific features. If the scientific comparison concerns conditions measured on the same people, the subject’s record—not each measurement separately—carries the pairing. A between-person shuffle of individual rows can destroy the correspondence between a person’s measurements. Conversely, if treatment was assigned to subjects and each subject contributes many measurements, treatment labels belong to subjects and must travel with each subject’s record.
Worked Example: Four Subjects Measured Twice
Suppose four hypothetical subjects have measurements before and after an intervention: subject 1 has 5 and 7 units, subject 2 has 8 and 9, subject 3 has 4 and 7, and subject 4 has 10 and 11. The observed within-subject changes are \(7-5=2\), \(9-8=1\), \(7-4=3\), and \(11-10=1\) units. Their mean change is
The four subject pairs are the units that preserve the observed pairing. Under a suitable null that permits exchanging the two condition labels within each subject, the transformations must retain each subject’s two values together; they must not freely distribute the eight measurements among unrelated rows. That within-subject exchangeability condition is substantive, not automatic: for example, a time trend or an intervention order effect may make the two labels non-interchangeable under the null. If treatment had instead been assigned to whole subjects, the treatment assignment labels would be reassigned across subjects, with both measurements remaining attached to each subject.
A Short Justification: Dependence Changes the Null Variability
The problem with treating correlated observations as independent can be seen from the variance of their mean. Suppose, for illustration, that \(m\) observations have common variance \(\sigma^2\) and every distinct pair has correlation \(\rho\). Then each pair has covariance \(\rho\sigma^2\), and
If \(\rho>0\), this variance is larger than the independent-observation value \(\sigma^2/m\). The expression is a simple illustration, not a universal model for family or repeated-measure dependence. It explains why a procedure that acts as though correlated rows were independent can understate uncertainty and make extreme statistics look too surprising. A valid permutation distribution must preserve or otherwise correctly represent the dependence relevant under the null.
Worked Example: Four Related Observations
Suppose, hypothetically, that four siblings’ outcomes have common variance \(\sigma^2=25\) square units and pairwise correlation \(\rho=0.5\). Under the illustrative equal-correlation model, the variance of their mean is
If the four outcomes were incorrectly treated as independent, the calculated variance would be \(25/4=6.25\) square units. The dependence-aware variance is \(15.625/6.25=2.5\) times as large. Thus, unrestricted shuffling of the four sibling rows as if they were unrelated can produce a reference distribution with the wrong spread. This calculation demonstrates the consequence of dependence; it does not, by itself, identify a valid permutation scheme for a particular family study.
Choosing and Checking the Unit
A practical decision begins with the study design and the null hypothesis. State what was assigned, what is observed repeatedly, which observations share a family or other cluster, and what features are held fixed under the null. Then specify the allowed transformation in words before computing a statistic. The following sequence helps make the decision explicit.
Ask whether condition labels were assigned to people, families, clinics, or another cluster. The randomization mechanism, when present, determines the assignment-level rearrangements.
Record which rows belong to the same subject or family, and identify ancestry or other grouping that may affect genotype or phenotype distributions.
Specify which labels, records, or residual quantities may be transformed and why their joint distribution should be unchanged under the null.
Verify that it does not detach measurements from subjects, split randomized clusters, or erase relevant group structure. If no defensible transformation is available, use a method whose assumptions model the dependence rather than forcing a shuffle.
Diagnostics should focus on the structure the proposed shuffle claims to ignore. Tabulate family sizes and repeated-measure counts; inspect phenotype and genotype distributions across ancestry or recruitment groups; and compare the observed design with the transformed data. If transformed data routinely create combinations that the study design could not have produced, or if they break the dependence under the null, the permutation scheme is suspect. A visually plausible histogram or a large permutation count is not evidence of exchangeability.
Alternatives, Limitations, and a Common Mistake
A design-based permutation test uses the actual assignment mechanism and is especially compelling when the assignment unit and allowable assignments are known. A model-based permutation test instead relies on distributional invariance under the null, sometimes conditional on covariates or structure. A parametric model can represent family correlation, ancestry effects, or repeated measures directly, but then its conclusions depend on its modeling assumptions. These approaches answer related questions but are not interchangeable: a permutation of labels is not a substitute for modeling structure that makes labels nonexchangeable.
Permutation tests also have limits. They cannot resolve unmeasured confounding simply by rearranging observed labels; conditioning on broad ancestry categories may leave residual structure; and a valid scheme for one test statistic or null hypothesis need not be valid for another. In high-dimensional analyses, correct permutation units are necessary for calibrated reference distributions, but they do not alone solve multiple-testing or model-selection problems.
- Common mistake: “There are 200 people, so I can shuffle 200 genotype labels independently.” This treats all people as exchangeable even if some are relatives, repeated observations, or drawn from different ancestry distributions.
- Correction: First justify invariance under the null and identify the unit the design permits to move. Keep related records together or use a model that accounts for their dependence; account for ancestry structure rather than assuming unrestricted exchangeability.
- Another warning: Permuting within a family or ancestry group is not automatically valid. The proposed within-group rearrangement still needs a null model or assignment mechanism that makes those transformations legitimate.
Check Your Understanding
Answer each question by identifying the unit and stating the key validity condition. Brief solutions follow.
- A trial assigns a dietary intervention to whole households; each household contributes three participants. What should be reassigned?
- In a genetic study, relatives appear in the sample and genotype frequencies differ across ancestry groups. Why may unrestricted individual-level genotype shuffling be invalid?
- For each of 30 people, a measurement is recorded at three time points. What must a proposed transformation preserve?
- What does exchangeability require, and why is it not established merely by having many observations?
- In the hypothetical ancestry example, verify the pooled genotype-1 mean and explain why it is not a within-group genotype contrast.
Solutions. (1) Reassign household labels according to the household-level assignment mechanism, keeping each household’s three participants together. (2) Relatives induce dependence, and ancestry can affect both genotype frequencies and phenotype distributions; unrestricted shuffling may break that structure and mix unlike groups. (3) It must keep each person’s repeated record linked; any condition-label transformation also requires a null that makes those labels exchangeable within person. (4) The joint distribution must be invariant under the specified allowed permutations under the null; sample size alone says nothing about that invariance. (5) The eight genotype-1 values sum to \(0+2+9+11+9+11+9+11=62\), so the mean is \(62/8=7.75\) units. Its group-specific means are 1 unit in A and 10 units in B, each matching genotype 0 within that group; the pooled contrast reflects the groups’ unequal representation, not a within-group difference.