Why a Linkage Peak Needs Scan-Wide Calibration
A linkage scan evaluates an NPL statistic at many genomic positions. Even if there is no linkage anywhere, random variation can make one position look unusually high. The largest statistic selected after searching the scan is therefore not distributed like the statistic at one position chosen in advance. A pointwise p-value answers a single-position question; it does not, by itself, account for selecting the most prominent result from many positions.
Permutation calibration addresses this selection by repeating the whole scan under a valid null procedure and recording the largest statistic from each repetition. The resulting distribution incorporates the dependence among nearby positions: it does not assume that statistics at different positions are independent. It also calibrates only the scan and analysis choices actually included in those repetitions.
For an excess-sharing test, the relevant tail is usually upper-tailed. If the analysis instead prespecifies a two-sided statistic, such as an absolute standardized statistic, the maximum must be taken over that two-sided statistic. The direction and statistic should be settled before interpreting the scan; changing them after seeing the peak changes the search being calibrated.
A valid permutation procedure must respect the null and the study design, including any restrictions imposed by family ascertainment. The earlier material on valid permutations under family ascertainment concerns which transformations are permissible. Once that procedure is chosen, apply it consistently: for every replicate, calculate the statistic at every position in \(G\), using the same analysis rules, and retain that replicate’s maximum.
From Permutation Scans to a Scan-Wide p-Value
Suppose \(B\) valid permutation replicates produce maxima \(M_1,\ldots,M_B\), and the observed scan has maximum \(M_{\mathrm{obs}}\). A standard Monte Carlo estimate of the upper-tail scan-wide p-value is
Here \(I(\cdot)\) equals 1 when its condition is true and 0 otherwise. The added 1 in the numerator and denominator avoids reporting a zero p-value just because none of the simulated maxima reached the observed maximum. The estimate is the proportion of null scans, with this finite-simulation correction, whose largest statistic is at least as large as the observed largest statistic.
A scan-wide critical value can also be obtained from the upper tail of the permutation maxima. At significance level \(\alpha\), a peak is scan-wide significant when it exceeds the maximum-statistic threshold calibrated for that level. With a finite number of replicates, the exact treatment of ties and the empirical quantile convention matter. One transparent approach is to report the adjusted permutation p-value and compare it directly with \(\alpha\); this makes the tail-count and tie handling explicit.
Why the Maximum Provides Family-Wise Calibration
The scan-wide error of interest is the chance of declaring at least one position significant when the null of no linkage across the scan holds. For a fixed threshold \(c\), that event is that at least one statistic reaches \(c\). Equivalently, the scan maximum reaches \(c\):
Thus, calibrating \(c\) from the null distribution of \(M\) directly calibrates the chance of one or more false peaks across the searched positions. This is family-wise error control under the global null, provided the permutation scheme correctly represents the null distribution of the entire scan. It does not require positions to be independent: their observed dependence is carried through each permutation scan.
This guarantee has a defined scope. A threshold calibrated for one genome-wide scan does not automatically account for trying several phenotypes, alternative statistics, different analysis settings, or additional regions and then reporting whichever result looks strongest. If those choices are part of the search, they must also be represented in the maximum used for calibration, or handled with an appropriate separate adjustment. Likewise, calibration under the global no-linkage null does not by itself guarantee strong family-wise error control under every configuration where some positions are linked and others are not. Such a stronger guarantee requires additional conditions, such as an appropriate subset-pivotality property.
Worked Example: Applying a Genome-Wide Threshold
In a hypothetical analysis, researchers perform \(B=999\) valid permutation scans over a prespecified genome-wide grid. The sorted permutation maxima have \(M_{(950)}=3.21\) and \(M_{(951)}=3.24\), with no ties at these values. The observed scan maximum is \(M_{\mathrm{obs}}=3.30\). The target scan-wide level is \(\alpha=0.05\).
Threshold comparison. With 999 replicates, an observed statistic strictly greater than \(M_{(950)}=3.21\) has at most 49 permutation maxima greater than or equal to it, under the stated no-ties condition. The corrected p-value is therefore at most
The observed maximum, \(3.30\), exceeds \(3.21\), so it meets this scan-wide 0.05 criterion. The value 3.21 is a threshold for the maximum statistic across this specified scan, not a pointwise threshold for each position.
Adjusted p-value from the tail count. Suppose exactly 37 of the 999 permutation maxima are at least 3.30. Then
Because \(0.0380<0.05\), the peak is significant at the 5% scan-wide level for the searched grid and analysis used here. This result does not give the probability that the null hypothesis is true, nor does it establish that the peak position is the causal location of a gene.
Pointwise and Scan-Wide Significance Are Different
A pointwise permutation p-value compares the observed statistic at one specified position with the null distribution of the statistic at that same position. It asks how unusual that value would be if that position had been chosen in advance. A scan-wide adjusted p-value compares the observed scan maximum with the null distribution of permutation maxima. It asks how unusual a peak at least that large would be after searching all positions in \(G\).
Because each permutation maximum is at least as large as the statistic at any one position in that permutation, accounting for the search will generally make the scan-wide p-value larger than the pointwise p-value at the observed peak. The amount of increase depends on the scan and its dependence structure; it is not generally correct to multiply a pointwise p-value by the number of positions.
Worked Example: A Pointwise Result That Is Not Scan-Wide Significant
In a hypothetical scan with \(B=1{,}999\) valid permutations, the observed peak is \(3.40\) at position \(x^\ast\). At that fixed position, 39 permutation statistics are at least \(3.40\). Across the full scan, however, 139 permutation maxima are at least \(3.40\).
The pointwise estimate at \(x^\ast\) is
For the scan-wide comparison, the observed maximum is also \(3.40\), and the relevant count is the number of permutation maxima at least that large:
At \(\alpha=0.05\), the statistic is pointwise significant at \(x^\ast\), but it is not significant after adjustment for searching the full scan. The two answers differ because the scan-wide calculation includes permutations in which a different position, rather than \(x^\ast\), produces a large statistic.
The Search Domain Is Part of the Calibration
The maximum depends on the set of positions being searched. If \(G_1\) is contained in \(G_2\), then for every scan,
Consequently, expanding the search domain cannot reduce any replicate’s maximum, and the resulting threshold will generally be at least as high. A threshold for a chromosome or selected region is therefore not automatically a valid genome-wide threshold. The same principle applies when the search includes additional markers, positions, or analysis choices.
Worked Example: Regional and Genome-Wide Searches
In a hypothetical study, researchers evaluate the same observed peak, \(3.30\), in two ways using the same 999 valid permutation replicates. For a prespecified region, 34 permutation maxima are at least 3.30. For the full genome, 73 permutation maxima are at least 3.30.
For the regional search,
For the genome-wide search,
The peak meets the 0.05 criterion for the regional search, but not for the genome-wide search. These conclusions are consistent because the two p-values answer different search questions. The regional result is appropriate only if the region was genuinely the prespecified search domain; choosing it after inspecting the genome-wide result would not justify using the smaller regional p-value as though the region had been chosen in advance.
What Each Permutation Must Repeat
The maximum-statistic method works only if the permutation replicates reproduce the procedure that generated the observed peak. A replicate should not evaluate just the observed peak position: that would calibrate a fixed-position test, not the act of searching. Instead, apply each valid null transformation, recalculate the NPL statistic across the same grid, and record the largest value.
Define the positions, statistic, tail, and any analysis choices included in the scan before using its calibrated result.
Use transformations justified by the null and the family-ascertained design; arbitrary shuffling is not automatically valid.
Calculate the statistic at every searched position using the same scan procedure, then retain the maximum.
Count permutation maxima at least as large as the observed maximum and use the corrected permutation p-value or a corresponding scan-wide threshold.
This procedure also clarifies why nearby positions need not be treated as independent tests. Their joint behavior is reproduced within each scan, and the maximum summarizes the scan’s most extreme result. The quality of the calibration still depends on having valid null permutations and enough replicates to estimate the relevant tail with useful precision.
Common Mistake
- Wrong reasoning: “The peak has a pointwise p-value below 0.05, so the linkage scan is significant at 0.05.” This ignores that the peak was selected from many searched positions.
- Correction: Compare the observed scan maximum with permutation maxima from complete scans, or report its maximum-statistic adjusted p-value. State the searched domain and the permutation procedure so the scope of the conclusion is clear.
A second version of the same error is to permute only after selecting the observed peak and then test that position as if it had been specified in advance. Selection has already used information from the full scan. The null replicates must reproduce that search to calibrate its false-positive risk.
Check Your Understanding
Answer each question using the distinction between a fixed position and a searched scan.
- What does a permutation maximum represent, and why must each permutation evaluate the full position grid?
- In \(B=999\) permutations, 24 maxima are at least as large as the observed scan maximum. Calculate the corrected scan-wide p-value.
- Why can a pointwise p-value of 0.02 coexist with a scan-wide p-value of 0.07?
- If a search is expanded from one region to the whole genome, what happens to each replicate’s maximum, and why?
- What additional search choices must be included or otherwise accounted for if several phenotypes or NPL statistics are tried before one result is reported?