Permutation Calibration Of NPL Statistics

Why Calibrate an NPL Statistic by Simulation?

An NPL statistic summarizes allele sharing among affected relatives. Its null distribution can be difficult to obtain from a simple reference table: pedigrees may differ in size and structure, marker information may be uneven, and the statistic may combine sharing across several relatives. A practical alternative is to construct the null distribution for the analysis at hand by repeatedly generating marker inheritance under no linkage.

The aim here is a pointwise p-value: evidence against no linkage at one specified marker or chromosome position. The observed pedigree structure and affection data are retained, while inheritance at the marker is generated as it would be under the no-linkage null. Each simulated data set is analyzed using the same NPL statistic and analysis procedure as the observed data.

Definition: Let \(T_{\mathrm{obs}}\) be the observed NPL statistic at a specified position, with larger values indicating greater excess allele sharing among affected relatives. The pointwise null distribution is the distribution of that same statistic at that position when inheritance at the marker is generated under no linkage, using the specified pedigree data and analysis procedure. The upper-tail pointwise p-value is the null probability of obtaining a statistic at least as large as \(T_{\mathrm{obs}}\).

In this setting, “permutation” is often used broadly for a randomization procedure, but it does not mean freely shuffling people or affection labels. A valid procedure randomizes marker inheritance according to the pedigree’s Mendelian transmission and recombination structure under the null. Depending on the data and implementation, it may simulate inheritance vectors directly or simulate marker data by gene dropping and then repeat the analysis.

What the Null Simulation Must Preserve

Under no linkage, marker inheritance is not associated with the affection pattern through linkage to the trait locus. The affection statuses, pedigree relationships, and analysis choices are therefore held fixed in a conditional simulation, while marker inheritance is regenerated under the null. “Held fixed” describes the conditioning used to form the reference distribution; it does not mean that inheritance is identical across replicates.

A valid simulation must reflect the features that determine the null behavior of the statistic. These commonly include the pedigree structures, Mendelian transmission probabilities, recombination between positions for multipoint analyses, and the marker information and missingness handled by the analysis. If marker genotypes are simulated, the simulation and analysis must also use an appropriate model for founder alleles and observed genotypes. The same NPL statistic, weighting, and other analysis steps must be applied to every replicate.

For a chromosome scan, inheritance at nearby positions is correlated because recombination is not independently redrawn at every position. Simulating each position independently would fail to preserve that correlation. At a single prespecified position, the calibration is pointwise; it does not account for having searched many positions and then reporting the largest statistic. A separate adjustment is needed for a scan-wide claim.

Calibration conditions: The reference distribution is appropriate when the simulated marker inheritance follows the no-linkage model for the pedigrees, the conditional information used in the analysis is handled consistently, and each replicate is analyzed just like the observed data. Validity also depends on representing the study’s sampling and ascertainment in the conditioning or simulation scheme.

Empirical Pointwise P-Values

Suppose \(B\) null replicates produce statistics \(T_1,\ldots,T_B\). For an excess-sharing test, count a replicate as at least as extreme as the observation whenever \(T_b \geq T_{\mathrm{obs}}\). This includes ties, because the test is asking for the probability of a value at least as large as the observed one.

\widehat{p}_{\mathrm{emp}} = \frac{1+\sum_{b=1}^{B} I(T_b \geq T_{\mathrm{obs}})}{B+1},

Here \(I(\cdot)\) equals 1 when the condition inside it is true and 0 otherwise. The added 1 accounts for the observed statistic when comparing it with the simulated statistics. Thus the estimate is never zero: its smallest possible value is \(1/(B+1)\). Reporting zero after observing no exceedances would incorrectly suggest that the null probability is known to be zero.

If the null distribution can be enumerated exactly over \(M\) equally likely configurations, the exact upper-tail probability is the number of configurations with statistic at least \(T_{\mathrm{obs}}\), divided by \(M\). The plus-one formula is for Monte Carlo calibration from \(B\) simulated replicates; it treats the observation and the null simulations symmetrically for the rank calculation when they are exchangeable under the null.

Worked Example: A Small Set of Null Replicates

Suppose a hypothetical analysis at one prespecified position gives \(T_{\mathrm{obs}}=1.80\). Nineteen null inheritance replicates, generated using the same two-family pedigree structure and NPL calculation, give statistics:

\(0.20,\ 0.40,\ 0.60,\ 0.70,\ 0.80,\ 0.90,\ 1.00,\ 1.10,\ 1.20,\ 1.30,\ 1.40,\ 1.50,\ 1.60,\ 1.70,\ 1.75,\ 1.80,\ 1.90,\ 2.00,\ 2.10.\)

Count the upper-tail replicates. The values at least as large as 1.80 are 1.80, 1.90, 2.00, and 2.10, so the exceedance count is 4.

Calculate the empirical p-value.

\widehat{p}_{\mathrm{emp}} = \frac{1+4}{19+1} = \frac{5}{20} = 0.25.

The result is a pointwise empirical p-value of 0.25. Under the simulated no-linkage reference, statistics at least this large are not unusual. With only 19 replicates the estimate is coarse: possible values occur in increments of \(1/20=0.05\).

Why the Plus-One Formula Is Used

The formula also has a simple rank justification. Under the null, suppose the observed statistic and the \(B\) simulated statistics are exchangeable: their joint distribution is unchanged if their labels are rearranged. If there are no ties, the rank of the observation among these \(B+1\) values is then equally likely to be any rank from 1 to \(B+1\). The number of simulated values at least as large as the observation is \(B+1\) minus its rank, so adding 1 gives its upper-tail rank count. Dividing by \(B+1\) gives the rank divided by the total number of values.

With ties, counting all tied values in the upper tail makes the result conservative rather than treating an observed tie as less extreme than an equal simulated value. The justification relies on exchangeability under the specified null simulation; a large number of replicates cannot repair a simulation model that fails to represent the null.

Worked Example: A More Precise Monte Carlo Calibration

In a hypothetical study, the observed NPL statistic at a prespecified position is \(T_{\mathrm{obs}}=2.35\). The analyst generates \(B=999\) null replicates, preserving the observed pedigrees and affection statuses and applying the same NPL procedure in each replicate. Eleven simulated statistics are at least 2.35.

The empirical p-value is

\widehat{p}_{\mathrm{emp}} = \frac{1+11}{999+1} = \frac{12}{1000} = 0.012.

This is evidence against no linkage at that position under the specified null model. It is not the probability that no linkage is true, and it is not a genome-wide p-value.

The simulation count also limits numerical precision. With 999 replicates, the smallest possible plus-one value is \(1/1000=0.001\). As a rough measure of Monte Carlo variability, the simulated exceedance fraction is \(11/999 \approx 0.0110\), with approximate standard error

\sqrt{\frac{0.0110(1-0.0110)}{999}} \approx 0.00330.

This approximation describes random variation from using a finite number of replicates; it is not a confidence interval for a population parameter. When a p-value near a decision threshold is important, increasing \(B\) can reduce this simulation noise.

Exact Enumeration and Pointwise Interpretation

Simulation is not always necessary. If the null distribution consists of a manageable number of equally likely inheritance configurations, one can calculate the tail probability by enumerating them all. For example, if a hypothetical pedigree analysis has 16 equally likely configurations and 3 yield a statistic at least as large as the observed value, the exact pointwise p-value is \(3/16=0.1875\). This value is discrete because only a finite set of configurations is possible.

When configurations are not equally likely, they must be weighted by their null probabilities; simply counting them would give the wrong reference distribution. Likewise, simulations from a complex inheritance process must reproduce its probabilities rather than treating every distinct configuration as equally likely.

Worked Example: Ties in an Exact Null Distribution

Consider a hypothetical pointwise NPL analysis with 20 equally likely null configurations. Their statistic values include six configurations with \(T=1.5\), four with \(T=2.0\), and ten with \(T<1.5\). The observed statistic is \(T_{\mathrm{obs}}=1.5\).

Because the upper tail includes ties, all six configurations with \(T=1.5\) and all four with \(T=2.0\) count. Therefore,

p_{\mathrm{exact}} = \frac{6+4}{20} = 0.50.

Counting only values strictly greater than 1.5 would give \(4/20=0.20\), which answers a different question and understates the specified upper-tail probability. The exact p-value here is 0.50.

Common Calibration Mistakes

Shuffling affection labels arbitrarily. This changes the trait data and may break the dependence and ascertainment structure within pedigrees. For this calibration, the central randomization is marker inheritance under the no-linkage null, with the relevant observed phenotype and pedigree information conditioned on as specified.
Treating inheritance draws as independent across linked positions. This discards recombination-induced correlation in a multipoint analysis. Simulated inheritance along a chromosome should follow the appropriate transition structure.
Using the fraction of exceedances without considering the observed statistic. The plus-one estimate is \((r+1)/(B+1)\), not zero when \(r=0\). It also makes clear that the simulation has finite resolution.
Ignoring ties. For an upper-tail test, count \(T_b \geq T_{\mathrm{obs}}\), not only \(T_b > T_{\mathrm{obs}}\).
Calling a pointwise p-value genome-wide evidence. Calibration at one position does not account for searching across positions and selecting a peak. The interpretation must match the analysis performed.

Key takeaway: A permutation-calibrated NPL p-value compares the observed statistic with statistics generated under a correctly specified no-linkage inheritance model. The simulation must respect pedigree and marker structure, ties belong in the upper tail, and the resulting p-value is pointwise unless the search process is also calibrated.

Check Your Understanding

Work through these questions using the definitions and procedure above.

At one position, the observed NPL statistic is 1.9. If 29 of 1,999 null replicates are at least this large, calculate the plus-one empirical p-value.
Why is freely shuffling affection labels not generally the same as simulating marker inheritance under no linkage?
What is the smallest plus-one empirical p-value attainable from 499 null replicates?
An exact null distribution has 40 equally likely configurations, 7 of which have statistics tied with or greater than the observed value. Calculate the exact upper-tail p-value.
Why does a pointwise empirical p-value not, by itself, account for selecting the largest statistic from a chromosome scan?