Extreme-Phenotype Sampling for Quantitative Traits

Why Sample From the Tails?

A genome-wide association study can recruit participants across the full range of a quantitative trait, or it can deliberately recruit people with unusually high and unusually low values. The second approach, often called extreme-phenotype sampling, concentrates recruitment where genetic differences may be easier to detect. If a variant shifts the trait upward, its dosage may be more frequent among people in the upper tail and less frequent among people in the lower tail.

This efficiency comes with a cost: the sampled participants no longer represent the trait distribution in the target population. The association observed in the tails therefore needs to be interpreted using the sampling design. In particular, the difference between the average trait values of the two selected groups is not itself an estimate of a genetic effect. The analysis must use genotype differences, a model that accounts for selection, or both.

Normal-Tail Quantities

Suppose the population trait \(Y\) is approximately normal with mean \(\mu\) and standard deviation \(\sigma_Y\). If the upper and lower tails each contain proportion \(q\), define \(c\) by \(P(Z>c)=q\), where \(Z\sim N(0,1)\). The cutoffs are then \(\mu+c\sigma_Y\) and \(\mu-c\sigma_Y\). The standard normal density and distribution functions are denoted by \(\phi\) and \(\Phi\).

For a standard normal variable, the mean above \(c\) is \(\lambda(c)=\phi(c)/[1-\Phi(c)]\). By symmetry, the mean below \(-c\) is \(-\lambda(c)\). Thus the selected upper and lower groups have a larger mean contrast than a random sample would typically show, even before considering genotype.

How Selection Enriches a Genotype Contrast

Consider a biallelic variant with allele frequency \(p\), and let \(G\) be the allele dosage, coded 0, 1, or 2. Under Hardy–Weinberg equilibrium, \(E(G)=2p\) and \(\operatorname{Var}(G)=2p(1-p)\). Suppose the trait follows an additive model: each additional effect allele changes the population mean by \(\beta\) trait units. Write \(b=\beta/\sigma_Y\) for the effect in standard-deviation units.

For the following approximation, assume the trait is approximately normal, the effect is small relative to \(\sigma_Y\), and the upper and lower thresholds are set at \(\mu\pm c\sigma_Y\). Also assume participants are sampled randomly within each tail. Under these conditions, a genotype dosage shifts the trait mean by approximately \(\beta(G-2p)\). The probability of entering the upper tail therefore changes with dosage; the lower-tail probability changes in the opposite direction.

Why the Dosage-Contrast Formula Holds

The approximation can be derived by expanding the tail probabilities for small \(b\). Let \(q=1-\Phi(c)\). For a person with dosage \(G\), the approximate standardized shift is \(b(G-2p)\). The probability of being above the upper cutoff is therefore \(1-\Phi\{c-b(G-2p)\}\). A first-order Taylor expansion around \(b=0\) gives \(q+\phi(c)b(G-2p)\). Similarly, the lower-tail probability is approximately \(q-\phi(c)b(G-2p)\).

To obtain the expected dosage among upper-tail participants, multiply each dosage by its upper-tail probability and average over the population genotype distribution. The constant part contributes \(qE(G)\), and the first-order part contributes \(\phi(c)b\operatorname{Var}(G)\). Dividing by the upper-tail probability, which is \(q\) to first order, gives an upper-tail mean dosage of approximately \(E(G)+\lambda(c)b\operatorname{Var}(G)\). The corresponding lower-tail mean is approximately \(E(G)-\lambda(c)b\operatorname{Var}(G)\). Subtracting yields \(2\lambda(c)b\operatorname{Var}(G)\), as stated.

The approximation shows why the allele frequency matters: when \(p\) is near 0 or 1, \(2p(1-p)\) is small, so there is less genotype variation to enrich. It also shows why more extreme cutoffs can produce a larger genotype contrast: \(\lambda(c)\) rises as the tails become more extreme. However, increasingly extreme sampling leaves fewer eligible people and can make recruitment or estimation less practical.

Planning and Analyzing an Extreme-Tail Sample

Extreme sampling is most useful when a study can define the target population and phenotype carefully, identify tail thresholds in a relevant reference distribution, and recruit enough people from both tails. The genotype association can be analyzed as a dosage contrast between the tails or with a model for the probability of tail membership. If the study uses a different sampling fraction from each tail, the sampling scheme must be reflected in the model; random sampling within each tail is a key condition for the simple contrast interpretation.

Here is a practical sequence for moving from design to interpretation:

Assumptions and Common Pitfalls

The conversion is not a universal correction that can be applied regardless of design. It relies on approximate normality of the trait distribution near the thresholds, an additive genotype effect that is small relative to the trait variation, suitable allele-frequency and genotype-variance information, and correct identification of the population cutoffs. Population structure or differences between recruitment sites can create genotype differences that are not due to the trait effect. Reliable phenotype measurement and alignment between the recruited sample and target population remain important.

For larger effects, substantial departures from normality, or asymmetric sampling, the first-order approximation may be inaccurate. In those settings, use the specified genotype-specific selection probabilities or an appropriate likelihood that represents the actual ascertainment process. Inference also needs a valid standard error and testing procedure for the chosen analysis; enrichment of the expected signal does not remove sampling uncertainty or other sources of bias.

• Mistake: treating the upper-minus-lower trait mean as the genetic effect. That contrast is large by construction. The correction is to estimate the genotype association and interpret it using a model that accounts for tail selection.
• Mistake: applying the conversion factor without its assumptions. The factor depends on the tail cutoff, allele frequency, and normal, small-effect approximation. Check these inputs and use a more suitable selection model if they do not fit.
• Mistake: interpreting a selected sample as a population-representative cohort. The proportions recruited from the two tails were set by design. They do not estimate the population prevalence of high or low trait values.