Prediction Targets Under Selective Sampling

Why the Prediction Target Must Name a Population

A polygenic score can be evaluated in a genome-wide association study cohort, in a clinic that recruits volunteers, or among people who will later receive predictions. These are not automatically the same population. Even when the score and outcome definition are unchanged, the distribution of scores, outcomes, and other predictors can differ between them. Consequently, a performance estimate is meaningful only when its target population is clear.

This distinction matters in practice. A score may explain a substantial share of outcome variation in a broadly sampled source population but much less in a recruited cohort whose participants have a narrow range of scores. A case-control evaluation may estimate discrimination for its sampled cases and controls, while a risk estimate intended for deployment may need calibration to a different outcome frequency. The numbers are not necessarily contradictory: they can describe different targets.

Definition: The source population is the population from which individuals could, in principle, have entered the study. The recruited cohort is the population represented by those who were actually enrolled or retained for analysis. The deployment population is the population in which predictions are intended to be used. A prediction-performance target specifies both the performance measure and the population distribution under which that measure is defined.

Let $G$ denote a fixed genetic score, $Y$ the outcome, and $R=1$ indicate recruitment into the analyzed cohort. A source-population quantity is calculated under the joint distribution of $G$ and $Y$ in the source population. A recruited-cohort quantity is calculated under the distribution conditional on $R=1$. A deployment quantity is calculated under the distribution among people eligible for use of the prediction. These distributions may coincide, but that is an assumption to examine, not a consequence of using the same score.

Three Distinct Properties of Prediction

Explained variance, discrimination, and calibration answer different questions. They should not be treated as interchangeable summaries or as though a high value for one guarantees a high value for another.

Definition: For a continuous outcome, the population explained variance of the optimal prediction based on $G$ is $$ R^2_G=\frac{\operatorname{Var}\{E(Y\mid G)\}}{\operatorname{Var}(Y)}. $$ It is the fraction of outcome variance captured by the conditional mean $E(Y\mid G)$ in the stated population. A validation-based $R^2$ for a particular fixed prediction rule can instead be defined as $1-E[(Y-\hat Y)^2]/\operatorname{Var}(Y)$; it may be negative if that rule predicts worse than the population mean.

For a binary outcome, discrimination is commonly summarized by the area under the receiver operating characteristic curve (AUC). It is the probability that a randomly selected person with the outcome has a higher score than a randomly selected person without it, with half credit for a tie. The relevant case and control score distributions are those in the target population. AUC does not, by itself, say whether predicted absolute risks agree with observed outcome frequencies.

Definition: A risk prediction $q(G)$ is calibrated in a population when $E(Y\mid q(G)=q)=q$ over prediction values $q$ in that population. Calibration concerns agreement between predicted risks and outcome frequencies. A calibration-in-the-large comparison addresses overall systematic over- or underprediction; a calibration slope summarizes whether predictions are too extreme or not extreme enough under the specified calibration model.

Calibration is intrinsically tied to the outcome prevalence and risk distribution of the target. Explained variance depends on the outcome variance and the variation in the score-linked conditional mean. AUC depends on how the score ranks cases against controls. Therefore, changing the population can affect each property in a different way.

How Selective Sampling Changes Explained Variance

Selection can change the spread of $G$ and $Y$, even if the conditional relationship between them is stable. If recruitment narrows the score distribution, the score may account for a smaller fraction of outcome variation within the recruited group. This does not require the score’s conditional mean association with the outcome to have weakened.

Worked Example: Restricted Score Variation

Hypothetical model. In a source population, suppose $Y=2G+\varepsilon$, where $G$ and $\varepsilon$ are independent, $E(\varepsilon)=0$, $\operatorname{Var}(G)=1$, and $\operatorname{Var}(\varepsilon)=3$. Suppose recruitment selects people with a narrower range of $G$, such that in the recruited cohort $\operatorname{Var}(G\mid R=1)=0.25$, while the residual distribution is unchanged and remains independent of $G$. The conditional mean remains $E(Y\mid G)=2G$ in both populations.

In the source population, the variance of the conditional mean is $$ \operatorname{Var}\{E(Y\mid G)\}=\operatorname{Var}(2G)=4\operatorname{Var}(G)=4. $$ The total outcome variance is $4+3=7$, so the source-population explained variance is $$ R^2_{G,\text{source}}=\frac{4}{7}=0.5714. $$

In the recruited cohort, the variance of the conditional mean is $4(0.25)=1$, while the outcome variance is $1+3=4$. Thus $$ R^2_{G,\text{cohort}}=\frac{1}{4}=0.2500. $$

The score has the same conditional mean relationship in both populations, but its explained variance is lower in the recruited cohort because the score varies less there relative to the residual variation. The cohort estimate describes recruited participants; it is not an estimate of source-population explained variance unless transport to that population is justified.

The calculation uses the variance decomposition for this model: because $Y=2G+\varepsilon$ and $G$ is independent of $\varepsilon$, $\operatorname{Var}(Y)=4\operatorname{Var}(G)+\operatorname{Var}(\varepsilon)$. This makes clear why the denominator matters: explained variance is a ratio, not an intrinsic property of a score that must stay fixed across populations.

How Selection Changes Discrimination

For binary outcomes, AUC is determined by the score distributions among cases and controls. Selection can alter either distribution—for example, if recruitment depends on score, outcome status, or factors associated with both. The AUC calculated among recruited cases and controls then describes those selected groups. It need not equal the AUC among all source-population cases and controls or among cases and controls in deployment.

Worked Example: AUC Before and After Selective Recruitment

Hypothetical data. Consider a binary outcome and a score that takes values 0, 1, and 2. In the source population, four cases have scores $0,1,2,2$, and four controls have scores $0,0,1,2$. Recruitment retains three cases with scores $1,2,2$ and three controls with scores $0,0,1$.

There are $4\times4=16$ case-control pairs in the source population. A case with score 0 earns one half-credit tie against the controls with score 0, for 1 point total. A case with score 1 earns 2 points against the two controls with score 0 and one half-point for the tie with score 1, for 2.5 points. Each of the two cases with score 2 earns 3.5 points: it beats the controls with scores 0, 0, and 1 and ties the control with score 2. Total credit is $1+2.5+3.5+3.5=10.5$, so $$ \operatorname{AUC}_{\text{source}}=\frac{10.5}{16}=0.6563. $$

In the recruited cohort there are $3\times3=9$ pairs. The case with score 1 earns $2+0.5=2.5$ points against the two controls with score 0 and the control with score 1. Each case with score 2 beats all three controls, earning 3 points. Total credit is $2.5+3+3=8.5$, and therefore $$ \operatorname{AUC}_{\text{cohort}}=\frac{8.5}{9}=0.9444. $$

The recruited-cohort AUC is higher because the selected case and control score distributions produce more favorable pairwise rankings. It does not establish that discrimination is this high in the source population or in deployment. Each AUC estimates the ranking performance for the particular case and control populations used in its calculation.

This example also shows why prevalence alone is not a definition of AUC: AUC compares case and control score distributions, rather than weighting everyone according to the population’s case fraction. However, selection can change those distributions and hence AUC. AUC is not guaranteed to remain constant merely because the score’s ordering rule has not changed.

How Selection Changes Calibration

A score can preserve useful ranking while becoming poorly calibrated in a new population. One common reason is a change in baseline outcome risk. If the score was converted to absolute risks using the source population’s baseline incidence, those predictions may systematically overstate or understate risk in a deployment population with a different baseline.

Worked Example: A Shift in Baseline Risk

Hypothetical risk model. Suppose a score gives source-population risks of 0.10 and 0.30 for two score groups, with half the source population in each group. The predictions are calibrated in the source population: the group outcome frequencies are 0.10 and 0.30, and the overall frequency is $(0.10+0.30)/2=0.20$.

Now suppose a deployment population has the same proportions in the two score groups, but its outcome odds at each score value are half the source-population odds. This is an intercept shift on the log-odds scale. For a source risk $p$, halving the odds gives a deployment risk $$ p_{\text{deploy}}=\frac{p}{2-p}. $$ For the first group, this is $0.10/(2-0.10)=0.05263$. For the second group, it is $0.30/(2-0.30)=0.1765$. The deployment outcome frequency averaged over the two equally sized groups is $$ \frac{0.05263+0.1765}{2}=0.1146. $$

The original predictions still average 0.20, so, using the unrounded deployment frequency, they overstate the deployment population’s overall frequency by about $0.20-0.114551=0.08545$, or 8.545 percentage points. On the log-odds scale, the relation between the deployment outcome risk and the source prediction has calibration intercept $\log(0.5)\approx-0.6931$ and slope 1 under the stated model. Thus the model’s relative ordering need not be discarded, but its source risks are not calibrated for deployment without adjustment.

This estimate of miscalibration describes the specified deployment population and shift. It does not show that the source-population predictions were miscalibrated there; they were stipulated to be calibrated in the source population.

The example assumes an exact odds shift, common across score values, and unchanged group proportions. Real population changes may also alter score distributions, conditional effects, outcome definitions, or selection mechanisms. In those cases, an intercept adjustment alone may not restore calibration. Calibration must be assessed using outcomes and predictions representative of the intended target, or its limitations must be stated.

State the Target Alongside the Estimate

A useful report identifies the population represented by the estimate, the outcome, the score or prediction rule, and the performance measure. It should distinguish the population used to fit or evaluate the score from the population in which its use is intended. For example, “AUC among recruited cases and controls” is more informative than “AUC of the score” when recruitment was selective.

Performance measure	What it summarizes	Population feature that can change it
Explained variance	Conditional-mean variation relative to outcome variation, for a continuous outcome	Score spread, outcome variance, and the score-outcome relationship
AUC	Pairwise ranking of cases above controls, with half credit for ties	Score distributions among cases and among controls
Calibration	Agreement between predicted risks and observed outcome frequencies	Baseline risk and the relationship between score and outcome in the target

A direct estimate in a recruited cohort is appropriate when the question is performance among people like those recruited. It does not automatically answer how the score will perform in the broader source population or in deployment. Conversely, a source-population or deployment claim requires data or assumptions that connect the evaluated sample to that population. The label on an estimate is part of its interpretation, not a formality.

Common Mistake: Treating Performance as a Fixed Score Property

Wrong reasoning: “The score is unchanged, so its AUC, explained variance, and calibration must be unchanged.”
Correction: The score rule may be fixed while the distributions used to calculate each metric change. Explained variance uses outcome and prediction variation; AUC uses case and control score distributions; calibration compares predicted risks with outcomes in the target population.
Wrong reasoning: “A high AUC in a recruited cohort establishes good absolute-risk predictions in deployment.”
Correction: AUC measures ranking, not agreement of predicted and observed risks. A deployment population can have different baseline risk and therefore require a separate calibration assessment.
Wrong reasoning: “The validation sample estimate applies to everyone in the source population because it came from that source.”
Correction: If inclusion or retention was selective, the estimate describes the recruited sample unless selection is ignorable for that metric or an appropriate transport analysis supports a broader target.

Key takeaway: Every performance estimate has a population target. Name whether it describes the source population, recruited cohort, or deployment population, and interpret explained variance, discrimination, and calibration as distinct, population-dependent properties.

Check Your Understanding

For each question, state which population or metric is at issue.

A recruited cohort has a narrower score distribution than its source population, while the conditional mean outcome model is unchanged. Why can explained variance differ?
What two score distributions determine the AUC for a binary outcome, and how can selective recruitment affect them?
A score has high AUC in a clinic sample. Does that establish calibration in a deployment population with a different baseline outcome risk? Explain.
In the calibration example, what does the deployment-population difference between the mean predicted risk and the mean outcome frequency describe?
Write one sentence that reports an AUC estimate without implying that it describes an unobserved population.