Defining Ancestry Portability Estimands

Why Portability Comparisons Need an Estimand

A statement such as “the polygenic score is less portable” is incomplete until it specifies where performance is being evaluated, which performance measure is meant, and what population provides the comparison. A score may perform differently across groups, but the size and meaning of that difference depend on the target populations and the chosen measure. Defining these before analysis makes the scientific question explicit rather than letting a validation sample silently determine it.

This tutorial concerns the target and quantity a portability analysis aims to describe. It does not assume that an ancestry category is a fixed biological type. In a particular study, the group definitions must be stated operationally and justified for the intended use. They should also be distinguished from the individual-level genetic data used to calculate a score.

Define the Score, Outcome, and Target Populations

Let \(S=f(X;w)\) denote a specified polygenic score computed from predictors \(X\) using fixed scoring rule \(f\) and weights \(w\). The scoring rule includes details that can affect the result, such as allele coding and any transformations applied to the score. Let \(Y\) be the outcome, defined in the same way for the populations being compared. A portability estimand concerns the performance of this specified score for this specified outcome in specified populations.

For example, a target population might be adults meeting stated eligibility criteria, recruited in specified health systems during a stated period, with group membership defined using a stated study rule. The population definition should clarify whether the groups are mutually exclusive and whether people outside the compared groups are included in the intended target. If the score, outcome definition, or eligibility criteria change, the estimand may change too.

Write \(P_A\) and \(P_B\) for the joint population distributions of score and outcome in two target populations. For a performance measure \(M\), define \(M(P_A,f)\) as the population performance of the fixed score \(f\) under \(P_A\). The measure could be an AUC, an explained-variance measure, or another clearly specified performance quantity. Its definition, direction, and units or scale must be stated. For an interpretable contrast, use the same measure and outcome definition in both populations.

Differences and Ratios Require a Reference

A performance difference and a performance ratio answer distinct questions. If \(A\) is the reference population and \(B\) is the comparison population, define the difference as \(M_B-M_A\) and, when meaningful, the ratio as \(M_B/M_A\). The order of subtraction or division must be stated: reversing the groups reverses a difference and takes the reciprocal of a ratio. A phrase such as “the portability difference” without a direction or reference can therefore be ambiguous.

A difference retains the scale of the measure: if the measure is an AUC, the difference is in AUC units. A ratio is dimensionless, but it is not automatically more informative. A ratio requires a nonzero reference value; it can be unstable when the reference performance is close to zero, and a ratio of a measure with a meaningful zero can have a different interpretation from a ratio of a measure whose zero is only a scale point. Both quantities should be reported with their component performances and with the reference population named.

The performance metric is part of the estimand, not a cosmetic reporting choice. For instance, a ranking metric describes one feature of predictions, whereas an error-based metric describes another. A difference in one does not imply an equal or even corresponding difference in another. The question here is to define the contrast for a selected measure; interpreting how measures relate across outcome scales is a separate issue.

Why the Target Population Can Change the Quantity

A target population is not merely a description attached after results are calculated. Changing its membership or composition can change the population performance itself. For a simple illustration, consider a performance measure defined as expected loss, where smaller values indicate better predictions. Let \(L(S,Y)\) be the loss for a person with score \(S\) and outcome \(Y\). If the target population consists of group \(A\) with proportion \(\pi\) and group \(B\) with proportion \(1-\pi\), its expected loss is a mixture of the group-specific expected losses.

The first equality follows by partitioning the expectation over the two groups; the second uses \(P(A)=\pi\) and \(P(B)=1-\pi\). Even if the group-specific losses stay unchanged, the overall expected loss can change when \(\pi\) changes. Consequently, “overall performance in the study sample” does not necessarily estimate performance in a target population with a different group composition. Other metrics need not combine by this particular weighted-average formula, so their population definitions must be used directly.

Performance Is Not an Individual Score Difference

The score \(S_i=f(X_i;w)\) is an individual-level prediction input or summary. A portability estimand instead describes how a specified score performs across a target population. Differences in genotype data can produce different numerical score values for two people even when the scoring rule and weights are unchanged. Conversely, similar score values do not guarantee similar performance, because performance also depends on how scores relate to outcomes across people in the target population.

An average score difference, such as \(E[S\mid A]-E[S\mid B]\), is therefore not the same quantity as \(M_A-M_B\). It concerns the distribution of score values, not a performance metric. Moreover, even if a person’s score changes after changing the scoring rule, that change alone does not reveal whether prediction performance improved or worsened in any target population.

Common Pitfalls in Defining Portability Estimands

• Leaving the target population implicit. The wrong reasoning is that any validation cohort represents “the ancestry group.” The correction is to state eligibility, setting, period, outcome definition, and the operational group rule, then identify the population to which the result is intended to apply.
• Reporting a contrast without naming its direction. “A difference of 0.12” does not say which group has higher performance. Write the subtraction order, such as \(M_B-M_A\), and name the reference.
• Treating a ratio as a universal summary. A ratio can be unhelpful or unstable if its denominator is zero or close to zero. Report the component performances and the difference as well when useful.
• Calling an individual score shift a performance loss. The wrong reasoning is that lower or higher score values in one group demonstrate worse portability. The correction is to evaluate a defined performance metric against outcomes in each target population.
• Assuming sample composition is irrelevant. For some metrics, the target distribution affects the performance quantity. Define the intended population rather than treating the validation sample’s mixture as automatically generalizable.

Uncertainty belongs alongside an estimated contrast, but it does not replace the estimand. A standard error or interval quantifies sampling uncertainty for a specified population quantity under a specified analysis; it cannot resolve an unclear target, metric, or reference. The first task is to define what the estimate is meant to describe.