Worked Analysis: Tracing a Score Portability Gap

Introduction: Trace the Gap Before Explaining It

A polygenic score can predict a phenotype less well in a target ancestry than in the population used to develop it. That difference is an observation, not yet an explanation. It may reflect the phenotype’s measurement scale, less variation in the score, differences in LD tagging or genetic effects, calibration failure, or ordinary sampling uncertainty. These possibilities have different implications and should not be collapsed into one claim about “portability.”

This worked analysis follows a hypothetical quantitative-trait score through those checks in sequence. The goal is to identify what the available summaries establish, what they do not establish, and what additional analysis would help distinguish competing explanations. The numerical results are illustrative, not findings from a real study.

Definition: For a score \(S\) and quantitative phenotype \(Y\), the coefficient of determination \(R^2\) in a linear regression with an intercept is the proportion of the sample variation in \(Y\) explained by the fitted regression. In a one-predictor regression, \(R^2=\operatorname{Corr}(S,Y)^2\). A portability gap is a difference in a specified performance measure between populations; it is not, by itself, evidence about why that difference occurred.

Start With Comparable Phenotypes and Performance Measures

Suppose a development population and an independently sampled target population each have \(n=400\) participants. The same score definition is applied in both. The hypothetical summaries are:

Summary	Development population	Target population
Phenotype mean	50 units	50 units
Phenotype standard deviation	10 units	12 units
Score standard deviation	1.00 score unit	0.70 score units
Regression slope of phenotype on score	2.00 phenotype units per score unit	1.714 phenotype units per score unit
Score–phenotype correlation	0.200	0.100
\(R^2\)	0.0400	0.0100

For a one-predictor regression with an intercept, the slope is the covariance divided by the score variance. Equivalently, it is the correlation multiplied by the ratio of the phenotype standard deviation to the score standard deviation. The summaries agree: in the development population, \(0.200(10/1.00)=2.00\); in the target population, \(0.100(12/0.70)=1.714\). Squaring the correlations gives \(0.0400\) and \(0.0100\), respectively.

Before attributing this difference to genetics, verify that the phenotype has the same meaning, ascertainment, and measurement procedure in both groups. An arbitrary linear change of units is not itself a predictive failure. If \(Y^*=a+cY\), where \(c\ne0\), then adding \(a\) leaves the correlation unchanged. Multiplying by a positive \(c\) also leaves it unchanged; multiplying by a negative \(c\) reverses its sign. In every case, the squared correlation, and thus \(R^2\), is unchanged. This invariance assumes the same observations and an intercept in the regression. It does not mean that genuine differences in phenotype variation or measurement quality are irrelevant.

Worked Example: Separating Units From Predictive Performance

In the target population, the score–phenotype correlation is \(0.100\), so \(R^2=0.100^2=0.0100\). Suppose the phenotype is reported instead as \(Y^*=20+0.5Y\). The new phenotype standard deviation is \(0.5(12)=6.00\) units. The regression slope becomes \(0.5(1.714)=0.857\) new phenotype units per score unit, while the correlation remains \(0.100\) and \(R^2\) remains \(0.0100\).

If the transformation were \(Y^*=20-0.5Y\), the correlation would be \(-0.100\), because the direction of the outcome scale has been reversed. Its square would still be \(0.0100\). Thus, a numerical change in slope can arise solely from the choice of outcome units, and a sign change can arise from reversing the scale. Neither change alone demonstrates weaker predictive performance as measured by \(R^2\).

Check Score and Phenotype Variation

The \(R^2\) identity makes the role of variation explicit. In a simple regression with an intercept, the fitted slope is \(b=\operatorname{Corr}(S,Y)s_Y/s_S\). Rearranging gives:

R^2=\operatorname{Corr}(S,Y)^2 =\frac{b^2s_S^2}{s_Y^2}.

Here \(s_S\) and \(s_Y\) are the sample standard deviations of the score and phenotype. This equation is an accounting identity for the observed regression; it does not identify a causal mechanism. A narrower score distribution can reduce the variation in the phenotype that the score predicts, even if the slope per score unit is unchanged. A larger phenotype standard deviation can also reduce \(R^2\) for a given slope and score spread. But a difference in \(s_Y\) is not automatically a mere unit difference: it can reflect genuine differences in phenotype variation, residual variation, or measurement.

Worked Example: How Much Could Score Spread Account For?

The development summaries give \(b=2.00\), \(s_S=1.00\), and \(s_Y=10\), so \(R^2=(2.00^2)(1.00^2)/10^2=4/100=0.0400\). In the target population, the score standard deviation is \(0.70\), the phenotype standard deviation is 12, and the observed slope is \(1.714\). Therefore:

R^2_{\text{target}} =\frac{(1.714)^2(0.70)^2}{12^2} \approx\frac{1.440}{144} =0.0100.

As a descriptive counterfactual, keep the development slope of \(2.00\) phenotype units per score unit, but use the target score and phenotype standard deviations. The resulting \(R^2\) would be \((2.00^2)(0.70^2)/12^2=1.96/144=0.0136\). This calculation shows that the narrower score spread and larger phenotype spread could produce a lower \(R^2\) even under that unchanged-slope scenario. The observed value is lower still, and the target slope is also smaller. The counterfactual does not prove why the slope differs; genetic tagging, effect differences, measurement, and sampling remain possible explanations.

Trace Genetic Components: LD Tagging and Effect Heterogeneity

A score combines genotype values using weights estimated in some discovery data. Even when those weights are fixed, their relationship to the phenotype can differ across populations. For a vector of genotypes \(G\), score weights \(w\), and a linear genetic phenotype component with effects \(\gamma\), write \(S=w^\mathsf{T}G\) and \(Y_{\text{gen}}=\gamma^\mathsf{T}G\). If the target-population genotype covariance matrix is \(\Sigma\), then:

\operatorname{Var}(S)=w^\mathsf{T}\Sigma w, \qquad \operatorname{Cov}(S,Y_{\text{gen}})=w^\mathsf{T}\Sigma\gamma.

The covariance depends both on how the score variants tag one another through LD, represented by \(\Sigma\), and on the effects being tagged, represented by \(\gamma\). A change in LD can alter how well score variants tag causal variants even if causal effects are shared. A change in causal effects can alter score–phenotype covariance even if LD is unchanged. In real analyses, both can change together, and finite-sample weight error or phenotype differences can complicate the comparison.

Worked Example: A Tagging Variant With Different LD

Consider a deliberately small, hypothetical two-variant model. Let \(G_1\) and \(G_2\) be standardized genotypes, each with variance 1. Variant \(G_1\) is included in the score, while \(G_2\) affects the phenotype: \(S=0.4G_1\) and \(Y=0.5G_2+\varepsilon\). Assume \(\varepsilon\) is independent of both genotypes, has mean 0 and variance 1, and the correlation between \(G_1\) and \(G_2\) is \(\rho\). The score weight 0.4 is the marginal association expected in a development setting with effect 0.5 and LD correlation 0.8.

In the development setting, \(\rho=0.8\). Thus \(\operatorname{Var}(S)=0.4^2=0.16\), \(\operatorname{Cov}(S,Y)=0.4(0.5)(0.8)=0.16\), and \(\operatorname{Var}(Y)=0.5^2+1=1.25\). The score slope is \(0.16/0.16=1.00\), and:

R^2=\frac{\operatorname{Cov}(S,Y)^2} {\operatorname{Var}(S)\operatorname{Var}(Y)} =\frac{0.16^2}{0.16(1.25)} =0.128.

Now keep the causal effect and residual variance unchanged but set target LD correlation to \(\rho=0.2\). The score variance remains 0.16, but the covariance becomes \(0.4(0.5)(0.2)=0.040\). The target slope is \(0.040/0.16=0.250\), and \(R^2=0.040^2/[0.16(1.25)]=0.00800\). In this toy model, weaker LD tagging alone lowers score performance. The calculation illustrates a mechanism; it does not establish that LD caused the gap in the larger cohort summaries above.

A separate effect-heterogeneity check asks whether the effects of relevant variants differ across populations, rather than assuming that an altered marginal score association must be an LD effect. For example, retain target LD correlation \(\rho=0.2\) in the toy model but change the causal effect from 0.5 to 0.35. Then \(\operatorname{Cov}(S,Y)=0.4(0.35)(0.2)=0.028\), while \(\operatorname{Var}(Y)=0.35^2+1=1.1225\). The resulting score \(R^2\) is:

R^2=\frac{0.028^2}{0.16(1.1225)} =\frac{0.000784}{0.1796} \approx 0.00437.

Compared with the LD-only target calculation (\(R^2=0.00800\)), this example shows how a smaller causal effect can further reduce performance. Actual evidence for heterogeneity requires estimates and uncertainty for effects, with attention to allele alignment, LD, phenotype definition, and the precision of the discovery and target estimates. Comparing marginal associations alone may confound effect differences with LD differences.

Check Calibration Separately From Ranking

A score may rank individuals reasonably while its development-population predictions are systematically too large or too small in a target population. For a quantitative phenotype, one useful check is to compare predictions from the development regression with an observed target regression. Suppose the development prediction is \(50+2S\), and in the target sample the fitted regression is \(50+1.714S\), with both scores centered so that the reference score mean is zero. The target slope is smaller than the development slope.

Worked Example: Interpreting a Target Calibration Slope

Define the calibration slope as the coefficient from regressing observed target phenotype values on the development predictions, with an intercept. In this simple case, the development prediction changes by \(2.00\) phenotype units for each score unit, whereas the target fitted value changes by \(1.714\). The calibration slope is therefore \(1.714/2.00=0.857\). A slope below 1 indicates that the development prediction contrast per score unit is too steep for the target regression: the predicted differences are larger than the observed differences on average.

This slope comparison does not show that the score has no ranking ability, nor does it identify whether LD, effects, score spread, or other factors produced the mismatch. An intercept check is also needed: a calibration slope alone does not reveal a systematic shift in the predicted phenotype level. For example, if target mean phenotype were 53 rather than 50 while the score were centered at zero, a prediction intercept of 50 would be low by 3 phenotype units on average even if the slope were correct. These are quantitative-trait predictions in phenotype units, not estimates of individual absolute disease risk.

Quantify Uncertainty Before Calling a Gap

A difference between two sample performance estimates is itself uncertain. As a simple illustration for independent samples, Fisher’s transformation \(z=\operatorname{atanh}(r)\) approximately has standard error \(1/\sqrt{n-3}\) when observations are independent and the usual large-sample conditions hold. A confidence interval can be formed on the transformed scale and converted back to a correlation interval. Since \(R^2=r^2\), squaring the correlation endpoints gives an interval for \(R^2\) when the interval endpoints have the same sign; if the correlation interval crosses zero, square the interval’s minimum and maximum values, accounting for the fact that the minimum square is then zero.

Worked Example: Is the Difference Precise?

For the hypothetical independent samples of \(n=400\), the observed correlations are \(r_D=0.200\) and \(r_T=0.100\). The Fisher-transformed values are approximately \(\operatorname{atanh}(0.200)=0.2027\) and \(\operatorname{atanh}(0.100)=0.1003\). Each has standard error \(1/\sqrt{397}=0.0502\). A 95% interval for the development correlation is obtained by taking \(0.2027\pm1.96(0.0502)\), or approximately \((0.1043,0.3011)\), and transforming back to get \(r_D\) approximately \((0.104,0.292)\). Squaring gives an approximate \(R^2\) interval of \((0.0108,0.0853)\).

For the target correlation, \(0.1003\pm1.96(0.0502)\) gives approximately \((0.0019,0.1987)\) on the transformed scale, or \(r_T\) approximately \((0.0019,0.196)\). The corresponding \(R^2\) interval is approximately \((0.000004,0.0384)\). These broad, overlapping intervals caution against treating the point estimates as precise population values.

For a direct approximate comparison of independent correlations, the transformed difference is \(0.2027-0.1003=0.1024\), with standard error \(\sqrt{1/397+1/397}=0.0710\). The test statistic is \(0.1024/0.0710=1.44\), which gives a two-sided \(p\)-value of about 0.15 under the equal-correlation null and the approximation’s assumptions. This does not establish that the population correlations are equal; it indicates that these samples provide limited evidence against equality at conventional significance levels. A formal analysis should account for the actual design, including dependence if samples overlap and any relevant covariates or ascertainment.

A Practical Diagnostic Sequence

The order of checks matters because later explanations can be misleading when basic comparability has not been established. A useful analysis keeps the performance measure fixed where possible, reports its uncertainty, and treats proposed explanations as hypotheses to investigate rather than conclusions inferred from one contrast.

Verify phenotype comparability.
Check trait definition, units, ascertainment, and measurement quality. Distinguish a simple change of units from real population differences in phenotype or residual variation.

Reconstruct the performance summary.
Report score and phenotype standard deviations, regression slopes, correlations, and \(R^2\). Use \(R^2=b^2s_S^2/s_Y^2\) to see how their observed values fit together.

Investigate genetic tagging and effects.
Examine whether LD patterns change the score’s tagging of relevant variants and whether estimated effects differ, while accounting for uncertainty and possible weight error.

Check target calibration.
Compare the target prediction intercept and slope with the development predictions. Keep calibration distinct from ranking and from absolute-risk interpretation.

Put uncertainty around the gap.
Estimate intervals for performance and differences using methods that reflect the sample design. Avoid ranking explanations by point estimates alone.

Common Pitfalls and Takeaway

Calling a unit change a performance change. A slope changes when the phenotype unit changes. Check correlations or \(R^2\) as well, and remember that multiplying the phenotype by a negative constant reverses correlation’s sign while leaving \(R^2\) unchanged.
Attributing every lower \(R^2\) to different genetic effects. Score spread, phenotype variation, LD tagging, effect heterogeneity, measurement, and sampling uncertainty can all matter. The observed \(R^2\) alone does not separate them.
Treating a calibration slope as a ranking statistic. The slope compares predicted and observed contrasts on a scale; it is not itself a measure of whether individuals are ordered correctly.
Reading overlapping intervals or a nonsignificant test as proof of equality. The result may simply be imprecise. State what the data support and how uncertain the comparison remains.

Key takeaway: Trace a portability gap in sequence: establish phenotype comparability, inspect score and phenotype variation, investigate LD tagging and effect heterogeneity, assess calibration, and quantify uncertainty. A careful analysis separates observed performance from explanations for it.

Check Your Understanding

Use the definitions and calculations in this tutorial to answer the following questions.

If a quantitative phenotype is multiplied by \(-2\) and shifted by 5, what happens to its correlation with the score and to \(R^2\)?
With \(b=1.5\), \(s_S=0.8\), and \(s_Y=10\), calculate the one-predictor regression \(R^2\).
In the two-variant example, which quantity changed in the LD-only comparison, and why did the score–phenotype covariance change?
What does a target calibration slope of 0.85 say about prediction contrasts, and what does it not establish about ranking?
Why does a nonsignificant comparison of two sample correlations not prove that the population correlations are equal?