Why Analyze Unmeasured Participation?
A polygenic score can appear more or less predictive in a study than it would be in the population the study is intended to represent. Inverse-probability weighting can address participation differences when the relevant predictors of participation are measured and the required probabilities can be estimated. But some factors may be unrecorded. If such a factor is associated with phenotype risk and also affects who participates, the observed participants can have a different score–phenotype relationship from the target population.
Sensitivity analysis does not reveal the unknown participation process from the observed data alone. Instead, it specifies plausible values for the unmeasured factor’s distribution, its relation to phenotype risk, and its relation to participation, then calculates the consequences. A useful question is not simply whether selection bias is possible, but how strong selection would have to be to account for the observed performance.
Set Up a Selection Sensitivity Model
Let \(G\) denote a polygenic-score group, such as a lower-score or higher-score category; \(Y\) denote a binary phenotype; \(U\) denote an unmeasured binary factor; and \(S=1\) indicate participation in the study. The target population is defined before selection, and the observed evaluation sample consists of individuals with \(S=1\). For each score group \(g\), define \(q_g=P(U=1\mid G=g)\) and \(r_{gu}=P(Y=1\mid G=g,U=u)\) in the target population.
A simple model assumes that participation depends on \(G\) and \(U\), but not additionally on \(Y\) once \(G\) and \(U\) are specified. This is a substantive assumption, not a consequence of having a large sample. Under it, participants are selected within each score group according to their \(U\) values. The model also assumes that the specified \(q_g\), \(r_{gu}\), and \(s_{gu}\) are appropriate for the target population and that participation probabilities are positive for the relevant groups. If participation depends directly on phenotype status even after conditioning on \(G,U\), a more general calculation is needed, shown later.
The target-population risk in group \(g\) is a mixture over the two values of \(U\):
In the selected sample, the probability mass contributed by \(U=1\) is proportional to \(q_gs_{g1}\), while the mass contributed by \(U=0\) is proportional to \((1-q_g)s_{g0}\). The corresponding selected risk is therefore
Only the relative participation probabilities within a score group affect this risk. Define \(k_g=s_{g1}/s_{g0}\), the participation ratio for \(U=1\) versus \(U=0\) in group \(g\). Dividing the numerator and denominator by \(s_{g0}\) gives
When \(k_g=1\), participation does not differentially select on \(U\) within that score group, so the selected and target risks agree under this model. When \(k_g>1\), participants are enriched for \(U=1\); when \(k_g<1\), they are depleted. If \(U\) predicts the phenotype, changing its representation can change the selected risk.
Worked Calculations
Worked Example: Calculating Selection-Induced Score Performance
Suppose the target population has lower-score and higher-score groups. In both groups, \(q_g=0.25\), the phenotype risks are \(r_{g0}=0.06\) for \(U=0\) and \(r_{g1}=0.30\) for \(U=1\). Thus the target risk in either group is \(0.75(0.06)+0.25(0.30)=0.12\); the score has no target-population risk contrast in this hypothetical example. Suppose participation ratios are \(k_{\text{low}}=0.5\) and \(k_{\text{high}}=2\).
For the lower-score group, the selected risk is
For the higher-score group, it is
The selected-sample risk difference, higher-score minus lower-score, is \(0.1560-0.09429=0.06171\), or about 6.17 percentage points. The selected risk ratio is \(0.1560/0.09429=1.655\). In this constructed example, differential participation creates an apparent score-group contrast even though the target risks are both 12%. This is a sensitivity calculation, not evidence that this particular selection process occurred.
Worked Example: Finding the Selection Strength Needed to Explain a Contrast
Continue with the same hypothetical target risks and \(q_g=0.25\). Suppose the lower-score group has \(k_{\text{low}}=0.5\), and the observed higher-score risk is \(0.1560\). To find the high-group participation ratio that would generate this risk, set the selected-risk formula equal to \(0.1560\), writing \(k\) for \(k_{\text{high}}\):
Multiplying through by the denominator gives \(0.1170+0.0390k=0.0450+0.0750k\). Hence \(0.0720=0.0360k\), so \(k=2.000\). The corresponding contrast requires a participation ratio of 2 in the higher-score group versus 0.5 in the lower-score group, a fourfold difference between those within-group ratios. For example, these ratios could arise from participation probabilities 0.40 versus 0.20 for \(U=1\) and \(U=0\) in the higher-score group, and 0.10 versus 0.20 in the lower-score group. The example quantifies the required selection under the stated assumptions; it does not establish that those probabilities are plausible in a particular study.
Worked Example: Checking a Model-Implied Risk Difference
Suppose a sensitivity model gives the following participation-weighted case and noncase contributions for two score groups. For the higher-score group, the contributions are 0.1425 for cases and 0.5685 for noncases; for the lower-score group, they are 0.0745 and 0.6435. The model-implied selected risks are each the case contribution divided by the sum of the case and noncase contributions.
For the higher-score group, \(r_{\text{high}}^S=0.1425/(0.1425+0.5685)=0.1425/0.7110\). For the lower-score group, \(r_{\text{low}}^S=0.0745/(0.0745+0.6435)=0.0745/0.7180\). Using the unrounded risks, the selected risk difference is
Thus, under these hypothetical contributions, the higher-score group’s selected risk exceeds the lower-score group’s by about 9.67 percentage points. Keeping the unrounded fractions until the subtraction avoids small discrepancies caused by rounding each risk first.
A Short Derivation and a Broader Selection Model
The key weighting formula follows directly from conditioning on participation. Under the simple model, the joint probability of phenotype and participation within score group \(g\) is
Substituting the definitions gives \((1-q_g)r_{g0}s_{g0}+q_gr_{g1}s_{g1}\). Likewise, \(P(S=1\mid G=g)=(1-q_g)s_{g0}+q_gs_{g1}\). Dividing the first expression by the second yields the selected-risk formula. The derivation makes the assumptions visible: in particular, it factors the joint probability of \(Y\) and \(S\) conditional on \(G,U\), which requires that participation not depend additionally on \(Y\) once \(G,U\) are fixed.
If participation may depend directly on phenotype status, define \(a_{gyu}=P(S=1\mid G=g,Y=y,U=u)\). The selected risk then becomes
This extension allows cases and noncases to have different participation probabilities even after accounting for \(G,U\). It requires additional sensitivity parameters, so the data generally cannot determine all of them when \(U\) is unmeasured.
Worked Example: Allowing Participation to Depend on Phenotype
For one score group, suppose \(P(U=1)=0.25\), \(P(Y=1\mid U=1)=0.40\), and \(P(Y=1\mid U=0)=0.08\). Let the participation probabilities be \(a_{1,1}=0.80\) for a case with \(U=1\), \(a_{0,1}=0.40\) for a noncase with \(U=1\), and \(a_{1,0}=a_{0,0}=0.50\) for either phenotype status with \(U=0\). The target risk is \(0.25(0.40)+0.75(0.08)=0.1600\).
The selected case contribution is \(0.25(0.40)(0.80)+0.75(0.08)(0.50)=0.0800+0.0300=0.1100\). The selected noncase contribution is \(0.25(0.60)(0.40)+0.75(0.92)(0.50)=0.0600+0.3450=0.4050\). Therefore the selected risk is \(0.1100/(0.1100+0.4050)=0.2136\). In this example, the selected risk is about 21.36%, compared with the target risk of 16.00%. The difference arises from the stipulated selection probabilities; it is not removed by knowing only that \(U\) predicts the phenotype.
How to Use and Interpret a Sensitivity Analysis
A useful analysis evaluates a range of assumptions rather than presenting one convenient parameter choice as established fact. Vary the prevalence of \(U\) across score groups, its phenotype risks, and its participation ratios; then recompute the score-performance measure for each combination. If the measure is a risk difference, report it in percentage points and state which group is subtracted from which. A tipping point is a parameter combination at which the adjusted contrast reaches a specified value, such as zero. It answers how strong the assumed selection must be to remove or produce a contrast under that model.
The calculations should use quantities on the same target-population and outcome definitions as the intended performance claim. A risk difference and a risk ratio answer different questions; a sensitivity result on one scale does not automatically settle the other. Likewise, a risk contrast across score groups is a predictive association, not evidence that changing the score or its underlying variants would cause a change in phenotype risk.
Selection diagrams provide a useful conceptual check: if both the score group and \(U\) affect participation, restricting analysis to participants can induce an association between them. If \(U\) also predicts the phenotype, that induced association can alter score performance. This mechanism is often described as collider bias. The sensitivity model translates the qualitative mechanism into numerical assumptions and consequences.
Common Pitfalls
- Treating assumed parameters as estimated facts. When \(U\) is unmeasured, its prevalence, risk association, and participation association are not generally identified from participant data alone. Label the values as assumptions and show how conclusions change across defensible values.
- Using a participation ratio without specifying its direction. State whether \(k_g\) compares participation for \(U=1\) with participation for \(U=0\). Reversing that ratio reverses the interpretation.
- Assuming selection depends only on \(U\) when it may also depend on \(Y\). The simpler formula requires no additional outcome-dependent selection conditional on \(G,U\). If that assumption is doubtful, use a model with \(a_{gyu}\) and acknowledge that more assumptions are needed.
- Calling a sensitivity result a correction. A single set of assumed parameters does not identify the true target performance. Describe it as a scenario or sensitivity estimate, and give the assumptions that produce it.
- Rounding intermediate risks too early. Compute the final contrast from unrounded fractions or values, then round the reported result. State the units, such as percentage points for a risk difference.
Check Your Understanding
Use the model definitions and assumptions above to answer the following questions.
- Under the simple selection model, what does \(k_g=1\) imply about the selected and target risks within score group \(g\)?
- If \(U\) is more common among participants than nonparticipants within a score group and \(U\) increases phenotype risk, what direction of change in selected risk would you generally expect?
- Why can a score show a selected-sample risk contrast even when the target-population risks are equal across score groups?
- What additional probabilities are needed if participation depends directly on phenotype status after conditioning on score group and \(U\)?
- What does a tipping point establish, and what does it not establish, when \(U\) is unmeasured?